Evaluation 2021
Evaluation results - Adult Education Academy 2021
External Evaluation: Katrin Riß
INTALL-Project Coordinator: Prof. Dr. Regina Egetenmeyer (Julius-Maximilian University Würzburg)
This outline summarizes the main results of the short-term evaluation of the Adult Education Academy (AEA) 2021 – until 2020 referred to as the “INTALL Winter School”. The respective evaluation step was the fifth out of six within the programme ‘INTALL - International & Comparative Studies in Adult Education & Lifelong Learning’ (2018-2021). Further details are spelled out in the evaluation report.
Since the beginning of our external evaluation in 2019, we continuously see a very positive feedback by the participants and even a rising tendency in most of the findings. All in all, the programme seems to be well geared to its target groups and successfully reaches its goals in fostering interest, knowledge and competences in the field of adult education and lifelong learning. It catches one’s eye that other than in 2020, when master students tended to rate highest, in 2021 the doctoral students seemed to be most satisfied with the AEA. Practitioners obviously were only slightly less content with the programme. Master students close the list of subgroups, still being very pleased with the AEA.
Following we will pick out some findings to illustrate this overall picture. Also, we will summarize some results regarding the effects of the AEA.
The AEA 2021 – programme and evaluation
In 2021, the worldwide coronavirus pandemic took its toll on the AEA. Due to the binding contact and travel restrictions, the programme had to be transferred from a blended learning module to a fully virtual programme. All participants and all staff attended the AEA via their digital devices at home. All lectures and group work (including excursions and socializing events) were realized via video chatting services and further online collaboration tools. Nevertheless, the organizers succeeded to implement the agenda approved during the previous year without any major changes. Some modules were further optimized (following previous evaluation results). For example, the module “Employability in adult education” was expanded and transferred to the online preparation phase.
The external evaluation continued to work with the questionnaire developed by the organizers in previous years. In 2021, the analysis is based on a total of 61 cases.
Participants’ profile
Quite some questions of the short-term evaluation deal with the participants’ profile and their run-up to the AEA. The following findings intend to give an idea of the group.
- At the time of the AEA, 46% of the respondents (28 persons) were studying for a master’s degree, 29% (18 respondents) were doctoral students, and 23% (14 respondents) were practitioners (in the field of adult education and lifelong learning).
- Again, the AEA impressed with its broad internationality. According to the registration data participants of 21 different countries of origin joined the programme, studying in 15 different countries. These included the countries of the INTALL consortium as well as others, a fact that again reveals the internationality of students studying at the INTALL partner universities.
- In general, participants apply for the AEA due to content-related reasons, due to the wish to meet student/lecturers/practitioners from other countries and due to career-related motives equally. Besides, half of the doctoral students (50% / N=18) and around one third of the practitioners (31% / N=13) indicated to have registered in the AEA 2021 because they had attended it before (respectively the predecessor winter schools). This applies to none of the master students. Also, recommendations of former participants seem to be a strong inducement to participate. Among the master and doctoral students this applies for roughly half of the group (master students: 46% / N=26, doctoral students: 56% / N=18). Also, 30% of the practitioners agreed to that (N=13).
- 59% (26 respondents / N=44) (2020: 70% / N=56) of the master and doctoral students indicated that they had received some kind of support by their home universities. 95% of them agreed that this support had been helpful (4 or 5 on the 5-point-scale, mean value 4,77 / STD 0,528 / N=22).
- Almost half of the group had not undertaken any international study mobilities before attending the AEA (49% / N=61). 38% indicated they did so once. Obviously, these shares differ between the subgroups. While for 61% of the master students (N=28) the AEA was their first study mobility, this is true for 44% of the doctoral students (N=18) and for 36% of the practitioners (N=14).
- In 2021, the share of participants who had never or only little been in touch with trans- or international topics in their study or work before was bigger than in 2020. All the same, 70% (N=54) of the respondents agreed they already had been highly interested in trans-/international and comparative adult education/lifelong learning in particular before the AEA. Also, 73% (N=51) had been highly interested in focussing on trans-/international aspects in their further studies and even 81% (N=52) in their future professional activities.
- Further, 70% of the master students (N=23) and 75% of the practitioners (N=8) stated they were highly motivated to take up PhD studies before the AEA. 81% of the doctoral students (N=16) were highly motivated to take up postdoc studies after their graduation.
Assessment of the programme
As mentioned before, the overall assessment of the AEA in 2021 turned out exceptionally well. The mean value at the question “What is your overall evaluation of the AEA?” was 4,68 (STD 0,831 / N=47) (five-point-scale, 1=poor, 5=excellent). The subgroup of doctoral students even rated with the highest possible rating at all (mean value 5,00 / STD 0 / N=15). This general picture is also confirmed by a set of questions asking how satisfied students were with the organizational, academical and didactical quality of the different programme modules. None of the modules achieved mean values lower than 4,20. Doctoral students were most satisfied.
In the evaluation participants were also consulted, how useful they considered the different components for their own development. Except for one component all mean values lie above 4,00 and mostly range around 4,50 on the five-point-scale. Compared to 2020, most of the results turned out higher.
Effects of the Adult Education Academy
The evaluation also keeps track of the effects and benefits of the AEA as assessed by its participants. In 2021, obviously the strongest effects were achieved with the group of doctoral students, followed by the practitioners and then the group of master students.
All results regarding the effects on the thematic interest in adult education and lifelong learning as well as its international aspects turned out higher than in the previous years. All mean values ranged between 4,31 and 4,56 (N=55) on the 5-point-scale (2020: 4,04 to 4,29, 2019: 4,16 to 4,31). The effects on the motivation to undertake further study mobilities continued to be high. 90% (N=41) of the master and doctoral students ticked a 4 or 5 at the respective question. With regard to further study plans 63% of the master students and practitioners (N=32) stated that the AEA had increased their motivation to pick up doctoral studies (4 or 5 on the 5-point-scale). 87% of the doctoral students (N=15) confirmed this effect with regard to a possible post-doc research after their graduation (4 or 5 on the 5-point-scale).
Effects on the interest in adult education and lifelong learning
Further, the findings on the effects on competences related to international study and work contexts in adult education and lifelong learning resulted very high and exceeded the values of the previous year. Again, the ratings of the doctoral students turned out to be the highest among the subgroups (mean values between 4,31 to 4,75 with a relatively low spread of data points). Little lower we find the results of master students (mean values from 3,54 to 4,27).
|
| Total |
| Master students |
| Doctoral Students |
| Practitioners | ||||
| N | Mean | STD | N | Mean | STD | N | Mean | STD | N | Mean | STD |
understanding of AE and LLL in other countries improved | 54 | 4,52 | 1,041 | 26 | 4,27 | 1,343 | 16 | 4,75 | 0,577 | 12 | 4,75 | 0,622 |
helped me to see AE and LLL in my own country from a different perspective | 53 | 4,42 | 1,082 | 24 | 4,21 | 1,414 | 16 | 4,75 | 0,447 | 13 | 4,38 | 0,870 |
methodological skills for conducting comparative research work improved | 54 | 4,26 | 1,013 | 25 | 4,08 | 1,320 | 16 | 4,56 | 0,512 | 13 | 3,85 | 0,725 |
analytical competencies improved | 54 | 4,00 | 1,229 | 25 | 3,88 | 1,453 | 16 | 4,31 | 0,704 | 13 | 3,85 | 1,281 |
professional networking competencies improved | 53 | 4,13 | 1,144 | 24 | 3,92 | 1,283 | 16 | 4,56 | 0,727 | 13 | 4,00 | 1,225 |
competencies in interacting with people from other cultural backgrounds improved | 54 | 4,30 | 1,039 | 25 | 4,08 | 1,222 | 16 | 4,75 | 0,447 | 13 | 4,15 | 1,068 |
Effects - Competences (by subgroups)
Perspectives
All in all the findings drew a very positive picture of the AEA, and it seems that the organizers successfully transferred the programme into an online concept.
In 2021 the AEA’s funding within the project ‘INTALL - International & Comparative Studies in Adult Education & Lifelong Learning’ will come to an end. Also, considering the current pandemic situation, it continues to be uncertain if the AEA – in case of further funding – will return to the face-to-face event in Wuerzburg next year. Nevertheless, before closing this report, we would like to highlight some final aspects worth mentioning or worth considering for future programmes. (1) As in the reports 2019 and 2020, at this point we will pick out the employability module. This module underwent several changes over the last years. In 2021 now, it was extended, split up in several sessions and shifted to the preparation phase. Along with these changes the participants feedback has improved since 2019 remarkably. The results in this year’s evaluation finally are on par with those of the other modules. All subgroups seem to have benefitted more than in the years before. (2) In the past evaluations, respondents often highlighted the connecting and networking effect of the Winter School/AEA. At this of course the informal parts of the programme and the socializing components did their bit, however were hard to compensate within the virtual setting. Nevertheless, the results show that the AEA obviously succeeded in its goal to foster networking among its participants. (3) On the other hand, the virtual setting also implied a number of learning opportunities. Various comments positively referred to the variety of online applications for digital teaching and learning introduced by the AEA. However, some participants also felt that this variety was too vast. (4) Some comments finally suggested to further exploit the chances of the online modus. For example, field visits to practice institutions could be extended to institutions in other countries than just Germany.
