Evaluation 2020
Evaluation results - Adult Education Academy (former Winter School) 2020
Summary of the external evaluation results
Author: Katrin Riß
INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer
The Winter School Programme 2020
In February 2020, the INTALL-Winter School was held for the second time. Compared to the previous year, three main modification were applied, which again proves the programme’s adaptability and the organizers’ constant effort to improve the Winter School according to evaluation results and observations made. First, the heterogeneous interests and needs of the target group of practitioners were taken more into account. By opening both Winter School weeks to the group of practitioners, more options were given especially to those who were interested most in dealing with theoretical approaches of adult education and lifelong learning (week 1). Second, in 2020 the organizers provided a more differentiated and extended programme regarding the topic of employability, a component that stuck out in the 2019 evaluation. And third, responding to the considerable popularity of the Winter School, the organizers increased the number of participants to a total of 83 (during week 2) compared to a total of 67 in 2019.
Approach, data basis and perspectives of the external evaluation
Some alterations aside, the short-term survey 2020 continued to take a quantitative approach. The survey was based on the very questionnaire used in the evaluation cycle 2019, implemented as an online survey at the end of the Winter School. In total, 70 questionnaires were filled in, which sum up to a response rate of 84%. To give a more detailed picture, in 2020 the study often differentiates between the subgroups of the survey’s population, i.e. the master students, the doctoral students and the practitioners.
Sample/group characteristics
The group of participants was again particularly characterized by its great internationality. The number of countries of the students’ home universities or employers totaled to 16 in 2020, a little more than half of them were European countries (56%). Further the participants backgrounds included 33 different countries of origin and 26 different countries of present residence. This range again far exceeded the consortium of the ten INTALL partners and reflects the considerable internationality of the INTALL partners as well as the remarkable outreach of the programme.
In 2020, a little less than half of the respondents were studying for a master’s degree at the time of the Winter School (31 respondents, 44%). Around one third were PhD students (25 respondents, 36%) and 1 out of 5 belonged to the group of practitioners working in the field of adult education and lifelong learning (20%).
Regarding the gender distribution, 71,4% of the group were female, 28,6% male.
Participants’ Satisfaction
The international Winter School in Wuerzburg has received remarkably good evaluation results during the last years. The Winter School in 2020 is no exception to this. In an overall perspective, the participants rated the Winter School programme with a mean value of 4,33 (SD 0,675) on the 5-point-scale and thus nearly as high as in 2019 (4,39). While the practitioners stayed very steady in their assessment (from 4,11 in 2019 to 4,14 in 2020), the doctoral students rated a little lower (from 4,47 in 2019 to 4,04 in 2020) and the master students a little higher (from 4,42 in 2019 to 4,65 in 2020).
With regard to the organisational, academical and didactical quality of the different programme modules, the results turned out to be rather high. The respective mean values ranged between 3,70 and 4,56. Strongest modules seemed to be the Preparatory Phase and “Lifelong Learning Strategies in Europe/ International Strategies in Adult Education”. At this point we see a rising tendency since 2019. 10 out of 15 mean values (5-point-scale) turned out to be higher than in the previous Winter School.
Also, the participants were asked how useful they considered the different programme components for their own development (rating on a 5-point scale with 1 = not useful and 5 = very useful). Almost all mean values of the components turned out to be higher than the value of 4,00. 23 out 35 items were rated higher than in the 2019.
Effects of the Winter School
The results of the short-term-survey in 2020 equal those of 2019 remarkably. Again, the Winter School had strengthened the participants’ interest in adult education and lifelong learning considerably and also increased their interest and motivation to focus on its trans-/international and comparative aspects in their further studies or future professional activities. All items asked for in the questionnaire received very high mean values between 4,04 and 4,29 on the 5-point scale (1=not at all, 5=very much). Taking a closer look at the subgroups the survey shows that doctoral students tended to rate these questions a little lower.
| Total | Masters students | Doctoral students | Practitioners | ||||
| N=70 | N=31 | N=25 | N=14 | ||||
| Mean | STD | Mean | STD | Mean | STD | Mean | STD |
Interest in adult education and lifelong learning increased
| 4,29 | 0,887 | 4,45 | 0,624 | 4,04 | 0,978 | 4,36 | 1,151 |
Interest in trans-/international and comparative adult education and lifelong learning increased
| 4,12 | 1,022 | 4,33 | 0,802 | 3,72 | 1,308 | 4,36 | 0,633 |
Motivation to focus on trans-/international aspects in your further studies
| 4,06 | 1,006 | 4,33 | 0,661 | 3,80 | 1,190 | 3,91 | 1,221 |
Motivation to have a trans-/international focus in future professional activities increased
| 4,14 | 1,004 | 4,32 | 1,013 | 4,00 | 0,978 | 4,00 | 1,038 |
Also, participating in the programme seemed to increase the respondent’s motivation to undertake further study mobilities considerably. 84% of the master and doctoral students (N=56) scored a 4 or 5 at this question. The respective mean value was 4,32 (STD 1,011).
Further, 66% of the master students and practitioners (N=45) ticked a 4 or 5 when being asked if their motivation to take up PhD studies had grown (2019: 56%). 64% of the doctoral students (N=25) chose a 4 or 5 to indicate that their interest in taking up postdoc research after graduation had increased due to the participation in the Winter School.
And finally, the Winter School also intends to have an effect on competencies related to international or internationally influenced working contexts. In 2020 again, a large share of respondents clearly agreed on these effects. First, the Winter School obviously has a large impact on the development of intercultural competencies. 82% of the participants indicated that the Winter School had helped to improve their “competencies in interacting with people from other cultural backgrounds” considerably by rating a 4 or 5 (mean value 4,21 / STD 1,001). Also, 87% of the respondents stated that their “understanding of adult education and lifelong learning in other countries” had improved by ticking a 4 or 5 (mean value 4,37 / STD 0,745). 74% scored a 4 or 5 when being asked in how far the Winter School had helped to “develop different perspectives on adult education and lifelong learning in their own country” (mean value 4,06 / STD 1,048).
Regarding methodological skills, analytical competencies and professional network competencies the results turn out to be a bit lower. The mean values at the respective items are 3,81 (STD 1,115), 3,78 (STD 1,104) and 3,76 (STD 1,173). The results of the different subgroups at these effects differ considerably. Except for a better understanding of adult education and lifelong learning in general, master students seem to profit most regarding the development of their competencies.
| Total | Master students | Doctoral Students | Practitioners | ||||
N | 70 | 31 | 25 | 14 | ||||
| Mean | STD | Mean | STD | Mean | STD | Mean | STD |
understanding of AE and LLL in other countries improved
| 4,37 | 0,745 | 4,48 | 0,724 | 4,16 | 0,850 | 4,50 | 0,519 |
helped me to see AE and LLL in my own country from a different perspective
| 4,06 | 1,048 | 4,45 | 0,675 | 3,76 | 1,200 | 3,71 | 1,204 |
methodological skills for conducting comparative research work improved
| 3,81 | 1,115 | 4,23 | 0,805 | 3,44 | 1,294 | 3,54 | 1,127 |
analytical competencies improved | 3,78 | 1,104 | 4,37 | 0,556 | 3,16 | 1,214 | 3,62 | 1,193 |
professional networking competencies improved | 3,76 | 1,173 | 4,16 | 0,860 | 3,32 | 1,376 | 3,67 | 1,155 |
competencies in interacting with people from other cultural backgrounds improved | 4,21 | 1,001 | 4,42 | 0,720 | 3,96 | 1,241 | 4,17 | 1,030 |
Table 1: Effects - Competencies (by subgroups)
Conclusions
The short-term evaluation of the INTALL-Winter School 2020 again shows that the programme is highly appreciated by its participants and the effects must be considered as high. In an overall perspective the results equal those of the previous year(s) and continue to be on a very high level. Again, the evaluation shows that especially the welcoming, respectful and encouraging learning atmosphere is of great value to the participants. Particularly good and noticeably higher ratings compared to 2019 were given to the module “Lifelong Learning Strategies in Europe/ International Strategies in Adult Education” which is realized during week 1 of the Winter School.
In an overall perspective, it seems that the master students are most satisfied with the Winter School experience. Looking at their motivation we saw that they started with the broadest interest in all aspects and learning opportunities of the programme and also have profited in a very broad and multiple sense. Doctoral students obviously seemed to be focused especially on the content level of the Winter School and in general gave lower ratings than the masters (and mostly practitioners) did. Practitioners finally entered the programme mostly for reasons of exchanging with other experts/fellows in the field as well as looking for expertise in a theoretical but also practical sense, and in the end seem to be fairly satisfied with the Winter School.
The evaluation 2020 produced the following conclusions for further improvement, which however only concern smaller aspects and do not show a need for greater modifications: (1) Since 2019, considerably chances have applied with regard to the practitioners and obviously already have led to various effects. The practitioners’ role seems to be clearer, the integration in the group obviously was improved and all in all we find the practitioners’ satisfaction with the programme to be quite high. Nevertheless, some (single) findings suggest that in week 2 the role of practitioners during the comparative groups could be further developed. (2) The Employability Day has been changed since 2019 considerably and in 2020 the respective evaluation results have improved. Nevertheless, this module seems to bear further developmental potential compared to the other parts of the programme. Unfortunately, the findings give no further hints in which way this component should be altered.
