Evaluation 2019

Evaluation results - Adult Education Academy (former Winter School) 2019

Summary of the external evaluation results
Author: Katrin Riß
INTALL-Project Coordinator: Julius-Maximilian University Würzburg, Prof. Dr. Regina Egetenmeyer

Approach

The short-term evaluation in 2019 took a mainly quantitative approach. The questionnaire used was based on the instrument applied in the years before and adjusted to the current Winter School by the organisers. The findings result from a total of 67 questionnaires (response rate 100%), filled in online at the end of the Winter School. 

Sample/group characteristics 

The group of participants was formed by students from 20 different countries, about two third of them being master students (43 persons, 64%), 1 out of 4 being a doctoral student (15 persons, 22%), and about 1 out of 7 working as practitioners in the field of adult education and lifelong learning (9 persons, 14%).

Participants’ Satisfaction

The evaluation clearly shows that, all things considered, the participants were highly satisfied with the Winter School. On a 5-point scale (1 = poor, 5 = excellent) their overall assessment reached a mean value of 4,39. These very high ratings confirm the results of the COMPALL evaluations in the years 2016 to 2018 assessed with similar mean values (2016: 4,20, 2017:4,24). Taking a closer look, we also see that no outliers at all can be found in that question and the high ratings apply to all student groups (master students, doctoral students, practitioners) with the ratings of the practitioners being the lowest.

With respect to the different programme components, especially Week 1 and the Comparative Groups during Week 2 achieved very high ratings in the categories organisational, academic, didactical quality (all means values ranging between 4,13 and 4,40 on a 5-point scale with 1 = not satisfied, 5 = very satisfied). Ratings a little lower were given to the Preparatory Phase (mean values for the three categories 4,04 – 3,93 – 3,99) varying from country to country. The lowest ratings finally were registered with regard to the Employability Day at the beginning of Week 2 (mean values 3,72 – 3,67 – 3,79).

According to this data but also to the participants’ qualitative feedback the Winter School appears to be a learning setting well accepted and appreciated by the students. Various comments highlighted the didactical quality of the sessions, a learner centric and participative approach including a broad variety of (innovative) teaching methods, as well as respectful, unformal and supportive atmosphere at the Winter School.

Effects of the Winter School

The International Winter School mainly intends to introduce its participants to international issues and comparative aspects within adult education and lifelong learning and to provide them with the respective competencies and skills required in international or internationally influenced working contexts. Based on the participants’ answers it can be concluded that the Winter School widely accomplishes these goals. The majority of the students strongly agrees that their interest in trans-/international and comparative aspects of adult education and lifelong learning has increased and that they are motivated to further focus on these aspects in their studies and even in their future professional activities (see table). Concerning the question if their motivation to undertake further study mobility increased, more than 90% of the master and doctoral students (N=58) scored a 4 or 5 on the 5-point scale. Also, the motivation to take up further studies was increased: 56% of the master students and practitioners rated a 4 or 5 on the scale expressing their motivation to take up a PhD. 62% of the doctoral students (9 persons) stated with a score of 4 or 5 that their interest in taking up postdoc research after their graduation increased due to the participation in the Winter School.

With regard to the effects of the Winter School, the questionnaire also asked about the development of competencies induced by participating in the programme. As the chart below illustrates all ratings are considerably high, especially concerning aspects that refer to “international” competencies and knowledge. For example, almost all participants (93%) claimed that their understanding of adult education and lifelong learning in other countries has increased very much or much. 78% of the group have ticked a 4 or 5 when being asked if their intercultural competencies have improved. The lowest values – even if still high – on the other hand we find for the question concerning the increase of analytical competencies. At this point, it has to be kept in mind that 2/3 of the participants had stated that these competencies had been high already before the Winter School.

When taking a closer look at the outcomes for the different groups of participants, we see that master and doctoral student obviously profited the most from participating in the Winter School. This regards the academic outcomes (mean value 4,38 on the 5-point scale) as well as the personal ones (mean value 4,43). For the group of practitioners, who were newly addressed by the INTALL Winter School the outcome, however, resulted a little lower. This group rated their academic outcome with a mean value of 3,56 (on the same 5-point scale), the personal outcome with 4,0, and finally the outcome for the employment with a mean value of 3,88.

Conclusions

The evaluation’s findings confirm and continue the consistent positive or more precisely rising results of the COMPALL evaluation and attest the Winter School to be a well-designed and effective learning opportunity for its participants. The results prove a respectful learning environment that obviously encourages its participants to develop confident research activities and improves employability skills. Apart from social and especially intercultural outcomes the evaluations points to considerable academic learning effects for the participants. This regards knowledge about international contexts of adult education and lifelong learning as well as the improvement of academic skills – most striking of course comparative skills.

Despite this already high standard, the evaluation’s results point to some modifications on the concept and programme. This concerns the following aspects: (1) Some findings suggest that the concept regarding the new target group of practitioners can be further adapted in order to increase the effects and to fully exploit the chances arising from bringing students and practitioners together. Obviously, the role of the practitioners was not very clear to the group at every point of the Winter School and chances to contribute their specific perspective on the subjects discussed should be inhanced. Also, it seems that the preparation of the practitioners should receive more attention. (2) Other findings indicate that the concept of the Employability Day needs to be revised. Obviously, this module did not fully match the participants’ needs and some of them were hoping for more profound information. At this point, some more thought might be given to the variety of participants and strategies should be found to take the differing prior knowledge more into account (splitting the group, using methods that allow more options and flexibility, using the expertise of the practitioners etc.). (3) This piece of work obviously plays an important part during the preparatory phase of the Winter School and contributes remarkably to the learning effects of this programme. In this process it requires considerable effort and time to prepare and thus seems to be highly important to the participants. At some points the findings of the evaluation suggest that during the intensive phase some more time and focus should be spent on these essays in order to provide more content-related feedback as well as appreciation for the effort taken. Considering the little data base this fact might only regard single comparative groups. In any case, we recommend ensuring the same intensity in dealing with the transnational essays in all comparative groups. (4) As in the years of the COMPALL Winter School, the programme schedule remains a quite striking aspects in the evaluation. Various comments suggest that especially during Week 1 efforts should be taken to keep participants attentiveness high. This might include changes in the schedule like shifting intense lectures to an earlier day time, but also using (even) more interactive teaching methods and providing more opportunities for interaction among the group.