Country report for SELFIE WBL piloting Hungary Maria João Proença (EfVET) Miha Zimšek (Skupnost VSŠ) Anita Goltnik Urnaut (Skupnost VSŠ) Alicia Leonor Sauli Miklavčič (Skupnost VSŠ) Ralph Hippe (JRC) 2021 EUR 30771 EN This publication is a report by the Joint Research Centre (JRC), the European Commission’s science and knowledge service. It aims to provide evidence-based scientific support to the European policymaking process. The scientific output expressed does not imply a policy position of the European Commission. Neither the European Commission nor any person acting on behalf of the Commission is responsible for the use that might be made of this publication. For information on the methodology and quality underlying the data used in this publication for which the source is neither Eurostat nor other Commission services, users should contact the referenced source. The designations employed and the presentation of material on the maps do not imply the expression of any opinion whatsoever on the part of the European Union concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Contact information Name: Yves Punie Address: Edificio Expo, C/ Inca Garcilaso 3, 41092 Sevil e, Spain Email: yves.punie@ec.europa.eu EU Science Hub https://ec.europa.eu/jrc JRC125788 EUR 30771 EN PDF ISBN 978-92-76-40324-1 ISSN 1831-9424 doi:10.2760/25503 Luxembourg: Publications Office of the European Union, 2021 © European Union, 2021 The reuse policy of the European Commission is implemented by the Commission Decision 2011/833/EU of 12 December 2011 on the reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Except otherwise noted, the reuse of this document is authorised under the Creative Commons Attribution 4.0 International (CC BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). This means that reuse is al owed provided appropriate credit is given and any changes are indicated. For any use or reproduction of photos or other material that is not owned by the EU, permission must be sought directly from the copyright holders. Al content © European Union, 2021, except: cover image © j-mel - stock.adobe.com How to cite this report: Proença, MJ, Zimšek, M, Goltnik Urnaut, A, Sauli Miklavčič, AL, and Hippe, R. Country report for SELFIE WBL piloting. Hungary, EUR 30771 EN, Publications Office of the European Union, Luxembourg, 2021, ISBN 978-92-76-40324-1, doi:10.2760/25503, JRC125788. Abstract: This report presents the results of the pilot study of SELFIE for work-based learning carried out in Hungary between September and December 2020. The study aimed at testing the tool before its launch online. In total, 14 VET col eges and 38 companies (operating in different sectors) were engaged in the pilot, involving 2090 users (teachers, students, school leaders and in-company trainers). In addition, 312 individuals (students, teachers, school leaders, school coordinators and in-company trainers) participated in the qualitative research carried out after the pilot. This research included interviews and focus groups, with the purpose of col ecting further feedback. The overal results indicate that SELFIE WBL tool is user-friendly and easy to understand, wel designed, and inclusive with its 360-degree reflection, as it engaged al those involved in WBL activities in the Hungarian WBL system. The SELFIE WBL tool and the report provided support to school leaders in the development and monitoring of the school’s digital strategy as wel as provide relevant information to al stakeholders in the SELFIE WBL pilot, contributing to increasing the effectiveness of learning in VET schools and companies. Contents Acknowledgements..................................................................................................................................................................................................................................... 1 Executive summary .................................................................................................................................................................................................................................... 2 1 Introduction............................................................................................................................................................................................................................................... 4 2 Digital education and WBL policies..................................................................................................................................................................................... 5 3 Set up of the pilot............................................................................................................................................................................................................................... 7 3.1 Methodology for selecting the pilot schools and companies.......................................................................................................... 7 3.2 Methodology for translating and adapting SELFIE materials ..................................................................................................... 11 4 Pilot implementation..................................................................................................................................................................................................................... 12 5 Fol ow up: quantitative and qualitative analyses................................................................................................................................................ 15 5.1 Methodology ............................................................................................................................................................................................................................ 15 5.2 Quantitative results........................................................................................................................................................................................................... 17 5.3 Qualitative results ..............................................................................................................................................................................................................20 5.3.1 Initial motivation from participants.................................................................................................................................................. 21 5.3.2 Strengths and weaknesses of the SELFIE WBL tool .......................................................................................................... 21 5.3.3 Questionnaire, content, and SELFIE WBL report.................................................................................................................... 22 5.3.4 Current and future use of SELFIE WBL..........................................................................................................................................23 5.4 Overal findings.....................................................................................................................................................................................................................23 6 Lessons learnt and suggestions for future development ............................................................................................................................26 7 Implications of COVID-19 pandemic ..............................................................................................................................................................................28 8 Conclusions and recommendations .................................................................................................................................................................................29 References...................................................................................................................................................................................................................................................... 30 List of abbreviations and definitions.......................................................................................................................................................................................32 List of figures .............................................................................................................................................................................................................................................. 33 List of tables..................................................................................................................................................................................................................................................34 Annexes..............................................................................................................................................................................................................................................................35 Annex 1. Key information on the WBL system ..................................................................................................................................................... 36 Annex 2. Dominant economic sectors in Hungary ..............................................................................................................................................43 Annex 3. Guidelines and templates for focus groups, semi-structured interviews, and list of chal enges..... 44 Annex 4. Analysis of open question “Suggestions for improvement” and examples of questions......................... 54 Annex 5. School report “Overview of areas” ............................................................................................................................................................56 Annex 6. Figures and tables with results of SELFIE WBL piloting quantitative data ..........................................................58 Annex 7. Overview of SELFIE WBL results in Hungary................................................................................................................................... 63 Annex 8. Country fiche................................................................................................................................................................................................................. 72 Annex 9. List of tools similar to SELFIE and other tools used in WBL.............................................................................................. 75 i Acknowledgements We would like to thank al vocational education and training (VET) schools and companies that, on a voluntary basis, have participated in the SELFIE for work-based learning (WBL) piloting experience during the most chal enging period of the last century. A word of appreciation goes to the national coordination team who, tirelessly, assured the engagement and continuous support to participating schools and companies and provided the valuable information without which the pilot experience and the report would not have been possible. We would also like to thank DG Employment, Social Affairs and Inclusion and the European Training Foundation (ETF) for the effective col aboration al along the piloting phase and DG Education and Culture for the support. In addition, we would like to thank the national coordinators of al nine piloting countries for the fruitful exchanges and opportunities of mutual learning that have facilitated the piloting process. Final y, the active involvement and support of national VET and WBL stakeholders has been crucial in this endeavour of piloting SELFIE WBL during the COVID-19 crisis. Disclaimer The aggregated and anonymised data which is presented in this document has been extracted by the European Commission from SELFIE database and does not necessarily reflect an official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this document. Neither the Commission nor any person acting on the Commission’s behalf may be held responsible for the use which may be made of the information contained therein. The views expressed in this report are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission. Note: This report was produced in the framework of the contract agreement 939681 – 2020 BE between European Forum of Technical and Vocational Education and Training (EfVET) and European Commission’s Joint Research Centre (JRC) "Piloting SELFIE for work-based learning contexts in VET (SELFIE WBL), Lot 3: Hungary. 1 Executive summary SELFIE is an online self-reflection tool developed to support schools, including Vocational Education and Training (hereafter, VET), to reflect on their digital readiness and preparedness by looking at different dimensions such as VET school strategies, infrastructure, teaching practices, equipment, and the experience of students. The tool was developed in 2018 by the Joint Research Centre (JRC) and the Directorate-General for Education, Youth, Sport and Culture. In early 2020, in cooperation with the Directorate-General for Employment, Social Affairs and Inclusion, it was adapted to include a module on work-based learning which adds the views of in-company trainers. The aim has been to help improving coordination between VET schools and training companies, and to discuss how they could jointly embed digital technology in their training and apprenticeship programmes. This also means bringing VET teachers and in-company trainers closer together. Throughout 2020, the JRC launched a pilot experience of SELFIE for work-based learning contexts in VET (SELFIE WBL) in nine different countries. EfVET in col aboration with JRC organised them in France, Poland, Hungary and Germany. In addition, JRC managed the pilot in Romania. 4 additional non-EU countries (Georgia, Montenegro, Republic of Serbia, and Turkey) piloted SELFIE WBL managed by ETF and JRC. The piloting of SELFIE WBL in Hungary was launched in July 2020 and effectively roled out in September 2020. It entailed three main phases: 1. Translation of al supporting documents and the tool itself. 2. Selection and engagement of stakeholders (including VET schools and companies). 3. Piloting of the SELFIE WBL in the selected VET schools and companies. a. Qualitative research consisting of the organisation of focus groups with students and teachers in each one of the VET schools, in-depth interviews with school leaders and in-company trainers and additional desk research on similar self- reflection and other digital tools in use in the country. The main emphasis of the piloting experience was on the qualitative research as it al owed to col ect quality information with the view of contributing to practice development and improving the SELFIE WBL tool and its further development. 14 schools were involved in the qualitative research, 28 focus groups (total ing 141 teachers and 141 students) and 16 semi-structured interviews with 12 school leaders and 4 company representatives were organised which al ow the col ection of relevant feedback regarding the tool. It is necessary to say that the outcomes of the pilot are not representative at national level for the education and training systems. The pilot process was disturbed by the Covid-19 pandemic with the confinement measures taken by the Hungarian government impacting on the data col ection process and requiring great effort from the national team and the school coordinators to assure the delivery, as planned, of al activities. This also had a massive impact on the educational community’s state making it difficult to motivate and engage participants to fil out the SELFIE WBL tool. The overal feedback received was that the SELFIE WBL tool is user-friendly and easy to understand, wel designed, and inclusive with its 360-degree reflection, as it engaged al those involved in WBL activities in the Hungarian WBL system (students, teachers, school leaders and in-company trainers). The main chalenges reported by school leaders, trainers and students proved to be pedagogical support and, resources related to the digital competences and knowledge of the teachers, the digital learning skil s of the students and the overal implementation of digital technologies in the classroom. For in-company trainers, the biggest chal enges mentioned were the continuing professional development (CPD), and pedagogical support and resources. The SELFIE WBL tool and the report provided support to school leaders in the development and monitoring of the school’s digital strategy as wel as provide relevant information to al stakeholders in the SELFIE WBL pilot, contributing to increasing the effectiveness of learning in VET schools and companies. School leaders have also expressed the intention to use it on a regular basis. 2 School leaders have also expressed their interest in the next steps of SELFIE WBL, to explore further opportunities to facilitate the engagement of and impact on al stakeholders. Next to the technological aspect and competences, teachers’ attitudes towards the “digital world” and digitalisation in general have to be taken into consideration. School leaders shared their perspective regarding the importance of digitalization not only because of the pandemic, but rather as encouragement for al stakeholders (schools, companies) to increase the effectiveness of teaching and learning. Feedback provided was that the SELFIE WBL pilot came at the right time, not only for schools and their leaders, but also for teachers, students, and in-company trainers. The next chal enge wil be to act based on the SELFIE WBL report results. 3 1 Introduction The pilot of SELFIE for work-based learning contexts was carried out in nine countries. The European Forum of Technical and Vocational Education and Training (EfVET) in col aboration with European Commission’s Joint Research Centre (JRC) have organised them in France, Poland, Hungary and Germany. JRC has managed the pilot in Romania. In addition, the European Training Foundation (ETF) in col aboration with JRC has piloted the tool in four non-EU countries namely Georgia, Montenegro, Republic of Serbia, and Turkey. EfVET carried out the overal management of the SELFIE WBL pilot in Hungary in col aboration with JRC. The pilot was coordinated at national level by Association for Hungarian Digital Education (AHDE), EfVET member in Hungary. The qualitative research and reporting of the pilot was led by EfVET member in Slovenia - Skupnost višjih strokovnih šol Republike Slovenije (Skupnost VSŠ). Overal Management of SELFIE WBL in Hungary - specific responsibilities alocated to each organisation were as fol ows: EfVET was the project coordinator and responsible for the overal project management, quality, and reporting. More specifical y, the Project Manager was responsible for the implementation of the work plan , for al administrative and financial management of the proposal and for assuring each member of the team was provided with the support needed to implement the tasks. EfVET had one member of the governance responsible for overseeing the piloting process and one project manager responsible for the operations, ongoing support to the national coordinators and the liaison with JRC. Skupnost VSŠ – Skupnost višjih strokovnih šol Republike Slovenije was a research partner. It was responsible for the qualitative research (including the conduction of the case studies) and for the final report, summarizing the process fol owed and lessons learnt from the piloting of SELFIE WBL in VET schools and companies and comprising the list of digital tools used in the work-based learning (WBL) sector for each country. Skupnost VSŠ had three members who were part of the research team (one senior plus one junior researcher, and a senior WBL expert), working directly with EfVET and the national coordinators. AHDE - The Association for Hungarian Digital Education was the national coordinator for Hungary. Its main responsibilities were the translation and adaptation of SELFIE WBL and supporting materials into Hungarian, reaching out and engaging the stakeholders, VET schools and companies, overseeing the piloting of the SELFIE WBL tool and supporting the research component. The national coordinator worked very closely with school coordinators providing ongoing support. The national coordinator had a pivotal role in the piloting process for the ongoing support to VET schools and companies. AHDE had two members of staff dedicated to the SELFIE WBL pilot - one senior VET expert supported and one additional member of the team responsible for overseeing the operations at national level. Management at national level - responsibilities are defined as fol ows: The national coordinator had a pivotal role in the SELFIE WBL piloting process and in the selection of VET schools and companies on national level. The national team was responsible for the ongoing support to VET schools, the engagement of national stakeholders and the preparation and delivery of planned webinars. It also acted as a liaison between Skupnost VSŠ and VET schools in everything related to the research component (including the translation of support materials developed for that effect). The national team was responsible for the conduction of the interviews with school leaders and company representatives. The school coordinators were the main organisational force on institutional level engaging and mobilising companies, school leaders, teachers and students and offering them ongoing support during the pilot process. The school coordinator was also responsible for the organisation of the focus groups that took place in schools – one with teachers and the other one with students. The school coordinators were also responsible for the management of the relationship with companies and the eventual support that might have been required throughout the SELFIE WBL pilot. 4 2 Digital education and WBL policies Historical evolution1 The dual vocational training has a long history and tradition in Hungary rooting back to early 1990ś, even though, a series of political and pedagogical changes have occurred since. The 2011 Act on VET (Act CLXXXVII of 2011 on VET) officially introduces the concept of dual-VET where students should have some practical training in the first year and in a company in the next 2 years of their training. In this sense, it is previewed that companies and the government have joint responsibilities regarding the training and its costs. Companies being responsible for the vocational education and training and the practical training lies with the companies. In 2015, with the purpose of boosting the attractiveness of the VET pathways and promoting it, the government issues a concept paper on VET, the Government Decree No 1040 of March 2015, on vocational education and training for the economy, that leads to the amendment of the 2011 Act on VET with a great emphasis on lifelong learning pathways and developing steps to answer the needs of the labour market in a more efficient way. Initiatives such as the review of the VET training programmes and the placement of VET schools under the supervision of the Ministry for National Economy were some of the measures taken. It aimed at widening up the opportunities of the dual system and to consolidate it by strengthening the relationship between economy and VET. The practical training (apprenticeship) is set in a twofold way namely: - Via an apprenticeship training contract between the learner and the company; - Via a cooperation agreement between VET schools and companies, where learners are not contractually linked to the company, not being subject to receive remuneration (exception made to summer holidays) The decision of where the practical training takes place lies entirely on the availability of the companies to provide the training and the decision of the learners to attend it. The preferred option expressed in the Vet Act of 2011 is the apprenticeship contract and measures to promote and incentivize it have been taken by the government. Until 2013, students could enrol in IVET pathways upon completion of the lower secondary level. In 2013, with the purpose of reducing school dropout and promote the VET pathways, the government introduced the “vocational bridging programmes” initiative, allowing students from age 14 and upon completion of the lower secondary level, to enrol in 2-years VET pathways with the opportunity to access practical training from an earlier age (Cedefop, 2020a and Bükki, 2019). Measures to extend dual training (apprenticeships) Several reforms and initiatives were held by the Hungarian Government aiming at boosting VET attractiveness. The priorities set for the period of 2016-2020, inspired by the policy paper of 2015, by the Director General for VET (CEDEFOP, 2020a) focused on the desired increase in the number of companies providing practical training and the number of apprenticeship contracts and in the quality of this training by enhancing the cooperation between schools and companies – a critical aspect for the success of the apprenticeship offer (Bükki, 2019). The targets set for IVET was to increase for 8% to 25% the share of apprenticeships in upper and post-secondary levels by 20182, ideally by the means of apprenticeship contracts. With this purpose, the “chamber guarantee” scheme was launched, where learners would be allowed to participate in the practical training offered by schools only if placements at companies were not available- which would then have to be confirmed in writing by the chamber (Bükki, 2019). 1 This subsection is based on Cedefop (2020a) and Bükki (2019). 2 At the time of this report, we did not have up to date data regarding the actual statistics. Attempts were made with the National Statistics Agency and the Ministry of Technology and Innovation, but this information was not available. 5 Digitalisation of VET – VET 4.03 In 2019, following the Government Decision No: 1168/2019 (I I.28) on the mid-term strategy “VET 4.0 for the renewal of VET and Adult Education”, mid-term strategy for VET 4.0 was adopted. The strategy outlined the vision for the VET system, where al students would be able to develop skil s matching the needs of the fast-changing labour market. The strategy also aimed at contributing to increasing the attractiveness of the VET pathways, even though according to CEDEFOP the trend has been one of increase as we observe “a rise in the upper secondary population, especial y in VET (40% against 36.4% in the general path in 2019)” (CEDEFOP, 2019) The mid-term strategy is organised around 3 mail pil ars:  Attractive learning environment by assuring that the training curricula embeds a digital curriculum and that schools have the equipment needed. It is also planned to assure that adults in upskil ing pathways have opportunities to develop their digital skills.  Career opportunities where learners have the support and guidance, they need to complete the VET education and transition to the labour market, choosing a career leading to a good income.  Teachers with up to-date-skil s to implement and perform at highest level, assuring not only the use of digital technology in the classroom as the development of digital skil s by the learners. The strategy previews also the possibility of continuous professional development courses for teachers organised by companies. The strategy foresees a close cooperation between industry and education via the sectoral skil s council, assuring that each VET qualification embeds digital and industry 4.0 skil s. The most Recent Government Decree for boosting of Digitalisation o f VET On the 28th of February 2020, the Hungarian Government set up a new agency – caled Centre for Digitalisation of Vocational and Adult Education - to boost the effectiveness of the process of digitalisation of VET education. The Centre for Digitalisation of Vocational and Adult Education supports the implementation of the Digital Education Strategy in VET and adult education. It is responsible for the digital transformation of vocational education and training and adult learning. The Centre supports the development of the IT infrastructure, organisational transformation, and content. It supports the VET schools in fulfil ing their requirements for digital competence, implements, coordinates the development of digital pedagogical methodologies, and supports their implementation. It also provides professional support for the Government in reforming the curriculum and participates in the development of the digital competence framework. 3 From here, this subsection is based on Csík (2020). 6 3 Set up of the pilot 3.1 Methodology for selecting the pilot schools and companies Selection criteria for VET schools were set to capture and reflect the diversity of VET schools (see Figure 1) and their environment according to: - Size of VET schools (as defined in the SELFIE WBL tool), - Location (as defined in the SELFIE WBL tool), - Geographical coverage (result of agreement within the SELFIE WBL pilot team), - Programme area coverage (result of agreement within the SELFIE WBL pilot team) and - Number of VET schools (at least 12 VET schools). Figure 1. Selection criteria for VET schools. Source: Skupnost VSŠ. (2020) Regarding the school size and location, the decision was to apply the same criteria as defined by JRC in the SELFIE WBL tool. Regarding the different programmes offered by the different VET schools, this was the result of a consultation with the SELFIE WBL pilot team in the 4 countries where the pilot is being overseen by EfVET. It does not intend to be an exhaustive list of al the programmes in the country but rather reflect the common areas identified by the SELFIE WBL pilot team. The agreed minimum number of VET schools to be engaged in the SELFIE WBL pilot was 12. One important consideration was the voluntary participation of schools in the pilots which meant, on a practical level, that the ultimate criteria would be the school’s availability and wil ingness to participate in the pilot and commitment to the proposed responsibilities. Mapping VET Schools in Hungary was done by the national coordinator AHDE and the Ministry of Innovation and Technology, engaged from a very early stage, and advised which regions could be involved in the piloting of the SELFIE WBL. Contacts of potential schools who would match the criteria were provided and one-to-one contacts initiated by AHDE. In addition, AHDE, via its own internal networks, including the Budapest Training Centre and the National Office of Vocational Education and Training and Adult Learning, reached out to VET schools in other Regions. Even though a public list of VET schools in Hungary4 exists, the above-mentioned approach was considered as best given the limited timeline of the SELFIE WBL pilot. The registration process was managed by the national team, in close communication with the Ministry of Innovation and Technology. The ultimate decision to participate was made by VET schools. Outreach and Engagement – AHDE has established one-to-one communication with each VET school that expressed interest and availability to participate in the SELFIE WBL pilot, providing additional information regarding the piloting process and the qualitative research, explaining the advantages and benefits of the SELFIE WBL pilot and also providing information on the type of support available for the participating VET schools. This on-going communication was critical to assure VET schools’ engagement and commitment to participate in the SELFIE WBL pilot. The decision was to engage al schools that registered until the deadline set. A Memorandum of Understanding was sent to al VET schools to be signed, to formalize the cooperation between EfVET, SEPR and each of the VET schools. 4 The public list of VET schools in Hungary is available at https://szakkepzes.ikk.hu/2019#intezmenyek. 7 Overal , 14 VET schools from 5 administrative divisions have been engaged in the SELFIE WBL pilot, covering only smal -sized VET schools. While most of them are located in urban areas, there is a diversity in terms of geography. Most programme areas were covered save from biotechnology and art and design. . The summary of VET schools engaged in the SELFIE WBL pilot and the diversity of coverage according to above set criteria can be seen in Figures 2 and 3. Figure 2. The diversity of selected VET schools according to size, location, and programme area. Source: Skupnost VSŠ. (2021) Figure 3. The diversity of selected VET schools and companies according to geographical coverage. 8 Source: Skupnost VSŠ. (2021) Selection criteria for companies were set to cover and reflect the diversity of companies prioritising the relevant national economic areas (see Annex 2) and the diversity thereof. The selection criteria for the diversity of companies (see Figure 4) were set to: - Company size (Commission Recommendation of 6 May 2003, 2003) and - Economic sector coverage (result of agreement within the SELFIE WBL pilot team). Figure 4. Selection criteria for companies. Source: Skupnost VSŠ. (2020) Engagement of companies was managed by selected VET schools from the pool of companies each VET school works with. In Hungary, VET schools have a very close relationship with the companies they work with which was key to reach out to companies and engage them. The above criteria were presented to each VET school by AHDE. The minimum requirement set for the SELFIE WBL pilot was to engage at least one company per VET school involved. Their engagement was based on their availability and wil ingness to participate and aligned with criteria set above, despite the additional measures taken as a result of the COVID 19 pandemic. The number of companies engaged was 38 and the diversity of coverage according to above set criteria can be seen in Figure 5. Figure 5. Selected companies per selection criteria. 9 Source: Skupnost VSŠ. (2021) Overal, there was an effort at national level to be as diverse as possible regarding the economic sectors. There is, as the figure reflects, a great diversity regarding the company size. Different economic sectors representing the most dominant sectors such as tourism and catering, business services, automotive industry, production of electronic products and agriculture are wel covered (see Annex 2). There is, as Figure 5 reflects, a great diversity regarding the company size as well as the in most dominant economic sectors represented. Initial y, it had been planned to have companies’ representatives signing a Memorandum of Understanding. Given the feedback received by the national coordinator regarding the chal enges that may be faced in the process of having companies signing this document, and the wish of VET schools to take responsibility for the management of the communication and relationship with the different companies engaged in the SELFIE WBL pilot, EfVET decided not to proceed with this formalisation on the basis that it was not needed, and it was adding an unnecessary administrative burden. 10 3.2 Methodology for translating and adapting SELFIE materials The translation and adjustment of SELFIE WBL consisted of 3 main actions namely: (1) linguistic translation, (2) content-focused translation and (3) contextual adaptation and usability. The first one refers to the translation of the documents provided by JRC and was carried out by AHDE. The second and third actions related to the translation were carried simultaneously and brought together VET and WBL experts from 3 different VET schools. The involvement of external VET and WBL experts was done to assure the language and the terminology used were clear and understandable by al those involved and in line with the official ones used in the country. The linguistic translation took place in the first 2 months of the project. There was an initial misunderstanding regarding the deadlines set for the different actions and some delays were observed on steps 2 and 3. Figure 6. Translation process. Source: Skupnost VSŠ. (2020) 11 4 Pilot implementation The SELFIE WBL pilot was implemented in the folowing steps (see Figure 7): Figure 7. Implementation process. Source: Skupnost VSŠ. (2020) Step 1) Translation of SELFIE WBL materials was done from August to September 2020 (see chapter 3.2 Methodology for translating and adapting SELFIE materials). Step 2) Mobilisation of VET schools and companies took place from July to September 2020 (see chapter 3.1 Methodology for selecting the pilot schools and companies). 12 Step 3) Selections of VET schools and companies were conducted from July to September 2020 (see chapter 3.1 Methodology for selecting the pilot schools and companies) and the Memorandums of Understanding were signed with each selected VET school defining roles and commitments of each VET school to formalize this cooperation after the selection in September 2020. Step 4) Preparatory webinar was organised by the national coordinator to bring together al national stakeholders, EfVET, JRC, European Commission as wel as VET schools, companies, and the research team on 22nd September 2020. The main objective was to present the aim of the SELFIE WBL, provide an overview of implementation steps, school self-reflection report, personalized certificates and digital badges, schools’ and companies’ commitments and timeline. Furthermore, feedback from each representative on eventual concerns and expectations was discussed as wel as the mapping of digital tools for WBL used in the country, schools, and companies. Step 5) Piloting of the SELFIE WBL self-reflection exercise began by VET schools registering into the SELFIE tool, planning the activation period, announcing the SELFIE WBL pilot within the school and among partner companies and motivating them to participate by explaining the benefits of their participation. When activating the SELFIE WBL self-reflection exercise, school coordinators monitored and reported the participation rate (40 % of WBL students, 40 % of VET teachers and at least 1 in-company trainer) and further motivated and promoted the participation among the target groups needed. Most difficult to motivate proved to be in-company trainers as they are not in school and under the management of the school. The SELFIE WBL process took place from September til October 2020, and the feedback from the exercise is presented in chapter 5.2 Quantitative results. Step 6) Follow-up and guidance webinar was organised by the national coordinator addressing only VET schools and company representatives on 8th October 2020. The aim was to fol ow-up the piloting experience, gather initial feedback from school coordinators, address eventual chal enges that may have arisen during the process, confirm the overal figures in terms of completion of the questionnaires and prepare school coordinators for the conduction of students and teachers focus groups and semi structured interviews for school leaders and company representatives. The school coordinators were asked to provide feedback on their experience during the implementation process through the list of chal enges provided by the research team. The research team also provided the guidelines and reporting templates for focus group implementation as wel as the list of chal enges to school coordinators, guidelines and reporting templates for semi-structured interview implementation to the national coordinator. The guidelines, report templates, and the list of chalenges can be found in Annex 3. Step 7) Focus groups were organised by school coordinators in November and December 2020. Two focus groups were organised per VET school, one with students and one with teaching staff to reflect and discuss their interpretation and in-depth understanding of the relevant report results. Due to COVID-19 pandemic the school coordinators struggled to organise focus groups and reach the agreed participation rate of 10 students/teachers per focus group (see chapter 7. Implications of COVID-19). In total 28 focus groups were organised involving 141 students and 141 teachers. The feedback from the focus groups is integrated in chapter 5.3 Qualitative results. Step 8) In-depth semi structured interviews were organised by national coordinators from November 2020 to February 2021. The aim was to conduct 14 interviews with 4 in-company trainers and decision-making staff in VET school (4 pedagogical managers/directors, 4 sector heads/managers, 4 board heads/directors) to reflect and discuss their interpretation and in-depth understanding of the report results and to plan improvements based on those results. Interviews were conducted online. Due to COVID-19 pandemic the national coordinators struggled to engage in-company trainers (see chapter 7. Implications of COVID-19). In total 16 interviews were conducted involving 12 decision-making staff in VET schools and 4 in-company trainers. The feedback from the interviews is integrated in chapter 5.3 Qualitative results. Step 9) Evaluation webinar brought together al national stakeholders, EfVET, JRC and the research team on 8th January 2021. The main purpose was to evaluate the experience, col ect information and recommendations regarding the SELFIE WBL tool from policy makers and other institutional representatives at national level, the opportunities they see for the broader use of the tool in the WBL sector and identify possible dissemination actions that could take place. The research team presented the preliminary results and discussed those with the participants. The feedback from the webinar is integrated in chapter 5.3 Qualitative results. 13 Step 10) Quantitative and qualitative research was conducted simultaneously and upon the receipt of feedback from al above activities from September 2020 to February 2021. The research team prepared the quantitative analysis based on the results of the SELFIE WBL self-reflection exercise provided by JRC and the qualitative analysis based on the feedback from focus groups (teachers and students), semi -structured interviews (school leaders and in-company trainers), the list of chal enges (school coordinators), the fol ow-up and evaluation webinars (for details see chapter 5 Fol ow up: quantitative and qualitative analyses). The timeline of the SELFIE WBL pilot was severely affected by the COVID-19 pandemic which delayed the implementation of focus groups, semi-structured interviews, the evaluation webinar and in consequence the qualitative and quantitative research. It also affected the engagement of participants (see chapter 7. Implications of COVID-19). 14 5 Follow up: quantitative and qualitative analyses 5.1 Methodology This project aimed to explore a broad scope of aspects of the SELFIE WBL tool to contribute to practice development and to improve the tool itself and its further development. To reach these aims and to increase the internal and external validity of the research results, the research design is based on methodological triangulation of using several different methods. The research team and its project partners used as approach of integrating the quantitative and qualitative methodology. Therefore, the fol owing methods and techniques were used (Majchrzak, 1990): — Analysis of primary sources: analysis of anonymized data provided by JRC. — Analysis of secondary sources prepared by JRC: 4 reports showing aggregated graphs of SELFIE WBL pilot data which were: Participation (numerus and percent according to different demographic variables), Satisfaction (percent and mean for values of overal score and further recommendations), Main Areas (percent of positive responses for area and each variable) and Additional Information (percent of answers). — Analysis of school reports generated by school coordinators, involved in SELFIE WBL pilot. — Semi-structured interview reports, involving 2 respondent groups (school leaders and in-company trainers) provided by the national coordinator. — Focus groups reports, involving the 2 other respondent groups (teachers and students). The quantitative data were col ected through the SELFIE WBL questionnaires, which were answered by school leaders, teachers, students, and in-company trainers. The SELFIE WBL tool provides state-of-the-art information as perceived by the respondent groups. Respondents were selected in a manner that it is possible to make a representative conclusion (Ragin, 2007) at institutional level. We used univariate methods in this study. They are primarily intended to present the distribution of variables’ values; hence the tables in chapter 5.2 and Annex 6 display the number of valid values and additional statistics that we selected: mean (the average value) and standard deviation. In our database, the number of valid responses varied between the variables. When answering the questions for which the quantitative analysis is presented, the respondents had a help text and answered mostly on a 5-level scale with the additional option “prefer not to say” or “not applicable” (and in two cases on a 10-level scale, one question being for al respondent groups and another for two respondent groups). For some questions they had the possibility to select the answer or not (multiple choice). In the folowing quantitative part (see chapter 5.2) we present frequency tables and descriptive statistics. The tables with descriptive statistics display: — N = number of valid responses from the respondents — Mean (M) = the average value of the data points or numbers — Standard deviation (SD) = a measure of the dispersion of a dataset relative to its mean The qualitative research component of the SELFIE WBL pilot had as goal to colect feedback in view of improving the SELFIE WBL tool before it is launched online. The qualitative data were col ected through desk research, feedback from school coordinators, focus groups and in-depth semi structured interviews. The main goal of the desk research was to map out existing similar self-reflection tools in the country used in WBL contexts and to identify other existing digital tools. This mapping and listing were done in two different ways. On the one hand the research team conducted a comprehensive online desk research on al official and available websites from governmental institutions responsible for overseeing WBL in the country. On the other hand, by col ecting this information from the different respondent groups engaged in the pilot (see Annex 9). 15 Focus groups brought groups of people together with the main purpose to col ect feedback regarding the SELFIE WBL tool from users’ perspective. The proposal was to conduct two separate focus groups in each VET school, one with teachers involved in the pilot and the other with students (each gathering 10 persons). The selection of the students and teachers did not fol ow any criteria. The selection was left to the school coordinators according to the guidelines, they invited the first 10 teachers/students who applied. Facilitators of focus groups were given guidelines (how to conduct focus groups, how and what to report) and templates for reporting the feedback of the focus groups (see Annex 3). The qualitative research method of in-depth semi structured interviews consisted in posing a series of open and closed questions to targeted individuals, i.e., pedagogical managers/directors, sector heads/managers, board heads/directors and in-company trainers, with the goal to gain some insight regarding their perspective on the topic of digitalisation, their wil ingness to further explore SELFIE WBL and to integrate the tool in their cur ent work, as wel as to gather recommendations regarding possible ways to improve it (see Annex 3). There were two open questions in SELFIE WBL for students (digital technology they find useful for learning and ideas and suggestions to further improve SELFIE WBL). We analysed them using thematic analyses. The thematic analysis is a method for examining the content of responses from data col ected from open-ended questions, focus group discussions, or interviews. It enables identifying emergent topics not explicitly stated in SELFIE WBL questions. It is based on organizing key issues in data and grouped under themes reflecting important relations in the research questions (Braun and Clarke, 2006). Results of the thematic analysis were included in the qualitative part of the report (see Annex 4). The qualitative research method of analysis of school reports generated by school coordinators consists of gathering chal enges, advantages of the implementation of SELFIE WBL, and further feedback on the SELFIE WBL process from the perspective of school coordinators, who organised and monitored the SELFIE WBL process within their institutions. To col ect feedback, a template was prepared and provided to school coordinators (see Annex 3). The data colection took place from September 2020 until February 2021. The analyses started in December 2020. Al responses to the SELFIE WBL self-reflection exercise, focus groups, semi-structured interviews, and analysis of school reports generated by school coordinators remained anonymous and disconnected from contact details to ensure confidentiality. 16 5.2 Quantitative results Participants in the quantitative analysis were from 14 VET schools. There were 2090 respondents in the database. The participation of school leaders, teachers, students, and in-company trainers was as fol ows: — 48 school leaders — 452 teachers — 1558 students — 32 in-company trainers. In the SELFIE WBL pilot the sample of respondents according to school management al of them were from public schools (100%). 23 % of VET students’ population studies in church and business entities, foundations, associations, etc., which can get funding from the central government budget based on an agreement with the minister responsible for VET. Nevertheless, they are al considered public. 41.9% of respondents were from schools located in towns (15001-100,000 people), almost one third (31.3.8%) were from cities (100,001 to 1,000,000 people), and 26.8% from large cities (more than 1,000,000 people). The SELFIE WBL self-reflection exercise consists of eight areas scored with a five-point Likert scale (1-5). Figure 8 displays the percentage of positive responses (i.e., responses on 4 and 5) by main areas. The most positive responses are in the area “Leadership” (73.5%), which is followed by the area “Col aboration and Networking” (72.7%) and “Infrastructure and Equipment” (60.6%). On the other hand, the least positive responses from the respondents are seen in the areas “Students digital competence” and “Assessment practises” (both 46.3%). Figure 8. P ercentage of positive responses by area. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 17 Table 1 displays average values for main areas per respondent group. The number of questions in the areas differ between the respondent groups. The area with the highest mean in the group of school leaders is “Continuing Professional Development” (M=4.3). Teachers and students rated the highest “Pedagogy: Supports and Resources” (teachers M=4.3, students M=3.6), in-company trainers “Infrastructure and Equipment” (M=4.2). The lowest mean for school leaders and teachers is the area “Assessment practices” (school leaders M=3.3 and teachers M=3.4), for students “Infrastructure and Equipment” (M=3.1) and for in-company trainers “Pedagogy: Supports and Resources” and “Assessment practices” (both M=3.3). Al area means are above the middle of the 5-level scale. Based on the SELFIE WBL pilot results in Hungary, the general opinion of al respondent’s groups except students was the same (M=3.8). Students’ average score is the lowest (M=3.2). Table 1. Descriptive statistics for main areas per respondent group. School In-company Teachers Students leaders trainers Main Area N=452 N=1558 N=48 N=32 M SD M SD M SD M SD Leadership 3.6 1.1 3.7 1.1 / / 3.4 1.3 Col aboration and Networking 4.0 0.9 3.7 1.0 3.2 1.2 4.0 1.2 Infrastructure and Equipment 3.7 1.1 3.6 1.1 3.1 1.4 4.2 1.1 Continuing Professional Development 4.3 0.8 3.9 1.0 / / 4.0 1.1 Pedagogy: Supports and Resources 4.2 0.9 4.3 0.9 3.6 1.1 3.3 1.3 Pedagogy: Implementation in the classroom 3.6 0.9 3.8 1.0 3.3 1.2 3.9 1.0 Assessment practices 3.3 1.1 3.4 1.3 3.2 1.2 3.3 1.2 Students digital competence 4.1 0.8 4.0 0.9 3.2 1.3 3.8 1.0 Al participants 3.8 1.0 3.8 1,08 3.2 1.4 3.8 1.1 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Note: M=mean, SD= Standard Deviation; Green: the highest score, Grey: the lowest score. Figure 9 displays means for overal satisfaction with SELFIE WBL on a 10-level scale per respondent group. The highest satisfaction is indicated by school leaders (7.6) and the lowest, yet stil above the middle of the 10-level scale, is given by students (6.4). Figure 9. Mean overall score for overall satisfaction with SELFIE WBL per respondent group. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 18 The likelihood for further recommendation of the SELFIE WBL on a 5-level scale was the highest among school leaders (M=3.4) and the lowest among teachers (M=2.8). The percent of positive responses (“Very likely” and “Extremely likely”) in the group of school leaders was 50.0%. On the other hand, the highest percent of negative responses (“Not at al likely” and “Not very likely”) was given by teachers (19.5%). The percent of answer “prefer not to say” was the highest among in-company trainers (15.6%). Students and in-company trainers were asked about their opinion about the questions included in SELFIE WBL (see Table 5 in Annex 6). They rated the relevance of questions on a 10-level scale. Students’ average score was slightly above the middle of the scale (M=5.7) and in-company trainers’ average score was a little higher (M=6.8). The SELFIE WBL self-reflection exercise included also questions about respondents. Teachers indicated usefulness of “Continuing Professional Development” (CPD) activities on the pedagogical use of digital technologies. The percentage of positive responses (i.e., responses on 4 and 5) was the highest for “Learning through col aborating” (78.1%), fol owed by “Online professional learning” (68.7%) and “Face-to-face professional learning” (68.1%). “Study visit” was chosen with the lowest percent of positive responses (56.2%). The answer “Did not participate” was the most often used for “In-house mentoring” (52.2%). Teachers and in-company trainers were asked about their confidence in the use of digital technologies.5 Teachers (86.5% positive responses) and in-company trainers (79.3%) feel the most confident in using technology for communication. Teachers are least confident in using digital technology for class teaching (60.5%) and in-company trainers for preparing lessons (74.1%). Furthermore, teachers and in-company trainers were asked “For what percentage of teaching/training time have you used digital technologies in class in the past 3 months?” There were five possible answers. 6 The highest percent of teachers (26.3%) chose answer “25-50%” and the highest percent of in-company trainers (37.5%) chose answer “0-10%”. 36.3% of teachers and 18.8% of in-company trainers chose answer chose answer “51-75%” or “76-100%”. The students reported that they used technology in and out of school most frequently for fun (87.9%). Most of them had access to technology outside the school (89.1 %,). Answers to the question “Is teaching/training with digital technologies in your school/company negatively affected by the fol owing factors?”7 displays school leaders (20.1%) and teachers (18.4%) found “Lack of funding” as the most influential negative factor. In-company trainers most frequently choose “Lack of time for trainers” (15.0%). Moreover, answers to the question “Is remote teaching and learning/training with digital technology negatively affected by the fol owing factors?”8 display that remote teaching and learning is most often negatively affected by “Limited student access to digital devices” (school leaders 23.0%%. teachers 19.0% and in-company trainers 21.2%). The least negative factor is the same in al respondent groups: “Teachers/Trainers lacking time to provide feedback to students” (school leaders 4.1%, teachers 6.1%, and in-company trainers 4.5%). The percent of chosen positive factors for remote teaching, learning, or training9 displays agreement between groups. The most positive factor is “The school and company’s experience in the use of Virtual Learning Environments” (school leaders 17.3%, teachers 15.0% and in-company trainers 15.6%). For more information on figures, tables, and data, see Annexes 6 and 7. 5 Teachers responded to the question regarding the situation in their school (teaching), in-company trainers regarding the situation in their company (training). 6 Answers: 0-10%; 11-25%; 26-50%; 51-75%; 76-100%; Prefer not to say. 7 School leaders and teachers responded to the question regarding the situation in their school (teachers), in-company trainers regarding the situation in their company (trainers). 8 School leaders and teachers responded to the question regarding the situation at their school (teachers, teaching), in -company trainers regarding the situation in their company (trainers, training). 9 School leaders and teachers responded to the question regarding the situation in their school and teaching, in-company trainers regarding the situation in their company and training. 19 5.3 Qualitative results Fourteen pilot schools were included in the qualitative part of the SELFIE WBL. The qualitative analysis was based on feedback from 28 Focus groups,16 semi-structured interviews, 14 school reports, the final evaluation webinar, constant e-mail communication with national coordinator and his assistant and answers to open questions in the SELFIE WBL (see chapter 5.2 Quantitative results). Focus groups, in which 141 teachers and 141 students participated, were moderated by national coordinators. The latter also conducted 16 semi-structured interviews with stakeholders: 12 with school leaders and 4 with in-company trainers. Additionaly, we received 14 reports of school coordinators identifying advantages and positive reflections to the SELFIE WBL tool, but also chal enges and possible improvements. The colection of qualitative data was seriously affected by the second wave of Covid-19, which pushed the implementation of the qualitative phase of SELFIE WBL pilot down the priority list both in schools and among participants. This manifested itself in a difficult access to participants and fewer opportunities for participants to participate actively in focus groups and semi-structured interview as they had already dealt with cases of Covid-19, conducting live schooling, and preparing for the transition to remote learning. However, it was extremely chal enging to engage in-company trainers in semi-structured interviews as companies demanded their ful focus on preparing the company to the new situation. The situation (might) influenced the answers in a negative way. As participants were frustrated and stressed, the first feedback was pessimistic and “aggressive”, but the atmosphere improved towards the end of the meetings. Based on the results of the SELFIE WBL self-reflection exercise, it was not possible to determine by deviation the best and worst performing school as the results were quite similar or differed only in individual parameters. Therefore, we decided to present the results of al covered schools as study cases in this qualitative part. Table 2. Number of students, teachers, school leaders, in-company trainers and school coordinators involved in the qualitative analysis. Semi-structured Semi-structured School Focus groups Focus groups interviews with coordinators School interviews with with students with teachers in-company (list of school leaders trainers challenges) School 1 10 10 1 School 2 10 10 1 School 3 10 10 1 School 4 10 10 3 1 1 School 5 10 10 1 School 6 10 10 3 1 1 School 7 11 11 1 School 8 10 10 3 1 1 School 9 10 10 1 School 10 10 10 1 School 11 10 10 3 1 1 School 12 10 10 1 School 13 10 10 1 School 14 10 10 1 TOTAL 141 141 12 4 14 Source: Own analysis. For details on focus groups, semi structured interviews, and chalenges see Annex 3. 20 5.3.1 Initial motivation from participants Almost 80% of students reported they did not have any prior expectations regarding the SELFIE WBL survey. The rest expected that their answers would help future students to have better learning opportunities. An additional expectation was to help improve digital capacity in the classrooms and to support their school since they were told that SELFIE WBL would help school leaders improve digital capacity (infrastructure and digital education) in their school. This created higher motivation to take the SELFIE WBL seriously. Likewise, teachers did not have any expectations regarding the survey, except that their answers would help the school to be more prepared for a quality digital education and that SELFIE WBL would contribute to speed up the digital transformation in their school. On the other hand, school leaders were positive that SELFIE WBL covered al important topics and that the results would help them develop a quality digital strategy of the school. The SELFIE WBL results could also be used as a starting point in developing teachers’ digital competences since school leaders’ state they want to have teaching staff with strong digital competences. One school wanted to be the first school in Hungary to have a digital curriculum in their area. In-company trainers were less motivated due to their lack of time. Their motivation was mainly to increase the effectiveness of dual training and the company’s involvements in the improvement of digital y enhanced teaching. Al students and majority of teachers saw completion of the SELFIE survey as an obligatory task. School leaders confirmed it was “strongly advised” to complete the survey, which was completed in the class after teacher’s request. The majority of students also received no or not enough information about the survey. Their motivation would be higher if they received prior briefing (e.g., topic of the SELFIE WBL, benefits for students, teachers, and school) by either school leader, teacher or at the SELFIE WBL website. Motivation would also be higher had they known future steps to be taken based on the results. That was also confirmed by some school leaders, although one school leader admitted that students were consciously not informed, as they wanted to receive “realistic picture and their first impression”. On the other half, approximately half of the teachers received short information about the survey. One third of students and teachers reported that they already completed the SELFIE in spring (without questions regarding the WBL), so the questions and the SELFIE portal were familiar to them. Fil ing out SELFIE the second time was a bit boring for some, especial y, because they are yet to receive the feedback about the spring results. Likewise, many have not seen the results of the second survey prior to focus group. 5.3.2 Strengths and weaknesses of the SELFIE WBL tool Participants filed out the SELFIE WBL self-reflection exercise on various devices, mostly on computers and smartphones. The time needed to complete the SELFIE WBL varied widely between the participants. Some dedicated over 45 minutes for a “reflective answer”, the others (students and teachers) admitted they only needed 10 minutes because they rushed through the survey. Also, some participants already fil ed out SELFIE in spring so the questions and the tool was familiar to them. School leaders general y do not see the length of the survey as a problem, as in their opinion everyone “shal take 30 minutes for thinking about the future of the school”. However, general opinion is that the participants lose their motivation if the survey takes more than 30 minutes. Almost al participants find SELFIE WBL easy to manage and completely understandable on both devices. Majority of them report the SELFIE WBL tool as user-friendly, very easy to use, transparent, with a good structure, wel designed, and with 360-degree evaluation. The “help” function makes the complex questions clear and understandable, while the examples help participants to differentiate between similar questions. Majority of participants believe that the SELFIE WBL results show school’s strengths, weaknesses and areas of improvement in digital education. As the strength they also mention a possibility of tailoring the survey to school’s needs and focus on eight areas of digital education. Participants came across inconveniences while filing out SELFIE WBL and express important challenges for an optimal functioning of SELFIE WBL. They propose the “save function” to avoid repeating the survey due to 21 internet troubles.10 Furthermore, some students mentioned the problem with fil ing out SELFIE WBL on mobile phone: the language of the questions could not be changed from English to Hungarian (other languages were listed, Hungarian was not an option). The participants did not notice any problems with translation. They find vocabulary and vocational and professional terminology appropriate, but every fourth student and every third teacher mention that some expressions sound a bit “foreign”, outdated or have a word which is not used anymore. The participants also had some recommendations to improve the SELFIE WBL tool: - Add option “this question does not concern me”.11 - Add option of descriptive answer, enabling the participants to explain their answer in detail. - Add condition(s) for questions to achieve that only certain persons answer certain questions (e.g., if only one teacher at school dealing with companies). 5.3.3 Questionnaire, content, and SELFIE WBL report Participants (especialy students and teachers) find SELFIE WBL comprehensive, long and tiring, which makes them lose their interest towards the end of the survey. Around half of teachers and a third of students believe the survey should not be longer than 20-30 minutes (some would even shorten it to 5-7 minutes). They propose to reduce the number of questions. Although some school leaders discussed the possibility to add additional questions to the SELFIE WBL, they decided against it as the survey was already long enough and they did not want to risk losing participants due to its excessive length. Furthermore, al participants had the feeling of repetitiveness, i.e., some topics/questions were asked repeatedly, “Only wording of the question was different”. Questions and statements in SELFIE WBL were mainly clear, understandable, and relevant for al participants (school leaders, students, teachers and in-company trainers). Only a few questions were mentioned as complex and hard to understand due to the difficult wording. Participants could not give a specific example of such question, but they confirmed that the practical examples of the “help” function helped them. The participants believe each question group (topic) is important, useful as there are mutual dependencies between the topics of SELFIE WBL. The participants find the questionnaire relevant as it highlights shortcomings, strengths, weaknesses, and areas of improvement. Students strongly believe that SELFIE WBL could help al stakeholders to increase the effectiveness of learning. They believe questions on the digital competencies of teachers are the most important. On the other hand, the most important topics for teachers are infrastructure and equipment as without them, there is no digital education and remote learning. Beside infrastructure, the three most important and useful questions from the SELFIE WBL for school leaders are: digital competences of the students (their digital learning skil s must be developed); knowledge sharing, network & col aboration and CPD and technology enhanced assessment practices. The most important questions of SELFIE WBL for in-company trainers are knowledge sharing, network & col aboration, CPD (digital teaching), and strategy. They believe there is a huge lack of quality digital content (videos, course materials, activities or simulations). Participants find SELFIE WBL thorough, and they generaly do not miss any topic. However, some of them suggested additional topics that could be included in the questionnaire: - Students: ICT availability at home and effectiveness of remote learning. - Teachers: conditions for studying at home (production of digital material, assessment of teacher’s tasks during remote teaching; devices and software provided by the school for the teachers); online vocational training; availability of OER/digital content; digital competences of the teachers; harmful effects of digital education; social disadvantage of out-of-classroom education; online grading and social status of students. 10 It shal be noted that SELFIE team has long been aware of this issue which is technical y currently not possible to solve and at the same time stil safeguard anonymity. 11 It shal be noted that the SELFIE WBL tool already includes this option: "non applicable”. 22 - School leaders: knowledge sharing and support system on macro level (between schools); digital content availability in technical courses; digital/mobile tools and apps in the company (WBL). - In-company trainers: availability of digital material (videotaping WBL). School leaders believe that from the results of SELFIE WBL, it wil become clear what the school is good at, what are the areas where it fal s behind and what are the areas which need improvement. The participants also had a recommendation to add benchmark function to the SELFIE WBL report for the comparison with other schools in the country and international y, to see where they fit, but it shal not be a ranking list. The PDF format of the SELFIE WBL report is colourful and appealing yet difficult to understand as question texts are not displayed and some scores are not ful y visible (see Annex 5, areas C and H for in-company trainers, and areas D, E and H for school leaders). 5.3.4 Current and future use of SELFIE WBL SELFIE WBL shal help school leaders develop and review their school’s digital strategy, but also help al stakeholders increase the effectiveness of learning. The biggest chal enges for school leaders are the CPD/upskil ing the digital competences and knowledge of the teachers, and the digital learning skil s of the students. Likewise, for in-company trainers, the biggest chal enges are the CPD/upskil ing the digital competences, and the development of the digital content. The pilot schools want to use SELFIE WBL as a digital strategy planning and progress control tool. Most of them want to use the tool annual y. There are several reactions based on the SELFIE WBL results anticipated. Some school leaders announced a quality digital strategy based on the survey results and plan to repeat the SELFIE WBL (yearly). Others believe results should be presented to each group with an explanation. On the other hand, some school leaders admit that after the spring, SELFIE survey no feedback was provided, nor any action plan developed. The participants (students, teachers and in-company trainers) confirm no information was received, which might have influenced their motivation and answers. For the SELFIE WBL pilot, students and teachers generaly do not have information when the results wil be presented to them. Nevertheless, every third participant stil hopes to be involved in the “reading” of the SELFIE WBL results and, if possible, also in the planning actions. Based on the SELFIE WBL results, some schools want to develop an appropriate hybrid learning system that wil work for remote learning and classroom teaching (also after Covid-19 pandemic). There were schools for which SELFIE WBL was the first self-evaluation system, but also schools that used SELFIE for the second time. They claim the differences between the first and the second results can be recognized. Another line can be drawn regarding digital strategies. In some schools, the management (school leaders) wil use the results to develop the first version of the digital strategy or digital transformation plan. Other schools already have basic strategy or plan on digital teaching. For the latter, the SELFIE WBL results wil be used for fine-tuning the existing plan and developing a detailed action plan. This wil not just mean rewriting the strategy, but also reasoning and justifying it. 5.4 Overall findings This chapter presents reflections and main findings from the pilot, gathered from both quantitative and qualitative analyses and the reflections from the participants. School coordinators report no difficulties during the registration, which might be due to the fact that several schools already used SELFIE in the past (spring 2020). They report the registration as logical, clear, and easy. One coordinator mentioned a difficulty, but admitted it was his mistake (he did not mark the company partners’ involvement and had to do the registration twice). Some also report problems with inputting the school and company data. As they already participated in SELFIE, the old data was kept in the SELFIE tool. When they 23 joined the SELFIE WBL pilot, they did not edit (or they forgot to do that) the old data already in the SELFIE WBL tool and replace it with new data/numbers for the current school year. The possibility to custom the survey (add optional and open questions) was not ful y used; majority of schools did not use extra questions and no open question was added. Around half of teachers did not know that this possibility existed and whether their school used this opportunity. School coordinators report several problems when reaching out to and motivating participants to fil out SELFIE WBL. The two biggest issues were holidays (due to summer holidays, there was not enough time for prior information and promotion and due to autumn holidays, there were problems to encourage participants to fil out the survey and achieve the desired percentage) and start of the new school year. If students are not in the classrooms, they are hard to reach as they do not have the adequate equipment. Moreover, there was no clear instruction for school coordinators which teachers shal be included in SELFIE WBL: al teachers or only teacher teaching VET school subjects. This was confirmed by one-quarter of teachers teaching general school subject, not vocational school subjects, who stated that they do not have the capacity to answer questions regarding the dual training partners. Moreover, one-quarter of students also confirmed that the answer N/A is not a sign of questions being incomprehensible but rather lack of information as they are yet to do their WBL. Thus, they did not see the rationale of fil ing out the questionnaire. Al students and majority of teachers saw completion of the SELFIE survey as an obligatory task, so they rarely report it as their inherent motivation. Their motivation would have improved if they knew the steps from school leaders would be taken based on the survey results. In some schools, the director developed a project team for managing SELFIE WBL and informed teachers about SELFIE WBL (during the webinar run by AHDE), its benefit for the students, teachers and school; the aims of the school with SELFIE WBL and the fact that the school was chosen as one of the pilot school in Hungary. School leaders’ motivation was mainly based on possibilities for development in the future on the SELFIE WBL findings and possibility to develop a quality digital strategy of their school. Students were invited to participate through their teachers. Most of the students fil ed out the questionnaire in the classroom, some on their own devices at home. Participants (who have seen the report) agree that the SELFIE WBL school report in PDF presents a clear overview, and problems a school is facing when implementing digital education. They believe they can recognize school’s advantages and disadvantages from the report. The participants of pilot schools find the SELFIE WBL report wel designed but they missed concrete questions under the graphs, so they could understand the content better. Results from the SELFIE WBL report are a good starting point for a debate during the focus groups and interviews. The latter were very useful to help school leaders understand what they can take out from the report and how they can use it for further steps. As such, the SELFIE WBL report serves as a basis to prepare the school digitization strategy. School coordinators and participants report no problems regarding certificates and badges. They report the school badges are a good and useful marketing tool. They want to use them on their homepages too. Almost al of them reported that there is an extra registration needed on a Spanish webpage to receive the SELFIE WBL badge, which is useless and very annoying. The certificates had no motivational drive for students and teachers. Only one group of students admit they saw the benefit of completing the survey in receiving the certificate. Participants believe SELFIE WBL is a nice feature providing assessment from al stakeholders and thus promoting discussion between them. School leaders clearly see the usefulness of SELFIE WBL, other participants to a lesser extent. A good half of students, but also some teachers and school leaders, could not determine the real usefulness until they see the SELFIE WBL results, actions taken and if these actions have any effect on school’s development. They believe the school shal use the SELFIE WBL report data to implement changes. If they wil not use it, future participation in SELFIE WBL is unnecessary. The SELFIE WBL ecosystem is not living yet, but the SELFIE WBL tool is a good basis for the start of this ecosystem. On the institutional level, school leaders report that the SELFIE WBL tool wil be included in their (digital) strategies. However, some participants suggested short (3-5 minutes) briefings, where school coordinator (or school leader) would present SELFIE WBL. Furthermore, it would be motivating if this prior information would focus on the objectives of the school management with SELFIE WBL and its benefit for al stakeholders. 24 The outcomes of SELFIE WBL were successful y presented to project partners in Hungary, which served as a good promotion. In cases where the lowest score was regarding the digital readiness of company, the top priority of the coming years wil be the improvement of learning, mentoring and guidance in companies. 25 6 Lessons learnt and suggestions for future development Covid-19 pandemic influenced and disturbed the process as lockdown heavily influenced participants’ mood during the focus groups and semi-structured interviews. Majority of students and teachers had no expectations regarding SELFIE WBL, most of them fil ed it out because they were asked to and find it obligatory. Prior information about the purposes and planned actions upon the SELFIE WBL results would rise participants’ motivation. Focus groups and semi-structured interviews were organised with expected number of participants in al participating schools. They were moderated by the national coordinators and therefore had the same structure, which is positive; however, written reports are quite similar. School reports were taken into account as wel , particularly when analysing case studies. The satisfaction with SELFIE WBL is high according to survey results, focus groups, and semi-structured interviews. Participants fil ed out the SELFIE WBL on computers and smartphones. Majority of participants report no problems or inconveniences on any of the used devices. Furthermore, participants find SELFIE WBL easy to use and handle, self-explanatory, user friendly, understandable, and transparent. Teachers also mention possibility of creating additional questions as a plus, as wel the fact that it is an online, paperless survey. Al participants (students, teachers, school leaders and in-company trainers) find the “help” function very useful as it makes complex questions clear and understandable and helps recognise the differences between similar questions. Most of the participants believe SELFIE WBL presents the shortcomings wel , especial y strengths, weaknesses, and areas of improvement in digital education. On the other hand, minority of students did not find it convenient to fil out SELFIE WBL on a smartphone. They claim that due to the screen’s smal size, it had to be readjusted constantly to make the survey transparent. A participant also stated that possible answers could not be ful y read, but they do not provide more specific information. Some (every sixth) students and teachers also noted, that when completing SELFIE WBL using a smartphone there was a problem that the language of the questions could not be changed from English to Hungarian. The participants (especial y students and teachers) also find SELFIE WBL long and tiring and suggest it shal not take more than 30 minutes to fil it out. On the other hand, some suggest adding an option of descriptive answer, thus enabling the participants to explain their answers. Especial y students propose the “save function” to avoid repeating the survey due to internet troubles. At the moment, it is also not possible to save and return to the previous page because the answers have to be re-entered.12 The participants find SELFIE WBL and its content relevant. The questions in SELFIE WBL are understandable and clear to majority of participants, the least for the students. Only half of students confirmed that the questions are understandable and clear. Some questions were complex but the practical examples of the “help function” helped. The majority of participants did not notice any problems with translation and characterized the topics (question groups) as important and useful. Questions regarding the remote learning took students the most time, as teachers had a different teaching strategy during the first lockdown. On the other hand, participants mostly state SELFIE WBL is extensive, and because of that, some students admit that, at the end of the survey, they only clicked automatical y without considering the answer. Majority of participants also had a feeling of redundancy, repetitiveness and that certain topics were asked multiple times. Questions in SELFIE WBL need to be clear, unambiguous, straight-forward and use up-to-date terminology. Translations in the final version shal be correct, without spel ing errors and outdated vocabulary.13 Participants also make suggestions for changes in SELFIE WBL, e.g., specific questions for different type of users (e.g., for in-company trainers only job and company related questions, not questions about the school). They also suggest adding new topics: conditions for studying/teaching at home should be better examined, ICT availabilities at home; availability of OER/digital content; digital competences of the teachers; effectiveness of remote learning and availability of digital materials connected to a specific profession. The students would like to receive the SELFIE WBL report and information about the results. They also suggest adding questions next to the results in the SELFIE WBL report. School leaders would like to see adding a chapter 12 It shal be noted that SELFIE team has long been aware of this issue which is technical y currently not possible to solve and at the same time stil safeguard anonymity. 13 The participants were asked to provide concrete details of questions with spel ing errors and outdated vocabulary, but they could not indicate specific items, words or questions. 26 on fol ow up and recommendations for schools at the end of the SELFIE WBL report. The average responses for each group for each of the 8 areas are sometimes not seen if the average is above 4.2 (see the chapter overview of areas in the SELFIE WBL school report in PDF). Participants did not comment features of SELFIE WBL (badge and certificate) or any possible suggestion for other features. Students and teachers generaly confirm that data were clearer when interpreted during the focus groups, as its usefulness was clearer. Some of them added that charts without explanation in the SELFIE WBL report are not always useful to them as the interpretations can vary. As schools and classrooms are changing their ways of teaching in the times of digital transition, the SELFIE WBL tool emerged as a useful tool for self-reflection. Around one third of the students and teachers think the SELFIE WBL results could provide valuable information to the school regarding the necessary changes. Around half of them (and also in-company trainers) are very interested in the next steps of SELFIE WBL and how the survey could help schools, students, teachers and companies increasing the effectiveness of learning. They are wil ing to be involved in the evaluation of the survey results and action planning. Lastly, SELFIE WBL could also serve as a basis when discussing with decision-makers who are financing schools. At the final webinar, a representative of the Hungarian Ministry of Innovation Technology confirmed that the Ministry wants to support the implementation of VET 4.0 programme with teachers’ trainings and developing a support system for the teachers focusing the digital pedagogy support. SELFIE WBL has a role in this process as a tool for schools and companies to improve their learning possibilities, facilitate coordination between the VET schools, companies, and students, and as a result give appropriate digital competences to the students. The participants decided they would cooperate in sharing their best practices and work together as a pilot knowledge sharing network for promoting SELFIE WBL and developing a framework of Digital VET teaching/learning strategy of the school. They also want to work together in the future. AHDE wil give a platform, support, and help for the future work. Moreover, AHDE wil offer 30 hours support training programme, free for participants of the pilot schools. Telenor (mobile communication company) wil additional y give each pilot school two mobile Wi-Fi routers with free Wi-Fi access. 27 7 Implications of COVID-19 pandemic Covid-19 highlighted the digital and pedagogical weaknesses of the schools, teachers, in-company trainers and students. Some teachers confirm that they fil ed out multiple, even much more focused questionnaires in relation to the Covid-19 pandemic (e.g., research on the infrastructure and the ICT equipment) during the first wave of the Covid-19 pandemic than SELFIE WBL. A group of teachers, students and school leaders state they did not expect so many questions in SELFIE WBL related to the Covid-19. They understood that it is a hot topic and very interesting area for a short-term research, but they believe it wil not be as relevant in the future. During a focus group, students explained that questions on remote learning took longer to answer, as every teacher had a different teaching strategy during the first lockdown due to the Covid-19 pandemic. Remarks from moderators include explanation of bad mood on focus group meetings and during the semi-structured interviews due to the lockdown. Some semi-structured interviews were conducted on the third day of the lockdown which heavily influenced interviewee’s mood. Some focus groups were conducted one day before or on the first day of the second lockdown, which made the teachers frustrated and stressed. Because of this, they gave more pessimistic and more “aggressive” feedback. Therefore, the situation influenced their answers in a negative way. In-company trainers reported, that out-of-school education and quarantine began in the spring, during the preparations for the exams. It was difficult to solve the preparation, because there were some topics for which only very old (black and white) videos were available on the internet. Prior to the exam, students had to catch up some of the missed WBL. The exam did not contain questions to those WBL parts which could not be taught. During the quarantine, in-company trainers had to organize tasks which had to be done by the students once a week. In-company trainers needed approximately 3-4 hours of preparation for each group (task development, adaptation to the curriculum and finding digital materials). School leaders saw the biggest chal enge in the CPD/upskil ing the digital competences and knowledge of the teachers, and in the digital learning skil s of the students. During the first Covid-19 period, an internal knowledge sharing was set up and the teachers helped each other using different digital tools and contents. It worked very wel and highlighted the importance and effectiveness of it. They also mention advantages of the Covid-19 lockdown, for example, the teachers were forced to use digital technology for teaching. It speeded up the digital transformation of the school. From September 2020, some school leaders prepared for the next lockdown, so they were ready when it came14, thus causing the teachers and the students less problems than they had in spring (2020). School leaders chose different online platforms for teaching (also confirmed by students), and some opted for “flipped classrooms”. School leaders notice a huge lack of vocational digital content, which is why the teachers had to develop videos of WBL in laboratories (the laboratories of the school were closed too, so there was no real possibility for WBL). The most useful topic in the SELFIE WBL for school leaders was the one regarding the measurement of the digital competencies of the teachers and students. The questions related to digital material and equipment were also important, as it is very hard to find good digital material. This was especial y noticeable at the start of Covid-19 pandemic. 14 Around November 2020. 28 8 Conclusions and recommendations Majority of participants labeled SELFIE WBL relevant and fit for national (Hungarian) circumstances. Even though, the completion of the survey was (seen as) an obligatory task and that only minority of students and teachers received information on SELFIE WBL in advance, they are stil interested in the next steps of SELFIE WBL and how it could help al stakeholders. The participants believe the SELFIE WBL measures school’s digital capability wel . Besides technology and competences, teachers’ attitudes should also be taken into consideration in SELFIE WBL. School leaders believe digitalization and thereby SELFIE WBL is a “hot” topic at the moment, not just because of the Covid-19 pandemic, but rather as a help for al stakeholders (teachers, students, schools and companies) to increase the effectiveness of (online) teaching and learning. They also understood the aims of the survey and recognized SELFIE WBL as a useful tool for developing digital strategy and practices. The SELFIE WBL pilot came at the right time, not only for schools and their leaders, but also for teachers, students, and in-company trainers. The next chal enge is to act based on the SELFIE WBL report results. As participants stated, “SELFIE WBL is worth as much as and as many things get implemented based on its results”. Recommendations: - General recommendation regarding SELFIE WBL is that it shal be shorter, and the questions sometimes simplified so the students would understand them. - SELFIE WBL shal al ow benchmarking – predominantly school leaders want a comparison with other schools to see where they fit, i.e., above or under the national average. However, it shal not be a ranking list. The pilot schools are keen to set up a knowledge network, which could be also used for promotion of SELFIE WBL. - SELFIE WBL shal include skip option, i.e., condition(s) for questions to achieve that only certain persons answer certain questions (e.g., only one teacher at the school is in contact with companies; teacher teaching general subject cannot answer questions regarding vocational subjects). - Participants wish the SELFIE WBL school report would contain conclusions and recommendations with fol ow up steps for their school. - Possible integration of the SELFIE WBL certificate/badge into Europass Digital Credentials (digital file to store in a wal et in Europass Library). 29 References Braun, V. & Clarke, V. (2006). Using thematic analysis in psychology. Qualitative Research in Psychology, 3(2), 77–101. Available: http:/ dx.doi.org/10.1191/1478088706qp063oa Bükki, Eszter. (2019). Vocational education and training in Europe : Hungary. Cedefop ReferNet VET in Europe reports 2018. Available: http:/ libserver.cedefop.europa.eu/vetelib/2019/Vocational_Education_Training_Europe_Hungary_2018_ Cedefop_ReferNet.pdf Business Europe. (2016). The cost effectiveness of apprenticeship schemes – making the business case for apprenticeships. Available: The cost-effectiveness of apprenticeship schemes (be-extranet- prod.s3.amazonaws.com) CEDEFOP. (2018). Hungary: encouraging apprenticeship take-up in initial vocational education and training. Available: Hungary: encouraging apprenticeship take-up in initial vocational education and training | Cedefop (europa.eu) CEDEFOP. (2019). The newly approved mid-term strategy (VET 4.0) for the renewal of vocational education and training and adult education programmes is the policy answer to the chal enges of the 4th Industrial Revolution. Available: https:/ www.cedefop.europa.eu/da/news-and-press/news/hungary-vocational- education-and-training-digital-era CEDEFOP. (2020a). Developments in vocational education and training policy in 2015-19: Hungary. Cedefop monitoring and analysis of VET policies. Available: https:/ www.cedefop.europa.eu/en/publications-and- resources/country-reports/developments-vocational-education-and-training-policy-2015-19-hungary CEDEFOP. (2020b). Digital gap during COVID-19 for VET learners at risk in Europe. Synthesis report on seven countries based on preliminary information provided by Cedefop’s Network of Ambassadors tackling early leaving from VET. Available: https:/ www.cedefop.europa.eu/files/digital_gap_during_covid-19.pdf Commission Recommendation of 6 May 2003 concerning the definition of micro, smal and medium-sized enterprises. (2003). Official Journal, L 124, 36-41. Available: https:/ eur-lex.europa.eu/legal- content/EN/TXT/?uri=CELEX:32003H0361. Csík, G.; Szűcsné Szabó, K. (2020). Vocational education and training for the future of work: Hungary. Cedefop ReferNet thematic perspectives series. Available: http:/ libserver.cedefop.europa.eu/vetelib/2020/vocational_education_training _future_work_Hungary_Cedefop_ReferNet.pdf European Commission. (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Farkas, P.et al. (2016). Vocational education and training in Europe – Hungary. Cedefop ReferNet VET in Europe reports. Available: http:/ libserver.cedefop.europa.eu/vetelib/2016/2016_CR_HU.pdf Figgou, L. & Pavlopoulos, V. (2015). Social Psychology: Research Methods. In: International Encyclopedia of the Social & Behavioral Sciences (Second Edition), Elsevier, Pages 544-552. Available: https:/ www.sciencedirect.com/science/article/pi /B9780080970868240282 Hungarian Government. (2019) Government Decision No: 1168/2019. (I I.28.) on the mid-term strategy ‘Vocational Education and Training (VET) 4.0 for the renewal of VET and Adult Education (AE). Available: https:/ kormany.hu/404#!DocumentBrowse Hungarian Government. (2016) Government Decree 1536/2016. The Digital Education Strategy of Hungary. Gyakorlati képzés - Tanulószerződéssel, együttműködési megálapodással és halgatói munkaszrződéssel a munka világában - Magyar Kereskedelmi és Iparkamara. (2019). Available: https:/ tanuloszerzodes.hu/wp-content/uploads/2020/06/Tanuloszerzodes_2019.pdf 30 Majchrzak, A. (1990). Methods for policy research. Applied social research methods series. v.3. Newbury Park, CA: SAGE Publications. Ragin, Charles C. (2007). Družboslovno raziskovanje: enotnost in raznolikost metode. Ljubljana: Fakulteta za družbene vede. SAGE Publications. (2019). Thematic Analysis of Survey Responses from Undergraduate Students. SAGE Research Methods Datasets. Available: https:/ methods.sagepub.com/base/download/DatasetStudentGuide/thematic-analysis-students- technology Skupnost VSŠ. (2020). Guide and Work Plan for National Coordinators. Available: https:/www.skupnost- vss.si/wp-content/uploads/2021/03/Guidelines-and-Work-Plan-for-National-Coordinators_V5.pdf Skupnost VSŠ. (2021). SELFIE WBL Hungary – Preliminary Results. SPIRIT Slovenija. (2020). Gospodarske panoge Madžarske. Available: https:/ www.izvoznookno.si/drzave/madzarska/gospodarske-panoge/ Szakképzés 4.0 – A szakképzés és felnőttképzés megújításának középtávú szakmapolitikai stratégiája, a szakképzési rendszer válasza a negyedik ipari forradalomra Szakképzés magyarországon - Gyakorlati útmutató a vál alati szakképzés megvalósításához A gyakorlati képzőhely és a szakképző iskola együttműködése – Német-Magyar Ipari és Kereskedelmi Kamara. (2019). Available: http:/ www.ahkungarn.hu/szolgaltatasok/hu/dualis-szakkepzes The Hungarian Chamber of Commerce and Industry. (2015). With Dual Training in the World of Work Learning by working. Available: Dual_ENG_A4_beliv.indd (tanuloszerzodes.hu) 31 List of abbreviations and definitions AHDE Association for Hungarian Digital Education CEDEFOP The European Centre for the Development of Vocational Training CPD Continuing professional development DES Digital Education Strategy EfVET European Forum of Technical and Vocational Education and Training EQF European Qualification Framework ETF European Training Foundation GDP Gross domestic product ICT information and communications technology ISCED International Standard Classification of Education JRC Joint Research Centre, European Commission M Mean - the average/central value of the data points or numbers MKIK Hungarian Chamber of Commerce and Industry MoU Memorandum of Understanding N Number of valid responses from the respondents NFA National Employment Fund NVQR National Vocational Qualification Register NAK Hungarian Chamber of Agriculture SD Standard deviation - a measure of the dispersion of a dataset relative to its mean Skupnost VSŠ Association of Slovene Higher Vocational Col eges SEN Special Educational Needs SME Smal and medium-sized enterprises STEM Science, technology, engineering, and mathematics TEL Technology Enhanced Learning VET Vocational education and training WBL Work-based learning 32 List of figures Figure 1. Selection criteria for VET schools. 9 Figure 2. The diversity of selected VET schools according to size, location, and programme area. 10 Figure 3. The diversity of selected VET schools and companies according to geographical coverage. 11 Figure 4. Selection criteria for companies. 11 Figure 5. Selected companies per selection criteria. 12 Figure 6. Translation process. 13 Figure 7. Implementation process. 14 Figure 8. Percentage of positive responses by area. 19 Figure 9. Mean overal score for overal satisfaction with SELFIE WBL per respondent group. 20 Figure 10. Share of learners in company-based programmes by programme type. 39 Figure 11. Hungarian vocational education and training. 40 Figure 12. Vocational education and training for the future of work - Hungary. 42 Figure 13. Distribution of economic sectors in Hungary. 44 Figure 14. Overview of areas snapshot from an Anonymous SELFIE WBL school report. 57 Figure 15. Mean score for al variables in main areas per respondent group. 59 Figure 16. Mean likelihood for further recommendation of SELFIE. 61 Figure 17. Negative factors for technology use in school and company - percent per respondent group. 61 Figure 18. Negative factors for technology use for remote teaching, learning, and training – percent per respondent group. 62 Figure 19. Positive factors for remote teaching, learning, and training - percent per respondent group. 63 33 List of tables Table 1. Descriptive statistics for main areas per respondent group. 20 Table 2. The number of students, teachers, school leaders, in-company trainers and school coordinators involved in the qualitative analysis. 22 Table 3. Thematic analysis of open question responded by students. 55 Table 4. Overal satisfaction with SELFIE - percentage distribution per respondent group. 59 Table 5. Relevance of questions per respondent group. 60 Table 6. Likelihood for further recommendation of SELFIE tool - percent per respondent group. 60 34 Annexes Annex 1. Key information on the WBL system Annex 2. Dominant economic sectors in Hungary Annex 3. Guidelines and templates for focus groups, semi-structured interviews, and list of chal enges Annex 4. Analysis of open question “Suggestions for improvement” and examples of questions Annex 5. School report “Overview per areas” Annex 6. Figures and tables of SELFIE WBL piloting quantitative analysis Annex 7. Overview of SELFIE WBL results in Hungary Annex 8. Country fiche Annex 9. List of tools similar to SELFIE and other tools used in WBL 35 Annex 1. Key information on the WBL system OVERVIEW OF HUNGARIAN CONTEXT OF WBL Definition The dual vocational training has a long history and tradition in Hungary rooting back to early 1990ś even though a series of political and pedagogical changes have occurred since. The 2011 Act on VET (Act CLXXXVII of 2011) officially introduces the concept of dual-VET where students should have some practical training in the first year and in a company in the next 2 years of their training. In this sense, it is previewed that companies and the government have joint responsibilities regarding the training and its costs. Companies being responsible for the vocational education and training and the practical training lies with the companies. In 2015, with the purpose of boosting the attractiveness of the VET pathways and promoting it, the government issues a concept paper on VET, the Government Decree No 1040 of March 2015, on vocational education and training for the economy. that leads to the amendment of the 2011 Act on VET with a great emphasis on lifelong learning pathways and developing steps to answer the needs of the labour market in a more efficient way. Initiatives such as the review of the VET training programmes and the placement of VET schools under the supervision of the Ministry for National Economy were some of the measures taken. It aimed at widening up the opportunities of the dual system and to consolidate it by strengthening the relationship between economy and VET. The practical training (apprenticeship) is set in a twofold way namely: - Via an apprenticeship training contract between the learner and the company; - Via a cooperation agreement between VET schools and companies, where learners are not contractually linked to the company, not being subject to receive remuneration (exception lade to summer holidays) The decision of where the practical training takes place lies entirely on the availability of the companies to provide the training and the decision of the learners to attend it. The preferred option expressed in the Vet Act of 2011 is the apprenticeship contract and measures to promote and incentivize it hav e been taken by the government. Until 2013, students could enrol in IVET pathways upon completion of the lower secondary level. In 2013 and with the purpose of reducing school dropout and promote the VET pathways, the government introduced the “vocational bridging programmes” initiative allowing students from age 14 and upon completion of the lower secondary level, to enrol in a 2years VET pathways with the opportunity to access practical training from an earlier age. Sources: Cedefop (2020); Bükki, Eszter (2019) Measures to extend dual training (apprenticeships) Several reforms and initiatives were held by the Hungarian Government aiming at boosting VET attractiveness. The priorities set for the period of 2016-2020, inspired by the policy paper of 2015, by the Director General for VET (CEDEFOP. 2020) focused on the desired increase in the number of companies providing practical training and the number of apprenticeship contracts and in the quality of this training by enhancing the cooperation between schools and companies – a critical aspect for the success of the apprenticeship offers. The targets set for IVET was to increase for 8% to 25% the share of apprenticeships in upper and post-secondary levels by 201815, ideally by the means of apprenticeship contracts. With this purpose, the “chamber guarantee” scheme was launched, where learners would be allowed to participate in the practical training offered by schools only if placements at companies were not available- which would then have to be confirmed in writing by the chamber. Source: Bükki, Eszter (2019). 15 At the time of this report, we did not have up to date data regarding the actual statistics. Attempts were made with the National Statistics Agency and the Ministry of Technology and Innovation , but this information was not available. 36 Provision of practical training The provision of practical training at upper secondary and post-secondary can take place in public or private schools, all funded by the central government budget. The Ministry of Innovation and Technology is, since 2018, responsible for regulating the provision of VET working in articulation with the Minister for Human Capacities regarding the regulation of the general education content. According to CEDEFOP, there are 44 VET centres with altogether 380 member schools maintained by the Ministry for Innovation and Technology. State-maintained VET schools in the sector of agriculture are operated by the Ministry of Agriculture and belong to the Network of Agricultural VET schools with 46 schools. The Ministries of Interior and of Defense as well as some universities also operate some VET schools that provide sector-specific programmes, reflecting the complexity and at the same time, the richness of the VET offer (Bükki , 2019). The apprenticeship pathways are supervised by the Hungarian Chamber of Commerce and Industry and the Hungarian Chamber of Agriculture whose main responsibilities are to accredit and register training providers and to register apprenticeship contracts while providing some guidance and support to potential learners. Concerning the VET curricula and the specific allocation of time for each component (theory and practice), these are defined in the vocational and examination requirement in line the National Vocational Qualification Register qualification. (Bükki , 2019) The practical training can be organised in 2 different ways: - Via an apprenticeship contract between the learner and the company which is the preferred option according to the VET Act of 2011, where learners are entitled to monthly payments and are also entitled to social insurance; - Via a cooperation agreement between VET schools and companies, where learners are not contractually linked to the company, not being subject to receive remuneration (exception made to summer holidays) The changes implemented in 2015 resulted in the decision of alowing learners to participate in in practical training offered by the school or to sign a cooperation agreement with a company in case there is no other option, this is to say in case there is no placement available to learners. This wil have to be confirmed in writing by the Chamber. These changes have also broadened up the profile of apprentices al owing learners enroled in adult’s education pathways and learners from grades 11 and 12 of vocational grammar school programmes to benefit from this practical training. In addition to the above-mentioned requirement, the conditions for cooperation agreements (between schools and companies) to be establish are related with the share of practical training (should be less than 40%) and the provision of practical training that should be done in a state-maintained school with companies providing practical training during the summer period. According to the latest data available, most secondary vocational school learners (EQF 4) participate in dual apprenticeship training (apprenticeship contract). Most of those enrol ed in vocational grammar school learners (upper secondary EQF 4 and post- secondary EQF 5) have their practical trainings at school or via a cooperation agreement with a company (Bükki , 2019). The share of learners in one of the two forms of company-based learning by programme type is shown in the Figure 10. 37 Figure 10. Share of learners in company-based programmes by programme type. Source: Bükki (2019). The above table refers to the year of 2017 and represents the share of VET learners by type of work-based learning and programme (%). As an example, when looking to the vocational grammar school column, it is possible to see that 61,7% of the students in this type of VET schools learn in WBL format. 37,7% of the students have apprenticeship contracts (blue) and 24% of them have cooperation contracts (red) Blue + Red = Green. Incentives for enterprises to provide practical training to VET learners. There are two main support systems for companies who provide practical training. On one hand, the support and guidance provided by the Chamber and, on the other hand, the financial incentives, and benefits, related with the possibility of deducting costs associated with the practical training from their training levy and the possibility of providing funding to companies to organise and develop training workshops. In addition, costs that are not covered by the training levy can also be claimed back to the National Employment Fund, being this possible also to other type of entities providing practical training such as hospitals, associations, foundations, churches, among others. Latest reforms Hungary has undergone several reforms in the VET system, since the early 2000ś, in recognition of the role VET and more specifical y the apprenticeship training plays in the countryś economy and society in general. In 2017, the amended Act on VET and Adult Training was adopted by the Government, with several changes being propose aiming at increasing the attractiveness, flexibility and quality of vocational grammar schools as wel as promoting and stimulating the provision of practical training, forging a closer connection with companies. More specificaly, the main changes adopted alowed (1) the extension of organisations eligible to provide practical training, offering organisations of the social sector to provide it; (2) companies to sign “pre-apprenticeship” contracts with lower secondary students; (3) the possibility for students in grades 11 and 12 of vocational grammar schools to access apprenticeship training and also (4)the possibility, particularly for SME, to provide practical training to more than 12 apprentices simultaneously. The adoption of the amendments also led to the creation of the sector skills councils (ágazati készségtanács), where key representatives of the economic sectors through their elected representatives are given the opportunity to contribute and monitor the content of VET programmes, contributing to an increased cooperation between VET Schools and Companies (and among companies) and excellence in VET. Source: Bükki. 2019. 38 Figure 11. Hungarian vocational education and training. Source: Bükki (2019). 39 DIGITALISATION OF VET – VET 4.0 In 2019, following the Government Decision No: 1168/2019 (III.28) on the mid-term strategy “VET 4.0 for the renewal of VET and Adult Education”, mid-term strategy for VET 4.0 was adopted. The strategy outlined the vision for the VET system, where all students would be able to develop skills matching the needs of the fast-changing labour market. The strategy also aimed at contributing to increasing the attractiveness of the VET pathways, even though according to CEDEFOP the trend has been one of increase as we observe “a rise in the upper secondary population, especially in VET (40% against 36.4% in the general path in 2019)” (CEDEFOP, 2019). The mid-term strategy is organised around 3 mail pillars:  Attractive learning environment by assuring that the training curricula embeds a digital curriculum and that schools have the equipment needed. It is also planned to assure that adults in upskilling pathways have opportunities to develop their digital skills.  Career opportunities where l earners have the support and guidance, they need to complete the VET education and transition to the labour market, choosing a career leading to a good income.  Teachers with up to-date-skills to implement and perform at highest level, assuring not only the use of digital technology in the classroom as the development of digital skills by the learners. The strategy previews also the possibility of continuous professional development courses for teachers organised by companies. The strategy foresees a close cooperation between industry and education via the sectoral skills council, assuring that each VET qualification embeds digital and industry 4.0 skills. The most Recent Government Decree for boosting of Digitalisation of VET On the 28th of February 2020, the Hungarian Government set up a new agency – called Centre for Digitalisation of Vocational and Adult Education - to boost the effectiveness of the process of digitalisation of VET education. The Centre for Digitalisation of Vocational and Adult Education supports the implementation of the Digital Education Strategy in VET and adult education. It is responsible for the digital transformation of vocational education and training and adult learning. The Centre supports the development of the IT infrastructure, organisational transformation, and content. It supports the VET schools in fulfilling their requirements for digital competence, implements, coordinates the development of digital pedagogical methodologies, and supports their implementation. It also provides professional support for the Government in reforming the curriculum and participates in the development of the digital competence framework. Source: Csík, G., (2020) . 40 Digital Education Action Plan for VET The Digital Education Strategy (DES) was formally launched in 2016 and its development took all levels of the education system into consideration including lifelong learning and VET. Embedded in the broader Digital Success Programme (DSP)16, being one if its pillars, it aims at addressing the challenges and opportunities of the 21st century caused by the impact of Industry 4.0. The main goal of DES development was to “create the possibility of the effective dissemination of digital literacy in harmony with the sectoral strategies and professional objectives at all levels of the education system, thus contributing to increasing Hungary’s competitiveness” (Hungarian Government, 2016). Figure 12. Vocational education and training for the future of work - Hungary. Source: Csík. (2020). The strategy aimed at responding to the changes in the labour market impacting the economy and lay down the foundation for a more competitive, efficient and forward-looking approach regarding the educational system, at all different levels, taking into consideration what were considered to be key factors of the process namely: 16 Government decision NO 2012/22015 of 29 of December 2015 41 physical infrastructure and availability of technological equipment in educational institutions; management of educational institutions; teachers preparedness and readiness to embrace digital technologies into teaching and learning; teachers training offer available and the curricula in the different educational levels. The SWOT analysis carried out to inform the strategy, and detailed in the DES document, outlined the challenges faced in the VET system regarding the digital technologies related with 4 main aspects: lack (or obsolete) of technological infrastructure and equipment in schools; lack of motivation and some resistance to change expressed by teachers; lack of basic digital skills of students when enrolling in VET pathways and lack of training and learning materials. This analysis led to the development of a logical framework outlining the strategic goals and outcomes of the digital strategy ambition for the VET sector, aiming at answering the challenges found. The main goal set for the VET sector is to ensure that students completing vocational education and training acquire general and vocational digital competences required by the labour market and necessary for continuing education. In addition, 3 other specific goals were set- as follows:  Development of the digital competences of teachers and vocational instructors in line with the technical and trade-specific requirements of the 21st century.  Development of the infrastructure necessary for digital education in vocational training institutions.  The availability of digital vocational content in respect of all trades. The following initiatives were proposed to be put in place with the purpose of achieving to goals set in the strategy (Hungarian Government, 2016): • set of VET output requirements that supports digital competence development. • Trade-specific learning materials that support digital competence development. • Development of digital methodological practice of teachers and vocational instructors • Digital infrastructure development in specialized classrooms and workshops • Development of management’s commitment towards supporting digital education These targets were set at the same time as the SWOT analysis as part of the Irinyi plan (Idők, Magyar, 2016). 42 Annex 2. Dominant economic sectors in Hungary Gross domestic product (GDP) structure Figure 13. Distribution of economic sectors in Hungary. 4% 31% Agriculture Services Industry 65% Source: SPIRIT Slovenija. (2020) Agricultural sector The agricultural sector accounts for 4% of GDP and employs 4.9% of the working population. The main agricultural crops are cereals, fruits, vegetables, and wine. Service sector The service sector contributes 65% of GDP and employs 63.9% of the workforce. The service sector has increased exports in recent years and become more competitive. Industrial sector Industry accounts for 31% of GDP and employs 31.2% of the working population. Hungary is very open to foreign investment. The most important segments of the industrial sector are the automotive industry and the production of electronic products, which together account for 30% of exports and 15% of GDP. In 2019, industrial production grew by 5.6%. For 2020, analysts predict a decline in industrial production by -10%. In 2021, industrial production is expected to grow by 6.8%, and in 2022 by 4.4%. Source: SPIRIT Slovenija. 2020. 43 Annex 3. Guidelines and templates for focus groups, semi-structured interviews, and list of challenges Focus Group Guidelines Objective The main objective of the focus groups is to spend some time with each of the 2 key target groups for the SELFIE WBL project - learners and teachers - and to discuss the “how” and “why” behind the main questions and answers of the survey. We want participants to elaborate further on the key questions of the survey (Pilot of SELFIE WBL tool) and explore participants’ views about the tool, the main chal enges they faced in using SELFIE tool and whether it helps them assess where they stand with learning in the digital age. We want them to speak freely and not be swayed by pre-conceived notions they may have about what are deemed desirable answers as there are no wrong answers. Moderators The focus group for teaching staff should be moderated by a peer teacher and the focus group for learners should be moderated by a tutor to create a comfortable and trustful atmosphere which enables open reflection and discussion. We advise that a note-taker is also assigned to each moderator to enable fluent moderation. Participants Each VET school organises 2 focus groups. One exclusively with teachers as participants and the other with learners. The diversity in terms of school’s size shal be taken into account. The only pre-condition to become a participant is that they have taken part in the SELFIE WBL pilot survey. The optimal size of each focus group is 10 participants which al ows al members to participate, and enables the moderator, i.e., institutional coordinator or learners’ tutor time to be able to tease out the nuance behind participants’ answers. For online focus groups where plenary discussions/interactions are less straightforward a slightly lower number of participants (minimum of 5) is acceptable to ensure there is opportunity for al participants to have their say, remain engaged, and reduce strain on the moderator. Duration Typical y, a focus group lasts between 60–90 minutes. This gives enough time to al ow for deeper conversations to take place but does not run too long which can lead to participant fatigue. In the case of online focus groups, it is advisable to keep the session time to maximum 60 minutes as it is just that little bit harder for people to stay focused. Moderation The focus group wil need to be wel moderated in order to guide the discussion, using a combination of questions and further probes. The participants should be encouraged to interact with each other as wel as to generate deeper insights about the different subtopics. With an online focus group, it is probably not possible to get the same type of feedback or interplay between participants as with face-to-face focus groups, so the role of the moderator is here even more important. The moderator wil give an overview of the project and its purpose, ask questions, fol ow up with more questions, and keep the conversation on track and on subject. Make sure to keep it relaxed, that participants are at ease and feel comfortable and safe in opening and sharing their thoughts. Reminding participants that there are no right or wrong answers is a good way to make sure they are not self-censoring. Make sure that the moderator also takes enough time for introductions and for participants to become comfortable in the session to ensure individuals to engage with one another. 44 Normal y, al discussions can take place in a normal plenary form, but if the moderator feels the need for it, they might use smal exercises like brainstorm activities in which the participants write down ideas on (virtual) post-it notes, plotting these post-it notes in a matrix or map to prioritize items, or simply keeping track of inspiration and solutions that come up during the session in a visual way. Themes/questions Based on experience with similar focus groups, we should have time to address three to four different themes with open-ended questions, fol ow-up questions and, especial y, discussion between participants. The topics that we would suggest are: The strengths and weaknesses of the SELFIE WBL tool Questions to the participants can include:  What works particularly wel in SELFIE tool? What does not?  What would you see as most important chal enges for an optimal functioning SELFIE tool? Discussion should be encouraged comparing different situations, shared experiences regarding strengths and weaknesses, concrete tips & tricks on how to make improvements. Discussion on relevant survey results Participants shal reflect and discuss their interpretation and in-depth understanding of the relevant survey results, for example going into different elements of SELFIE tool (e.g., Leadership, Infrastructure and Equipment, Teaching and Learning etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey if it is optimal or more out of necessity and if there are intentions to become either more or less involved in SELFIE tool in the future. Areas where further support is needed/useful Questions to the participants can include:  What are the areas of SELFIE tool where more information, knowledge, guidance, training etc. would be welcome for them and/or col eagues in similar roles?  What potential changes do you anticipate based on the survey results? Again, discussion should be encouraged comparing different situations, experiences and visions. Equipment/facilities Chairs set up in a circular pattern around a table is the most ideal set up for a focus group as you want al the participants to be able to easily see each other. In case of online focus group, a Zoom room can be set up by the Research Team (contact us17 at least 1 week prior to the event providing exact date and timeslot). The amount of information that is shared in focus groups is not easily captured by a note-taker, as there are numerous side conversations that happen. The best way to scrutinize data at a later time is to audio and video record the focus group sessions. Please do not forget to get a consent from the participants to be recorded and let them know their responses wil remain anonymous and no names wil be mentioned in the report. 17 Research Team contacts: miha.zimšek@skupnost-vss.si and/or alicia.miklavcic@skupnost-vss.si. 45 Focus Group Report Date: Country: School: Moderator(s): Participant Name and Surname Teacher/Student Subject/Programme 1 2 3 4 5 6 7 8 9 10 46 Discussion Themes - Discussion 1: Icebreakers - Discussion 2: The strengths and weaknesses of the SELFIE WBL tool - Discussion 3: Discussion on relevant survey results - Discussion 4: Areas where further support is needed/useful Theme 1: Icebreakers Suggestions for discussion: Questions to the participants can include: - What were your expectations of Selfie WBL? - Do you think your expectations were met? Common responses/general consensus: Areas of disagreement/lack of consensus: Other notes & observations Theme 2: The strengths and weaknesses of the SELFIE WBL tool Suggestions for discussion: Questions to the participants can include:  What works particularly wel in SELFIE WBL tool? What does not?  What would you see as most important chal enges for an optimal functioning SELFIE WBL tool? Discussion should be encouraged comparing different situations, shared experiences regarding strengths and weaknesses, concrete tips & tricks on how to make improvements. Common responses/general consensus: Areas of disagreement/lack of consensus: Other notes & observations Theme 3: Discussion on relevant survey results Suggestions for discussion: Participants shal reflect and discuss their interpretation and in-depth understanding of the relevant survey results, for example going into different elements of SELFIE WBL tool (e.g., Leadership, Infrastructure and Equipment, Teaching and Learning etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey, if it is optimal or more out of necessity and if there are intentions to become either more or less involved in SELFIE WBL tool in the future. Common responses/general consensus: 47 Areas of disagreement/lack of consensus: Other notes & observations Theme 4: Areas where further support is needed/useful Suggestions for discussion: Questions to the participants can include:  What are the areas of SELFIE WBL tool where more information, knowledge, guidance, training etc. would be welcome for them and/or col eagues in similar roles?  What potential changes do you anticipate based on the survey results?  What kind of technology you are using when you are working in the company? (State specific examples about the use of technology in company and in school?)  Did you start with digital learning because of COVID-19?  What problems did you face because of COVID-19?  Did you include blended learning?  Did you perform apprenticeships during the lockdown (remote mode/distance mode)?  Wil you use SELFIE WBL in the future? Again, discussion should be encouraged comparing different situations, experiences, and visions. Common responses/general consensus: Areas of disagreement/lack of consensus: Other notes & observations Additional themes/discussions/ideas/observations (Only if the content does not fal into any previous categories/themes above) Notes & observations: 48 In-depth Semi Structured Interviews Guidelines Objective In-depth, semi-structured interviews intend to elaborate further on the report results and foreseen improvements based on those results. The interviews are verbal interchanges where the national coordinator attempts to elicit information from 4 in-company trainers and decision-making staff in VET school by asking questions. Even though the national coordinator shal prepare a list of predetermined questions, in-depth, semi-structured interviews usual y unfold in a conversational manner offering participants the chance to pursue issues they feel are important. In-depth interviews are conducted in order to gain a thorough insight about a particular issue, in our case future improvements. Interviews are conducted individual y and focused on each organization separately. Interviewer The interview shal be done by national coordinator. People wil talk more when they feel more relaxed and at ease, so the questions are not asked in any given order, rather they are asked in a way that develops the conversation. Interviewee In-depth semi structured interviews are done with 4 in-company trainers and decision-making staff in VET school (4 Pedagogical Managers/Directors, 4 Sector Heads/Managers, 4 Board Heads/Directors). The pre-condition to become an interviewee is that they have taken part in the SELFIE WBL pilot survey. Duration Typical y, a semi-structured interview lasts between 30–60 minutes. This gives enough time to al ow for deeper conversations to take place but does not run too long which can lead to interviewee fatigue. Before the interview When recruiting interviewees, indicate that you would be happy to conduct the interview at a time and place which best suits them. Do not forget to remind the interviewee of the time, date, and location of the interview (online). Before the interview commences national coordinator should ask the interviewee if they consent to the interview being digital y recorded. Informed consent can be confirmed by the interviewer reading the consent form and the interviewee verbally indicating that they agree. During the interview You need to listen careful y to what the interviewee is saying, for their response might not actual y answer the question. Alternatively, the interviewee may give you a vague response, to which, you might have to ask for clarification or further explanation. The most important thing to remember when conducting an interview is not to rush through the questioning. Also, do not interrupt participants when they are in the middle of a sentence or when they stop in order to col ect their thoughts. “Could you tel me” is always a good way of starting an interview or asking an interviewee to explain a particular point of view. Do not disclose the details or discuss the comments of another interviewee during an interview. This not only breaches past interviewees’ confidential y, but the present interviewee wil doubt your ability to maintain their confidence. This is not to say that you cannot talk in generalities (e.g., if an interviewee asks you “what have other people said” in relation to particular point, you could say “wel , a lot of interviewees have indicated that” etc.). 49 Have your notepad and pen ready because sometimes interviewees can say the most insightful things when the digital recorder has been switched off. After the interview It is extremely important that you write the report immediately after the interview, whilst you can stil remember vividly al the aspects of the interview. The recorded audio of the interview should help you prepare an accurate report. Use your experience from each interview to improve the next interview. Themes/questions A semi-structured in-depth interview is usual y one in which the interviewer has a checklist of topic areas or questions. The themes that we would suggest are: Icebreakers Questions to the interviewees can include:  What were your expectations of the participation in the survey?  Do you think your expectations were met? Discussion on relevant survey results Interviewees shal reflect and discuss their interpretation and in-depth understanding of the relevant survey results, for example going into different elements of SELFIE tool (e.g., Leadership, Infrastructure and Equipment, Teaching and Learning etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey if it is optimal or more out of necessity and if there are intentions to become either more or less involved in SELFIE tool in the future and/or use its results. Future improvements After interviewees discuss pilot results, they should consider implementing proposed solutions. This means that they (plan to) improve process/WBL and continue to look for ways to make it even better for their organization. Questions to the interviewees can include:  What would be your potential reactions based on the survey results?  Is there an action plan to support the implementation of the proposed solutions?  How wil you prioritize your reactions to the results? Wil resources (e.g., financial, capacity etc.) play a role in prioritization process? Equipment/facilities In case of online interview, a Zoom room can be set up by the Research Team (contact us18 at least 1 week prior to the event providing exact date and timeslot). 18 Research Team contacts: miha.zimšek@skupnost-vss.si and/or alicia.miklavcic@skupnost-vss.si. 50 In-depth Semi Structured Interviews Report Date: Country: School: Facilitator(s): Interviewee: Discussion Themes - Discussion 1: Icebreakers - Discussion 2: Discussion on relevant survey results - Discussion 3: Areas where further support is needed/useful Theme 1: Icebreakers Suggestions for discussion: Questions to the interviewees can include: - What were your expectations of the participation in the survey? - Do you think your expectations were met? Common responses/general consensus: Areas of disagreement/lack of consensus: Notes & observations: Theme 2: Discussion on relevant survey results Suggestions for discussion:  What kind of technology you are using when you are working in the company? (State specific examples about the use of technology in company and in school?)  Did you start with digital learning because of COVID-19?  What problems did you face because of COVID-19?  Did you include blended learning?  Did you perform apprenticeships during the lockdown (remote mode/distance mode)?  Wil you use SELFIE WBL in the future?  What are the things you liked about SELFIE WBL? What could be improved? Interviewees shal reflect and discuss their interpretation and in-depth understanding of the relevant survey results, for example going into different elements of SELFIE tool (e.g., Leadership, Infrastructure and Equipment, Teaching and Learning etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey, if it is optimal or more out of necessity and if there are intentions to become either more or less involved in SELFIE tool in the future and/or use its results. Common responses/general consensus: 51 Areas of disagreement/lack of consensus: Notes & observations: Theme 3: Future improvements Suggestions for discussion: Questions to the participants can include:  What would be your potential reactions based on the survey results?  Is there an action plan to support the implementation of the proposed solutions?  How wil you prioritize your reactions to the results? Wil resources (e.g., financial, capacity etc.) play a role in prioritization process? Again, discussion should be encouraged comparing different situations, experiences, and visions. Common responses/general consensus: Areas of disagreement/lack of consensus: Notes & observations: Additional themes/discussions/ideas/observations (Fil in only if the content does not fal into any previous categories/themes above) Notes & observations: 52 List of Challenges The fol owing tables are to be fil ed in by the corresponding participants in the pilot process from the beginning of their engagement til the November 15th, 2020. They wil serve to the research team to identify advantages and positive reflections to SELFIE WBL but foremost to identify chal enges and possibilities of improvement. School Coordinator/Leadership Country: School: Process Advantages Challenges School registration process Supporting materials and info Input of School data Customising survey Motivating participants - Students - Teachers - Leaders - Companies Generating links Survey content Survey technical issues Monitoring participation - Students - Teachers - Leaders - Companies SELFIE WBL Report - Usefulness - Features lacking Reaching objectives (40% of students and 40% of teachers) Certificates/Digital badges - Participants - School Findings (unexpected issues) Lessons learnt Covid 19 impact How Covid 19 affected /experience with blended learning, description of the profile of school, remote teaching and learning Other Add rows, as necessary. Source: Skupnost VSŠ, 2020. 53 Annex 4. Analysis of open question “Suggestions for improvement” and examples of questions Thematic analysis, defined as a method for identifying, analysing, and reporting patterns (themes) within data (Braun and Clarke, 2006) was used for analysing Open-ended question on Suggestions for improvement given by students. Description of process: We read al answers from students to open question: »How can we improve SELFIE further? Share your ideas and suggestions with us. « We have got familiarised with the data and prepared list of key issues/themes and codes. Text answers of students was tabulated, and each answer was classified in themes (code). Then we counted the number of answers with the same code and prepared Table 3. Categories/themes: S – About SELFIE TOOL (satisfaction, critics, missing themes, items to add) Q – Opinion about questions (length, repeating, complicated) A – Opinion about answers (number of answers, option others: ____ …) L – Language (terminology, understandable, more languages) D – Devices – problems with using different devices for SELFIE T – Timing of involvement I – Design W – Internet connection DT – Digital technology P – Praises 0 – Nothing to change K – Critics C – Linked with Covid-19 / - Prefer not to answer X – Not classified Table 3. Thematic analysis of open question responded by students. Code Key words, answers Frequency S SELFIE (simplify, long, make shorter, concentrated text, make profession specific) 21 Q Questions (to general, better, understandable, normal, concise, clarifications needed, 40 add explanation, more specific, too general, add questions for instance »What have we learned with ICT? «, about distance learning, personal tailoring, many unnecessary) A Answers (wider choice of options about placement in company; more and 19 understandable answers, option »other« and possibility to write the answer, more detailed answer option, or a wider scale for reviews, 10-level scale9 L Language, vocabulary (formal wording, translation, to complicated) 7 D Devices (not possible to read the ful answer on mobile) 1 T Timing (not yet in the company for practice, 4 I Design (grey/dark theme, colourful, add music) 7 W Wi-Fi connection 0 Dt Digital technology (equipment, use, teacher’s knowledge for using DT and for remote 9 teaching) P Praises (perfect, understandable, good interface, OK, just fine, no shortcomings, liked 95 SELFIE, everything clear, was satisfied, good) 0 No proposals, Nothing left out, No need for changes, No ideas 113 k Critics (imperfect, unnecessary, irrelevant, boring, no sense) 13 C Linked with Corona-19 0 / I'd rather not answer 7 x Unclassified 27 Total 363 Source: Own analysis. Examples of questions considered repetitive: 54 In our school, I have access to the internet for learning In my company, I have access to the Internet for learning In our school, there are computers or tablets for me to use In my company, I can learn operating the relevant (digital) equipment In our school, I use technology in different subjects In our school, we use technology for projects that combine different subjects Examples of questions considered too long and complex: In our school, I have access to a database of companies providing traineeships, apprenticeships, and other opportunities In our school, teachers give us different activities to do using technology that suit our needs In our company, in-company trainers use digital technologies to tailor the training to our individual needs In our company, I gain experience in using digital technologies, which makes me more prepared for my future profession In our school, we talk with teachers about the advantages and disadvantages of using technology for learning In our school, I use technology to understand my strengths and weaknesses as a learner In our company, I use digital technology to understand my strengths and weaknesses as a learner In our school, I use technology to keep a record of what I have learned relevant to my field of study 55 Annex 5. School report “Overview of areas” Figure 14. Overview of areas snapshot from an Anonymous SELFIE WBL school report. 56 Source: Anonymous SELFIE WBL school report. (2020) 57 Annex 6. Figures and tables with results of SELFIE WBL piloting quantitative data Figure 15 displays average values per respondent group for al variables. The mean on a five-point Likert scale (1-5) was equal y high for school leaders, teachers, and in-company trainers (M=3.8) and the lowest for students (M=3.2). Figure 15. Mean score for all variables in main areas per respondent group. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Table 4 displays the percent of answers about the overal satisfaction with SELFIE WBL on 10-level scale per respondent group and means for satisfaction with SELFIE WBL per respondent group. The percent of scores above the middle of the scale is the highest in the group of school leaders (87.0%) and the lowest in the group of students (66.9%). The highest satisfaction is in the group of school leaders (M=7.6) and the lowest, yet stil above the middle of the 10-level scale, is in the group of students (M=6.4). Mean of al respondents’ satisfaction is 6.5. Table 4. Overall satisfaction with SELFIE - percentage distribution per respondent group. Overall School In-company Teachers Students Total satisfaction with leaders trainers N=421 N=1475 N=1973 SELFIE N=46 N=31 1 0.0% 1.4% 5.6% 3.2% 4.6% 2 0.0% 1.4% 2.8% 3.2% 2.4% 3 0.0% 5.0% 4.9% 3.2% 4.8% 4 0.0% 4.5% 5.4% 3.2% 5.1% 5 13.0% 14.0% 14.3% 9.7% 14.0% 6 8.7% 10. 5% 12.3% 9.7% 11.8% 7 21.7% 16.4% 19.2% 22.6% 18.7% 8 28.3% 29. 5% 18.2% 25.8% 21.1% 9 21.7% 8.8% 6.4% 12.9% 7.3% 10 6.5% 8.6% 10.8% 6.5% 10.2% Summary 1-5 13.0% 26.4% 33.1% 22.6% 30.9% Summary 6-10 87.0% 73.6% 66.9% 77.4% 69.1% Mean 7.6 6.9 6.4 7.2 6.5 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 58 Students and in-company trainers were asked about their opinion of the questions included in the SELFIE WBL self-reflection exercise (Table 5). They rated the relevance of questions on a 10-level scale. Students provided 65.4 % of responses in the range of 6-10 (M=5.7), and in-company trainers in 77.4% (M=6.8). Table 5. Relevance of questions per respondent group. Students N=1391 In-company trainers N=31 Score Frequency Percent Frequency Percent 1 76 5.5% 1 3.2% 2 43 3.1% 1 3.2% 3 68 4.9% 1 3.2% 4 87 6.3% 1 3.2% 5 207 14.9% 3 9.7% 6 153 11.0% 3 9.7% 7 242 17.4% 7 22.6% 8 242 17.4% 8 25.8% 9 128 9.2% 4 12.9% 10 145 10.4% 2 6.5% Summary 1-5 481 34.6% 7 22.6% Summary 6-10 910 65.4% 24 77.4% Mean 5.7 6.8 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Table 6 presents the percent of answers about the likelihood for further recommendation of SELFIE WBL per respondent group on a 5-level scale. The highest percent of positive responses (“Very likely” and “Extremely likely”) is in the group of school leaders (50.0%). In the group of in-company trainers is the share of positive responses 25% and in the groups of teachers 19.3%. In the group of teachers 19.5% of responses are negative responses (“Not at al likely” and “Not very likely”). The percent of answer “Prefer not to say” is the highest among in-company trainers (15.6%). The average likelihood for further recommendation of the SELFIE WBL self-reflection exercise is the highest among school leaders (M=3.7) and the lowest among teachers (M=3.2). Table 6. Likelihood for further recommendation of SELFIE tool - percent per respondent group. Recommending Leaders Teachers In-company Total SELFIE N=48 N=452 trainers N=32 N=532 Not at al likely 0.0% 6.4% 6.3% 5.8% Not very likely 2.1% 13.1% 0.0% 11.3% Somewhat likely 39.6% 33.6% 34.4% 34.2% Very likely 33.3% 27.7% 34.4% 28.6% Extremely likely 16.7% 6.2% 9.4% 7.3% Prefer not to say 8.3% 13.1% 15.6% 12.8% Mean 3.7 3.2 3.5 3.3 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 59 Figure 16 displays likelihood for further recommendation of the SELFIE WBL self-reflection exercise. Means in al groups are above the middle of the 5-level scale. School leaders have the highest mean (3.7) and teachers the lowest (3.2). Figure 16. Mean likelihood for further recommendation of SELFIE. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Figure 17 displays shares of factors which negatively affect digital technologies use in schools and companies. There is disagreement between respondent groups. School leaders (20.1%) and teachers (18.4%) chose “Lack of funding” most frequently and in-company trainers chose “Lack of time for trainers” most frequently (15.0%). Figure 17. Negative factors for technology use in school and company - percent per respondent group. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 60 Figure 18 displays shares of factors which negatively affect remote teaching, learning, or training. Results show that participants share the opinion that the most influential positive factor for remote teaching, learning, and training with digital technology is “Limited students access to digital devices” (school leaders 23.0%, teachers 19.0% and in-company trainers 21.2%). Al participants agree that the positive factor that least affects remote teaching and learning is the “Teachers/Trainers lack of time to provide feedback to students” (school leaders 4.1%, teachers 6.1% and in-company trainers 4.5%). Figure 18. Negative factors for technology use for remote teaching, learning, and training – percent per respondent group. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 61 Figure 19 displays shares of factors which negatively affect remote teaching, learning, or training. Results show that school leaders and teachers share the opinion that the most influential positive factor for remote teaching and learning and training with digital technology is “Teachers’/Trainers’ participation in professional development programs” (school leaders 17.3%, teachers 15.0% and in-company trainers 15.6%). Figure 19. P ositive factors for remote teaching, learning, and training - percent per respondent group. Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 62 Annex 7. Overview of SELFIE WBL results in Hungary The outcomes of the pilot are not representative of the national education and training systems. They provide useful insights for schools and companies participating in the pilot and, overal , for schools and companies providing similar WBL programmes and belonging to the specific economic sectors covered by the pilot. Details on al questions can be found in the questionnaires on the SELFIE tool website. User participation Note: The six participation categories were answered by school coordinators during school registration. Categories for ‘disadvantaged homes’ and ‘different language’ are: fewer than 10 %, 10 -25%, 26-50%, above 50 %, I don’t know. ‘Didn’t answer’ is also possible, as the questions were optional. 63 SELFIE WBL - Main areas Note: positive responses = answers on 4 or 5 on a five-point scale 64 65 66 SELFIE WBL - Additional areas Note: positive responses = answers on 4 or 5 on a five-point scale 67 68 69 70 Satisfaction Note: Satisfaction with SELFIE WBL, on a scale from 1 to 10 71 Likelihood of recommending SELFIE Note: on a scale from 1 to 5 Annex 8. Country fiche 72 73 74 Annex 9. List of tools similar to SELFIE and other tools used in WBL The goal was to map out existing self-reflection tools and other existing digital tools in the country and schools used in WBL contexts. This mapping and listing shal include official and available websites from Governmental Institutions responsible for overseeing the WBL in the country and with different stakeholders engaged in the pilot. Name of WBL tool Link Aim Advantages SELFIE is a free, online SELFIE al ows a school tool to help schools to monitor its progress https:/ ec.europa.eu/educ assess how they use over time and can help SELFIE WBL ation/schools-go- digital_en digital technologies for start a dialogue within innovative and effective the school on potential learning. areas for improvement. The label system of “Digital School” was developed in order to provide information on This system linked to the the digital compliance DigCompOrg framework Digitáls Névjegy level of public education to ensure international Rendszer (DNR) - The DNR (dpmk.hu) institutions, including the comparability. It is https:/ dnr.dpmk.hu/page. availability of internet implemented into the label system of “Digital School” php?pid=77 access and ICT tools, the Hungarian digital readiness and circumstances. The digital teaching practices system is for al type of of teachers as wel as schools. the digital extra- curricular activities they offer, etc. The SELFIE WBL system is a unique opportunity as it • Improves learning in both school and company and improves mentoring and guidance • Facilitates coordination among the VET school, the company, and the student • Supports to find new ways to lower training costs Currently in Hungary the pilot schools do not know and use any other existing self-reflection tools and other existing digital tools in the country and schools used in WBL contexts. There is a Hungarian self-reflection tools based on DigCompEdu/DigCompOrg, cal ed “Digitáls Névjegy Rendszer (DNR)” - The label system of “Digital School” - but it is not dealing with WBL aspect of the education. The system is piloted currently. 75 GETTING IN TOUCH WITH THE EU In person Al over the European Union there are hundreds of Europe Direct information centres. You can find the address of the centre nearest you at: https:/ europa.eu/european-union/contact_en On the phone or by email Europe Direct is a service that answers your questions about the European Union. You can contact this service: - By freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these cal s), - At the fol owing standard number: +32 22999696, or - By electronic mail via: https:/ europa.eu/european-union/contact_en FINDING INFORMATION ABOUT THE EU Online Information about the European Union in al the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en EU publications You can download or order free and priced EU publications from EU Bookshop at: https:/ publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en). KJ-NA -30771 -EN -N Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID doi:10.2760/25503 72836355 IS BN 978-92-76-40324-1 ISBN 978-92-76-40324-1 (PDF)