Country report for SELFIE WBL piloting France Maria João Proença (EfVET) Miha Zimšek (Skupnost VSŠ) Anita Goltnik Urnaut (Skupnost VSŠ) Alicia Leonor Sauli Miklavčič (Skupnost VSŠ) Ralph Hippe (JRC) 2021 EUR 30933 EN This publication is a report by the Joint Research Centre (JRC), the European Commission’s science and knowledge s erv ice. I t a ims to provide evidence-based scientific support to the European policymaking process. The scientific output expressed does not imply a policy position of the European Commission. Neither the European Commission nor any p erson a ctin g o n b ehalf o f th e Co mmis sio n is responsible for the use that might be made of this publication. For information on the methodology and quality underlying the data used in this publication for which the source is neither Eurostat nor other Commission services, users should contact the referenced source. The designations employed and the presentation of material on the maps do not imply the expression of any opinion whatsoever on the p art of the European Union concerning the legal status of any country, territory, city or area or of its authorities, or concerning the delimitation of its frontiers or boundaries. Contact information Name: César Herrero Address: Edificio Expo, C/ Inca Garcilaso 3, 41092 Sevil e, Spain Email: cesar.herrero@ec.europa.eu EU Science Hub https://ec.europa.eu/jrc JRC125789 EUR 30933 EN PDF ISBN 978-92-76-45988-0 ISSN 1831-9424 doi:10.2760/424070 Luxembourg: Publications Office of the European Union, 2021 © European Union, 2021 The reuse policy of the European Commission is implemented by the Commission Decision 2011/833/EU of 12 December 2 0 1 1 o n th e reuse of Commission documents (OJ L 330, 14.12.2011, p. 39). Except otherwise noted, the reuse of this document is authorised u nder the Creative Commons Attribution 4.0 International (CC BY 4.0) licence (https://creativecommons.org/licenses/by/4.0/). T h is me ans th at reuse is al owed provided appropriate credit is given and any changes are indicated. For any use or reproductio n o f photo s o r o ther material that is not owned by the EU, permission must be s ought directly from the copyright holders. Al content © European Union, 2021, except: cover image © j-mel - stock.adobe.com How to cite this report: Proença, M. J., Zimšek, M., Goltnik Urnaut, A., Sauli Miklavčič, A. L., and Hippe, R., Country report fo r S ELFI E WB L piloting – France , EUR 30933 EN, Publications Office of the European Union, Luxembourg, 2021, ISBN 978 -92-76-45988-0, doi:10.2760/424070, JRC125789. Contents Executive summary.................................................................................................................................................................................................................................. 4 1. Introduction ...........................................................................................................................................................................................................................................6 2. Digital education and WBL policies ................................................................................................................................................................................8 3. Setting up the pilot........................................................................................................................................................................................................................10 3.1 Methodology for selecting the pilot schools and companies.....................................................................................................10 3.2 Methodology for translating and adapting SELFIE materials................................................................................................... 15 4. Pilot implementation.....................................................................................................................................................................................................16 5. Fol ow up – quantitative and qualitative analyses........................................................................................................................................19 5.1 Methodology..............................................................................................................................................................................................................................20 5.2 Quantitative results............................................................................................................................................................................................................ 22 5.3 Qualitative results................................................................................................................................................................................................................ 27 5.3.1 Initial motivation from participants.....................................................................................................................................................29 5.3.2 Strengths and weaknesses of the SELFIE WBL tool ..............................................................................................................29 5.3.3 Questionnaire, content and SELFIE WBL report ..................................................................................................................... 30 5.3.4 Current and future use of SELFIE WBL ............................................................................................................................................ 30 5.4 Overall findings ......................................................................................................................................................................................................................31 6. Lessons learnt and suggestions for future development........................................................................................................................34 7. Implications of the COVID-19 pandemic................................................................................................................................................................ 36 8. Conclusions and recommendations.............................................................................................................................................................................37 References..................................................................................................................................................................................................................................................... 38 List of abbreviations and definitions ................................................................................................................................................................................... 41 List of figures...............................................................................................................................................................................................................................................43 List of tables ................................................................................................................................................................................................................................................. 45 Annexes............................................................................................................................................................................................................................................................. 47 Annex 1. Key information on the WBL system..................................................................................................................................................48 Annex 2. Dominant economic sectors in France .............................................................................................................................................53 Annex 3. Guidelines and templates for focus groups, semi-structured interviews and a list of challenges 55 Annex 4. Analysis of open question ‘ Suggestions for improvement’ and examples of questions......................67 Annex 5. School report ‘Overview of areas’..........................................................................................................................................................70 Annex 6. Figures and tables with results of SELFIE WBL piloting quantitative data..................................................73 Annex 7. Overview of SELFIE WBL results in France.................................................................................................................................81 Annex 8. Country fiche..............................................................................................................................................................................................................94 Annex 9. List of tools similar to SELFIE and other tools used in WBL...................................................................................... 96 Acknowledgements We would like to thank al vocational education and training (VET) schools and companies that, on a voluntary basis, have participated in the SELFIE for work-based learning (WBL) piloting experience during the most chal enging period of the last century. A word of appreciation goes to the national coordination te am who, tirelessly, assured the engagement with and continuous support to participating schools and companies and provided the valuable information without which the pilot experience and the report would not have been possible. We would also like to thank DG Employment, Social Affairs and Inclusion and the European Training Foundation (ETF) for the effective col aboration throughout the piloting phase and DG Education and Culture for the support. In addition, we would like to thank the national coordinators of al nine piloting countries for the fruitful exchanges and opportunities of mutual learning that have facilitated the piloting process. Lastly, the active involvement and support of national VET and WBL stakeholders has been crucial in this endeavour to pilot SELFIE WBL during the COVID-19 crisis. 1 Disclaimer The aggregated and anonymised data which is presented in this document has been extracte d by the European Commission from the SELFIE database and does not necessarily reflect an official opinion of the Commission. The Commission does not guarantee the accuracy of the data included in this document. Neither the Commission nor any person acting on the Commission’s behalf may be held responsible for the use which may be made of the information contained therein. The views expressed in this report are purely those of the authors and may not in any circumstances be regarded as stating an official position of the European Commission. Note: This report was produced within the framework of contract agreement 939680 – 2020 BE European Forum of Technical and Vocational Education and Training (EfVET) and the European Commission’s Joint Research Centre (JRC) – ‘Piloting SELFIE for work-based learning contexts in VET (SELFIE WBL), Lot 2: France’. 2 Abstract This report presents the results of the pilot study of SELFIE for work-based learning carried out in France between September and December 2020. The study aimed at testing the tool and quality of the questionnaires before its launch online. In total, 15 VET col eges and 18 companies (operating in differe nt sectors) were engaged in the pilot, involving 3 365 users (teachers, students, school leaders and in-company trainers). In addition, 192 individuals (students, teachers, school leaders, school coordinators and in-company trainers) participated in the qualitative research carried out after the pilot. This research included intervie ws and focus groups, with the purpose of col ecting further feedback. The overal results indicate that the SELFIE WBL tool is user-friendly and easy to understand, wel designed and inclusive with its 360-degree reflection, as it engaged al those involved in WBL activities in the French WBL system. Nevertheless, responde nts also requested a variety of changes to the questionnaires and made recommendations on how to improve the tool. The SELFIE WBL tool and the report were seen as providing support to school leaders in the development and monitoring of the school’s digital strategy as wel as providing relevant information to al stakeholders in the SELFIE WBL pilot, contributing to increasing the effectiveness of learning in VET schools and companies. 3 Executive summary SELFIE is an online self-reflection tool developed to support schools, including VET, to assess their digital capacity and preparedness by looking at different dimensions such as school strategies, teaching and learning practices, equipment and infrastructure and student competences. The tool was developed in 2018 by the JRC and the Directorate-General for Education, Youth, Sport and Culture. In early 2020, in cooperation with the Directorate-Gene ral for Employment, Social Affairs and Inclusion, it was adapted to include a module on work-based learning which adds the vie ws of in-company trainers. The aim has been to help improve coordination between VET schools and training companies, and to discuss how they could jointly embed digital technology in their training and apprenticeship programmes. This also means bringing VET trainers1 and in-company trainers closer together. Throughout 2020, the JRC launched a pilot experience of SELFIE for work-based learning contexts in VET (SELFIE WBL) in nine different countries. The EfVET in col aboration with the JRC organised this in France, Poland, Hungary, Germany. In addition, the JRC managed the pilot in Romania. Four additional non -EU countries (Georgia, Montenegro, the Republic of Serbia and Turkey) piloted SELFIE WBL managed by the ETF and JRC. The aim of the pilot was to test the WBL extension of the tool, which first meant an e xtended questionnaire for the existing groups but also a new questionnaire for a new respondent group. Ensuring the high quality of the new questionnaires was a key objective of the piloting. In addition, a range of te chnical changes to the tool were piloted. In particular, involving companies in the SELFIE self-reflection e xercise in addition to schools was a new and important endeavour, and it was important to test its concrete practical implementation. The piloting of SELFIE WBL in France was launched in July 2020 and effectively rol ed out in September 2020. It entailed three main phases: the first related to the translation of al supporting documents and the tool itself; the second to the selection and engagement of stakeholders (including VET schools and companies); and the third related to the piloting of the SELFIE WBL in the selected VET schools and companies and the qualitative research consisting of the organisation of focus groups with learners2 and trainers in each one of the VET schools, in-depth interviews with school directors and in-company trainers and additional desk research on similar self- reflection and other digital tools in use in the country. The main emphasis of the piloting experience was on the qualitative research as it al owed quality information to be col ected with the view to contributing to practice development and improving the SELFIE WBL tool and its further development. 13 schools were involved in the qualitative research, 21 focus groups (total ing 84 trainersand 86 learners) and 18 semi-structured interviews were organised with 14 school leaders and 4 in-company trainers which al owed the col ection of relevant feedback regarding the tool. The pilot process was disturbed by the COVID-19 pandemic with the confinement measure s take n by the government, impacting on the data col ection process and requiring great efforts from the national team and the school coordinators to assure the delivery, as planned, of al activities. This also had a massive impact on the educational community’s state of mind, making it difficult to motivate and engage participants to fill out the SELFIE WBL tool. The overal feedback received was that the SELFIE WBL tool is user-friendly and e asy to understand, well designed and inclusive with its 360-degree reflection, as it engaged al those involved in WBL activities in the French WBL system (learners, trainers, school leaders and in-company trainers). Nevertheless, re spondents 1 In France, VET school teaching staff are addressed as trainers and not teachers, therefore the term ‘trainers’ is the term u s ed in th is report related to the results provided during the SELFIE WBL survey under the term ‘teachers’. 2 In France, VET school students are addressed as learners and not students, therefore the term ‘learners’ is the term used in this re port related to the results provided during the SELFIE WBL survey under the term ‘students’. 4 also requested a variety of changes to the questionnaires (for example, a lower total number of q uestions) and made recommendations on how to improve and increase the scope of the tool, such as providing the possibility to network with other schools. Additional materials such as guides for learners were also mentioned. The main chal enges identified through the quantitative and qualitative research by school leade rs, trainers and learners proved to be pedagogical support and resources with the digital competences and knowledge of trainers, the digital learning skil s of learners, the overal implementation of digital te chnologies in the classroom and the infrastructure and equipment. Likewise, for in-company trainers, the biggest challenges mentioned were the infrastructure and equipment. The SELFIE WBL tool and the report provided support to school leaders in the development and monitoring of the school’s digital strategy as wel as providing relevant information to al stakeholders in the SELFIE WBL pilot, contributing to increasing the effectiveness of learning. School leaders and trainers have also expressed their intention to use it on a regular basis. School leaders have also expressed their interest in the next steps of SELFIE WBL and exploring further opportunities of SELFIE WBL to facilitate engagement of and impact on al stakeholders. In addition to the technological aspect and competences, trainers’ attitudes towards the ‘digital world ’ and digitalisation in general also have to be taken into consideration. School leaders shared their perspectives regarding the importance of digitalisation not only as a result of the pandemic, but rather as a means to encourage al stakeholders (schools, companies) to increase the effectiveness of teaching and learning. Feedback provided was that the SELFIE WBL pilot came at the right time, not only for schools and their leaders, but also for trainers, learners and in-company trainers. The next chal enge wil be to act based on the SELFIE WBL report results. 5 1. Introduction SELFIE is a self-reflection tool supporting the use of digital technologies for teaching and learning in schools. It was launched by the European Commission in October 2018 as part of its Digital Education Action Plan and is available in more than 30 languages. After the successful start of SELFIE, a feasibility study (Broek and Buiskool 2020) was conducted that explored how the tool could be adapte d to the needs of work-based learning (WBL) contexts, in which a student is learning both in a school and in a company. Following the positive results of this study, the extension of the tool was developed. The aim of the SELFIE WBL pilot was to test this WBL extension of the tool, which first meant an extended questionnaire for the existing groups (school leaders, teachers and students) but also a new questionnaire for a new respondent group (in-company trainers). Ensuring the high quality of the new questionnaires was a key objective of the piloting. In addition, a range of technical changes to the tool were pilote d. In particular, involving companies in the SELFIE self-reflection exercise in addition to schools was a new endeavour, and it was crucial to test its concrete practical implementation before the final release of SELFIE WBL. The pilot experience of SELFIE WBL was launched in nine countries. The European Forum of Te chnical and Vocational Education and Training (EfVET), in col aboration with the European Commission’s Joint Re search Centre (JRC), organised this in France, Poland, Hungary and Germany. The JRC managed the pilot in Romania. In addition, the European Training Foundation (ETF), in col aboration with the JRC, piloted the tool in four non-EU countries, namely Georgia, Montenegro, the Republic of Serbia and Turkey. The overal management of the SELFIE WBL pilot in Germany was carried out by the EfVET in collaboration with the JRC. The pilot was coordinated at national level by the Societé dÉnseignement Profe ssionnelle du Rhône (SEPR), one of EfVET’s members in France. The qualitative research and reporting of the pilot was led by an EfVET member in Slovenia – Skupnost višjih strokovnih šol Republike Slovenije (Skupnost VSŠ). Overal management of SELFIE WBL in France – specific responsibilities al ocated to each organisation were as fol ows. EfVET – The European Forum of Technical and Vocational Education and Training was the project coordinator and responsible for the overal project management, quality and reporting. More spe cifically, the project manager was responsible for the implementation of the work plan and for al administrative and financial management of the proposal and for assuring each member of the team was provided with the support needed to implement the tasks. The EfVET had one member of the governance responsible for overseeing the piloting process and one project manager responsible for the operations and ongoing support to the nat ional coordinators and the liaison with the JRC. Skupnost VSŠ – Skupnost višjih strokovnih šol Republike Slovenije was a research partner and, as such, responsible for the qualitative research including the conducting of the case studies as wel as the final report summarising the process of and lessons learnt from the piloting of SELFIE WBL in VET schools and companies, and compiling the list of digital tools used in the work-based le arning (WBL) sector for each country. Skupnost VSŠ had three members who were part of the research team (one senior and one junior researcher, and a senior WBL expert), working directly with the EfVET and national coordinators. SEPR – The Societé dÉnseignement Professionnel e du Rhône was the national coordinator for France and, as such, responsible for the translation and adaptation of SELFIE WBL and supporting materials into French; for reaching out and engaging the stakeholders, VET schools and companies; and for overseeing the piloting of the SELFIE WBL tool and supporting the research component. The national coordinator worke d very close ly with school coordinators, providing ongoing support. The national coordinator had a pivotal role in the piloting process for the ongoing support to VET schools and companies. The SEPR had two members of staff 6 dedicated to the SELFIE WBL pilot – one senior VET expert supporting and one additional member of the team responsible for overseeing the operations at national level. Management at national level – responsibilities were defined as fol ows. The national coordinator had a pivotal role in the SELFIE WBL piloting proce ss and the sele ction of VET schools and companies at national level. The national team was responsible for the ongoing support to VET schools, the engagement of national stakeholders, the preparation and delivery of planned webinars, acting as a liaison between Skupnost VSŠ and VET schools in everything related to the research component, including the translation of support materials developed for that purpose. The national team was responsible for the conducting of the interviews with school leaders and company representatives. The school coordinators were the main organisational force at institutional level e ngaging and mobilising companies, school leaders, trainers and learners and offering them ongoing support during the pilot process. The school coordinator was also responsible for the organisation of the focus groups that took place i n schools – one with trainers and the other with learners . The school coordinators were also responsible for the management of the relationship with companies and the eventual support that might have been re quire d throughout the SELFIE WBL pilot. 7 2. Digital education and WBL policies Work-based learning (WBL) refers to knowledge and skil s acquired through carrying out – and reflecting on – tasks in a vocational context, either in the workplace or in a VET institution. In France, there are two diffe rent pathways for work-based learning, namely (Ministry of Labour, 2017): — ful -time study in a vocational school with placements in a company (from 4 to 10 weeks, depending on the qualification level). There are public and private vocational schools, both delivering diplomas from the State; — apprenticeship in an apprenticeship training centre (ATC). Most ATCs are private as the legal status is non-profit organisation, but the National Board of Education created some public ATC s that are integrated into public training centres. In any case, the same diplomas are awarded at the end of the training. The only difference is the method of training and the time spent at school and in a company. The recognition of the value of apprenticeships in France has increased over the years, currently being se en as a way to gain work experience and excel ence. The apprenticeships programmes and curricula are set by the certifier, which may be (Ministry of Labour, 2017): — for the most part, the Ministry of National Education, Youth and Sport — the Ministry of Labour — the Chambers of Trades and Crafts — the Chamber of Commerce — professional unions. An apprenticeship is a very efficient way to enter the labour market with lasting effect: 70% of appre ntices have found a job 7 months after the apprenticeship contract ends, and 60% have a permanent contract. Companies confirm that an apprenticeship is a great opportunity: 80% are satisfied with their apprentices and they hire a new one at the end of the previous contract (Ministry of Labour, 2017). DIGITALISATION STRATEGY FOR VET AND APPRENTICESHIPS IN FRANCE The topic of digitalisation of education fal s under the responsibility of the French Digital Plan for Education established in 2015, responsible for the implementation and deployment of the digital services in the Education sector. This is part of a bigger initiative of the French Government related to the digitalisation of the public services in recognition of the need to keep up with the latest technological trends contributing to the economic development of the country (French Government, 2017). The digital strategy for the education sector (LÉcole Numérique) was official y launched in 2015, after an extensive consultation carried out by the Ministry of Education in articulation with other Ministries as it aims to bring schools into the digital age (French Government, 2017). Its implementation relies on the coordinated mobilisation and action of different stakeholders (schools, teachers, apprentices, managers, ministries and local, regional and other national authorities, universities, research centres, industry) to put in place the optimal conditions for an efficient development and deployment of resources, including access to training. The ambition is to create an e-Education ecosystem so as to enable proper use of resources and effective development of skil s and services (French Government, 2017). 8 Renewed in 2018, the strategy is organised around five main axes: place school data at the heart of the ministry's digital strategy; teach in the 21st century with digital; support and strengthen the profe ssional development of teachers; develop apprentices' digital skil s and create new links with the school's actors and partners. — The strategy envisions the continued and renewed support for the establishment of digital (educational) workplaces across the national territory. — Schools have the resources (equipment and materials) to provide digital services to e xp and and enrich the educational offer and to customise the support to apprentices. — Teachers have access to diversified educational resources that can be used on a daily basis, as well as to initial and continued training and tools, al owing them to monitor their apprentices and communicate with families. The French Government approved the deployment of 1 bil ion euros for the period from 2016 to 2019 to support the different initiatives and project development related to the digitalisation of education. Since 2016, several different initiatives and projects have been launched by the Ministry of Education. A very brief overview of the diversity landscape of these projects is shared as fol ows (French Government, 2017): — equipping schools. Huge investment was made by national, local and regional authorities to support schools with digital equipment and services (individual materials for apprentices, te achers such as cabling, internet connections, tablets, digital workspaces, access to digital resources and training) including support to schools on how to use this equipment and resources; — digital platforms. Integrated set of digital services made available to the educational community of schools. It constitutes a unified entry point al owing users to access (according to their profile and level of authorisation) their digital content. It is also a platform for exchange and collaboration between users (from the same school or other schools). This platform offers services such as digital textbooks, common workspaces and storage for apprentices and teachers, access to digital resources, col aborative tools, blogs, forums, virtual classrooms, etc.; support to school management – notes, absences, timetables, agendas, etc.; and communication, messaging, staff and family information, videoconferencing, etc. These digital workplaces can be accessed by apprentices, parents, te achers, administrative staff. This initiative is also related to one of the strategy goals of simplifying administrative formalities and facilitating the communication with the broader educational community; — training resources for teachers and trainers. Several online platforms have been develop ed with different purposes such as: providing digital training to teachers, managers, trainers on different areas; sharing resources that can be used daily; disseminating information on existing practices and research carried out at national level on the topic of digitalisation. Some of these platforms are fully dedicated to the topic of inclusiveness; — the Digital Competence Reference Framework (CRCN). Digital skil s reference framework applicable to al EQF levels, inspired by the European framework (DigComp) and launched at the start of 20 19 school year; — data protection and safety. Appointment in August 2018 of a data protection officer (DPD) for the Ministry of National Education and for the Ministry of Higher Education, Research and Innovation to assure compliance with the European guidelines and support the educational community in understanding how personal data should be col ected, processed and stored. In addition, an ethics committee on digital data was set up to advise and support on issues re lated to the use of data col ected and processed in the school context. In this context, training targeting school’s management and teachers related to the chal enges of using digital data were also deve loped and part of the portfolio of online courses were available to teachers on the different platforms. For more details on French WBL policies and digital education, see Annex 1. 9 3. Setting up the pilot 3.1 Methodology for selecting the pilot schools and companies Selection criteria for VET schools were set to capture and reflect the diversity of VET schools (see Figure 1) and their environment according to: - size of VET schools (as defined in the SELFIE WBL tool), - location (as defined in the SELFIE WBL tool), - geographical coverage (result of agreement within the SELFIE WBL pilot team), - programme area coverage (result of agreement within the SELFIE WBL pilot team), and - number of VET schools (at least 12 VET schools). Figure 1. Selection criteria for VET schools Source: Skupnost VSŠ (2020). With regard to the school size and location, the decision was to apply the same criteria as defined by the JRC in the SELFIE WBL tool. In terms of the different programmes offered by the different VET schools, this was the result of a consultation with the SELFIE WBL pilot team in the four countries where the pilot has been overseen by the EfVET. It does not intend to be an exhaustive list of al the programmes in the country but rather reflect the common areas identified by the SELFIE WBL pilot team. The agreed minimum number of VET schools to be engaged in the SELFIE WBL pilot was 12. One important consideration was the voluntary participation of schools in the pilots which meant, on a practical level, that the ultimate criteria would be the school’s availability and wil ingness to participate in the pilot and its commitment to the proposed responsibilities. Mapping VET Schools in France was achieved by the national coordinator, the SEPR, with the support of: - the Fédération Nationale des Directeurs de Cerntres de Formation dÁpprentis FNADIR (National Federation of the Apprenticeship Training Centres Leaders) which has advertised the opportunity to participate in the piloting process among its members (600 training centres); - the Services de láutomobile et de la mobilité ANFA (National Association for Training in the Automotive Sector), which supported the dissemination among its members, encouraging them to engage in the piloting; - the Association Filiére Formation de líndustrie papier carton AFIFOR (French Union Organisation for Vocational Training and Apprenticeship in the Sector of Paper and Cardboard), which supported the dissemination among its members, encouraging them to engage in the piloting. The SEPR exclusively presented the SELFIE WBL pilot during an annual meeting of the AFIFOR Steering Committe e with school leaders and also at an additional meeting focusing on technology-enhanced learning with the coordinators of the training centres responsible for digitalisation. 10 Even though a public list of VET schools in France3 exists, the above-mentioned approach – consisting of reaching out to existing national networks of VET schools – was considered as best given the limited timeline of the SELFIE WBL pilot. The registration process was managed by the national team in close communication with the above stakeholders. The ultimate decision to participate was made by VET schools. Outreach and engagement – the SEPR has established one-to-one communication with e ach VET school that expressed interest and availability to participate in the SELFIE WBL pilot, providing additional information regarding the piloting process and the qualitative research, explaining the advantages and benefits of the SELFIE WBL pilot and also providing information on the type of support available should VET schools decide to participate. This ongoing communication was critical to ensure VET schools’ engagement and commitment to participate in the SELFIE WBL pilot. A Memorandum of Understanding was sent to all VET schools to be signed, to formalise the cooperation between the EfVET, SEPR and each of the VET schools. Overal , 15 VET schools from 7 different regions have been engaged in the SELFIE WBL pilot, covering all sizes of VET schools. While most of them are located in urban areas, there is diversity in terms of geography and also in terms of programme areas. The summary of VET schools engaged in the SELFIE WBL pilot and the diversity of coverage according to above set criteria can be seen in Figures 2 and 3. Figure 2. The diversity of selected VET schools according to size, location and programme area Source: Skupnost VSŠ (2021). 3 The public list of VET schools in France is available at https:/ www.lapprenti.com/. 11 Figure 3. The diversity of selected VET schools and companies according to geographical coverage 12 Source: Skupnost VSŠ (2021). Selection criteria for companies were set to cover and reflect the diversity of companies prioritising the relevant national economic areas (see Annex 2) and the diversity the reof. The selection criteria for the diversity of companies (see Figure 4) were set to: - company size (Commission Recommendation of 6 May 2003), and - economic sector coverage (result of an agreement within the SELFIE WBL pilot team). Figure 4. Selection criteria for companies Source: Skupnost VSŠ (2020). Company engagement was managed by selected VET schools from the pool of companies each VET school works with. The above criteria were presented to each VET school by the SEPR. The minimum requirement set for the SELFIE WBL pilot was to engage at least one company per VET school involved. Their engagement was based on their availability and wil ingness to participate and aligned with criteria set above, despite the 13 additional measures taken as a result of the COVID-19 pandemic. The number of companies engaged was 18 and the diversity of coverage according to the above criteria set can be seen in Figure 5. Figure 5. Selected companies per selection criteria Source: Skupnost VSŠ (2021). Overal , there was an effort at national level to be as diverse as possible regarding the economic sectors. As Figure 5 reflects, great diversity was achieved regarding the companies’ size but rather mode rate diversity regarding economic sectors. Nevertheless, the most dominant sectors such as agriculture, tourism, telecommunications, electronics and automotives are represented (see Annex 2). Each VET school engaged at least one company resulting in 18 companies from seven different regions (see Figure 3). Initial y, it had been planned to have companies’ re presentatives signing a Memorandum of Understanding. Given the feedback received by the national coordinator regarding the chal enges the proce ss of having companies signing this document would represent, and the wish of VET schools to take responsibility for the management of the communication and relationship with the companies engaged in the SELFIE WBL pilot, the EfVET decided not to proceed with this formalisation on the basis that it was not needed, and it was adding an unnecessary administrative burden. 14 3.2 Methodology for translating and adapting SELFIE materials The translation and adjustment of SELFIE WBL consisted of 3 main actions, namely: (1) linguistic translation, (2) content-focused translation and (3) contextual adaptation and usability. The first refers to the translation of the documents provided by the JRC and was carried out by an external company (Kern Lyon) and the national team from the SEPR. The second and third actions re lated to the translation were carried out simultaneously and brought together VET and WBL experts from three different VET schools. The involvement of external VET and WBL experts was done to ensure that the language and the terminology used were clear and understandable by al those involved and in line with the official language and terminology used in the country. The linguistic translation took place in the first 2 months of the project. There was an initial misunderstanding regarding the deadlines set for the different actions and some delays were observed in steps 2 and 3. Figure 6. Translation process Source: Skupnost VSŠ (2020). 15 4. Pilot implementation The SELFIE WBL pilot was implemented in the fol owing steps (see Figure 7). Figure 7. Implementation process Source: Skupnost VSŠ (2020). Step 1) Translation of SELFIE WBL materials was completed from August to September 20 20 (see Chapter 3.2 Methodology for translating and adapting SELFIE materials). 16 Step 2) Mobilisation of VET schools and companies took place from July to September 2020 (see Chapter 3.1 Methodology for selecting the pilot schools and companies). Step 3) Selections of VET schools and companies were conducted from July to September 20 20 (see Chapter 3.1 Methodology for selecting the pilot schools and companies) and the Memorandums of Understanding were signed with each selected VET school defining the roles and commitments of e ach VET school to formalise this cooperation after the selection in September 2020. Step 4) A preparatory webinar was organised by the national coordinator to bring together all national stakeholders, the EfVET, JRC, European Commission as wel as VET schools, companies and the research team on 14 September 2020. The main objective was to present the aim of the SELFIE WBL, provide an overview of implementation steps, school self-reflection report, personalised certificates and digital badges, schools’ and companies’ commitments and the timeline. Furthermore, feedback from each representative on possible concerns and expectations was discussed as wel as the mapping of digital tools for WBL used in the country, schools and companies. Step 5) Piloting of the SELFIE WBL self-reflection exercise began by VET schools registering with the SELFIE tool, planning the activation period, announcing the SELFIE WBL pilot within the school and among partner companies, motivating them to participate by explaining the benefits of their participation. When activating the SELFIE WBL self-reflection exercise, school coordinators monitored and reported the participation rate (40% of WBL learners, 40% of VET trainers and at least 1 in-company trainer) and further motivated and promoted the participation within the target groups needed. In-company trainers proved to be the most difficult to motivate as they are not in school and under the management of the school. The SELFIE WBL process took place from September until December 20 20, and the feedback from the exercise is presented in Chapter 5.2 Quantitative results. Step 6) A fol ow-up and guidance webinar was organised by the national coordinator addressing only VET schools and company representatives on 20 October 2020. The aim was to fol ow-up the piloting experience, gather initial feedback from school coordinators, address eventual chal enges that may have arisen during the process, confirm the overal figures in terms of comple tion of the questionnaires and pre pare school coordinators for the conducting of focus groups for learners and trainers and semi-structured interviews for school leaders and company representatives. The school coordinators were asked to provide feedback on their experience during the implementation process through the list of chal enges provided by the rese arch team. The research team also provided the guidelines and reporting templates for focus group implementation as wel as the list of chal enges to school coordinators, guidelines and reporting templates for semi -structure d interview implementation to the national coordinator. The guide lines, report templates, and the list of chal enges can be found in Annex 3. Step 7) Focus groups were coordinated by school coordinators in November 2020 and January 2021. Two focus groups were organised per VET school: one with learners and one with teaching staff to re flect and discuss their interpretation and in-depth understanding of the relevant report results. Due to the COVID-19 pandemic, the school coordinators struggled to organise focus groups and reach the agreed participation rate of 10 learners/trainers per focus group (see Chapter 7. Implications of COVID-19). In total, 21 focus groups were organised involving 86 learners and 84 trainers. The feedback from the focus groups is integrate d into Chapter 5.3 Qualitative results. Step 8) In-depth semi-structured interviews were managed by national coordinators from November 2020 to February 2021. The aim was to conduct 16 interviews with 4 in-company trainers and decision-making staff in VET schools (4 pedagogical managers/directors, 4 sector heads/managers, 4 board heads/directors) to reflect and discuss their interpretation and in-depth understanding of the re port re sults and to plan improvements based on those results. Interviews were conducted online. Due to the COVID-19 17 pandemic, the national coordinators struggled to engage in-company trainers (see Chapter 7. Implications of COVID-19). In total, 18 interviews were conducted involving 14 decision-making staff in VET schools and 4 in-company trainers. The feedback from the interviews is integrated into Chapter 5.3 Qualitative results. Step 9) An evaluation webinar brought together al national stakeholders, the EfVET, JRC and the research team on 11 January 2021. The main purpose was to evaluate the e xperience, collect information and recommendations regarding the SELFIE WBL tool from policy-makers and other institutional representatives at national level, the opportunities they see for the broader use of the tool in the WBL sector and identify possible dissemination actions that could take place. The research team presented the preliminary results and discussed those with the participants. The feedback from the webinar is integrated in to Chapter 5.3 Qualitative results. Step 10) Quantitative and qualitative research was conducted simultaneously and upon the re ceipt of feedback from al above activities from September 2020 to February 2021. The research team prepared the quantitative analysis based on the results of the SELFIE WBL self-reflection exercise provided by the JRC and the qualitative analysis based on the feedback from focus groups (trainers and learners ), semi-structured interviews (school leaders and in-company trainers), the list of chal enges (school coordinators), the fol ow-up and evaluation webinars (for details see Chapter 5 Fol ow up – quantitative and qualitative analyses). The timeline of the SELFIE WBL pilot was severely affected by the COVID-19 pandemic and the October 2020 terrorist attack in France, which delayed the implementation of focus groups, semi-structured interviews, the evaluation webinar and as a consequence the qualitative and quantitative research. It also affected participants’ engagement (see Chapter 7. Implications of COVID-19). 18 5. Follow up – quantitative and qualitative analyses 19 5.1 Methodology This project aimed to explore a broad scope of aspects of the SELFIE WBL tool to contribute to practice development and to improve the SELFIE WBL tool and its further development. To reach these aims and to increase the internal and external validity of the research re sults, the re se arch design is based on the methodological triangulation of using several different methods. The research team and its project partners used this as an approach to integrate the quantitative and qualitative methodology. Therefore, the following methods and techniques were used (Majchrzak, 1990): - analysis of primary sources – analysis of anonymised data provided by the JRC; - analysis of secondary sources prepared by the JRC – 4 reports showing aggregated graphs of SELFIE WBL pilot data which were: ‘Participation’ (numerus and percentage according to different demographic variables), ‘Satisfaction’ (percentage and mean for values of overal score and further recommendations), ‘Main Areas’ (percentage of positive responses for area and each variable) and ‘Additional Information ’ (percentage of answers); - analysis of school reports generated by school coordinators involved in the SELFIE WBL pilot; - semi-structured interview reports, involving 2 respondent groups (school le aders and in -company trainers) provided by the national coordinator; - focus group reports, involving the 2 other respondent groups (trainers and learners). The quantitative data were col ected through the SELFIE WBL questionnaire s, which were answered by school leaders, trainers, learners and in-company trainers. The SELFIE WBL tool provide s state-of-the-art information as perceived by the respondent groups. Respondents were selected in a manner so that it is possible to make a representative conclusion (Ragin, 2007) at institutional level. The quantitative results provide detailed information on the number of respondents, their distribution and the differences in the responses of the different groups (school leaders, teachers, students and in -company trainers). We used univariate methods in this study. They are primarily intended to present the distribution of variables’ values; hence, the tables in Chapter 5.2 and Annex 6 display the number of values and additional statistics that we selected: mean (the average value) and standard deviation. In our database, the number of responses varied between the variables. When answering the questions for which the quantitative analysis is presented, the respondents had a help text and mostly answered on a 5-level scale with the additional option ‘prefer not to say’ or ‘not applicable’ (and in two cases on a 10-level scale, one question being for al respondent groups and another for two respondent groups). For some questions they had the possibility to select or not select the answer (multiple choice). In the fol owing quantitative part (see Chapter 5.2) we present frequency tables and descriptive statistics. The tables with descriptive statistics display: - N = number of valid responses from the respondents - mean (M) = the average value of the data points or numbers - standard deviation (SD) = a measure of the dispersion of a dataset relative to its mean. The qualitative research component of the SELFIE WBL pilot had a goal to collect feedback in view of improving the SELFIE WBL tool before it is launched online. The qualitative data were col ected through desk research, feedback from school coordinators, focus groups and in-depth semi-structured interviews. 20 The main goal of the desk research was to map out existing similar self-reflection tools in the country used in WBL contexts and to identify other existing digital tools. This mapping and listing was done in two different ways: on the one hand, the research team conducted comprehensive online desk research on al official and available websites from governmental institutions responsible for overseeing WBL in the country; on the other hand, by col ecting this information from the different respondent groups engaged in the pilot (see Annex 8). Focus groups brought groups of people together with the main purpose of col ecting feedback regarding the SELFIE WBL tool from the users’ perspective. The proposal was to conduct two separate focus groups in each VET school, one with trainers involved in the pilot and the other with learners (each gathering 10 persons). The selection of the learners and trainers did not fol ow any criteria. The selection was left to the school coordinators according to the guidelines; they invited the first 10 trainers/learners who applied. Facilitators of focus groups were given guidelines (how to conduct focus groups, how and what to report) and templates for reporting the feedback from the focus groups (see Annex 3). The qualitative research method of in-depth semi-structured interviews consisted in posing a series of open and closed questions to targeted individuals – i.e. pedagogical managers/directors, sector heads/managers, board heads/directors and in-company trainers – with the aim of gaining some insight regarding their perspective on the topic of digitalisation, their wil ingness to further explore SELFIE WBL and to integrate the tool into their current work, as wel as gathering recommendations regarding possible ways to improve it (see Annex 3). There were two open questions in SELFIE WBL for learners (digital technology they find useful for learning and ideas and suggestions to further improve SELFIE WBL). We analysed them using thematic analyses. Thematic analysis is a method for examining the content of responses from data col ected from open-ende d questions, focus group discussions or interviews. It al ows emergent topics not explicitly stated in SELFIE WBL questions to be identified. It is based on organising key issues in data and grouping them under themes reflecting important relations in the research questions (Braun and Clarke, 2006). Results of the the matic analysis were included in the qualitative part of the report (see Annex 4). The qualitative research method of analysis of school reports generated by school coordinators consists of gathering challenges, advantages of the implementation of SELFIE WBL and further feedback on the SELFIE WBL process from the perspective of school coordinators, who organised and monitore d the SELFIE WBL process within their institutions. To col ect feedback, a template was prepared and provided to the school coordinators (see Annex 3). Data col ection took place from September 2020 until February 2021. The analyses started in December 2020. Al responses to the SELFIE WBL self-reflection exercise, focus groups, semi-structured interviews and analysis of school reports generated by school coordinators remained anonymous and separate from contact details to ensure confidentiality. The outcomes of the pilot are not representative of the national education and training syste ms. However, they provide useful insights for schools and companies participating in the pilot and, overal , for schools and companies providing similar WBL programmes and belonging to the specific economic sectors covered by the pilot. 21 5.2 Quantitative results Participants in the quantitative analysis were from 13 VET schools. There were 3 365 re spondents in the database. The participation of school leaders, trainers, learners and in-company trainers was as fol ows: — 53 school leaders — 262 trainers — 3 033 learners — 17 in-company trainers. In the SELFIE WBL pilot, the sample of respondents from private schools prevails at 77.9%; 11.2% of respondents originated from public VET schools and 10.9% identified themselves as ‘not applicable’. The repondents’ sample is comparable to the national rate of public (20.5%) and private (79.5%) apprentice ship VET centres in France (Ministry of Education, 2019). 53% of respondents were from schools located in cities (100 001-1 000 000 inhabitants), 20.1% of respondents from towns (15 001-100 000 inhabitants), 7.9% of respondents from small towns (3 001-15 000 inhabitants), 7.8% from rural areas (1 000 inhabitants or fewer), 7.5% from large cities (more than 1 000 000 inhabitants) and 3.7% from vil ages (1 001-3 000 inhabitants). The SELFIE WBL self-reflection questionnaire consists of eight areas scored on a 5-point Likert scale (1-5). Figure 8 displays the percentage of positive responses (i.e. responses on 4 and 5) by main areas. The most positive responses are in the area ‘Pedagogy – Supports and resources’ (60.0%), which is fol owed by the area ‘Infrastructure and equipment’ (46.3%) and ‘Students’ digital competence” (42.6%). On the other hand, the least positive responses from the respondents are seen in the area ‘Continuing professional developme nt’ (29.8%). Figure 8. P ercentage of positive responses by area 22 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Table 1 displays average values for main areas per respondent group. The number of questions in the areas differ between the respondent groups. There are some differences in the areas in which different respondent groups rated the highest. The area with the highest mean in the group of school leaders and in-company trainers is ‘Infrastructure and networking’ (M=3.7 for both groups). Trainers and learners rated ‘Pedagogy – Supports and resources’ the highest (trainers M=4.3, learners M=3.6). The lowest mean is in the area ‘Assessment practices’ for al groups (school leaders M=2.6, trainers M=2.7, learners M=2.8 and in-company trainers M=2.5). Average values per respondent groups for al variables are the highest for school leaders (3.3) and the lowest for trainers (3.1). Table 1. Descriptive statistics for main areas per respondent group School Trainers Learners In-company leaders trainers Main area N=262 N=2 789 N=53 N=17 M SD M SD M SD M SD Leadership 3.5 1.3 2.9 1.4 / / 2.9 1.1 Col aboration and networking 3.2 1.1 2.8 1.3 2.8 1.4 3.3 1.1 23 Infrastructure and equipment 3.7 1.2 3.3 1.3 3.2 1.6 3.7 1.2 Continuing professional development 3.4 1.2 3.0 1.8 / / 2.8 1.3 Pedagogy – Supports and resources 3.6 0.9 3.7 1.3 3.6 1.4 2.6 1.3 Pedagogy implementation in the classroom 3.0 1.0 3.1 1.5 3.1 1.8 3.1 1.1 Assessment practices 2.6 1.0 2.7 1.5 2.8 1.5 2.5 1.4 Students’ digital competence 3.2 0.9 3.0 1.4 3.1 1.6 3.3 2.8 Al areas 3.3 1.1 3.1 1.6 3.1 1.6 3.1 1.3 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Note: M=mean, SD=standard deviation; green: the highest score, grey: the lowest score. Figure 9 displays means for overal satisfaction with SELFIE WBL on a 10-level scale per respondent group. The highest satisfaction is indicated by school leaders (7.5) and the lowest, yet stil above the middle of the 10-level scale, is given by learners (5.8). Figure 9. Mean overall score for overall satisfaction with SELFIE WBL per respondent group Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. The likelihood of further recommendation of the SELFIE WBL on a 5-level scale was the highest among school leaders (M=4.1) and the lowest among trainers (M=2.9) (see Table 5 in annex 6). The percentage of positive responses (‘Very likely’ and ‘Extremely likely’) in the group of school leaders was 84.9%. On the other hand, the highest percentage of negative responses (‘Not at al likely’ and ‘Not very likely’) was given by trainers (22.1%). The percentage of answer ‘Prefer not to say’ was the highest among trainers (14.9%). Learners and in-company trainers were asked about their opinion on the questions included in SELFIE WBL (see Table 3 in Annex 6). They rated the relevance of questions on a 10-level scale. Learners’ average score is low (M=5.4); in-company trainers rated questions a little higher (M=6.1). 24 The SELFIE WBL self-reflection exercise also included questions about respondents. Trainers indicated the usefulness of ‘Continuing professional development’ (CPD) activities on the pedagogical use of digital technologies. The percentage of positive responses (i.e. responses on 4 and 5) was the highest for ‘Learning through col aborating’ (57.2%), fol owed by ‘Other in-house training’ (56.8%) and ‘Online professional learning’ (56.2%). ‘Study visit’ was chosen with the lowest percentage (31.3%). The answer ‘Did not participate’ was most often used for ‘Study visits’ (68.3%). Trainers and in-company trainers were also asked about their confidence in the use of digital technologies 4. Trainers (73.2% positive responses) and in-company trainers (62.5%) feel the most confident in using technology for ‘preparing lessons’. Trainers (56.4%) and in-company trainers (52.9%) are least confident in using digital technology for ‘Feedback and support’. Trainers and in-company trainers also answered the question ‘For what percentage of teaching/training time have you used digital technologies in class in the past 3 months?’5. They had five possible answers. The highest percentage of trainers (21.4%) chose answer ‘11-25%’ and the highest percentage of in-company trainers (64.7%) chose answer ‘0-10%’. 37.2% trainers and only 6.3% of in-company trainers chose answer ‘51-75%’ or ‘76-100%’. The learners reported that they most frequently used technology in and outside of school for fun (76.4 %). Most of them had access to technology outside the school (62.2%). Only 27.9% report using technology at home for school. Answers to the question ‘Is teaching/training with digital technologies in your school/company negative ly affected by the fol owing factors?’6 displays some differences in the evaluation of factors. School leaders rated ‘Low digital competence of teachers’ (21.1%) and trainers ‘Lack of funding ’ (18.4 %) as the most influential negative factors, and in-company trainers ‘Lack of time for trainers’ (30.8%). The negative factor that al school leaders (3.5%) and trainers (5.7%) rated lowest is ‘Limited or no technical support’. Three factors are least effective according to in-company trainers (‘Lack of funding’, ‘Unreliable or slow internet connection’ and ‘Low digital competence of students’) with 5.1%. Answers to the question ‘Is remote teaching and learning/training with digital technology negatively affected by the fol owing factors?’7 display that remote teaching and learning is most often negatively affected by ‘Limited student access to digital devices’ (school leaders 19.7%, trainers 19.7% and 24.3% in-company trainers) and ‘Limited student access to reliable internet connection’ (school leaders 20.8%, trainers 16.3% and in-company trainers 13.5%). In-company trainers rated ‘Trainers lacking time to develop material for remote training’ as the second most negative factor (19.9%). The percentage of chosen positive factors for remote teaching, learning or training8 displays agreeme nt between groups. The most positive factor is ‘Teachers col aborating within the school on digital te chnology use and creation of resources’ (school leaders 16.1%, trainers 17.5%) and ‘Trainers col aborating within th e company on digital technology use and creation of resources’ (in-company trainers 17.1%). 4 Trainers responded to the question regarding the situation in their school (teaching), in -company trainers regarding the situation in their company (training). 5 Answers: 0-10%; 11-25%; 26-50%; 51-75%; 76-100%; Prefer not to say. 6 School leaders and trainers responded to the question regarding the situation in their school (trainers), in -company trainers re gard ing the situation in their company (in -company trainers). 7 School leaders and trainers responded to the question regarding the situation at their school (trainers, teaching), in -company tra iners regarding the situation in their company (in -company trainers, training). 8 School leaders and trainers responded to the question regarding the situation in their school a nd te achin g, in -co mpany tra iners regarding the situation in their company and training. 25 For more information on figures, tables and data, see Annex 6. 26 5.3 Qualitative results Thirteen pilot schools were included in the qualitative part of the SELFIE WBL pilot in France. The qualitative analysis was based on feedback from 21 focus groups,18 semi-structured interviews, 13 school reports, the final evaluation webinar, constant communication (emails, zoom cal s) with national coordinators as well as answers to open questions in the SELFIE WBL self-reflection exercise (see Chapter 5.2 Quantitative re sults). Focus groups, in which 84 trainers and 86 learners participated, were moderated by national coordinators. The latter also conducted 18 semi-structured interviews with stakeholders: 14 with school leaders and 4 with in - company trainers (see Table 2). Additional y, we received 4 reports of school coordinators identifying advantages of and positive reflections on the SELFIE WBL tool, but also chal enges and possible improvements. The COVID-19 pandemic influenced and disturbed the process as lockdown made it hard to re ach out to participants. School leaders believe that deviations in answers were caused by the fact that the SELFIE WBL pilot was conducted during the lockdown in France, which forced trainers and learners to implement distance learning. Consequently, some participants, especial y trainers, are worried and wonder what their job wil look like in the near future. Moreover, school coordinators also mention an impact of terrorist attacks (teacher Samuel Paty was beheaded) on learners and their fear of returning to the school. Based on the results of the SELFIE WBL self-reflection exercise, it was not possible to determine, by deviation, the best and worst performing school as the results were quite similar or differed only in individual parameters. Therefore, we decided to present the results of all schools covere d as study cases in this qualitative part. Table 2. Number of learners, trainers, school leaders, in-company trainers and school coordinators involved in the qualitative analysis Semi- School coordinators Focus Semi-structured Focus groups structured (list of challenges) groups interviews with School with interviews learners with in-company trainers with school trainers leaders School 1 10 5 1 1 School 2 7 3 1 1 School 3 11 10 1 1 School 4 5 6 1 School 5 1 School 6 5 6 1 1 School 7 10 9 1 School 8 10 1 School 9 11 12 2 1 1 School 11 9 2 10 27 School 8 7 1 1 1 11 School 1 12 School 13 School 14 School 8 7 15 TOTAL 86 84 14 4 4 Source: Own analysis. For details on focus groups, semi-structured interviews and chal enges, see Annex 3. 28 5.3.1 Initial motivation from participants Learners were general y happy to be consulted and understood the importance of results to see where they stand in the use of digital technologies. It is difficult to comment on learners’ expectations, as trainers and school leaders asked them to fil out the SELFIE WBL survey. Almost half of learners did not receive clear information on survey objectives and goals, which had a negative impact. They would welcome a meeting before filing out the SELFIE WBL survey to be better prepared. Likewise, trainers participated because their school leaders asked them to, although some were glad to be consulted and saw the SELFIE WBL self-reflexion exercise important for the future. Trainers’ prior expectations were primarily to gain an overview of established practices, an overview of the digital ‘situation’ in their school and to improve the use of digital tools and practices. The SELFIE WBL pilot was therefore seen as a way to meet these expectations and move forward together to improve teaching practices. On the other hand, school leaders expected to see how the use of digital technology for te aching and learning was running in other VET schools (and abroad), to share good practices and experiences with peers (at national and European level) and to benefit from an overview of their use of digital practices. In-company trainers also expected to identify, better understand and learn new teaching methods at national and European level. Some participants could not imagine that such tools exist and see the added value of the SELFIE WBL in the inclusion of multiple stakeholders, especial y learners. What participants would like to see is how the surve y results wil be processed and used for future improvements at school level. 5.3.2 Strengths and weaknesses of the SELFIE WBL tool Participants fil ed out the SELFIE WBL tool on various devices (computers, smartphones and tablets). General y, they did not report technical problems when connecting and completing the questionnaire. What works particularly wel is that participants are able to complete the survey on their smartphones, especially since many learners do not own a personal computer. Participants further mention that the SELFIE WBL tool is easy to understand, complete, allows smooth navigation and has an advantage of being anonymous. The ‘help text’ feature is useful to help participants understand more complex questions. The SELFIE WBL self-reflection exercise is a good basis for an inventory – opening the debate, provoking internal exchanges and opening up a dialogue between trainers, learners, in-company trainers and the school leaders on the use of digital technology for teaching and learning. As such, it has a unifying effect. Moreover, the SELFIE WBL report offers school leaders an overview of strengths and weaknesses in the use of digital technologies. On the other hand, some questions are too general and lack details according to participants9. Some trainers believe the current assessment scale (1-5) directs the participant to choose option 3 in case of a doubt and does not require that participants clearly express their position. That could also explain the surve y re sults around the median. For more precision, they propose a Likert scale from 0 to 10. A group of learners wished that the interface of the SELFIE WBL tool was more dynamic and modern. 9 The participants were asked to provide concrete details of such questions, but they could not indicate them. 29 The participants also had some recommendations to improve the SELFIE WBL tool: - since some learners do not own a computer or smartphone, it would also be good to have a ‘pape r questionnaire’ enabling equal opportunities for al learners; - clearly differentiate the questions for teachers of general subjects and for trainers of vocational/professional subjects. One group cannot answer the questions related to the other group and vice versa; - add an option of providing a descriptive answer, al owing participants to explain their given answer. If a participant disagrees with the statement, there should be an option to justify and clarify their choice. 5.3.3 Questionnaire, content and SELFIE WBL report The SELFIE WBL survey is seen as comprehensive. Learners and trainers envisage that no other topic or question shal be added to the questionnaire. On the other hand, school leaders suggested that questions regarding blended learning should be included in the survey as a n independent a dditional s ection. Participants (around half of learners and trainers) find the SELFIE WBL self-reflection exercise too long. Some of them would rather at least half the number of questions, which would – according to them – make the survey results more meaningful as people would not lose focus during the se lf-reflection exercise. Furthermore, participants had the feeling that the questions were redundant and that they were filling out ‘the same thing twice’. A minority group of learners mention that questions are not always easy to understand and that the y would not be able to complete the questionnaire without explanation from the trainers. Their trainers confirm that questions are often complex, terminology and vocabulary difficult and in parts not appropriate for learners. Each of the six trainers and school leaders think the translation and the vocabulary used were not adapted to the apprenticeship system. A few learners, trainers and school leaders reveal that some statements are not clear enough and it is not always easy to understand to which topic they relate. Hence, participants believe the questions and the questionnaire should be shorter and the vocabulary simple. It was difficult for some school leaders to understand and interpret the SELFIE WBL school report with all the figures and bars. They claim that the averages are not so relevant and do not mean much when presenting the survey results to the col eagues. It takes time to go deeper into each question or stateme nt and real y understand the answers. There are some learners who have the impression that school leaders overestimated the equipment they have at the school (‘or maybe they have it, but learners cannot see it’). A few trainers also believe some averages are overestimated. 5.3.4 Current and future use of SELFIE WBL Learners and trainers have general y not seen the SELFIE WBL school report yet but are very interested to see the results and the actions that wil fol ow. Their main expectation is to use conclusions and findings from the report. For some school leaders, the SELFIE WBL pilot occurred at the right moment as they wanted to write a digital strategy but had no specific tools to do so. Therefore, some school leaders plan to distribute the SELFIE WBL school report to co-workers in order to identify the problems and jointly determine the action plan. Some schools already have digital strategies formalised and written, and a separate digital committee that assesses the strategy in order to continuously improve it. Some schools regularly review their strategies; thus, the SELFIE WBL report wil be taken into consideration when discussing digital transformation. However, school leaders admit the resources al ocated for these improvements are not yet sufficient. On the other hand, the questions and statements in the SELFIE WBL tool made traine rs re flect on the ir practice and subsequently (revealed) their training needs. While they did not expect much at the beginning, 30 they hoped for an evolution in the future. Based on the SELFIE WBL results , school le ade rs anticipate awareness-raising regarding the lack of digital tools for learners and trainers at the school. For some schools, the results highlighted a need to upgrade the skil s of trainers and other e mployees (e.g. contract staff). Therefore, several school leaders state training of trainers (e.g. basic software) and learners (e.g. use of social networks, digital tools and data exchange) as their priority. They are aware that the fol owing steps will not always be easy as some participants wil have to step out of their comfort zones. One school leader has already contacted the project manager regarding digital training, while other school leaders proposed 10 concrete actions that wil be implemented in the short and medium term (e.g. trainer training, use of tools, support for learners, improved internet speed). Another school leader decided to elect three digital delegates (representative of learners, trainers and companies) to identify the needs and report to them directly without an intermediary. Like other school leaders, they believe the lack of dialogue between the school and companies could also be improved thanks to digital technology. Several trainers and school leaders would like to know how they compare to other schools with some sort of ranking. Moreover, they would like to share practical information and good practices with other schools at national and European level, and benefit from advice on important levers as wel as pitfal s and dangers to avoid. The digital transformation of schools and companies is seen as an effective way to better re spond to the specificities of the most vulnerable and can therefore lead to better employability. Nevertheless, many school leaders point out that remote learning can broaden inequalities between learners as they do not have a comparable level of equipment and work environment. In addition, the dropout rate seems to be increasing with the transition to remote learning. In this regard, one school leader provides in-class teaching at least once a week, so that the ‘social link’ makes it possible to avoid dropping out of school. 5.4 Overall findings This chapter presents reflections and main findings from the pilot, gathered from both quantitative and qualitative analyses and the reflections from the participants. School coordinators confirm that school registration process was considered very easy and clear, but they admit they had to go through and read the instructions very careful y. For some, the number of learners and trainers they have to insert upon registration in SELFIE WBL pilot was not clear: 40% (the expected respondent rate) or 100% (the total number of learners and trainers at the VET school) of le arners and trainers. Namely, upon registration some schools entered 40% of learners and trainers which resulted in respondent rates of over 100%. Additional confusion was whether to include 40% of al learners or 4 0% of WBL learners. Furthermore, there was a difference between the recommendations written in the SELFIE WBL tool where it is recommended to reach 20-30% of the learner population, and the recommendations of the SELFIE WBL pilot where the percentage was set at 40%. Inputting the school and company data was also easy to understand. One school coordinator had a problem in that the link was not generated for in-company trainers (for trainers, learners and school leaders it was), so they could not ask them to fil out the SELFIE WBL survey. Other school coordinators did not re port problems when generating links and claimed that it was very easy. Even though they believe that the SELFIE WBL is already very complete, some school coordinators decided to add two questions for the learners and two questions for the trainers in order to focus more on topics important to their school. Customising the surveys (i.e. adding the questions) was easy. However, school coordinators recommend adding the ‘help text’ here as wel , to make sure everybody understands the question. Additionally, schools have problems with the limit of the 3-week timeslot for the SELFIE WBL survey, because learners are out of 31 school at the company for 2 or 3 weeks, so the school cannot reach them. If VET learners are not at school, they do not tend to answer emails and are unreachable. School coordinators report several problems when reaching out to participants and motivating them to fil out the SELFIE WBL survey. The four biggest barriers were holidays, start of the new school year, the terrorist attack in France and the COVID-19 pandemic. Due to the summer holidays, there was not enough time for prior information and promotion of the SELFIE WBL. Some VET schools were closed during last we ek of October as it was the autumn holidays, so they only had 2, weeks to pilot SELFIE WBL. The COVID-19 pandemic brought ordinances with restrictions and a shift in priorities to solve the COVID-19 situation, so the SELFIE WBL pilot was no longer seen as a priority. Considering these circumstances, participants felt under pressure which could have influenced the overal data gathered. Moreover, spending additional time on SELFIE WBL in such circumstances might have led to participants’ demotivation and even potential funding would likely not have helped in motivating them. The trainers and learners saw SELFIE WBL as an additional burde n in difficult times when the focus was on how to start remote learning. Although the trainers were happy to be asked and to give their opinion, they were mostly focused on how to deal with the COVID-19 crisis and remote learning. On the contrary, the SELFIE WBL pilot came ‘at the right time’ for the management of the schools. School leaders were therefore motivated, convinced by the interest in having a state-of-the-art school digital practices, and saw the added-value of the SELFIE WBL and its process. School coordinators organised meetings to explain to learners and trainers the aim of SELFIE WBL and its importance regarding schools’ digital strategy and practices. While learners were very glad and interested in giving their opinion, it was later shown that the explanation was not so clear for everyone. Moreover, it is difficult for school coordinators to reach learners when they are not in the classrooms, as not all le arne rs have (adequate) equipment at home. Due to the lockdown, there was also a problem for school coordinators to include companies. School coordinators did not manage to get in contact with al in-company trainers to ask them to fil out the survey. Additional y, in-company trainers had to be reminded several times to fill out the survey. Monitoring participants was very time-consuming for school coordinators, but it was not difficult to fol ow and monitor the participation rates. Some school coordinators were monitoring the comple tion rate daily and were regularly sending emails to trainers, school leaders and the pedagogical team to encourage the participants. Other school coordinators went to the classrooms to meet the learners and motivate them. Participants find the SELFIE WBL school report in PDF very useful and exhaustive. It opens up new perspectives like the necessity to improve learners’ and trainers’ digital skil s and the need to help and train the trainers who do not feel comfortable enough. The report is furthermore described as clear, instructive, informative and a good starting point for discussion with al stakeholders (learners, trainers, manageme nt team and companies). Participants agree that the SELFIE WBL tool gene rally highlighted strengths and weaknesses and is a relevant basis for schools to introduce their digital strategies. The divergent answers between trainers and school leaders are very informative as wel , even if it is not so easy for school leaders to realise that something is not going as wel as they expected. During the next SELFIE WBL exercise, they will add more questions to get more precise information from respondents. There was also a consensus within schools on the stimulative role of personalised certificates and digital badges. School coordinators do not report problems regarding personalised certificates and open badges. Certificates were downloaded in volumes; al trainers in particular have downloaded their participation certificates according to school coordinators. Some schools have already downloaded open badge and they are going to insert it in their electronic signature. Others have not done it yet but are also planning to insert the badge in their digital signature, on their school website and on their social networks. Participants mostly praised the SELFIE WBL tool as being very useful and would recommend it as a powerful self-reflection tool to al schools that need their digital practices to be state-of-the-art. One ve ry 32 positive aspect of the SELFIE WBL tool is that they can use it again and thus assess their progress. Some school coordinators report that they have to discuss the usefulness of the SELFIE WBL with the companies as wel (they could not do it yet as companies are currently closed due to the COVID-19 situation). As a result of the SELFIE WBL report, some school leaders decided to invest in new equipment and infrastructure; they are planning training modules on basic digital skil s for learners and trainers and beginning to introduce their digital strategy. According to many participants, the chal enge of the SELFIE WBL remains to include a comparison component between schools. There was a question mark around the usefulness of the SELFIE WBL tool for smal schools (e.g. less than 50 learners) that are very specific and have their own specialities. In such schools, discussion and open conversation with learners might be better than the survey. Participants see the SELFIE WBL as an approach quite similar to the quality approach advocated by QUALIOPI10. Schools are audited in order to get the national QUALIOPI certification, which certifies that a school complies with quality processes implemented and thus receives public funds. As such, the SELFIE WBL tool could serve as a basis in discussion with decision-makers who are financing schools, bearing in mind that after al it is a self-reflection tool, not an auditing tool. Moreover, the SELFIE WBL report is very useful for schools as both the SELFIE WBL and QUALIOPI are continuous improvement processes. In that sense, the SELFIE WBL could be the ‘armed wing’ of schools’ digital policy, rather than its redundancy. The SELFIE WBL tool is a good basis for the start of this ecosystem. While SELFIE WBL already provides many things, more is needed. Participants (school leaders, trainers, in -company trainers) would like to integrate SELFIE WBL into other national initiatives and processes that already exist. They would also appreciate direct links with other schools to share practical information, good practic es and create partnerships in technology-enhanced learning. What they would particularly li ke to know is what kind of experiments (in the use of digital technologies) have been implemented, what works and what does not. Based on that, they would ideal y like to start new projects, also with other European partners. 10 QUALIOPI is quality certification for training providers (la Certification des Organismes de Formation). 33 6. Lessons learnt and suggestions for future development The SELFIE WBL pilot strengthens the opinion that it is absolutely necessary to involve trainers and learners in such self-reflection exercises because they have many things to say, ideas to bring forward and can help gain better results. Besides, people are wil ing to give their opinion, participate in writing the strategy and like to feel integrated in the school community. Hence, participants believe that the SELFIE WBL pilot confirms that the more you involve your staff and school community, the more relevant the work done wil be. Schools have problems with the 3-week timeslot for the SELFIE WBL survey, because learners are out of school at the company for 2 or 3 weeks, so the school cannot reach them. VET specifics shal be acknowledged: VET learners are not like high-school students as they come to VET school only around once a month (depending on their study), which is a crucial aspect to take into consideration. Additional y, the COVID-19 pandemic influenced the process because learners were even less present at their VET school than usual. It is very time-consuming to motivate participants each week and it was also stressful because the SELFIE WBL pilot took place during the second lockdown and the priority of the school leaders and the trainers was to organise remote learning and deal with the pandemic. Some learners also suggest that SELFIE WBL should take place later in the school year, rather than at the very beginning. General y, participants had no problems and/or technical issues with SELFIE WBL. It was ve ry practical for them to be able to complete the questionnaire online, either on a smartphone or personal compute r. They mostly stated smooth and easy navigation, the importance of ‘help text’ to better understand the question and the fact that SELFIE WBL offers a good basis for opening up the debate and internal exchanges of opinion regarding the use of digital technology for teaching and learning. The fact that SELFIE WBL also includes companies and considers the opinion of the learners is a positive point, as is its anonymity. Some participants propose the ‘save function’ when fil ing out the survey, having the option to save and return to the previous page without having to re-enter al the answers11. Some learners also reported to the school coordinator that they could not activate the link on their smartphone, so they had to come to the school’s library and use a computer. A school coordinator further mentions a problem of ‘a need to disconnect before leaving the page, otherwise you have to wait 12 to 24 hours to be able to connect again’12. There were also some comments regarding the scale. The current scale from 1 to 5 is not optimal be cause when participants did not know what to answer, they chose 3. They suggest it would be more useful to have a scale from 1 to 4, because it stimulates the participant to take a stand. Participants lack an option to compare with other schools at a national and European level. They would also like an option to add a descriptive answer, al owing participants to comment on their choice (especial y where they disagree with the statement). As the texts in the drop-down menus are not fully visible on mobile devices, the learners propose that the SELFIE WBL tool to adopt non-uniform memory access (NUMA) to enable access to al the data13. The participants find the content of the SELFIE WBL survey exhaustive and complete. The majority would not add any topic or question. The majority also did not have any problem understanding the questions. On the other hand, participants believe the questions and the questionnaire should be shorter and less re petitive. They believe the SELFIE WBL questionnaire is too long at the moment, especial y for learners. The y re port a feeling of many similar questions. Moreover, the questions are sometimes too general and lack details (e.g. 11 It shal be noted that the SELFIE team has long been aware of this issue which is technical y currently not possible to solve and a t th e same time stil safeguard anonymity. 12 Unfortunately, we did not receive more information on the problem. 13 NUMA (non-uniform memory access) is a method of configuring a cluster of microprocessors in a multiprocessing system so that th ey can share memory local y, improving performance and the ability of the system to be expanded. The suggestion was identified when analysing answers in the database, received by the JRC. 34 two school leaders understood the same question differently). Each of the six trainers and school leader think that the translation and the vocabulary used were not adapted to the French national apprenticeship context14. Trainers who teach vocational/professional school subjects would have liked the SELFIE WBL to focus more precisely on their subject, as they felt the SELFIE WBL is a bit too general. Some participants suggest adding new topics, i.e. prevention of digital addiction, social media, dark side of technology, wel -being of learners and online assessment of learners. Participants predominantly believe that the SELFIE WBL school report is a great way to see their strengths and weaknesses and, on that basis, improve their practices. Results are encouraging, informative and can also be surprising for some stakeholders. In some schools, surprising results led to interesting discussion s at col ective meetings where participants could debate the divergence of opinions. For some school le aders, it was not very easy to understand and interpret the SELFIE WBL school report with al the figures and charts. They claim charts without explanations are not always useful. Some participants also state that survey results in the report are general y median (due to the 1-5 scale, see also explanation above), which does not me an much. These ‘average scores’ disable clear decision-making. What is also missing from the report is an option to compare themselves with other VET schools to see where they stand. School coordinators indicate that the certificates were downloaded in volumes. Other than that, participants did not comment on features of SELFIE WBL (badge and certificate) or any possible suggestion for other features. School coordinators state that data were clearer and better understood when discussed during the focus groups. Data comprised no unexpected results; they rather confirmed participants’ thoughts that the state of the art was as they expected. In some schools, results highlighted divergences between school leaders and trainers, and this al owed them to reflect on why the results differed so much. According to participants, SELFIE WBL represents an added value to better identify the directions to take, seek funding and support schools. The survey results are general y considered interesting. For some school leaders the results wil act as a springboard for a (digital) strategy. Others are already discussing the best ways to use the SELFIE WBL school report and are setting up a digital committee, involving trainers and companies (in some cases also learners). One very positive aspect of the SELFIE WBL tool is that it can be use d again an d thus asse ss a school’s progress. But trainers would also like support in concrete examples of uses of tools in other schools. Furthermore, trainers would like to discuss the report with other schools, cre ate partnerships and share practical information with them. 14 The participants were asked to provide concrete details of questions with spel ing errors and outdated vocabulary, but they c o uld not indicate specific items, words or questions. 35 7. Implications of the COVID-19 pandemic Before the COVID-19 crisis, schools general y did not have a common strategy for using technology for teaching and learning. It depended on the trainer’s sensibility and ability to use digital tools, but it was not official y formalised. The situation was heterogeneous15, and even the use of smartphones during the courses for pedagogical purposes was a source of conflict. However, the COVID-19 crisis has been a real accelerator to initiate the digital approach16. The lockdown made people realise that they have to adapt extremely quickly and move forward even faster. During the COVID-19 crisis, some schools could not have learners in the classroom and al courses had to be done remotely. It highlighted the fact that there is a lot of disparity re garding digital practices between trainers and between learners within the same school. Trainers in particular had to adapt themselves to digital tools. The situation and implementation of remote learning was (is) very disturbing for the trainers who have to change their way of teaching. In some schools, a multimedia library played a key role to demonstrate to trainers how to use digital tools, and meetings were organised to share good practices among trainers . Some schools also implemented a training module to give learners basic digital knowledge. They also had hybrid training and projects for teaching and learning, fostered teamwork to improve practices and offered a strong digital service. Other schools are searching for tools and methodologies to share al of these experiences, while some, thanks to the SELFIE WBL report, are starting to write their own digital strategy. Some schools are going to buy computers that wil be lent to learners who do not have any and who cannot afford to buy one17. They are also planning to set up a databank with both free digital resources they can use and specific resources they would like to create and wil be adapted to their own school subjects. At the moment, some schools do approximately 70% remote teaching and are constantly implementing and updating digital resources. They feel remote learning has its limits as participants report more work (overload) because of remote teaching and learning. Because of the COVID-19 crisis, remote teaching and learning will probably remain for some time in the future. There is stil a very important discussion on how to deal with it and how to balance al of these experiences. For some, blended learning wil become a standard, while others would like to take a step back as they were forced to rush into it during the crisis. 15 To some extent it stil is as learners (and trainers) state that there are a lot of different tools, which causes confusion and problems. 16 According to participants, other accelerators were the 2018 vocational training reform in France and the QUALIOPI certification, wh ich support distance learning. 17 More schools report that there is significant economic and social disparity in access to d igital technologies (co mputer e quipment, internet, etc.) as wel as in the handling of the digital tools by learners. 36 8. Conclusions and recommendations The SELFIE WBL pilot came at the right time, as the SELFIE WBL school reports highlight the strengths and areas for improvement. Participants believe the advantage of the SELFIE WBL tool is that it opens up the discussion between stakeholders (also trainers and learners). It was very interesting to see that trainers and learners felt very concerned, are general y very interested to know the SELFIE WBL results and would like to further explore the use of digital technologies. However, they do not want to get involved in the SELFIE WBL tool in the future unless they see a considerable improvement at their school based on these results. Participants are also interested in the SELFIE WBL results from other countries. The majority expressed the ir interest in having a global reflection involving learners, companies and school leaders. More than e ver, they remain interested in comparing the results, exchanging good practices with other schools (and countries), receiving regular updates on each other's practices and proposing learners’ meetings to discuss the use of digital technologies. In that sense, learners could be ‘digital ambassadors’. One of the findings of the SELFIE WBL school report is that trainers need to be better skilled to use digital technology for teaching and learning. On this basis, some schools wil organise trainer training. Despite the urgency felt, the idea is not to go too fast and not to skip steps. Maybe they wil focus on a few issues at the time, as involvement in SELFIE WBL can be seen as a long-term activity including fol ow-up meetings, actions taken based on the results and a SELFIE WBL self-reflection exercise over the years. For now, schools are discovering SELFIE WBL and are learning to use it. Recommendations - SELFIE WBL shal have fewer and shorter questions. - SELFIE WBL shal al ow benchmarking – school leaders and trainers want to compare themse lves with other schools in France and other countries and see where they are positioned. - SELFIE WBL shal enable the possibility of networking with other schools in France (and other countries) to compare practices, methods and tools used, share examples and have regular updates. - SELFIE WBL shal be more adapted to professions, because the differences in studies and practices cannot be highlighted in the current SELFIE WBL (e.g. digital needs are very different for professions such as florist, computer scientist, car mechanic). - Customising the surveys in the SELFIE WBL (adding the questions) is easy, but schools did not realise the possibility of adding a help text to their own questions and statements as well to make sure everybody understands the question. This should be made clearer in the guidelines. - SELFIE WBL shal have an option to also provide a descriptive answer, al owing participants to explain their given answer. If a participant disagrees with a statement, there should be an option to justify and clarify the decision. - There shal be a short guide on the SELFIE WBL tool for learners, to help them understand th e tool before they use it. - The SELFIE WBL certificate/badge could also be integrated into the Europass Digital Credentials (digital file to store in a wal et in the Europass Library). The results of the French pilot are very useful inputs for the SELFIE team in the finalisation of the tool and the questionnaires, which reviews al content and recommendations of all pilots in view of re leasing an enhanced and final version to the public. 37 References Braun, V. & Clarke, V., ‘Using thematic analysis in psychology’, Qualitative Research in Psychology, Vol. 3(2), 2006, pp. 77-101. Available at: http:/ dx.doi.org/10.1191/1478088706qp063oa Broek, S. & Buiskool, B., Adapting the SELFIE tool for work-based learning systems in Vocational Education and Training, Hippe, R. and Kampylis, P. editor(s), EUR 30079 EN, Publications Office of the European Union, Luxembourg, 2020, ISBN 978-92-76-10623-4 (online), doi:10.2760/934724 (online), JRC119707. Centre Inffo, Vocational education and training in Europe – France, Cedefop ReferNet VET in Europe re ports 2018, 2019. Available at: https:/ www.refernet.de/dokumente/pdf/2018_CR_FR.pdf Commission Recommendation of 6 May 2003 concerning the definition of micro, small and medium-sized enterprises, Official Journal, L 124, pp. 36-41, 2003. Available at: https://eur-lex.europa.eu/legal- content/EN/TXT/?uri=CELEX:32003H0361 European Commission, SELFIE database, special extraction for SELFIE WBL national coordinators, 2020. Figgou, L. & Pavlopoulos, V., Social Psychology: Research Methods, In: International Encyclopedia of the Social & Behavioral Sciences (Second Edition), Elsevier, 2015, pp. 544-552. Available at: https:/ www.sciencedirect.com/science/article/pi /B9780080970868240282 French Government, Law of 8 July 2013 for orientation and programming for the refoundation of the school of the Republic, Légifrance, 2013. Available at: https:/ www.legifrance.gouv.fr/loda/id/JORFTEXT000027677984/ French Government, L'école numérique, 2017. Available at: https:/ www.gouvernement.fr/action/l -ecole- numerique French Government, Law of 5 September 2018 for the freedom to choose one's professional future, 20 18, Légifrance. Available at: https:/ www.legifrance.gouv.fr/jorf/id/JORFTEXT000037367660/ Majchrzak, A., (‘Methods for policy research’, Applied social research methods series, Vol. 3, 19 90, Ne wbury Park, CA: SAGE Publications. Ministry of Education, Repères et références statistiques sur les enseignements, la formation et la recherche 2019, Direction de l'évaluation, de la prospective et de la performance, 2019. Available at: https:/ www.education.gouv.fr/reperes-et-references-statistiques-sur-les-enseignements-la-formation- et-la-recherche-2019-3806 Ministry of Labour, Dossier de Presse 04/02/2020 – Lápprentissage en 2019, 2020 . Available at: http:/ hauts-de-france.direccte.gouv.fr/sites/hauts-de- france.direccte.gouv.fr/IMG/pdf/dp_apprentissage_web-2.pdf 38 Ministry of Labour, L’apprentissage, un véritable outil d’insertion dans l’emploi, au service des jeunes et de l’entreprise, 2017. Available at: https:/ travail-emploi.gouv.fr/actualites/l-actualite-du-ministere/article/l- apprentissage-un-veritable-outil-d-insertion-dans-l-emploi-au-service-des Ragin, C. C., Družboslovno raziskovanje: enotnost in raznolikost metode, Ljubljana: Fakulteta za družbene vede, 2007. SAGE Publications, ‘Thematic Analysis of Survey Responses from Undergraduate Students’, SAGE Rese arch Methods Datasets, 2019. Available at: https:/ methods.sagepub.com/base/download/DatasetStudentGuide/thematic -analysis-students- technology Skupnost VSŠ, SELFIE WBL Pilot – Guide and Work Plan for National Coordinators, Version 5, 2020. Available at: https:/ www.skupnost-vss.si/wp-content/uploads/2021/03/Guidelines-and-Work-Plan-for-National- Coordinators_V5.pdf SPIRIT Slovenija, Francija – Gospodarske panoge Francije, 2020. Available at: https:/ www.izvoznookno.si/drzave/francija/gospodarske-panoge/ 39 40 List of abbreviations and definitions ATC Apprenticeship training centre CEDEFOP The European Centre for the Development of Vocational Training CPD Continuing professional development CRCN The Digital Competence Reference Framework CVET Continuing vocational education and training DigComp A framework for developing and understanding Digital Competence in Europe EfVET European Forum of Technical and Vocational Education and Training EQF European Qualifications Framework ETF European Training Foundation ISCED International Standard Classification of Education IVET Initial vocational education and training JRC Joint Research Centre, European Commission N Number of valid responses from the respondents M Mean - the average/central value of the data points or numbers OPCO Opérateur de Compétences SD Standard deviation – a measure of the dispersion of a dataset relative to its mean SEPR Societé dÉnseignement Professionnel e du Rhône Skupnost VSŠ Skupnost višjih strokovnih šol Republike Slovenije / Association of Slovene Higher Vocational Col eges SME Smal and medium-sized enterprises STEM Science, technology, engineering and mathematics TEL Technology-enhanced learning VET Vocational education and training WBL Work-based learning 41 42 List of figures Figure 1. Selection criteria for VET schools 8 Figure 2. The diversity of selected VET schools according to size, location and programme area 9 Figure 3. The diversity of selected VET schools and companies according to geographical coverage 10 Figure 4. Selection criteria for companies 10 Figure 5. Selected companies per selection criteria 11 Figure 6. Translation process 12 Figure 7. Implementation process 13 Figure 8. Percentage of positive responses by area 18 Figure 9. Mean overal score for overal satisfaction with SELFIE WBL per respondent group 19 Figure 10. Increase in number of training centres 38 Figure 11. The trend of enrolment onto apprenticeships at EQF Levels 3 and 4 39 Figure 12. Increase in apprenticeships across all EQF levels 39 Figure 13. French vocational education and training 40 Figure 14. Distribution of economic sectors in France 43 Figure 15. Overview of area snapshot from an anonymous SELFIE WBL school report 56 Figure 16. Mean score for al variables in main areas per respondent group 58 Figure 17. Mean likelihood of further recommending SELFIE 60 Figure 18. Negative factors for technology use in school and company – percentage per re spondent group 60 Figure 19. Negative factors for technology use for remote teaching, learning and training – percentage per respondent group 61 Figure 20. Positive factors for remote teaching, learning and training – percentage per respondent group 62 43 44 List of tables Table 1. Descriptive statistics for main areas per respondent group 19 Table 2. The number of learners, trainers, school leaders, in-company trainers and school coordinators involved in the qualitative analysis 21 Table 3. Thematic analysis of open questions responded to by learners 54 Table 4. Overal satisfaction with SELFIE – percentage distribution per respondent group 58 Table 5. Relevance of questions per respondent group 59 Table 6. Likelihood of further recommendation of the SELFIE tool –- percentage per respondent group. 59 45 46 Annexes Annex 1. Key information on the WBL system Annex 2. Dominant economic sectors in France Annex 3. Guidelines and templates for focus groups, semi-structured interviews and list of chal enges Annex 4. Analysis of open question ‘Suggestions for improvement’ and examples of questions Annex 5. School report ‘Overview by area’ Annex 6. Figures and tables for the SELFIE WBL piloting quantitative analysis Annex 7. Overview of SELFIE WBL results in France Annex 8. Country fiche Annex 9. List of tools similar to SELFIE and other tools used in WBL 47 Annex 1. Key information on the WBL system WORK-BASED LEARNING IN FRANCE In France, apprenticeship training centres were created in 1971; 2 laws gave a legal and a financial framework to apprenticeships to make it a ‘modern training pathway’. Since 1971, many important re forms have been implemented. In 1983, apprenticeships were transferred from national/state level to regional level. In 1987, a law stipulated that al qualification levels (from basic level to unive rsity level) are eligible for apprenticeships. The latest key reform took place in 2018 (French Government, 2018), with two main goals: - developing a training supply in apprenticeships; - making apprenticeships more attractive for learners, families and companies and showing this training pathway as a reliable and excel ent one. This was a substantial reform as: - new legal and financial rules were set up. The regional authorities are no longer re sponsible for financing apprenticeships. A new financial and governance system has been set up and new actors have emerged such as Opérateur de Compétences (OPCO)18; - new opportunities were introduced for companies that lack a qualified workforce. Now they have the right to create their own apprenticeship training centre, without asking for any specific and previous authorisations on educational or administrative matters; - apprenticeships are growing as people can sign an apprenticeship contract between the ages of 16 and 29 (vs 16 and 25 before 2018); - the value of apprenticeships is reasserted through: o increase in apprentices’ wages; o financial support for apprentices older than 18 who want to take their driving test; o financial support for companies with less than 250 employees that sign an apprenticeship contract with a learner to prepare for a diploma for EQF levels up to 4. Figure 10. Increase in number of training centres Source: Ministry of Labour (2020). Figure 10 displays the increase in training centres before the reform in 2018 (with a total of 965 training centres and on 31 December of 2019 a total of 1 200 training centres). 18 OPCO = Opérateur de Compétences whose mission is to finance the apprenticeship and support the professional bra nches to set u p their curricula and support SMEs defining their required training. More information at https://www.francecompetences.fr/la-formation- professionnel e/qui-fait-quoi/. 48 Apprenticeships strongly progress in France. Since 1990, the number of apprentices has more than doubled in France. In February 2020, there were 491 000 apprentices in France vs 220 000 in 1991. Figure 11 re flects the apprenticeship trend over the last 7 years and highlights the increase in apprenticeships at EQF Le vel 3 and Level 4. After the decline observed in 2013 (-9.3%) and 2014 (-6.8%), the number of learners who took the apprenticeship path and signed an apprenticeship contract rapidly increased in 2019 (+8% in comparison to 2018). Figure 11. The trend of enrolment onto apprenticeships at EQF Levels 3 and 4 Source: Ministry of Labour (2020). Figure 12 shows the increase of the apprenticeship pathway at all EQF le vels displaying a double-digit increase rate (+16%) in 2019. In 2019 there were 491 000 apprentices in France, and 368 000 new apprenticeship contracts were signed. This trend benefits al the regions with very high rates in some of them (+78.3% in French Guiana, +55.8% in Guadeloupe, +27.3% in Corsica and +23.8% in Auvergne-Rhône-Alpe s) and economic sectors (+3% in trades and crafts, + 13% in construction industry, + 11% in metal industry). Figure 12. Increase in apprenticeships across all EQF levels 49 Source: Ministry of Labour (2020). The apprentices split their learning time between the company and the apprenticeship training centre. The apprenticeship training centre is freely chosen by the company and the apprentice. Most of the time is spent in the company. During the time at the apprenticeship training centre, the apprentice s are considered as employees of the company and are paid for their apprenticeship. Vocational education and training consist of two elements, which are relatively independent of one another: - initial vocational education and training (IVET) for young people within a school context (full -time education) or under an employment contract (apprenticeship); - continuing vocational education and training (CVET) for young people who have left or completed initial education and for adults on the labour market. Among the factors that distinguish IVET and CVET are the difference between the certifier, the sources of funding and even the objectives. IVET includes various levels of training from lower secondary up to the higher level. There are more than 600 vocational diplomas in France that are managed by the National Board of Education. They are classified from EQF Level 3 to EQF Level 5. Vocational qualifications are developed and awarded by: - Ministries o the Ministry of National Education, Youth and Sport o the Ministry of Higher Education, Research and Innovation o the Ministry of Labour - Professional unions - Chamber of Trades and Crafts and Chamber of Commerce. Figure 13. French vocational education and training. 50 Source: Centre Inffo (2019). Work-based learning refers to knowledge and skil s acquired through carrying out – and reflecting on – tasks in a vocational context, either at the workplace or in a VET institution. In France, there are two different pathways for work-based learning, namely: - ful -time study in a vocational school with placements in a company (from 4 to 10 weeks, depending on the qualification level). There are public and private vocational schools, they both grant diplomas from the State; - apprenticeship at an apprenticeship training centre (ATC). Most ATCs are private and their legal status is that of a non-profit organisation, but the National Board of Education created some pu blic ATCs that are integrated into public training centres. In any case, the same diplomas are awarded at the end of the training. The only difference is thetraining method and the time spent at school and at the company. The recognition of the value of apprenticeships in France has increased over the years, currently being se en as a way to gain work experience and excel ence. 51 The apprenticeships programmes and curricula are set by the certifier, that can be: - for the most part the Ministry of Education, Youth and Sport - Ministry of Labour - Chamber of Trades and Crafts - Chamber of Commerce - professional unions. An apprenticeship is a very efficient way to enter the labour market with lasting effect: 70% of appre ntices have found a job 7 months after the apprenticeship contract ended, and 60% have a permane nt contract. Companies confirm that apprenticeships are a great opportunity (80% are satisfied with their apprentices and they hire a new one at the end of the previous contract) (Ministry of Labour, 2017). DIGITALISATION STRATEGY FOR VET AND APPRENTICESHIPS IN FRANCE The topic of digitalisation of education fal s under the responsibility of the French Digital Plan for Education established in 2015. It is responsible for the implementation and deployment of the digital services in the education sector. This is part of a bigger initiative by the French Government related to the digitalisation of the public services in recognition of the need to keep up with the latest technological trends contributing to the economic development of the country. The digital strategy for the education sector (LÉcole Numerique) was official y launched in 2015, after an extensive consultation carried out by the Ministry of Education in articulation with other ministries at it aimed at bringing schools into the digital age. Its implementation relies on a coordinated mobilisation and action of different stakeholders (schools, teachers, apprentices, managers, ministries and local, regional and other national authorities, universities, research centres, industry) to put in place the optimal conditions for efficient development and deployment of resources, including access to training. The ambition is to cre ate an e - education eco-system to enable proper use of resources and the effective development of skil s and services. Renewed in 2018, the strategy is organised around 5 main pil ars, namely: - place school data at the heart of the ministry's digital strategy; - teach digital in the 21st century; - support and strengthen the professional development of teachers; - develop apprentices' digital skil s; - create new links with the school's actors and partners. The strategy envisions the continued and renewed support to the establishment of digital (educational) workplaces across the national territory, ensuring that: - schools have the resources (equipment and materials) to provide digital se rvices to expand and enrich the educational offer and to customise the support to apprentices; - teachers have access to diversified educational resources that can be used on a daily basis, as we l as initial and continued training and tools al owing them to monitor their apprentices and communicate with families; - apprentices with special needs, and in particular apprentices with disabilities, are offered customised solutions that wil al ow them to benefit from education and educational services in the exact same way as other apprentices. The French Government approved the deployment of 1 bil ion euros for the period of 2016-2019 to support the different initiatives and project development related to the digitalisation of education. Since 2016, several 52 different initiatives and projects have been launched by the Ministry of Education. A very brief overview of the diversity landscape of these projects is shared as fol ows: ● equipping schools. Huge investment was made by natio nal, lo cal and reg io nal au thorities to support schools with digital equipment and services (individual materials for apprentice s and teachers such as cabling, internet connections, tablets, digital workspaces, access to digital resources and training) including support to schools on how to use equipment and resources; ● digital platforms. Integrated set of digital services made availab le to th e e du catio nal s ch ool community. It constitutes a unified entry point al owing users to access (according to their profile and level of authorisation) digital content. It is also a platform for exchange and col aboration between users (from the same school or within schools). This platform offers services such as digital textbooks, common workspaces and storage for apprentices and teachers, access to digital resources, col aborative tools, blogs, forum, virtual classroom, etc.; support to school management – notes, absences, timetables, agendas, etc.; and communication, messaging, staff and family information, videoconferencing, etc. These digital workplaces can be accessed by apprentices, parents, te acher s and administrative staff. This initiative is also related to one of the strategy goals for simplifying administrative formalities and facilitating the communication with the broader educational community; ● training resources for teachers and trainers. Several online platforms h ave b een de ve lo ped with different purposes, such as providing digital training to teachers, managers, trainers on different areas; sharing resources that can be used on a daily basis; disseminating information on e xisting practices and research carried out at national level on the topic of digitalisation. Some of these platforms are ful y dedicated to the topic of inclusiveness. ● The Digital Competence Reference Framework (CRCN) – the digital skil s reference framework applicable to al EQF levels, inspired by the European Framework (DigComp) and launched at the start of the 2019 school year. ● Data protection and safety – appointment in August 2018 of a data protection officer for the Ministry of National Education and for the Ministry of Higher Education, Research and Innovation to ensure compliance with the European guidelines and support the educational community in understanding how personal data should be col ected, processed and stored. In addition, an e thics committee on digital data was set up to advise on and support issues related to th e use of data col ected and processed in the school context. In this context, training targeting school’s management and teachers related to the challenges of using digital data were also developed and part of the portfolio of online courses available to teachers on the different platforms. Specifical y focused on the VET sector and acknowledging that, curre ntly, the attractiveness of training centres largely depends on the ability to build a quality digital educational offer adapted to the challenge s and needs of the labour market, a new digital space was create d within the e xisting ONISEP platform dedicated to the trends in terms of professional paths. In addition, and to unde rstand the changes that automation and digitalisation wil have in current professions, different research studies were conducted and wil be made available to training centres in the e-Fran projects platform. As mentioned earlier, the approach to al of the above initiatives was to ensure the engageme nt of all key educational stakeholders, of which local authorities. The establishment of a public e -educational service required the review of the existing governance approach, ensuring concertation and sharing between al levels of public authorities. To achieve this, a ‘committee of digital partners for education’ was set up aiming at structuring and systematising exchanges between al representatives, al owing them to be involved in the definition of a global and shared strategy to deploy digital education within the territories. The partners' committee contributes to the national digital education strategy at several levels: cal s for projects, reposi tories and guides, forward thinking. Annex 2. Dominant economic sectors in France 53 Gross domestic product (GDP) structure Figure 14. Distribution of economic sectors in France Source: SPIRIT Slovenija (2020). Agricultural sector France is the largest agricultural power in the European Union and accounts for a quarter of its total agricultural production. Nevertheless, the agricultural sector accounts for only 2% of the country's GDP and employs 2.6% of the population. The French agricultural sector re ceives significant subsidies from the European Union. The main agricultural products are wheat, corn, meat and wine. Service sector The service sector accounts for 79% of the GDP and employs 77.3% of the workforce. The most important segment in the service sector is tourism. Industrial sector The industrial sector accounts for 19% of the GDP and employs 20.1% of the active workforce. The Fre nch manufacturing industry is very diverse. The country is currently in the process of de-industrialisation, which has led to the outsourcing of many activities. The key industries in France are telecommunications, electronics and the automotive, aerospace and military industries. In 2019, industrial production grew by 0.4%. For 2020, analysts predict a decline in industrial production by -13%, fol owe d by 9.2% growth in 2021 and 2.3% growth in industrial production in 2022. Source: SPIRIT Slovenija. 2020. 54 Annex 3. Guidelines and templates for focus groups, semi-structured interviews and a list of challenges Focus group guidelines Objective The main objective of the focus groups is to spend some time with each of the two key target groups for the SELFIE WBL project – learners and trainers – and discuss the ‘how’ and ‘why’ behind the main questions and answers in the survey. We want participants to elaborate further on the key questions in the survey (SELFIE WBL tool pilot) and explore participants’ views about the tool, the main chal enges they faced in using the SELFIE tool and whether it helps them assess where they stand with regards to learning in the digital age. We want them to speak freely and not be swayed by pre-conceived notions they may have about what are deemed desirable answers as there are no wrong answers. Moderators The focus group for teaching staff should be moderated by a peer trainer and the focus group for le arne rs should be moderated by a tutor to create a comfortable and trust ing atmosphere which e nables ope n reflection and discussion. We advise that a note-taker is also assigned to each moderator to e nable fluent moderation. Participants Each VET school organises two focus groups: one exclusively with trainers as participants and the other with learners. Diversity in terms of a school’s size shal be taken into account. The only pre-condition to become a participant is that they have taken part in the SELFIE WBL pilot survey. The optimal size of each focus group is 10 participants which al ows al members to participate, and enables the moderator, i.e. institutional coordinator or learners’ tutor, time to be able to tease out the nuances behind participants’ answers. For online focus groups where plenary discussions/interactions are less straightforward, a slightly smaller number of participants (minimum of 5) is acceptable to ensure there is an opportunity for al participants to have their say, remain engaged and reduce strain on the moderator. Duration Typical y, a focus group lasts between 60–90 minutes. This gives enough time to al ow for deeper conversations to take place but does not run for too long, which can lead to participant fatigue. In the case of online focus groups, it is advisable to keep the session time to a maximum of 60 minutes as it is just that little bit harder for people to stay focused. Moderation The focus group wil need to be wel moderated in order to guide the discussion, using a combination of questions and further probes. The participants should be encouraged to interact with each other as wel as to generate deeper insights about the different sub-topics. With an online focus group, it is probably not possible to get the same type of feedback or interplay between participants as with face-to-face focus groups, so the role of the moderator is even more important here. The moderator wil provide an overview of the project and its purpose, ask questions, fol ow up with more questions and keep the conversation on track and on subject. Make sure to keep it relaxed, that participants are at ease and feel comfortabl e and safe in opening and sharing their thoughts. Reminding participants that there are no right or wrong answers is a good way to 55 make sure they are not self-censoring. Make sure that the moderator also takes enough time for introductions and for participants to become comfortable in the session to ensure individuals engage with one another. Normal y, al discussions can take place in a normal plenary form, but if the moderator feels the need for it, they might use smal exercises like brainstorm activities in which the participants write down ideas on (virtual) post-it notes, plotting these post-it notes in a matrix or map to prioritise items, or simply keeping track of inspiration and solutions that come up during the session in a visual way. Topics/questions Based on experience with similar focus groups, we should have time to address three to four different topics with open-ended questions, fol ow-up questions and, in particular, discussion between participants. The topics that we would suggest are as fol ows. The strengths and weaknesses of the SELFIE WBL tool Questions to the participants can include: − what works particularly wel in the SELFIE tool? What does not? − what would you see as the most important chal enges for optimal functioning of SELFIE tool? Discussion should be encouraged comparing different situations, shared experiences regarding strengths and weaknesses, concrete tips & tricks on how to make improvements. Discussion on relevant survey results Participants shal reflect and discuss their interpretation and in-depth understanding of the re levant survey results, for example going into different elements of the SELFIE tool (e.g. Le adership, Infrastructure and Equipment, Teaching and Learning, etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey if it is optimal or more out of necessity and if there are intentions to become either more or less involved in the SELFIE tool in the future. Areas where further support is needed/useful Questions to the participants can include: − what are the areas of the SELFIE tool where more information, knowledge, guidance, training , e tc. would be welcomed by you and/or col eagues in similar roles? − what potential changes do you anticipate based on the survey results? Again, discussion should be encouraged comparing different situations, experiences and visions. Equipment/facilities Chairs set up in a circular pattern around a table is the most ideal setup for a focus group as you want al the participants to be able to see each other easily. In case of an online focus group, a Zoom room can be set up by the Research Team (contact us19 at least 1 week prior to the event providing an exact date and timeslot). The amount of information that is shared in focus groups is not easily captured by a note-taker, as there are numerous side conversations that occur. The best way to scrutinise data at a later date is to audio and video record the focus group sessions. Please do not forget to get consent from the participants to be recorded and let them know that their responses wil remain anonymous and no names wil be mentioned in the report. 19 Research Team contacts: miha.zimšek@skupnost-vss.si and/or alicia.miklavcic@skupnost-vss.si. 56 Focus group report Date Country School Moderator(s) Participant Name and surname Trainer/Learners Subject/Programme 1 2 3 4 5 6 7 8 9 10 57 Discussion topics - Discussion 1 – Icebreakers - Discussion 2 – The strengths and weaknesses of the SELFIE WBL tool - Discussion 3 – Discussion on relevant survey results - Discussion 4 – Areas where further support is needed/useful Topic 1 – Icebreakers Suggestions for discussion Questions to the participants can include: - what were your expectations of Selfie WBL? - do you think your expectations were met? Common responses / general consensus Areas of disagreement / lack of consensus Other notes & observations Topic 2 – The strengths and weaknesses of the SELFIE WBL tool Suggestions for discussion Questions to the participants can include: − what works particularly wel in the SELFIE WBL tool? What does not? − what would you see as most the important chal enges for an optimal functioning SELFIE WBL tool? Discussion should be encouraged comparing different situations, shared experiences regarding stre ngths and weaknesses, concrete tips & tricks on how to make improvements. Common responses / general consensus Areas of disagreement / lack of consensus Other notes & observations Topic 3 – Discussion on relevant survey results Suggestions for discussion Participants shal reflect and discuss their interpretation and in-depth understanding of the relevant survey results, for example going into different elements of the SELFIE WBL tool (e.g. Leadership, Infrastructure and Equipment, Teaching and Learning etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey, if it is optimal or more out of necessity and if there are intentions to become either more or le ss involved in 58 the SELFIE WBL tool in the future. Common responses / general consensus Areas of disagreement / lack of consensus Other notes & observations Topic 4 – Areas where further support is needed/useful Suggestions for discussion Questions to the participants can include: − what are the areas of the SELFIE WBL tool where more information, knowledge, guidance, training etc. would be welcomed by you and/or col eagues in similar roles? − what potential changes do you anticipate based on the survey results? − what kind of technology are you using when you are working in the company? (state specific examples about the use of technology in company and in school) − did you start with digital learning because of COVID-19? − what problems did you face because of COVID-19? − did you include blended learning? − did you perform apprenticeships during lockdown (remote mode / distance mode)? − wil you use SELFIE WBL in the future? Again, discussion should be encouraged comparing different situations, experiences and visions. Common responses / general consensus Areas of disagreement / lack of consensus Other notes & observations Additional topics/discussions/ideas/observations (Only if the content does not fal into any previous categories/topics above) Notes & observations 59 In-depth semi-structured interview guidelines Objective In-depth, semi-structured interviews intend to elaborate further on the report results and foreseen improvements based on those results. The interviews are verbal interchanges where the national coordinator attempts to elicit information from 4 in-company trainers and decision-making staff in a VET school by asking questions. Even though the national coordinator prepares a list of predetermined questions, in-depth, semi-structured interviews usual y unfold in a conversational manner, offering participants the chance to pursue issues they feel are important. In-depth interviews are conducted in order to gain a thorough insight into a particular issue, in our case future improvements. Interviews are conducted individual y and focused on each organisation separately. Interviewer The interview shal be carried out by a national coordinator. People wil talk more when they feel more relaxed and at ease, so the questions are not asked in any given order, rather they are asked in a way that develops the conversation. Interviewee In-depth semi-structured interviews are carried out with 4 in-company trainers and decision-making staff in a VET school (4 pedagogical managers/directors, 4 sector heads/managers, 4 board heads/directors). The pre-condition to become an interviewee is that they have taken part in the SELFIE WBL pilot survey. Duration Typical y, a semi-structured interview lasts between 30–60 minutes. This gives e nough time to allow for deeper conversations to take place but does not run too long which can lead to interviewee fatigue. Before the interview When recruiting interviewees, indicate that you would be happy to conduct the interview at a time and place which best suits them. Do not forget to remind the interviewee of the time, date and location of the interview (online). Before the interview commences, the national coordinator should ask the interviewee if they consent to the interview being digital y recorded. Informed consent can be confirmed by the interviewer reading the consent form and the interviewee verbal y indicating that they agree. During the interview You need to listen careful y to what the interviewee is saying, for their response might not actually answer the question. Alternatively, the interviewee may give you a vague response, to which you might have to ask for clarification or further explanation. The most important thing to remember when conducting an interview is not to rush through the questioning. Additional y, do not interrupt participants when they are in the middle of a sentence or when they stop in order to col ect their thoughts. ‘Could you tel me’ is always a good way of starting an interview or asking an interviewee to explain a particular point of view. 60 Do not disclose the details or discuss the comments of another interviewee during an interview. This not only breaches past interviewees’ confidential y, but the present interviewee wil doubt your ability to maintain their confidence. This is not to say that you cannot talk in generalities (e.g. if an interviewee asks you ‘what have other people said’ in relation to particular point, you could say ‘wel , a lot of interviewees have indicated that’ etc.). Have your notepad and pen ready because sometimes interviewees can say the most insightful things when the digital recorder has been switched off. After the interview It is extremely important that you write the report immediately after the interview, whilst you can stil vividly remember al of the aspects of the interview. The recorded audio of the interview should help you to prepare an accurate report. Use your experience from each interview to improve the next interview. Topics/questions A semi-structured in-depth interview is usual y one in which the interviewer has a checklist of topic areas or questions. The topics that we would suggest are the fol owing. - Icebreakers Questions to the interviewees can include: − what were your expectations of the participation in the survey? − do you think your expectations were met? - discussion on relevant survey results. Interviewees shal reflect and discuss their interpretation and in-depth understanding of the relevant survey results, for example, going into different elements of the SELFIE tool (e.g. Le adership, Infrastructure and Equipment, Teaching and Learning, etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey if it is optimal or more out of necessity and if there are intentions to become either more or less involved in the SELFIE tool in the future and/or use its results. - Future improvements After interviewees discuss pilot results, they should consider implementing proposed solutions. This means that they (plan to) improve the process/WBL and continue to look for ways to make it even better for their organisation. Questions to the interviewees can include: − what would be your potential reactions based on the survey results? − is there an action plan to support the implementation of the proposed solutions? − how wil you prioritise your reactions to the results? Wil resources (e.g. financial capacity, etc.) play a role in the prioritisation process? Equipment/facilities 61 In case of an online interview, a Zoom room can be set up by the Research Team (contact us20 at least 1 week prior to the event providing exact date and timeslot). 20 Research Team contacts: miha.zimšek@skupnost-vss.si and/or alicia.miklavcic@skupnost-vss.si. 62 In-depth semi-structured interviews report Date Country School Facilitator(s) Interviewee Discussion Topics - Discussion 1 – Icebreakers - Discussion 2 – Discussion on relevant survey results - Discussion 3 – Areas where further support is needed/useful Topic 1 – Icebreakers Suggestions for discussion Questions to the interviewees can include: - what were your expectations of the participation in the survey? - do you think your expectations were met? Common responses / general consensus Areas of disagreement / lack of consensus Notes & observations Topic 2 – Discussion on relevant survey results Suggestions for discussion − What kind of technology are you using when you are working in the company? (state specific examples about the use of technology in company and in school) − Did you start with digital learning because of COVID-19? − What problems did you face because of COVID-19? − Did you include blended learning? − Did you perform apprenticeships during the lockdown (remote mode / distance mode)? − Wil you use SELFIE WBL in the future? − What are the things you liked about SELFIE WBL? What could be improved? Interviewees shal reflect and discuss their interpretation and in-depth understanding of the re levant survey results, for example going into different elements of the SELFIE tool (e.g. Leadership, Infrastructure and Equipment, Teaching and Learning, etc.). Further fol ow-up questions can be asked about the reasons why they took part in the SELFIE survey, if it is optimal or more out of necessity and if there are intentions to become either more or le ss involved in 63 the SELFIE tool in the future and/or use its results. Common responses / general consensus Areas of disagreement / lack of consensus Notes & observations Topic 3 – Future improvements Suggestions for discussion Questions to the participants can include: what would be your potential reactions based on the survey results? − is there an action plan to support the implementation of the proposed solutions? − how wil you prioritise your reactions to the results? Wil resources (e.g., financial, capacity, etc.) play a role in prioritisation process? Again, discussion should be encouraged comparing different situations, experiences and visions. Common responses / general consensus Areas of disagreement / lack of consensus Notes & observations Additional topics/discussions/ideas/observations (Fil in only if the content does not fal into any previous categories/topics above) Notes & observations 64 List of challenges The fol owing tables are to be fil ed in by the corresponding participants in the pilot process from the beginning of their engagement until 15 November 2020. They wil serve the research team in identifying advantages of and positive reflections on SELFIE WBL but, foremost, identify chal enges and possibilities of improvement. School Coordinator/Leadership Country: School: Process Advantages Challenges School registration process Supporting materials and info Input of school data Customising survey Motivating participants - Learners - Trainers - Leaders - Companies Generating links Survey content Survey technical issues Monitoring participation - Learners - Trainers - Leaders - Companies SELFIE WBL Report - Usefulness - Features lacking Reaching objectives (40% of learners and 40% of trainers) Certificates / Digital badges - Participants - School Findings (unexpected issues) Lessons learnt 65 How COVID-19 was affected /experienced with blended learning, COVID-19 impact description of the profile of school, remote teaching and learning Other Add rows, as necessary. Source: Skupnost VSŠ, 2020. 66 Annex 4. Analysis of open question ‘ Suggestions for improvement’ and examples of questions Thematic analyses, defined as a method for identifying, analysing and reporting patterns (themes) within data (Braun and Clarke, 2006) was used for analysing an open-ended question on ‘Suggestions for improvement’ provided by learners. Description of the process We read al answers from learners to open question: ‘How can we improve SELFIE further? Share your ideas and suggestions with us.’ We have become familiarised with the data and prepared a list of key issues/themes and codes. Text answers of learners was tabulated, and each answer was classified in themes (code). Then we counted the number of answers with the same code and prepared the Table 3. Categories/themes S – about SELFIE TOOL (satisfaction, critics, missing themes, items to add) Q – opinion about questions (length, repeating, complicated) A – opinion about answers (number of answers, option of others…) L – language (terminology, understandable, more languages) D – devices – problems with using different devices for SELFIE T – timing of involvement I – design W – internet connection DT – digital technology P – praises 0 – nothing to change K – critics N - prefer not to answer C – linked with COVID-19 X – not classified. Table 3. Thematic analysis of open question responded to by learners. Cod Key words and answers summary Frequenc e y S SELFIE (too many questions, make less questions/issues, too long, provide short tutorial for 71 learners, adapt to professions, add explanation and examples) Provide training before use Q Questions (improve the question sentences / the issue, too long, clarify, simplify, some 112 unnecessary, too similar, better connection to school profile, expand the questions, add open questions, open-ended questions, more detailed, precise, focused questions, easily to understand), Add: ‘about availability of our trainers’, ‘if we were fol owing the course / exercises online for 67 our learning’ A Answers – add more answers, adoption other, option to write own answer, add box for 16 comment, even scale, add more options about profession, not 5-level scale, not numbers, add 0 L Language (vocabulary, do not write in English) 2 D Devices (we do not see al the proposals on the mobile version in the drop- down me nus ; 7 transform website for mobile application, adopt for NUMA) T Timing 0 I Design (new layout, make course in video, better brightness, add pictures, improve visual) 10 W Wi-Fi (bad) 2 Dt Digital technology (we use little DT in school/practice, equip the school, digital training, bad 11 quality of computer) P Praises (good, very good, perfect, main there is, wel asked, efficient, fair) 13 0 No ideas, no proposals/suggestions, nothing, nothing to add, I don't know, nothing to change 90 K Critiques (useless, boring, stop it) 12 N Do not wish to answer 2 C Linked to COVID-19 0 X Not sorted – not understandable, poor translation 61 Source: Own analysis. Examples of questions considered repetitive In our school, I have access to the internet for learning In my company, I have access to the internet for learning In our school, there are computers or tablets for me to use In my company, I can learn about operating the relevant (digital) equipment In our school, I use technology in different subjects In our school, we use technology for projects that combine different subjects Examples of questions considered too long and complex In our school, I have access to a database of companies providing traineeships, apprenticeships and other opportunities In our school, trainers give us different activities to do using technology that suits our needs In our company, in-company trainers use digital technologies to tailor the training to our individual needs In our company, I gain experience in using digital technologies, which makes me more prepared for my future profession 68 In our school, we talk with trainers about the advantages and disadvantages of using technology for learning In our school, I use technology to understand my strengths and weaknesses as a learner In our company, I use digital technology to understand my strengths and weaknesses as a learner In our school, I use technology to keep a record of what I have learned relevant to my field of study 69 Annex 5. School report ‘Overview of areas’ Figure 15. Overview of areas snapshot from an anonymous SELFIE WBL school report 70 71 Source: Anonymous SELFIE WBL school report (2020). 72 Annex 6. Figures and tables with results of SELFIE WBL piloting quantitative data Figure 16 displays average values per respondent group for al variables. The mean on a 5-point Likert scale (1-5) was the highest for school leaders (M=3.3), and equal for trainers, learners and in -company trainers (M=3.1). Figure 16. Mean score for all variables in main areas per respondent group Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Table 4 displays the percentage of answers concerning the overal satisfaction with SELFIE WBL on 10-level scale per respondent group and means for satisfaction with SELFIE WBL per respondent group. The percentage of scores above the middle of the scale is the highest in the group of school leaders (90.6%) a nd the lowest in the group of learners (56.4%). The highest satisfaction is in the group of school leaders (M=7.5) and the lowest, stil above the middle of the 10-level scale, is in the group of learners (M=5.8). Table 4. Overall satisfaction with SELFIE – percentage distribution per respondent group Overall School In-company leaders Trainers Learners Total satisfaction with trainers N=252 N=2 939 SELFIE N=53 N=16 N=3 260 1 0.0% 2.0% 8.6% 6.3% 7.9% 2 0.0% 4.0% 2.9% 0.0% 2.9% 3 0.0% 4.0% 5.5% 12.5% 5.3% 4 0.0% 4.0% 7.1% 6.3% 6.7% 5 9.4% 18.7% 19.6% 6.3% 19.3% 6 11.3% 15.5% 12.0% 12.5% 12.2% 73 7 24.5% 23.0% 19.5% 18.8% 19.9% 8 34.0% 21.0% 15.2% 6.3% 15.9% 9 17.0% 4.8% 3.6% 25.0% 4.0% 10 3.8% 3.2% 6.1% 6.3% 5.8% Summary 1-5 9.4% 32.7% 43.7% 31.4% 42.1% Summary 6-10 90.6% 67.5% 56.4% 68.9% 57.8% Mean 7.5 6.2 5.8 6.4 5.9 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Learners and in-company trainers were asked about their opinion of the questions included in the SELFIE WBL self-reflection exercise (Table 5). They rated the relevance of questions on a 10-level scale. Learners provided 50.8% of responses in the range of 6-10 (M=5.4), and in-company trainers in 6 2.5% of re sponses in the range of 6-10 (M=6.1). Table 5. Relevance of questions per respondent group Learners N=2 578 In-company trainers N=16 Score Frequency Percent Frequency Percent 1 260 10.1% 0 0.0% 2 147 5.7% 1 6.3% 3 179 6.9% 1 6.3% 4 243 9.4% 2 12.5% 5 440 17.1% 2 12.5% 6 363 14.1% 2 12.5% 7 401 15.6% 4 25.0% 8 325 12.6% 3 18.8% 9 91 3.5% 0 0.0% 10 129 5.0% 1 6.3% Summary 1-5 1 269 49.2% 6 37.5% Summary 6-10 1 309 50.8% 10 62.5% Mean 5.4 6.1 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 74 Table 6 presents the percentage of answers concerning the likelihood of further recommendation of SELFIE WBL per respondent group on a 5-level scale. The highest percentage of positive responses (‘Very likely’ and ‘Extremely likely’) is in the group of school leaders (84.9%). In the group of trainers, the share of positive responses is 28.3% and in the group of in-company trainers it is 29.4%. There are 23.5% negative responses of ‘Not at al likely’ and ‘Not very likely’) in the group of in-company trainers. The percentage of answer ‘Prefer not to say’ is the highest among in-company trainers (14.9%). The average likelihood for further recommendation of the SELFIE WBL self-reflection exercise is the highest for school leaders (M=4.1) and the lowest for trainers (M=2.9). Table 6. Likelihood of further recommendation of SELFIE tool – percentage per respondent group. School Trainers In-company Total Recommending SELFIE leaders trainers N=17 N=332 N=262 N=53 Not at al likely 0.0% 3.4% 5.9% 3.0% Not very likely 3.8% 18.7% 17.6% 16.3% Somewhat likely 7.5% 17.9% 11.8% 16.0% Very likely 49.1% 31.7% 35.3% 34.6% Extremely likely 35.8% 13.4% 17.6% 17.2% Prefer not to say 3.8% 14.9% 11.8% 13.0% Mean 4.1 2.9 3.1 3.1 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 75 Figure 17 displays the likelihood of further recommendation of the SELFIE WBL self -reflection e xercise. Means in al groups are above the middle of the 5-level scale. School leaders have the highest mean (3.7) and trainers the lowest (3.2). Figure 17. Mean likelihood of further recommending SELFIE Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Figure 18 displays shares of factors which negatively affect digital technology use in schools and companies. There is disagreement between respondent groups. School leaders chose ‘Low digital competence of teachers’most frequently (21.1%). Trainers think that ‘Insufficient digital equipment’ is the main affective factor (18.4%) and in-company trainers chose ‘Lack of time for trainers’ most frequently (30.8%). Figure 18. Negative factors for technology use in school and company – percentage per respondent group 76 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. Figure 19 displays shares of factors which negatively affect remote teaching, learning or training. There was agreement between school leaders and trainers that the most affective factor is ‘Limited student access to digital devices’. Figure 19. Negative factors for technology use for remote teaching, learning and training – percentage per respondent group 77 Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 78 Figure 20 displays shares of factors which negatively affect remote teaching, learning or training. Results show that school leaders and trainers share the opinion that the most influential positive factor for re mote teaching and learning and training with digital technology is ‘Teachers col aborate within the school/company on digital technology use and creation of resources’ (school leaders 16.1%, trainers 17.5%). In -company trainers chose ‘Trainers’ participation in professional networks’ most frequently (17.1%). Figure 20. P ositive factors for remote teaching, learning and training - percentage per respondent group Source: European Commission (2020). SELFIE database, special extraction for SELFIE WBL national coordinators. 79 80 Annex 7. Overview of SELFIE WBL results in France The outcomes of the pilot are not representative of the national education and training systems. They provide useful insights for schools and companies participating in the pilot and, overal , for schools and companies providing similar WBL programmes and belonging to the specific economic sectors covered by the pilot. Details on al questions can be found in the questionnaires on the SELFIE tool website. User participation 81 Note: The six participation categories were answered by school coordinators during school registration. The categories for ‘disadvantaged homes’ and ‘different language’ are: less than 10%, 10-25%, 26-50%, above 50%, I don’t know. ‘Didn’t answer’ is also possible, as the questions were optional. 82 SELFIE WBL - Main areas Note: positive responses = answers on 4 or 5 on a five-point scale 83 84 85 SELFIE WBL – Additional areas Note: Positive responses = answers on 4 or 5 on a five-point scale 86 87 88 89 90 91 Satisfaction Note: Satisfaction with SELFIE WBL, on a scale from 1 to 10. 92 Likelihood of recommending SELFIE Note: On a scale from 1 to .5 93 Annex 8. Country fiche 94 95 Annex 9. List of tools similar to SELFIE and other tools used in WBL The goal was to map out existing self-reflection tools and other existing digital tools in the country and schools used in WBL contexts. This mapping and listing shal include official and available websites from governmental institutions responsible for overseeing WBL in the country and with different stakeholders engaged in the pilot. Name of WBL tool Link Aim Advantages SELFIE is a free, online SELFIE al ows a school https:/ ec.europa.eu/educ tool to help schools to monitor its progress SELFIE WBL ation/schools-go- assess how they use over time and can he lp digital_en digital technologies for start a dialogue within innovative and effective the school on pote ntial learning. areas for improvement. PIX is the online public service for assessing, The tool supports developing and assessment in 5 are as, certifying digital skil s including 16 skil s at 6 and basic digital levels. Areas covered – knowledge and is information and data, PIX https:/ pix.fr/ included in the French communication and Digital Competence col aboration, content Reference Framework creation, protection and (CRCN= Cadre de security, digital référence des environment. compétences numériques). 96 GETTING IN TOUCH WITH THE EU In person Al over the European Union there are hundreds of Europe Direct information centres. You can find the address of the centre nearest you at: https:/ europa.eu/european-union/contact_en On the phone or by email Europe Direct is a service that answers your questions about the European Union. You can contact this service: - by freephone: 00 800 6 7 8 9 10 11 (certain operators may charge for these cal s), - at the fol owing standard number: +32 22999696, or - by electronic mail via: https://europa.eu/european-union/contact_en FINDING INFORMATION ABOUT THE EU Online Information about the European Union in al the official languages of the EU is available on the Europa website at: https://europa.eu/european-union/index_en EU publications You can download or order free and priced EU publications from EU Bookshop at: https:/ publications.europa.eu/en/publications. Multiple copies of free publications may be obtained by contacting Europe Direct or your local information centre (see https://europa.eu/european-union/contact_en). KJ-NA -30933 -EN -N Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID 89685763 ISBN 978-92-76-45988-0 (PDF) doi:10.2760/424070 IS BN 978-92-76-45988-0