Scientific article Andragoška spoznanja/Studies in Adult Education and Learning, 2022, 28(1), 43-55 DOI: https://doi.org/10.4312/as/10547 Sabina Ličen, Igor Karnjuš, Mirko Prosen AN EVALUATION STUDY OF NURSE EDUCATORS’ LEARNING EXPERIENCE IN A DIGITAL MODULAR COURSE ABSTRACT The Covid-19 pandemic poses a new challenge in the adoption of ICT in education. The need for ed- ucational institutions to develop standards and implement quality assurance in digital education has suddenly gained new significance. The aim of this study is to evaluate the learning experiences of nurse educators in Slovenia acquired during a nine-week modular online course on the design, delivery and evaluation of online study units, based on a quality standard for digital education developed for this pur- pose. A mixed methods approach was used. The evaluation of the nurse educators’ learning experiences was examined quantitatively (cross-sectional study) and qualitatively (focus group). The Cronbach’s alpha for the entire questionnaire was 0.921, indicating that the internal consistency of the questionnaire developed for evaluation was excellent. The thematic analysis identified three main themes. At the end of the course, the nursing teachers expressed their desire for additional courses in which they could gain and deepen their knowledge of the preparation, implementation, and evaluation of the educational unit with the help of digital technology. Keywords: e-learning, online learning, nursing teachers, quality, higher education EVALVACIJA IZKUŠENJ VISOKOŠOLSKIH UČITELJEV ZDRAVSTVENE NEGE, PRIDOBLJENIH NA DIGITALNEM MODULARNEM TEČAJU – POVZETEK Pandemija covida-19 je prinesla nov izziv za uvajanje IKT v izobraževanje. Hkrati se je pokazala tudi potreba izobraževalnih ustanov po razvijanju standardov in na podlagi teh zagotavljanju kakovosti pri načrtovanju uporabe digitalne tehnologije. Namen raziskave je bil evalvirati izkušnje visokošolskih učiteljev zdravstvene nege v Sloveniji, pridobljene na devettedenskem modularnem spletnem tečaju o oblikovanju, izvajanju in vrednotenju spletnih učnih enot, ki je bil pripravljen na podlagi standarda Sabina Ličen, PhD, Assoc. Prof., Department of Nursing, Faculty of Health Sciences, University of Primorska, sabina.licen@fvz.upr.si Igor Karnjuš, PhD, Asst. Prof., Department of Nursing, Faculty of Health Sciences, University of Primorska, igor.karnjus@fvz.upr.si Mirko Prosen, PhD, Assoc. Prof., Department of Nursing, Faculty of Health Sciences, University of Primorska, mirko.prosen@fvz.upr.si AS_2022_1_FINAL.indd 43 13. 04. 2022 09:00:04 44 ANDRAGOŠKA SPOZNANJA/STUDIES IN ADULT EDUCATION AND LEARNING 1/2022 kakovosti za digitalno izobraževanje. Uporabljen je bil pristop mešanih metod dela. Evalvacija izkušenj učiteljev zdravstvene nege je potekala s kvantitativno (presečna raziskava) in kvalitativno (fokusna sku- pina) metodo. Cronbachov koeficient alfa za celoten vprašalnik je bil 0,921, kar nakazuje na odlično no- tranjo skladnost vprašalnika, razvitega za namen evalvacije. Na podlagi tematske analize smo opredelili tri teme. Ob zaključku tečaja so učitelji zdravstvene nege izrazili željo po dodatnih tečajih, na katerih bi pridobili in poglobili znanje o pripravi, izvedbi in evalvaciji izobraževalne enote s pomočjo digitalne tehnologije. Ključne besede: e-izobraževanje, spletno učenje, učitelji zdravstvene nege, kakovost, visokošolsko izobraževanje INTRODUCTION In March 2020, over one billion learners were affected by school or university closures, with Covid-19 cases being confirmed worldwide. Eighty percent of learners were ex- cluded from their educational institutions when physical campuses closed due to social distancing measures (UNESCO, 2020b). Moreover, UNESCO (2020a) reports that higher education closures have affected over 91 percent of students worldwide and that in the 2021 academic year, 23.8 million students were at risk of abandoning their studies or not gaining admission to schools. Digitally mediated teaching and learning have gradually been demanded and implemented, especially in the context of higher education. To alle- viate the education crisis, schools around the world have adopted online teaching methods to preserve educational opportunities and protect the health and lives of students. Background Digital education has allowed educational institutions to continue functioning during the global pandemic (Yeo et al., 2021). Many years of slow progress in digital education have recently experienced a sudden shift. At the outbreak of the pandemic, proponents of on- line learning, who had invested considerable time and energy in convincing stakeholders, quality standards committees, funders, and academic staff of the benefits of digital educa- tion and the need to develop new approaches to teaching and learning in order to meet the needs of 21st century learners, immediately gained global support (Barbour, 2021). On- line designers, developers, teachers, and researchers were sought out, and evidence-based benefits of online education were touted as equal to or better than face-to-face learning (Zalat et al., 2021). Academics with previous digital experience managed to quickly em- brace e-learning and took advantage of its flexibility, accessibility, and adaptability in transferring their course content from face-to-face classes to online platforms (Coman et al., 2020). Today, teachers and academics are well aware of the fact that quality, rigour and high levels of student engagement and satisfaction can be achieved in well-designed online environments (Alqurashi, 2019). Because of Covid-19, online learning abruptly shift- ed from being a convenient option to becoming a necessity. Educators who were less AS_2022_1_FINAL.indd 44 13. 04. 2022 09:00:04 45S. Ličen, I. Karnjuš, M. Prosen: An Evaluation Study of Nurse Educators’ Learning Experience... technologically savvy found themselves in a stressful situation, having to convert face-to- face learning sessions into online learning sessions without sufficient knowledge on how to design or deliver digital content. However, despite some initial reservations, online learning came to the rescue of educational institutions overnight with an unprecedented adoption of online educational solutions (Rapanta et al., 2020). In some cases, online learning has been seen as emergency remote teaching rather than effective e-learning due to a certain sense of desperation or lack of appropriate knowledge, skills and training. Emergency remote teaching refers to a teaching modality carried out partially or entirely online due to circumstances of crisis (such as natural disasters, wars, or health emergen- cies) (Slapac et al., 2021). This includes the use of distance-only teaching methods to replace established face-to-face teaching methods, as well as hybrid or combined teach- ing methods for the duration of the emergency situation (Ličen, 2021; Misirli & Ergulec, 2021). Either way, since the outbreak of the pandemic, many educational institutions have been forced to review and change their existing policies and procedures in need of clear and comprehensive long-term strategies for implementing digital education. Suddenly, compelled by overwhelming pressure on higher education institutions to provide appro- priate infrastructure which would enable quality digital education experiences through a targeted development of interactive pedagogical tools, effective online student engage- ment tools, and training for their academic staff (Rapanta et al., 2020), a new teaching paradigm began to emerge. The emergence of this new teaching paradigm means that all educational institutions need to develop a focused implementation strategy and revise their quality assurance protocols, extending them to include digital education methods and ensuring that they focus on appropriate inputs, processes and outcomes (European Commission, 2020). One of the possible solutions, which is in line with the reforms in higher education that focus on providing flexible approaches to education and training, could be an interac- tive framework which includes the best practices from existing frameworks and models in the literature from around the world, together with the experiences of digital educa- tion researchers from different countries, namely the Digital Education Quality Standard Framework (MacDonald, Backhaus, et al., 2021). This framework includes definitions of variables and sub-variables to facilitate context and understanding and has been proposed as a quality standard to guide the design and delivery of effective online learning (Mac- Donald, Backhaus, et al., 2021). The aim of the study is to use the Digital Education Quality Standard Framework to as- sess the learning experiences of nurse educators in Slovenia acquired in a digital modular course. METHODS A mixed methods approach was used. Quantitative and qualitative methods were used to assess the nurse educators’ learning experiences. We adopted a convergent parallel design AS_2022_1_FINAL.indd 45 13. 04. 2022 09:00:04 46 ANDRAGOŠKA SPOZNANJA/STUDIES IN ADULT EDUCATION AND LEARNING 1/2022 as proposed by Creswell and Plano Clark (2017), in which quantitative and qualitative data are collected simultaneously and both approaches are prioritised. The results of both analyses are then compared and/or combined into a unified whole. This type of research design supports triangulation, in which the researcher attempts to balance the weaknesses of the qualitative and quantitative approaches by taking advantage of the strengths of both. Quantitative data were collected using an online survey and qualitative data were collected using the focus group method. Settings Based on the Digital Education Quality Standard Framework (MacDonald, Ličen, et al., 2021), a nine-week modular online course was offered to nurse educators in Slovenia. The nine-week training series “Designing, Delivering and Evaluating Online Study-Units” for nurse educators was divided into practical topics: Introduction; Content; Delivery; Support, Structure, Community, and Outcomes. One topic was covered each week except for Delivery, which was covered in two weeks. The purpose of the modular online course was to provide learners with the basic knowledge and skills needed to design, deliver and evaluate an e-learning session. After completing the course, participants completed a questionnaire to evaluate their learning experience. The questionnaire was divided into two parts. The first part consisted of a total of 57 items divided into six sections (content, delivery, support, structure, community, and outcomes) based on the modular course do- mains (MacDonald, Ličen, et al., 2021). Participants rated these items using a Likert scale with 5 alternatives for each item, ranging from Strongly Disagree (1) to Strongly Agree (5). The final section of the survey included the participants’ demographic data: gender, age, education level, and length of service. Participants The modular online course, which was based on the Digital Education Quality Stand- ard Framework (MacDonald, Ličen, et al., 2021), was only offered to a limited number of nurse educators. Therefore, the purposive sample consisted of 24 nurse educators from tertiary educational institutions in Slovenia who had registered for the course. The sample consisted of 2 men (8.3%) and 22 women (91.7%). The age of respondents ranged from 31 to 61 years (x̄ = 43.50, SD = 8.113) and their total years of employment in higher education ranged from 1 to 40 years (x̄ = 14.63, SD = 11.294). They indicated that they had never participated in e-learning courses before, but described their com- puter skills as mostly average, despite having incorporated computer technologies into their teaching/work activities (e.g., they reported that they had used the virtual learning environment or Moodle to share their course readings and organise discussion groups, etc.) (Table 1). AS_2022_1_FINAL.indd 46 13. 04. 2022 09:00:04 47S. Ličen, I. Karnjuš, M. Prosen: An Evaluation Study of Nurse Educators’ Learning Experience... Table 1 The level of computer literacy and daily use of computer technologies in the classroom/work setting Which of the following best describes your level of computer literacy: n % Novice - I have attempted to use computer technologies, but I still need help on a regular basis. 1 4.2 Beginner - I am able to perform basic functions in a limited number of computer applications. 2 8.3 Average - I have general skills in a number of computer applications. 9 37.5 Advanced - I have the ability to competently use a wide range of computer technologies. 7 29.2 Expert - I am highly proficient in using a wide range of computer technologies. 5 20.8 How often do you use computer technologies in teaching/work: n % Rarely 3 12.5 Occasionally 5 20.8 Frequently 4 16.7 Almost always 6 25.0 All the time 6 25.0 Quantitative data collection and analysis After completing the digital modular course, the participants received an email invitation to complete an online survey (1ka.si). The questionnaire contained information about the aim of the study and their rights as participants. The study was conducted in compliance with the Declaration of Helsinki – Ethical Principles for Medical Research. The partici- pants had the opportunity to complete the questionnaire by the end of May 2021. During data collection, a database of the participants’ responses was created, excluding their surnames, first names and email addresses so as to ensure anonymity. The data were then exported to the IBM Statistical Package for the Social Sciences (SPSS) Version 26 for macOS for analysis. The quantitative data were analysed using descriptive statistics and linear regression. A p-value ≤ 0.05 was considered significant. Qualitative data collection and analysis In the second part of the study, prior to the focus group interviews, all 24 participants were emailed an informed consent form detailing the aims of the study, the method of data collection, and the measures taken to ensure anonymity and confidentiality. The participants were given the opportunity to select several possible dates for the focus group session. Based on the dates selected by the participants, two focus groups were formed, which is considered adequate to generate sufficient evidence (Guest et al., 2017). The focus groups, which took place in June 2021, were conducted via Zoom© and audio-recorded. AS_2022_1_FINAL.indd 47 13. 04. 2022 09:00:04 48 ANDRAGOŠKA SPOZNANJA/STUDIES IN ADULT EDUCATION AND LEARNING 1/2022 Data were collected in a semi-structured interview. The topic guide used in the focus groups consisted of thematic sections related to the content, delivery, and support provid- ed during the course, the structure of the course and its outcomes, based on the Digital Education Quality Standard Framework (MacDonald, Ličen, et al., 2021). Both focus groups lasted approximately one hour and were transcribed verbatim. During the prepa- ration, organisation and reporting phases of the study, several considerations were taken into account regarding credibility and trustworthiness according to the trustworthiness checklist proposed by Elo et al. (2014). The qualitative analysis was conducted separately by two authors who then merged their reflective notes and individual findings to produce the final results. These results were then triangulated with the quantitative results and the results of the literature review. This analysis was conducted using the qualitative data analysis software NVivo ver. 12 (QRS International). Mixed method approach Data analysis and integration in the convergent design consists of merging the qualitative and quantitative results. This is done in three phases: (1) analysis of the qualitative data by coding the data and grouping the codes into general themes; (2) statistical analysis of the quantitative data; (3) a mixed methods data analysis (Creswell & Creswell, 2018). According to Creswell and Creswell (2018), there are several ways to do this. After care- ful consideration and elaboration, we opted for the “side-by-side” comparison where re- searchers make the comparison within the discussion after first reporting the quantitative results and then the qualitative results which either confirm or refute the quantitative results. During the analysis, much attention was devoted to the importance of integrating both perspectives. RESULTS Quantitative findings The ratings of the digital modular course by the 24 participating nurse educators (x̄ = 4.42, SD = 0.419 [95% CI 4.24, 4.60], p = 0.000) indicate that the learning experience was perceived as very good. The Cronbach’s alpha for the entire questionnaire was 0.921, indicating that the internal consistency of the questionnaire developed for evaluation pur- poses was adequate. Table 2 shows the descriptive statistics of respondents based on the questionnaire domains. The results show that the participants rated the Outcomes, Structures, and Community domains highest (x̄ = 4.63, 4.42, and 4.42, respectively) and the Support domain lowest (x̄ = 4.28), but still very high. Moreover, the highest rated items were those stating that as a result of their participation in the modular online course, they were now able to or- ganise the course content for an effective online learning session (x̄ = 4.75; SD = 0.442), establish a discussion forum to facilitate learning and reflection (x̄ = 4.74; SD = 0.449), and create learning activities to effectively deliver online content and engage learners (x̄ = 4.71; SD = 0.464). Even the lowest rated items, such as those stating that the amount AS_2022_1_FINAL.indd 48 13. 04. 2022 09:00:04 49S. Ličen, I. Karnjuš, M. Prosen: An Evaluation Study of Nurse Educators’ Learning Experience... of time allocated for this modular online course was adequate (x̄ = 3.87; SD = 1.191), that the modular online course included (provided) useful and regular feedback from other learners (x̄ = 4.00; SD = 1.103), that the modular online course included (provided) collab- orative learning opportunities (x̄ = 4.08; SD = 0.830) and that learning occurred through discussion, reflection, collaboration and by taking initiative and responsibility to listen, question and think critically within the community of fellow learners (x̄ = 4.08; SD = 0.925), still scored very high. Further, we assumed that participants with longer tenure would perceive the course more favourably and therefore rate it higher. Therefore, a linear regression analysis was con- ducted to estimate the relationships between the participants’ overall length of service and their course ratings. Although the results show that those with low seniority rated the modular course higher, no significant regression equation was found (p > 0.05). Qualitative findings Using a descriptive approach to qualitative analysis (Doyle et al., 2020), the codes were reviewed and grouped into three overarching themes concerning the participants’ experi- ences of attending a nine-week modular online course, namely (1) perceptions of content structure and delivery, (2) perceived support and interaction, and (3) impact on profes- sional development. Perception of the content, structure, and delivery The participants reported that they were pleased with the content of the modular course and found it helpful in learning how to design, deliver and evaluate online lectures. All participants described their experience as very positive and very well organised with relevant resources and examples of best practice guiding them through the course. One of the participants stated: “The course content was very interesting; my experience was very positive, and I think this was a very well-organised course.” The course was Table 2 Digital Education Quality Standard Framework assessment questionnaire (n = 24) - descriptive statistics Domains N of items x̄ 95% CI p Cronbach α Lower Upper Content 13 4.41 4.21 4.61 0.000 0.896 Delivery 9 4.30 4.07 4.52 0.000 0.883 Support 6 4.28 4.01 4.56 0.000 0.867 Structure 11 4.42 4.21 4.63 0.000 0.899 Community 8 4.42 4.21 4.62 0.000 0.861 Outcomes 10 4.63 4.43 4.83 0.000 0.925 Note. Participants rated the questionnaire on a 5-point Likert scale ranging from 5 - Strongly Agree to 1 - Strongly Disagree; x̄ = Mean Value; SD = Standard Deviation. AS_2022_1_FINAL.indd 49 13. 04. 2022 09:00:04 50 ANDRAGOŠKA SPOZNANJA/STUDIES IN ADULT EDUCATION AND LEARNING 1/2022 perceived by the participants as interactive and engaging, and they felt that it gave them enough autonomy to follow their own learning needs and further polish their teaching style. Furthermore, it was recognised as balanced in terms of learning activities within each module: The number of assignments was reasonable and the tasks were interactive. I do not feel that anything else could have been added to the content or that anything was left out. The course was well-balanced and gave me enough space to follow my own teaching needs. At the beginning I was worried that we would be overwhelmed because of COVID, but in the end it was manageable. It followed a progres- sive approach that I liked and included several tips on how to keep me further motivated. Even though some participants had already been familiar with some of the e-tools used in the modular course, the course provided them with a new perspective and approach to using these tools in online teaching: […] I was familiar with the e-tools I used in preparing my online lecture, how- ever, I had never used them before, because I had only given face-to-face lec- tures. For me, this was an excellent opportunity to explore them to their full extent. I really enjoy learning new things, and am trying to improve my skills and competencies. Perceived support and interaction The participants found the level of support and timely and constructive feedback highly motivational. During the modular course, technical and media support was available. They also received support and feedback on their assignments from the course instruc- tors, which was perceived as timely, focused, constructive and motivating. Some participants commented thus: I think the feedback was very friendly and very supportive and constructive. The course instructors provided us with timely feedback after our tasks. […] [O]verall, the feedback I received was well-balanced and appropriate. […] [T]he instructors encouraged us and kept us motivated throughout the course. I think the instructors did their job perfectly, especially in terms of letting us work on our own assignments based on our practice. […] I felt that this was a highly autonomous way of learning. […] Their support motivated me to finish the assignments. AS_2022_1_FINAL.indd 50 13. 04. 2022 09:00:04 51S. Ličen, I. Karnjuš, M. Prosen: An Evaluation Study of Nurse Educators’ Learning Experience... Despite the mostly positive feedback from the participants, some of them missed face-to- face interaction between the participants and the course instructors: “I was very pleased with the recorded voice introduction at the beginning of each module, but I missed human interaction between fellow participants. I think more live interaction would have made a big difference.” The impact on professional development Overall, the participants felt that the course met their needs and expectations and would help them improve their pedagogical work as teachers. For example, one of the partici- pants remarked: I joined the course because I wanted to improve my teaching methods and tech- niques [...]. I felt that I needed to obtain more knowledge about e-learning, especially because we were forced to adopt e-learning due to the COVID situa- tion. I think that my expectations have clearly been met, and I feel confident in using this knowledge and skills in preparing my online lectures. It was also evident that the course contributed to the participants’ critical analysis of their own teaching (style, methods, and practice), as well as empowered them to explore e-learning approaches on their own and share this knowledge with their colleagues who did not attend the course. Furthermore, the course was recognised as an important step in their personal and professional development: The course gave me the opportunity to reflect on my own teaching practice and helped me improve my teaching approach. It opened up a whole new dimension of teaching for me. Learning new pedagogical methods and techniques of e-learning has given me the confidence to prepare my own e-courses and empowered me to share this knowledge with my colleagues as well. The modular course was a trigger for me to transfer my usual face-to-face teaching to online teaching. By the end of the course, I was able to do this independently. For me, this was also an excellent opportunity in terms of my personal and professional development. DISCUSSION Education is often perceived as a sector that resists change while struggling through a productivity and efficiency crisis (OECD, 2016). The transition from mostly traditional or teacher-centred pedagogy to learner-centred pedagogy is a long and slow process that depends on a number of factors. With this in mind, our aim was to evaluate the learning experiences of nurse educators in Slovenia acquired through a digital modular course based on a quality standard for digital education. In addition, we also aimed to fill the gap AS_2022_1_FINAL.indd 51 13. 04. 2022 09:00:04 52 ANDRAGOŠKA SPOZNANJA/STUDIES IN ADULT EDUCATION AND LEARNING 1/2022 in the literature, which has focused mainly on the quantitative perspective and included few studies dealing with the evaluation e-learning experiences using other methodologi- cal approaches (Frazer et al., 2017). The data from our questionnaire supports the data from our focus group interviews relat- ed to the course content, structure and delivery. The results show that participants liked the course, that the course was relevant, useful and helpful, and that they improved their learning skills in online teaching. They found the content of the modular course practical and the learning outcomes very clear. They felt that the course included relevant readings and practical tips. The participants were very satisfied with the course content, even those with previous experience in online teaching. However, there are a few things which need to be modified in the future. Some participants found the course very demanding, espe- cially during the COVID crisis, as they were heavily burdened with workload associated with the transition from face-to-face to online teaching. Nevertheless, they rated individ- ual items very positively. Nursing education today is in a constant state of change when it comes to new learning and teaching methods. The number of online nursing education programmes is rapidly increasing and faculty must keep up with the needs, desires, and demands of new genera- tions of students, while also taking into account the standards of professional regulation, as nursing is one of the most regulated professions in the European Union. Therefore, one of the greatest challenges for educational institutions is to integrate innovative e-learning methods which will strengthen and support both teaching and learning (Coman et al., 2020). To date, many studies have investigated online learning. However, few have ex- amined the learning effects before and after online learning (Kim et al., 2021). As nurse educators need to be competent in their professional roles and possess the necessary skills to positively influence student learning outcomes (Frazer et al., 2017), it is necessary to provide them with adequate training to support them in this process. At the same time, effective teaching strategies have been shown to enhance student learning, satisfaction, and achievement of outcomes (Authement & Dormire, 2020; Frazer et al., 2017). Recent studies have identified several variables which may intervene in the process of online education, such as the proper adaptation of teaching to the online format, including the correct delivery of course content (Baltà-Salvador et al., 2021). Although participants scored high on the “Community” domain in the quantitative study, the qualitative study revealed that some participants missed face-to-face interaction, es- pecially as this modular course was their first more comprehensive e-learning experi- ence. The learning environment can undoubtedly have an impact on the effectiveness of e-learning, especially in terms of internal factors such as learning motivation, personal preferences, etc. (Wang et al., 2021). However, immersion in e-learning and the acqui- sition of its principles is an ongoing process of connection, adaptation, guidance and feedback that needs to take place between the instructor and the student through an online platform, and this takes time (Frazer et al., 2017), especially if it is one’s first experience with e-learning. Future design of such modular courses should take this into account. AS_2022_1_FINAL.indd 52 13. 04. 2022 09:00:04 53S. Ličen, I. Karnjuš, M. Prosen: An Evaluation Study of Nurse Educators’ Learning Experience... Attention should also be paid to the implementation of different e-learning strategies which would support different types of interaction, in particular in the context of nursing (e.g., the patient-nurse relationship) in terms of the necessary competencies to be ac- quired. It is also important to note that other studies (Zalat et al., 2021) conducted among healthcare or medical students have reached similar conclusions regarding the lack of face-to-face interaction. This study confirms that the modular course promoted participant engagement and, in conjunction with ICT, stimulated professional and, in some cases, personal development. The course increased nurse educators’ confidence and enabled them to use the acquired knowledge and skills in their own teaching and adapt it to the learning needs of their students. This also contributes to the development of competencies required for effective online teaching in the future. Key competencies include communication skills, techno- logical literacy, provision of informative feedback, administrative skills, responsiveness, learning monitoring and provision of support (Roddy et al., 2017). Overall, e-learning provides teachers with the opportunity to take control of their teaching and learning pro- cess, deepen their understanding of the content, and improve their pedagogical skills, especially their skills in using technology in teaching and learning (Ahmad et al., 2014). The study has certain limitations which need to be taken into account. The sample of the quantitative study is not representative of all teachers in higher education, but is relevant to nursing teachers in Slovenia and therefore allows for certain conclusions to be drawn that explain teachers’ experiences with e-learning. In addition, the teachers’ prior experi- ence of using ICT varied, which could have had an impact on their perceived experience of the modular course. Similarly, their learning experience might have been influenced by the Covid-19 pandemic and the overnight shift to online teaching, which had forced many to become familiar with e-learning before enrolling in the modular course. Another relat- ed limitation is the socio-cultural context of the study and the level of adoption of e-learn- ing in higher education, which must also be considered when interpreting the results. Two focus groups were used in the qualitative study, which is still considered acceptable (Guest et al., 2017). However, we would suggest forming at least three focus groups in the future to stimulate greater group dynamics and to glean different perspectives. Future studies should therefore include a more representative sample in the quantitative part and explore the perspectives highlighted in the present study (the impact of different types of interaction during e-learning, teachers’ competencies for e-learning, etc.). It is also evident that research in this field must be interdisciplinary. CONCLUSION The Covid-19 pandemic has forced teachers worldwide to shift from physical classrooms to online learning environments, thus drastically changing the way we teach and learn. This study used a systematic and structured modular approach to demonstrate the impor- tance of preparing teachers for e-learning and encouraging them to adopt this “new”’ way AS_2022_1_FINAL.indd 53 13. 04. 2022 09:00:04 54 ANDRAGOŠKA SPOZNANJA/STUDIES IN ADULT EDUCATION AND LEARNING 1/2022 of teaching in higher education. The Digital Education Quality Standard Framework and its questionnaire also proved to be very useful for the development, implementation, and evaluation of e-learning. Funding The research was supported by the Erasmus+ Programme of the European Union “Digital Education Initiatives and Timely Solutions” (grant number 2019-1-MT01-KA203-051171). REFERENCES Ahmad, I., Said, H., Khan, F., & Al-Balushi, F. (2014). Effect of e-learning on professional development of university teachers. Research Journal of Recent Sciences, 3(7), 1–4. Alqurashi, E. (2019). Predicting student satisfaction and perceived learning within online learning en- vironments. Distance Education, 40(1), 133–148. https://doi.org/10.1080/01587919.2018.1553562 Authement, R. S., & Dormire, S. L. (2020). Introduction to the Online Nursing Education Best Practices Guide. SAGE Open Nursing, 6, 2377960820937290. https://doi.org/10.1177/2377960820937290 Baltà-Salvador, R., Olmedo-Torre, N., Peña, M., & Renta-Davids, A.-I. (2021). Academic and emotional effects of online learning during the COVID-19 pandemic on engineering students. Education and Information Technologies, 26(6), 7407–7434. https://doi.org/10.1007/s10639-021-10593-1 Barbour, M. K. (2021). The shift to distance learning: Tracing the roots of 100+ years of practice and opportunity. TechTrends, 65(6), 919–922. https://doi.org/10.1007/s11528-021-00670-0 Coman, C., Țîru, L. G., Meseșan-Schmitz, L., Stanciu, C., & Bularca, M. C. (2020). Online teaching and learning in higher education during the coronavirus pandemic: Students’ perspective. Sustainability, 12(24), 10367. https://doi.org/10.3390/su122410367 Creswell, J. W., & Creswell, J. D. (2018). Research design: Qualitative, quantitative, and mixed methods approaches (5th ed.). SAGE Publications. Creswell, J. W., & Plano Clark, V. L. (2017). Designing and conducting mixed methods research (3rd ed.). SAGE Publications. Doyle, L., McCabe, C., Keogh, B., Brady, A., & McCann, M. (2020). An overview of the qualitative descriptive design within nursing research. Journal of Research in Nursing, 25(5), 443–455. https:// doi.org/10.1177/1744987119880234 Elo, S., Kääriäinen, M., Kanste, O., Pölkki, T., Utriainen, K., & Kyngäs, H. (2014). Qualitative con- tent analysis: A focus on trustworthiness. SAGE Open, 4(1), 2158244014522633. https://doi. org/10.1177/2158244014522633 European Commission. (2020). Digital education action plan (2021-2027). https://ec.europa.eu/ education/education-in-the-eu/digital-education-action-plan_en Frazer, C., Sullivan, D. H., Weatherspoon, D., & Hussey, L. (2017). Faculty perceptions of online teach- ing effectiveness and indicators of quality. Nursing Research and Practice, 2017, 9374189. https:// doi.org/10.1155/2017/9374189 Guest, G., Namey, E., & McKenna, K. (2017). How many focus groups are enough? Building an evidence base for nonprobability sample sizes. Field Methods, 29(1), 3–22. https://doi. org/10.1177/1525822X16639015 Kim, S.-Y., Kim, S.-J., & Lee, S.-H. (2021). Effects of online learning on nursing students in South Ko- rea during COVID-19. International Journal of Environmental Research and Public Health, 18(16), 8506. https://doi.org/10.3390/ijerph18168506 AS_2022_1_FINAL.indd 54 13. 04. 2022 09:00:04 55S. Ličen, I. Karnjuš, M. Prosen: An Evaluation Study of Nurse Educators’ Learning Experience... Ličen, S. (2021). Nursing education in the time of COVID-19: What has it taught us? Obzornik Zdravst- vene Nege, 55(1), 4–6. https://doi.org/10.14528/snr.2021.55.1.3080 MacDonald, C. J., Backhaus, I., Vanezi, E., Yeratziotis, A., Clendinneng, D., Seriola, L., Häkkinen, S., Cassar, M., Mettouris, C., & Papadopoulos, G. A. (2021). European Union digital education quality standard framework and companion evaluation toolkit. Open Learning: The Journal of Open, Dis- tance and e-Learning. https://doi.org/10.1080/02680513.2021.1936476 MacDonald, C. J., Ličen, S., Clendinneng, D., Prosen, M., Filomeno, L., Mifsud, N., James, C., Back- haus, I., Cassar, M., Karnjuš, I., & Giuseppe, L. T. (2021). Program evaluation of a continuing edu- cation course to support academic staff transitioning to online teaching. IJSPE, 3(2), 54–80. Misirli, O., & Ergulec, F. (2021). Emergency remote teaching during the COVID-19 pandemic: Parents experiences and perspectives. Education and Information Technologies, 26(6), 6699–6718. https:// doi.org/10.1007/s10639-021-10520-4 OECD. (2016). Innovating Education and Educating for Innovation: The Power of Digital Technol- ogies and Skills. https://www.oecd.org/education/innovating-education-and-educating-for-innova- tion-9789264265097-en.htm Rapanta, C., Botturi, L., Goodyear, P., Guàrdia, L., & Koole, M. (2020). Online university teaching during and after the covid-19 crisis: Refocusing teacher presence and learning activity. Postdigital Science and Education, 2(3), 923–945. https://doi.org/10.1007/s42438-020-00155-y Roddy, C., Amiet, D. L., Chung, J., Holt, C., Shaw, L., McKenzie, S., Garivaldis, F., Lodge, J. M., & Mundy, M. E. (2017). Applying best practice online learning, teaching, and support to intensive online environments: An integrative review. Frontiers in Education, 2, 59. https://doi.org/10.3389/ feduc.2017.00059 Slapac, A., Balcerzak, P., & O’Brien, K. (2021). Handbook of research on the global empow- erment of educators and student learning through action research. IGI Global. https://doi. org/10.4018/978-1-7998-6922-1 UNESCO. (2020a). COVID-19 educational disruption and response. https://en.unesco.org/news/ covid-19-educational-disruption-and-response UNESCO. (2020b). 1.3 billion learners are still affected by school or university closures. https://en.unesco. org/news/13-billion-learners-are-still-affected-school-university-closures-educational-institutions Wang, C.-Y., Zhang, Y.-Y., & Chen, S.-C. (2021). The empirical study of college students’ e-learning effectiveness and its antecedents toward the COVID-19 epidemic environment. Frontiers in Psychol- ogy, 12, 3212. https://doi.org/10.3389/fpsyg.2021.573590 Yeo, S. C., Lai, C. K. Y., Tan, J., & Gooley, J. J. (2021). A targeted e-learning approach for keeping uni- versities open during the COVID-19 pandemic while reducing student physical interactions. PLOS ONE, 16(4), e0249839. https://doi.org/10.1371/journal.pone.0249839 Zalat, M. M., Hamed, M. S., & Bolbol, S. A. (2021). The experiences, challenges, and acceptance of e-learning as a tool for teaching during the COVID-19 pandemic among university medical staff. PLOS ONE, 16(3), e0248758. https://doi.org/10.1371/journal.pone.0248758 AS_2022_1_FINAL.indd 55 13. 04. 2022 09:00:04