E-Learning for a Digital Society Dr Lea Bregar Margerita Zagmajster, MA Dr Marko Radovan University of Nova Gorica Press 2022 E-Learning for a Digital Society Original title: E-izobraževanje za digitalno družbo Authors: Lea Bregar, Margerita Zagmajster and Marko Radovan Editor of translated version: Lea Bregar Reviewers of original: Nives Ličen and Dejan Dinevski English translation: Miha Odar Proofreading: David Rix Text formatting and cover layout: Ksenija Konvalinka Publisher: University of Nova Gorica Press, Vipavska 13, SI-5000 Nova Gorica, Slovenia Publication year: 2022 ISBN: ISBN 978-961-7025-26-2 (PDF) Published in PDF : http://www.ung.si/sl/zalozba/ http://www.ung.si/en/publisher/ 15 December 2022 CIP Cataloging in Publication, National and University Library, Ljubljana COBISS.SI-ID 132365059 ISBN 978-961-7025-26-2 (PDF) This work is licensed under a Creative Commons Attribution-Non- Commercial-ShareAlike 4.0 International License. How to cite: Bregar, L., Zagmajster, M., & Radovan, M. (2022). E-Learning for a Digital Society. University of Nova Gorica Press. https://www.ung.si/en/publisher/ iii COntent Preface � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �1 Introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �5 About the Authors � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 11 List of Abbreviations � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 13 1� Theoretical and Developmental Aspects of E-Learning � � � � � � � � � � � � � 15 1�1 What is E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 15 1�1�1 Definitions of E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �16 1�1�2 Characteristics of E-Learning in the Narrow Sense � � � � � � � � � � � � � �17 1�1�3 E-Learning Classifications � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �19 1�1�4 Potential Benefits of E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �25 1�1�5 Realising the Benefits of E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � �26 1�2 Overview of the State of E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � 29 1�2�1 The Global E-Learning Market � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �29 1�2�2 E-Learning in the European Union and Slovenia � � � � � � � � � � � � � � �31 1�3 Web Development and the Future of E-Learning � � � � � � � � � � � � � � � � � 40 1�3�1 The Evolutionary Stages of the Web and E-Learning � � � � � � � � � � � �41 1�3�2 Web 3�0 and Trends in the Development of E-Learning � � � � � � � � � �43 2� Business and Organisational Aspects of E-Learning Planning � � � � � � � 49 2�1 Preparation of the E-Learning Development Strategy � � � � � � � � � � � � � 49 2�1�1 General Aspects of Strategic Planning � � � � � � � � � � � � � � � � � � � � � � � � �49 2�1�2 Strategic Planning Aspects of E-Learning � � � � � � � � � � � � � � � � � � � � � �54 2�2 Creating a Business Plan � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 60 2�2�1 Economic Aspects of E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � �60 2�2�2 Business Plan in E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �71 3� Pedagogical Aspects of E-Learning Planning � � � � � � � � � � � � � � � � � � � � � � 79 3�1 Instructional Design Models for E-Learning � � � � � � � � � � � � � � � � � � � � � 79 3�1�1 Instructional Design � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �79 3�1�2 Instructional Models for E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � �80 3�2 Theories of Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 88 3�2�1 Theory of Behaviourism � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �88 3�2�2 Cognitive Theory � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �89 3�2�3 Theory of Constructivism � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �90 3�2�4 Theory of Connectivism � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �91 3�2�5 The Theory of Community of Inquiry � � � � � � � � � � � � � � � � � � � � � � � � �92 3�2�6 Pedagogy of Abundance � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �95 3�2�7 Heutagogy � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �97 iv 3�2�8 The Relevance of Learning Theories for the E-Learning Instructional Design � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �98 3�3 Educational Needs Analysis � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 100 3�3�1 Objectives of the Educational Programme � � � � � � � � � � � � � � � � � � � �100 3�3�2 Programme Analysis � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �101 3�3�3 Analysis of Learners � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �102 3�3�4 Conducting an Educational Needs Analysis � � � � � � � � � � � � � � � � � � �104 3�4 Instructional Design of an E-Learning Programme � � � � � � � � � � � � � � 106 3�4�1 Framework of an Instructional Design for E-Learning � � � � � � � � �106 3�4�2 Learning Objectives � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �107 3�4�3 Learning Theories and Learning Objectives � � � � � � � � � � � � � � � � � � �110 3�4�4 Learning and Teaching Strategies and E-Learning � � � � � � � � � � � � �111 3�4�5 Linking the Pedagogical and Business-Organisational Components � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �114 4� The Development of E-Learning Programmes � � � � � � � � � � � � � � � � � � � �115 4�1 Learning Material � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 115 4�1�1 Starting Points for the Development of Learning Material � � � � � �115 4�1�2 Content of the Learning Material � � � � � � � � � � � � � � � � � � � � � � � � � � � �117 4�1�3 Storyboard for Learning Material � � � � � � � � � � � � � � � � � � � � � � � � � � � �119 4�2 Learning Activities � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 123 4�2�1 The Role of Learning Activities in E-Learning � � � � � � � � � � � � � � � � �123 4�2�2 Types of Learning Activities� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �123 4�2�3 Learning Activities in E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � �126 4�2�4 Learning Activities as an Integral Part of the Learning Material � �127 4�3 Assessment Methods in E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � 131 4�3�1 General Aspects of Assessment � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �131 4�3�2 Bloom’s Taxonomies of Learning Objectives � � � � � � � � � � � � � � � � � �133 4�3�3 Traditional and Alternative Methods of Assessment � � � � � � � � � � � �138 4�3�4 The Benefits of Technology-Enhanced Testing and Assessment � �146 4�3�5 Specific Features of Assessment in E-Learning � � � � � � � � � � � � � � � �151 4�4 Selecting and Integrating Media into the Programme � � � � � � � � � � � � 153 4�4�1 Media and Technologies � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �153 4�4�2 Criteria for Media Selection � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �155 4�4�3 Usability and Limitations of Media in Education � � � � � � � � � � � � � �157 4�5 Digital Tools for E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 160 4�5�1 Digital Tools for E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �160 4�5�2 Learning Management Systems � � � � � � � � � � � � � � � � � � � � � � � � � � � � �162 4�5�3 Authoring Tools � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �167 4�5�4 The Choice of Digital Tools � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �168 5� Pedagogical Support � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �173 5�1 General Features of Pedagogical Support in E-Learning � � � � � � � � � � 173 v 5�2 The Changing Role of Teachers and Learners � � � � � � � � � � � � � � � � � � � 177 5�3 The Tutor’s Competencies in E-Learning � � � � � � � � � � � � � � � � � � � � � � � 181 5�4 Gil y Salmon’s E-Moderating Model � � � � � � � � � � � � � � � � � � � � � � � � � � � 182 5�5 Planning the Tutor’s Work � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 186 6� Learning Approaches and Methods in a Digital Society � � � � � � � � � � � �189 6�1 Introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 189 6�2 Open Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 191 6�2�1 Opening up Education and Open Education � � � � � � � � � � � � � � � � � �191 6�2�2 Categories of Open Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �194 6�2�3 Open Educational Resources (OER) � � � � � � � � � � � � � � � � � � � � � � � � �199 6�2�4 Massive Open Online Courses (MOOCs) � � � � � � � � � � � � � � � � � � � �205 6�2�5 The State of Open Education in the World, the European Union and Slovenia� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �220 6�3 Artificial Intelligence � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 226 6�3�1 What is Artificial Intelligence � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �226 6�3�2 The Impact of Artificial Intelligence on Education � � � � � � � � � � � � �229 6�3�3 Artificial Intelligence and E-Learning � � � � � � � � � � � � � � � � � � � � � � � �231 6�3�4 The Limits and Challenges of Artificial Intelligence � � � � � � � � � � � �232 6�3�5 Artificial Intelligence in Slovenia � � � � � � � � � � � � � � � � � � � � � � � � � � � �234 6�4 Learning Analytics � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 236 6�4�1 The Concept and Importance of Learning Analytics � � � � � � � � � � �236 6�4�2 The Design and Development of Learning Analytics � � � � � � � � � � �237 6�4�3 Learning Analytics Implementation Circle � � � � � � � � � � � � � � � � � � � �240 6�4�4 Learning Analytics Models by Level of Complexity � � � � � � � � � � � �243 6�4�5 The State of Learning Analytics in Practice � � � � � � � � � � � � � � � � � � �250 6�5 Intelligent Tutoring Systems � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 252 6�5�1 What are ITSs � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �252 6�5�2 Virtual Assistants � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �253 6�5�3 Using ITS in Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �254 6�5�4 Benefits and Limitations of ITSs � � � � � � � � � � � � � � � � � � � � � � � � � � � � �258 6�6 Mobile Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 259 6�6�1 What is M-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �259 6�6�2 The Benefits of M-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �261 6�6�3 Limitations of M-Learning� � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �263 6�6�4 Planning M-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �264 6�6�5 Tools for M-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �267 6�7 Microlearning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 269 6�7�1 Characteristics of Microlearning � � � � � � � � � � � � � � � � � � � � � � � � � � � �269 6�7�2 Benefits and Limitations of Microlearning � � � � � � � � � � � � � � � � � � � �272 6�7�3 The Preparation of Microlearning � � � � � � � � � � � � � � � � � � � � � � � � � � �273 6�8 Gamification � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 277 vi 6�8�1 Introduction � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �277 6�8�2 Gamification in Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �278 6�8�3 Types of Gamification � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �280 6�8�4 Benefits and Limitations of Gamification � � � � � � � � � � � � � � � � � � � � �282 6�8�5 Gamification Tools � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �283 6�9 Simulations in Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 284 6�9�1 Concept and Meaning of Simulations � � � � � � � � � � � � � � � � � � � � � � � �284 6�9�2 Simulations in Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �285 6�9�3 E-Learning and Simulations � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �286 6�9�4 Design and Development of Simulations � � � � � � � � � � � � � � � � � � � � �286 6�9�5 Authoring Tools for Developing Simulations � � � � � � � � � � � � � � � � � �287 6�10 Virtual and Augmented Reality � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 290 6�10�1 The Difference Between Virtual or Augmented Reality � � � � � � � � �290 6�10�2 Virtual Reality � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �290 6�10�3 Augmented Reality � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �291 6�10�4 Using Virtual and Augmented Reality in E-Learning � � � � � � � � � � �293 6�10�5 Virtual Reality and Augmented Reality Tools� � � � � � � � � � � � � � � � � �294 6�11 Digital Storytelling � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 296 6�11�1 What is Digital Storytelling � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �296 6�11�2 Elements of Digital Storytelling � � � � � � � � � � � � � � � � � � � � � � � � � � � � �297 6�11�3 Types of Digital Stories � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �298 6�11�4 Digital Storytelling in Education � � � � � � � � � � � � � � � � � � � � � � � � � � � � �298 6�11�5 Tools for Creating Digital Stories � � � � � � � � � � � � � � � � � � � � � � � � � � � �299 7� Management in the Delivery of E-Learning � � � � � � � � � � � � � � � � � � � � � �303 7�1 Management in E-Learning – General Aspects � � � � � � � � � � � � � � � � � � 303 7�1�1 The Importance of Management � � � � � � � � � � � � � � � � � � � � � � � � � � � � �303 7�1�2 Definition and Levels of Management � � � � � � � � � � � � � � � � � � � � � � � �304 7�1�3 Specific Features of E-Learning Management � � � � � � � � � � � � � � � � �306 7�1�4 Project Management � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �309 7�2 Services for Learners � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 311 7�2�1 Administrative and Technical Services � � � � � � � � � � � � � � � � � � � � � � �311 7�2�2 Library � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �314 7�2�3 Counselling and Information Services� � � � � � � � � � � � � � � � � � � � � � � �315 7�3 E-Learning Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 317 7�3�1 Educational Success Rate and Drop-Out Rate � � � � � � � � � � � � � � � � �318 7�3�2 The Quality of the Results � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �320 7�3�3 Business Efficiency � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �324 7�3�4 Access to Education and Training � � � � � � � � � � � � � � � � � � � � � � � � � � �327 7�4 Copyright and E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 330 7�4�1 The Origins of Copyright Law � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �330 7�4�2 International Conventions and Agreements � � � � � � � � � � � � � � � � � � �331 7�4�3 Copyright in Slovenia � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �333 vii 7�4�4 Copyright and E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �337 7�5 Quality Assurance in E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � � � 345 7�5�1 General Aspects of the Quality in Education � � � � � � � � � � � � � � � � � �345 7�5�2 Quality Assurance in E-Learning � � � � � � � � � � � � � � � � � � � � � � � � � � � �346 7�5�3 Domains of Quality Assurance in E-Learning � � � � � � � � � � � � � � � � �350 7�5�4 Quality Assurance and Evaluation � � � � � � � � � � � � � � � � � � � � � � � � � � �353 7�5�5 Quality and Development of E-Learning � � � � � � � � � � � � � � � � � � � � �354 List of Figures � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �357 List of Tables � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �361 Literature � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � � �363 Preface ”E-Learning for a Digital Society” was published in Slovenian by the Slove- nian Institute for Adult Education (Andragoški center Slovenije - ACS) and with the support of the Ministry of Education, Science and Sport of the Re- public of Slovenia, in March 2020 on the ACS’s website https://www.acs.si/ digitalna-bralnica/e-izobrazevanje-za-digitalno-druzbo/, and in print in Sep- tember 2020. It is designed as an e-learning compendium, primarily intended for adult education providers in Slovenia when deciding on the development and implementation of e-learning programmes. The aim of this book is to pre- sent the status, the basic concepts and the wide range of possible e-learning formats, teaching methods and approaches, as well as the related pedagogical, technological and business-organisational issues and solutions faced by educa- tors at each stage of the e-learning lifecycle, from integrating e-learning into an organisation’s strategy to evaluation procedures. The work has been well received by users and its relevance and interest have been further enhanced by the rise of emergency remote learning and teaching as an involuntarily introduced form of e-learning, which allowed the continu- ity of the educational process during the outbreak of the Covid-19 pandemic. To ensure that this work is accessible to the largest possible audience of us- ers the School of Engineering and Management, University of Nova Gorica, proposed a translation of the work into English with the status of an Open Educational Resource (OER). This would be an additional contribution of Slo- venia to the promotion of open education, where the University of Nova Gor- ica is actively participating together with the Jožef Stefan Institute, including through the high-profile Open Education for a Better World (OE4BW) project and the Leadership in Open Education master programme. The leadership of the University of Nova Gorica supported this initiative by providing the neces- sary funds for the translation and publication of the book ”E-Learning for a Digital Society” online and in printed format. As authors, we welcomed the opportunity to make our work available global y. We were aware that transla- tion into English is a great recognition of our work, but that translation does not only entail the translation and design work, but also careful editorial work. 2 The first, crucial editorial dilemma was whether to include in the English ver- sion the developments and innovations related to e-learning that have been brought about during the two-year pandemic period, or to keep the translation within the scope and structure of the original. We prepared the book in the years 2018 and 2019, in the run-up to the Cov- id-19 pandemic, which has had a significant impact on education, with many short- and long-term, positive and negative consequences at all levels, from the formal and non-formal education at individual educational organisations and national education systems, to the global level. The modest knowledge of the basics of e-learning and the specificities of emer- gency remote learning and teaching compared to online and traditional learning and teaching, the lack of or inadequate training of teachers and other staff, the inaccessible and inadequate technological infrastructure and equipment, cou- pled with the scale and severity of the epidemic, all of these circumstances raised many problems, dilemmas and even doubts about the professional validity of e- learning and online education in general during the time of the pandemic. In this exceptional period, some of the latent issues of modern, technology-enhanced education have been highlighted, such as equal educational opportunities and the inclusiveness of technology-enhanced education, digital literacy and digital competencies, digital security and digital ethics. The emergency remote learning and teaching has, however, contributed to a broad awareness of the importance and potential of technology for modernis- ing education. The pandemic has seen a substantial increase in the use of pre- viously available tools (e.g., videoconferencing systems) and digital resources (e.g., OERs and MOOCs). Previously known forms of e-learning, such as blended learning and hybrid learning, have come to the fore in a fresh and in- novative way, with a clearly clarified role. Considering the dynamic changes in the e-learning landscape over the last two years, it would be useful to build on our work by presenting the latest techno- logical and social trends and their impact on the development of e-learning by assessing the state in the field of e-learning in the post-Covid period and by reviewing the innovative methods and approaches that have recently emerged. However, after a lot of consideration and discussion between the authors and the initiators of the translation, we have decided to keep the content and scope of this translation essential y unchanged as regards the original. There are several reasons for this decision. We consider the original to be a coherent whole on the issue of adult e-learning during the time before the pandemic. However, we hypothesise that the developments in the field of e-learning during and after the pandemic are opening a new stage of development towards the convergence of e-learning with digital education and the gradual expanding and penetration of certain advanced forms of e-learning, such as modern blended and hybrid 3 learning into areas where traditional education was dominant before 2020 (i.e., in the formal education of children and young people). Such developments are in line with the demands of today’s volatile, uncertain, complex and ambiguous (VUCA) world, which cal s for a serious transformation of the education system with more flexibility, robustness and resilience at all levels and across all seg- ments of education. A quality treatment of this issue requires further research and study of the vast amount of literature, particularly that produced in the last two years. Clearly, this goes beyond the timeframe we had available for the trans- lation, and beyond the feasible scope of the editorial work. The translation is therefore identical in content and structure to the original. In places, we have improved the clarity and accuracy of the original text. We have paid special attention to the links. The sources of information referred to in the text and listed in the Literature section have been kept unchanged. We have only checked whether the URLs are still active. In the case of broken links, we found equivalent alternative web addresses and updated the dates of access. In the same way, we checked and updated the links in the text (recommended links and links to additional information), the only exception being that we deleted the broken links and found other sources that were also relevant to the topic. We checked whether the links were active in September and October 2022. As an OER in English, ”E-Learning for a Digital Society” is accessible to a significantly wider circle of audience interested in e-learning than the original in Slovenian. We would be delighted if this work could be of use to all those who are interested or involved in or are planning to be engaged in e-learning in one way or another. We will be particularly pleased if this OER is used by stu- dents who are exploring e-learning in all its diversity of forms and affordances through independent, self-directed study, with the intention of using creative- ly and constructively on the principles of open education the knowledge and competencies they have acquired for a better, sustainable world. Authors Ljubljana, November 2022 IntroductIon In recent decades, modern technologies have become our constant compan- ion, designing the fabric of our everyday lives. These technologies have a sig- nificant impact on how we organise our work, how we spend our leisure time, how we find personal and other contacts, how we get information as consum- ers, as citizens and as users of public services, and what gadgets and tools we use in the workplace and in educational contexts. Terms like e-business, e-government, e-banking and e-library have been part of our everyday vocabulary for more than a decade. One of these terms is e-learn- ing. But what do we actual y mean by e-learning? Can we talk about e-learning when a teacher uses a video clip, they have downloaded from the Internet to enhance the lesson? Perhaps e-learning needs more than the simple use of technology in education! What are the benefits and advantages of e-learning for the learner, for the school or educational institution, and for the company? Is e-learning not alienating and does it not bring about the orwellisation of learning? What makes a good e-learning course? These and similar questions were answered in our first book, ” The E-Learning Essentials” handbook, pub- lished in 2010 (in Slovenian language: priročnik Osnove e-izobraževanja). In the last decade, society has been changing even faster. Social processes, products and services are increasingly more complex, connected, unpredict- able and digitalised, and modern society is changing from an information society to a digital one. In such a context, professional and personal success can only be achieved by those who, in addition to professionalism, possess the capacity to adapt quickly to new circumstances, to make decisions in the face of uncertainty and constant changes, and to operate effectively, including through the mastery of technology and its services. When it comes to the transformation into a digital society, Slovenia is not among the most advanced countries. The 2019 Digital Economy and Society Index (DESI) ranks Slovenia 16th in the European Union. In terms of provid- ing educational opportunities in line with the needs of a digital society, it is particularly worrying that we are lagging in the development of human capital 6 and the use of Internet services. The education programmes offered today by our educational institutions and other providers are not yet largely designed to enable learners to acquire the competencies needed in an open, fast-changing and digitalised society. Education is still predominantly guided by a teacher- centred educational paradigm that places the teacher at the centre, transfer- ring their knowledge to their students (learners). Educators in Europe general y agree that the teacher-centred educational paradigm has outlived its usefulness and that it is time for it to be replaced by a so-called learner-centred paradigm. There is a growing awareness that the population of learners is heterogeneous, with different backgrounds, mo- tivations and interests in education, generational y diverse, and with differing educational needs and expectations. The educational process or learning pro- cess must be designed and implemented in such a way that the learner is an active creator of knowledge and new competencies, and the teacher will act as a guide and facilitator in this process. The guiding concepts underpinning the implementation of this educational paradigm are the personalisation of learn- ing and adaptive learning, creative learning, active, independent and authentic learning, col aborative and open learning, and ubiquitous learning. A range of methods and approaches are now available for introducing these concepts into education, mainly based on the technological advances of the last decade. Our research and monitoring of the development of technology-enhanced edu- cation in Slovenia shows that the knowledge of new methods and approaches is modest, and their use is limited to isolated cases. The 2010 ”The E-Learning Essentials” handbook is also outdated in this respect. We have therefore decided to prepare a new work on e-learning that will introduce interested users to the most significant innovative methods and approaches to technology-enhanced e-learning that have been introduced to e-learning (and traditional education) in the last decade. This content accounts for almost 40% of the new work. The knowledge we have gained through research and development work, as wel as through monitoring the literature, and our first-hand experience of designing and delivering e-learning courses over the last ten years, have also prompted us to critical y review and evaluate the content in the handbook from 2010 in the light of new findings and trends, and to add to, update or omit content where necessary. We have omitted content that is no longer relevant due to technologi- cal possibilities or where there have been no significant new developments. In this way, we have been able to keep the new work within a still acceptable volume in relation to the available resources, and to keep it transparent. This was the background for the creation of the new book, which from the substantive point of view provides a rounded and updated view of e-learning, and also covers edu- cational perspectives in Slovenia on the way to a digital society. Hence, the title ”E-Learning for a Digital Society”. 7 This work is based on the narrower concept of e-learning: by e-learning, we mean the pedagogical y effective use of the potential of ICT in education, so that the technology must be subordinated to the learning and teaching strate- gies and learning objectives. Such an understanding of e-learning allows us to identify the characteristics of e-learning and its associated strategic ben- efits in a comprehensive and consistent way. These include: spatial and tem- poral independence of the educational process, the accessibility and openness of knowledge resources and flexibility in the choice of learning and teaching methods, and the flexibility and diversity of communication and col aboration methods. The extent to which e-learning courses exploit the potential benefits in practice depends on a number of circumstances: the characteristics of the target groups, the content and objectives of the course, the technological con- ditions, the resources available and, above al , the capacity of all those involved to use technology creatively in the learning process. There is therefore no sin- gle recipe for a good and effective e-learning course, as several specific circum- stances need to be considered in practice, and the definition of comprehensive e-learning serves as a conceptual framework for the consistent and systematic planning, development, implementation and evaluation of e-learning courses. ”E-Learning for a Digital Society” aims to provide the information on e-learn- ing that adult education providers in Slovenia need when deciding whether to introduce an e-learning programme, develop their own programme, or adapt an already developed programme; how the programme should be implement- ed from a managerial, pedagogical and technical point of view; what the tech- nological support for e-learning should be; what elements should be included in an e-learning course; how to train staff for active and efficient cooperation in an e-learning course; what are the development perspectives for e-learn- ing on the way to a digital society. These are pedagogical, technological and business-organisational (managerial) issues faced by the leaders of educational institutions, adult education teachers, counsellors, heads of adult education departments in the public and private sectors, and adult education organisers. We expect that the handbook will also be of interest to those interested in this form of education, whether in formal or non-formal contexts. The work has been designed to cover all the main topics relevant to each stage in the lifecycle of an e-learning course as defined by the general instructional design model (ADDIE): educational needs analysis, design, development, implementa- tion and evaluation. It is divided into seven thematical y distinct parts, which are further divided into sections. We have also added a list of recommended web links to each section. The first part, ”Theoretical and Developmental Aspects of E-Learning”, pro- vides basic information about e-learning in terms of its conceptual definition and basic characteristics, as well as an overview of the state and future of e- learning in the world, in Europe and in Slovenia. The first part concludes with 8 an overview of the development stages or generations of e-learning, from the first generation of static e-learning, through the second and third generations (social and semantic e-learning), to the fourth generation, whose name has not yet been established in professional circles. In this book, we have paid attention to all the generations or development levels of e-learning. Knowledge of the different levels of development is essential for understanding and correctly as- sessing the state of e-learning in educational practice, which in Slovenia largely does not go beyond the second generation at the level of study programmes. At the same time, it provides information for educators on how to plan and im- plement e-learning at a developmental level that suits their educational needs and available resources, and what the prospects are for further development. In the second part, ”Business and Organisational Aspects of E-Learning Plan- ning”, we first outline the general characteristics of strategic planning and its importance and specificities in e-learning. In this part, we present the eco- nomic aspects of e-learning (specifics of costs and savings in e-learning) and the basic procedures and components of a business plan and its preparation in the e-learning context. In the third part, ”Pedagogical Aspects of E-Learning Planning”, we first pre- sent e-learning instructional design models, learning theories and educational needs analysis, all in the light of the specificity and relevance for e-learning. Within this framework, we then define the learning objectives and the ele- ments of the learning and teaching strategy, thereby determining the design of the e-learning courses. In the fourth part, ”The Development of E-Learning Programmes”, we aim to provide a brief and transparent overview of the information needed to im- plement the design, i.e., to develop the individual elements of an e-learning course – the learning material, learning activities, assessment methods, and the selection and integration of media in an e-learning course. We focus more on digital tools, which have fundamental y changed the processes and possi- bilities for developing quality e-learning courses over the last decade. Pedagogical support is briefly discussed in the fifth part, mainly from the per- spective of tutoring support in e-learning. The most comprehensive part of the publication, the sixth part, is devoted to presenting the characteristics and possibilities of learning approaches and methods based on technologies specific to the digital society. From the wide range of possible approaches and techniques for learning and teaching in a digital society, we have presented in more detail those that have received the most attention in the literature and are already being implemented in the prac- tice of modern education. These include open education with open educa- tional resources (OER) and massive open online courses (MOOCs), artificial intelligence, learning analytics, intelligent tutoring systems, mobile learning, 9 microlearning, gamification, simulations, virtual and augmented reality, and digital storytelling. In the last, seventh part ”Management in the Delivery of E-Learning”, we first discuss the general tasks of an e-learning manager related to the efficient and effective performance of the functions of planning, organising, leading and controlling. We also look at the specific tasks of the manager in e-learning and the services for which the management is responsible. We also present the main approaches and methods for monitoring the results of e-learning. We conclude the last part with an overview of copyright management and quality assurance in e-learning. In preparing this work, we have consulted hundreds of different sources. De- pending on how they are documented, we have divided them into two groups. The first group consists of those materials from which we have directly ob- tained information for the preparation of each topic and that we reference in the text. For this group, we have also indicated the date of access for materials published online, except for those marked ’doi’. The references for these mate- rials are listed in the reference list. The second group of sources consists of recommended links, which provide additional, more in-depth information or a broader view of the issues at stake. The recommended links have been grouped thematical y by sections. The us- ability and activity of all the links was checked for the last time during the edit- ing and preparation of the book for print, i.e., from October 2019 to January 2020. As the work has been in progress for a longer period, it is possible that certain links may no longer be active. A particular problem with this handbook is the terminology used, as many terms have not yet been translated into Slovenian, or there are different translations, since there is considerable inconsistency in the English-language literature. For the sake of clarity, we have added the original English term to each translation. We will be pleased if this work stimulates the debate regarding e-learning termi- nology in Slovenia to a greater extent than the handbook has done. We are aware that the book ”E-Learning for a Digital Society” cannot provide all the answers to the questions that may be of interest to educators and others interested in e-learning, nor can it meet all expectations. The field of e-learning is too large and too fast-moving for that. Thus, exploring the latest trends in in- novative education and e-learning shows that the development of technology, with the emergence of big data, open data and artificial intelligence, has reached a stage where a thorough reflection and in-depth study of the ethical, social, legal and other humanistic aspects of the use of technology is necessary as wel . But these topics will remain our research issue in days to come. 10 We will be pleased if ”E-Learning for a Digital Society” will also be a useful source of information for those who have e-learning as part of their education programmes, just as the handbook ”The E-Learning Essentials” was. We also hope that ”E-Learning for a Digital Society” will move the professional debate on e-learning in Slovenia to a deeper level, increase interest in this form of education and contribute to a greater number of quality e-learning courses. Authors Ljubljana, March 2020 About the Authors 11 about the authors Dr Lea Bregar, Associate Professor, until her retirement in 2008 worked at the School of Business and Economics, University of Ljubljana, where she taught courses in statistics and research methodology. She was the initiator and leader of the project to introduce distance education at the Professional Degree Pro- gramme in Business Administration at the School of Business and Economics (former Faculty of Economics) in Ljubljana in 1994. In the first phase of the Phare Programme for Distance Education, she led the National Contact Point for Distance Education. She is the author of numerous scientific and profes- sional articles in the field of statistics, distance education and e-learning that she presented at expert meetings in Slovenia and abroad and that had been published in national and international professional journals. She has devel- oped a range of study materials in statistics for self-study, including the CEES (Course on European Economic Statistics), an interactive online programme that was launched in 1999. Since her retirement, she has continued her re- search and development work in the field of statistics and e-learning. With the staff of the Slovenian Institute for Adult Education, she has designed and con- ducted training courses for the introduction of e-learning in adult education as well as prepared and implemented several online courses. In 2010, she was the editor and first author of the ”The E-Learning Essentials” handbook. She has been cooperating with the Doba Faculty, online accredited higher education institution, for many years as a provider of postgraduate courses (E-Learning Management, E-Learning Models, and Trends in Innovative Education), as a mentor for postgraduate students and as head and a researcher at the Institute for e-learning at this institution. In the recent period, she took part as invited lecturer in master programme Leadership in Open Education and as a co- mentor in the international project Open Education for Better World, both led by School of Engineering and Management, University of Nova Gorica. Dr Marko Radovan is an Associate Professor at the Department of Educa- tional Sciences, Faculty of Arts, University of Ljubljana. At the Faculty, he teaches various subjects related to teaching and learning for adults and the use of ICT in education, such as Andragogical Didactics, Methods of Adult Edu- cation and E-Learning. His research also focuses on issues of adult learning and motivation of adults for education, participation in adult education and the use of educational technology. Recently, he has been involved in various international and national projects dealing with the integration of technology in education, such as the ”Blended Learning in Vocational Education and Trai- ning” (BlendVET), ”Blended Learning International Entrepreneurship Skil s Programme” (BLUES), ”Digital Transformation in Humanities” (HUM@N), and ”Digital UL – Innovative use of ICT for Excellence” (Digital UL). He is also the author or co-author of numerous scientific articles in the field of e- education and co-author of the handbook ”E-Learning Essentials”, published 12 in 2010, and ”E-Learning for a Digital Society”, published in 2020, both by the Slovenian Institute for Adult Education. Margerita Zagmajster, MA, was Head of the Research and Development Centre at the Slovenian Institute for Adult Education until her retirement in 2021. She worked in various areas of adult education development, e-learning and the use of modern ICT in adult education. She was one of the pioneers of distance education in Slovenia while working at the Centre for University Development at the University of Ljubljana and was the Head of the National Contact Point for Distance Education at the School of Business and Econom- ics at the University of Ljubljana in the second phase of the Phare Programme for Distance Education. She has written several papers on distance education and e-learning and has presented the topic at national and international con- ferences. She has participated in several international projects in the field of e-learning and adult education, such as Improving Open and Distance Learn- ing in a Network (Netcampus), Multi-country Integrated System for Improved ODL Networking (MISSION), and Toolkit for Developing, Implementing and Monitoring Adult Education Strategies (DIMA). At the Slovenian Institute for Adult Education, she led the working group for the development of the e- learning online course supported by European Social Fund and was co-author of the course. In cooperation with external col aborators, she has delivered several e-learning training courses in the form of e-learning. She is co-author of the 2010 ”The E-Learning Essentials” handbook. She was the editor of the E- corner Portal and co-editor of the SLC Portal, both of which she also designed in terms of content. List of Abbreviations 13 List of abbreviations ACS Slovenian Institute for Adult Education (Andragoški center Slovenije) ADDIE Analysis, Design, Development, Implementation, Evaluation (Model) AR Augmented Reality BYOD Bring Your Own Device CAI Computer-Assisted Instruction CAL Computer-Assisted Learning CC Creative Commons CEDEFOP European Centre for Development of Vocational Training cMOOC connectivist Massive Open Online Course CoI Community of Inquiry EADTU European Association of Distance Teaching Universities ECTS European Credit Transfer System ENQA European Association for Quality Assurance in Higher Education EUA European University Association FAO Food and Agriculture Organisation of the United Nations GPA Grade Point Average ICT Information and Communication Technology IDM Instructional Design Model IoT Internet of Things ITS Intelligent Tutoring System LCMS Learning Content Management System LA Learning Analytics LMS Learning Management System LTS Learning and Teaching Strategy MIT Massachusetts Institute of Technology 14 MIZŠ Ministry of Education, Science and Sport (Ministrstvo za izobraževanje, znanost in šport) MOOC Massive Open Online Course OEF Open Education Framework PASS Personalised Adaptive Study Success POKI Offering Quality Education to Adults (Ponudimo odraslim kakov- ostno izobraževanje) NGDLE Next Generation Digital Learning Environment OECD Organisation for Economic Co-operation and Development OER Open Educational Resources PPT PowerPoint ROI Return on Investment RSS Real y Simple Syndication SAM Successive Approximation Model SIO Slovenian Education Network (Slovensko izobraževalno omrežje) SLC Self-Directed Learning Centres (Središča za samostojno učenje) SoLAR Society for Learning Analytics Research VR Virtual Reality VUCA Volatility, Uncertainty, Complexity, Ambiguity ZASP Copyright and Related Rights Act (Zakon o avtorskih in sorodnih pravicah) xMOOC Extended Massive Open Online Course What is E-Learning 15 MIZŠ Ministry of Education, Science and Sport (Ministrstvo za izobraževanje, znanost in šport) MOOC Massive Open Online Course OEF Open Education Framework theoretIcaL and PASS Personalised Adaptive Study Success POKI Offering Quality Education to Adults (Ponudimo odraslim kakov- deveLoPmentaL asPects 1 ostno izobraževanje) NGDLE Next Generation Digital Learning Environment of e-LearnIng OECD Organisation for Economic Co-operation and Development OER Open Educational Resources PPT PowerPoint ROI Return on Investment 1.1 What is e-Learning RSS Real y Simple Syndication SAM Successive Approximation Model Before we start looking at e-learning, we first need to clarify what we mean by SIO Slovenian Education Network (Slovensko izobraževalno omrežje) the term. SLC Self-Directed Learning Centres (Središča za samostojno učenje) A closer look at the literature and various materials related to the definition . SoLAR Society for Learning Analytics Research of e-learning (Fee, 2009; Lynch and Roecker, 2007; Moore et al., 2010; Sangra, 2012) reveals a considerable lack of uniformity and confusion in the inter- VR Virtual Reality pretation of the term. This causes problems not only in professional debate VUCA Volatility, Uncertainty, Complexity, Ambiguity but also in attempts to put e-learning into practice, as unclear and unresolved views on the concept of e-learning itself cause difficulties in setting objectives ZASP Copyright and Related Rights Act (Zakon o avtorskih in sorodnih for the implementation of e-learning and in choosing the ways and means to pravicah) achieve them. Of course, the presentation of the basics of e-learning in this xMOOC Extended Massive Open Online Course book cannot be done without a clear and unambiguous explanation of what is meant by e-learning, as the understanding of this concept itself has a direct impact on the topics covered. The divergence related to the conceptual definition of e-learning emerged at the very beginning of e-learning at the turn of the twenty-first century. The first definitions of e-learning date back to 2001. That year saw the publication of Marc Rosenberg’s e-learning Strategies for Delivering Knowledge in the Digital Age, which can be described as the first comprehensive study on e-learning. In this work, Rosenberg (2001) defines e-learning as the use of web-based technologies in a variety of solutions to enhance knowledge and performance. In the same year, Rossett and Sheldon defined e-learning more loosely as train- ing whose essential characteristic is the use of the Internet, and used the term 16 Theoretical and Developmental Aspects of E-Learning ’online’ education or ’online’ learning as a synonym. According to this understand- ing, an essential characteristic of e-learning is that it is delivered whol y or partly using electronic hardware or software, or a combination of the two (Rossett and Sheldon, 2001). Both definitions already indicate a dichotomy in the understanding and in- terpretation of e-learning that persists to this day. Moreover, the rapid devel- opment of technology and its use in different educational contexts make it increasingly difficult to adopt a single definition. 1.1.1 definitions of e-Learning On the one hand, we are dealing with views that consider e-learning to be any education that uses the Internet or technology (e-learning in the broad sense). Technology is only one component of the learning process and is designed to complement it, but it does not interfere with the conceptual foundations and pedagogical doctrine of the traditional y designed learning process. This definition is based on the technological component only. At the other end of the wide range of definitions of e-learning are definitions that deal with e-learning in a narrower, more specific way. Technology is in the function of education, so e-learning is the integration of technology into education (a narrower defini- tion of e-learning). An example of a broad definition of e-learning is that of the European Centre for the Development of Vocational Training (CEDEFOP): ”E-learning is learn- ing supported by information and communication technology (ICT). It may also include different forms and combined methods: the use of software, the Internet, CD-ROMs, online learning and any other electronic or interactive devices or media”. According to CEDEFOP, e-learning can be used not only as a tool of distance education but also to support face-to-face learning (CEDE- FOP, 2014, p. 72). E-learning is similarly defined in the 2013 and 2014 European University As- sociation (EUA) surveys on e-learning in the higher education sector in Eu- ropean countries. The term e-learning is defined as a overarching term for all educational forms based on the use of ICT to support both learning and teaching. It may refer to the use of technologies and tools to support learning in different contexts, ranging from face-to-face settings and distance learning or a combination of both, commonly referred to as blended learning (Gaebel et al., 2014, p. 17). A review of e-learning terminologies shows that a broad definition of e-learning is more widely used (EADTU, 2019). However, in our view, such a definition is outdated. Given the accessibility and ubiquity of ICT, almost all education today What is E-Learning 17 can be defined as e-learning, as technology is used to a greater or lesser extent (except in the less developed world) in virtual y all educational institutions and programmes. This loose definition of education is also not useful for research on e-learning. It also has a negative impact on the development of e-learning, as it ignores the fact that the use of ICT in education is not an end in itself, but is used for the sake of education. Researchers from the Open University of Catalonia set out to find a definition of e-learning that would suit the research objectives, through an extensive literature review, using the Delphi method to analyse the perceptions of e-learning among 33 prominent experts from around the world. According to the definition that is almost entirely accepted by the experts who took part in this study, e-learning is conceived narrowly, the purpose of use being essential: ” E-learning is an approach to teaching and learning, representing all or part of the educational model applied, that is based on the use of electronic media and devices as tools for improving access to training, communication and interaction and that facilitates the adoption of new ways of understanding and developing learning” (Sangra. et al., 2012, p. 152). 1.1.2 characteristics of e-Learning in the narrow sense In this book, we will start from a narrower concept of e-learning. By e-learning, we mean the pedagogical y effective use of the potential of ICT in education, so that the technology must be subordinated to the learning objective and learning and teaching strategies. The impacts of ICT on teaching and learning can be expressed in different ways (Kirkwood and Price. 2014, pp. 9 - 10): • replicating existing teaching and learning activities (for example, publish- ing textbook online); • supplementing teaching and learning activities (for example, posting online a video recording of a traditional lecture); • transforming teaching and learning processes and outcomes (for example, interactive online lectures). Supplementing leads to improving the quality of existing educational process- es and outcomes, while transformation leads to innovation, which ultimately leads to a change in the educational paradigm. Replicating existing processes and achievements with technology is not e-learning (in a narrow sense), even though it may have a business benefit for the organisation. Similarly, the SAMR model (Puentedura, 2014) classifies the effects of technol- ogy on learning and teaching. 18 Theoretical and Developmental Aspects of E-Learning The model consists of four levels of change in the learning process: • substitution, • augmentation, • modification, • redefinition. Technology-enhanced teaching and learning can only be defined as e-learning (in a narrower sense) when technology as an indispensable element enables a new quality in teaching and learning processes and outcomes (for example, the learner’s preparation of a seminar assignment enriched with sound and audio recordings and presented live, or the presentation of a seminar assignment in an interactive webinar). E-learning can only be considered when, without the use of technology, it would be impossible to implement the designed educational programme and achieve the learning objectives . The purpose and objectives of the use of tech- nology must therefore be clearly defined in the design stage of the learning programme. The importance of ICT for the achievement of learning objectives in e-learning is also highlighted by Elkins and Pinder (2015, p. 1): ”E-learning is any edu- cational programme, course or structured learning event that uses electronic media to achieve its objectives. Such education may have the same compo- nents as traditional education (text, audio, tests, assignments), but the learning objectives are achieved or even enhanced by the use of a computer.” The Organisation for Economic Co-operation and Development (OECD), (2018b) points to research findings (Fisher, 2006; Law, 2008) that technology per se does not improve learning achievement, the pedagogical aspect is still paramount. Clark and Mayer (2016, pp. 8–9) state that e-learning has the following charac- teristics: • ”Stores and/or transmits lessons on CD-ROM, local internal or external memory, or servers on the Internet or intranet. • Includes content relevant to the learning objective. • Uses media elements such as words and pictures to deliver the content. • Uses instructional methods such as examples, practice, and feedback to promote learning. • May be instructor-led (synchronous e-learning) or designed for self-paced individual study (asynchronous e-learning) • Helps learners build new knowledge and skil s linked to individual learning goals or to improved organizational performance”. What is E-Learning 19 The potential of ICT enables the improvement and innovation of the educa- tional process in three segments (Bregar, 2013): • spatial y and temporal y independent delivery of the educational process; • flexibility and diversity in the ways in which all stakeholders in the educa- tion process communicate and col aborate; • accessibility and openness of knowledge resources and flexibility and diver- sity in the choice of teaching and learning methods. E-learning that pedagogical y effectively deploys technology in all three seg- ments is what we call comprehensive e-learning. E-learning is understood as a generic, umbrel a concept that encompasses many implementation options and forms. The extent to which and the characteris- tics with which a particular implementation model will approach the concept of comprehensive e-learning depends on a number of factors, not only tech- nological possibilities; primarily pedagogical framework, but also institutional constraints and opportunities, financial and human resources, legislation, etc. 1.1.3 e-Learning classifications The variety of technological systems, tools and services and the diversity of the contexts in which e-learning is developed and delivered lead to a plethora of e-learning manifestations in practice today. Classifications are a fundamental tool for the systematic study of mass phenomena. comprehensive e-Learning and traditional and Blended Learning In this book, we use as a conceptual basis a classification based on the scope and degree of integration of technology in education. This classification di- vides e-learning into three basic groups: • traditional education (with limited use of technology); • blended education; • comprehensive e-learning. In traditional education, technology is one of the components of the learn- ing process, intended only to replicate it and sometimes make it more cost- efficient, without interfering with the conceptual basis and traditional (teacher centred) paradigm of the learning process. This is e-learning in a broader sense, which we have called partly technology-enhanced learning. Examples of partly technology-enhanced educational programmes range from the simplest use of ICT, such as publishing course or teaching material on CD-ROM or the 20 Theoretical and Developmental Aspects of E-Learning web, using e-mail and online resources, to more complex and sophisticated uses such as online discussions and web-based projects. The main features of partly technology-enhanced learning are: • The purpose and role of technology from a pedagogical point of view are general y not defined. • Technology is used in a piecemeal and disjointed way, either for individual elements of the learning process or for its administrative support. • The learning process is based on unchanged pedagogical concepts of tradi- tional (face-to-face) education. • The amount of direct teaching in the classroom is virtual y unchanged. The problems with the simple, unrelated and partial use of technology in education, and its equation with e-learning, were pointed out in particu- lar by representatives of the large open universities in the first period of the introduction of e-learning, i.e., at the turn of the twenty-first century. Carol A. Twigg of the University of Phoenix in Arizona, USA, pointed to an example of the inappropriate use of ICT in education that is un- fortunately still common today. ” The vast majority of online courses are organized in much the same manner as are their campus counterparts: developed by individual faculty members, with some support from the IT staff, and offered within a semester or quarter framework. Most follow traditional academic practices (”Here’s the syl abus, go off and read or do research, come back and discuss.”), and most are evaluated using tradi- tional student satisfaction methods. Using old approaches and concepts, even with new technology, cannot deliver better quality, greater acces- sibility and lower costs” (Twigg, 2001, pp. 3–4). The potential of partly technology-enhanced learning depends on the extent to which and how technology is integrated into the educational process, and how efficiently and effectively the process makes use of technological affordances. For example, publishing lecture notes on the Internet instead of in a printed publication will reduce the cost of the material and increase its accessibility. But simply being on the Internet does not make learning more active and independent. The preparation of e-learning material re- quires thorough preparation in terms of content and design, which must be based on the relevant pedagogical theories and respect also the princi- ples of preparing material for self-learning. The essential difference between partly technology-enhanced education and comprehensive e-learning is that in comprehensive e-learning, technological support is not only sporadic, for individual elements of the educational pro- What is E-Learning 21 cess, but is holistical y and intentional y integrated into all the elements of the process. This means that it is embedded in the pedagogical and administrative support and learning material, which also allows the learning process to be carried out with physical separation of teacher and learner. e-Learning and distance education The spatial separation of the teacher and the learner allows for greater flex- ibility in education. The spatial flexibility of traditional distance education, of course in a different technological context, has significantly improved the educational opportunities of important segments of the population (e.g., the working population, people in geographical y remote locations and people with disabilities) and thus contributed to openness of education. Physical separation has also brought some disadvantages, particularly in terms of reduced interaction in the educational process. However, these shortcomings can now be tackled relatively successful y using modern technology, through various forms of technology-enabled synchronous and asynchronous communication formats. Comprehensive e-learning, which enables the spatial y independent delivery of the learning process, has another important characteristic: through innova- tive forms of technology-enhanced communication and col aboration, the ac- cessibility of new sources of knowledge and a variety of technology-enhanced learning approaches and methods, it enables the implementation of modern pedagogical models in pedagogical practice, oriented towards the learner and the creation of new knowledge and competencies. The intermediate stage between partly technology-enhanced learning and comprehensive e-learning is so-called blended learning . Blended learning does not exclude direct (traditional) forms of instruction, but these can only occur in a complementary to and relatively limited way. 22 Theoretical and Developmental Aspects of E-Learning Figure 1: the extent and degree of integration of technological support at different forms of e-learning ning Comprehensive e-learning -lear Types of e Blended learning Partly technology- enhanced learning Traditional teaching Scope and integration of technological support in the teaching and learning process and learning These basic forms of e-learning thus differ in terms of the relative importance of technological support in the educational process and the extent to which the technology used is supported by appropriate pedagogical, organisational, personnel and financial solutions. other e-Learning classifications Below are some typical and more common classifications of e-learning. The classification used by the Sloan Foundation, led by Babson College, for the annual collection of data on e-learning in the US is widely used for its simplic- ity and pragmatism. The individual categories of this classification are based on how much of the course is delivered online (Allen, 2016, p. 7): • traditional course (no online technology); • web-facilitated course: between 1% and 29% of the content is delivered on- line; • blended or hybrid course: 30% to 79% of the content is delivered online, the rest in the classroom; • online course: 80% or more of the content is delivered online. The application of this classification in practice is rather arbitrary, as there is no defined way of measuring the scope of content delivered by categories. Mayadas et al., (2015) classify e-learning into seven groups according to the spatial flexibility of delivery at the course level: • classroom course: co-teaching with minimal use of ICT; What is E-Learning 23 • synchronous distributed course: a combination of classroom and distance learning delivery with synchronous communication (e.g., webinars); • classroom-based course, partly with web-enhanced activities: the amount of classroom hours remains unchanged; • blended/hybrid classroom course: classroom time is significantly reduced, replaced by online classes and other online activities; • blended/hybrid online course: most of the delivery is online, but some is mandatory in the classroom; • online course: all course activities are online; • flexible delivery: the course can be delivered in a variety of ways, giving learners a choice. Stephen Downes (2012a), the father of the new theory of connectivism, classi- fies e-learning into 6 generations based on the development of technology and related ICT uses: • online content (only learning content is online, programmed learning); • online interactions and network; • computer games; • new online content and/or online interactions (based on learning manage- ment systems (LMS) and learning content management systems (LCMS)); • web 2.0 combined with e-learning 2.0; • massive open online courses (MOOCs). Elkins and Pinder (2015) discuss e-learning in three basic categories: synchro- nous, asynchronous and group learning (cohort learning). The classifications shown are so-called linear (one-dimensional), as they take into account only one criterion, mainly the intensity of the use of technology and/or the spatial flexibility of the learning process. A step further is the UNESCO classification, which breaks down the degree of integration of technology in education according to the characteristics of com- munication and the organisation of content in the educational programme. 24 Theoretical and Developmental Aspects of E-Learning Figure 2: classification of e-learning according to the integration of Ict in communication and learning content Communication Rich, professional Communities communication via of practice communities of practice Blended learning Enriched communication within a course Online courses Little or no human interaction E-resources Content Content not Content Content organized for pre-structured co-constructed by instruction within a course learners and teachers Source: Anderson, 2010, p. 41. European University Association (EUA) uses three criteria for classifying e- learning in its surveys on e-learning in the higher education sector (Sursock, 2015, p. 74): • level of delivery (study programme, course); • delivery method (online, blended); • the extent of delivery in the organisation (predominant delivery method by department, by individual teacher). The EUA classification has the advantage of also considering organisational aspects, but it only covers one functionality of technology in education, that of increasing spatial flexibility. A comprehensive view of the state of e-learning would only be possible with a classification that would classify e-learning programmes according to the level of ICT integration across all the core functional areas of e-learning1 (e.g., content and learning material, assessment, communication and collaboration, management/organisation of the learning process). 1 A similar approach was used by Amy Wilson in her categorisation of e-learning for the higher education sector in New Zealand (Wilson, 2012). What is E-Learning 25 1.1.4 Potential Benefits of e-Learning In the early stages of the use of modern ICT in education, dating back to the second half of the 1990s, theoreticians and practitioners enthusiastical y cited the many advantages of e-learning, which were to fundamental y change tra- ditional methods of education. But they forgot about the limitations of human cognitive abilities in harnessing the potential of technology for learning and teaching. The real effects of technology use depend on its compatibility with the characteristics of the cognitive processes, and the consideration of the in- structional design principles, as well as a range of other factors, such as man- agement and environmental support, appropriate technological infrastructure, etc., need to be taken into account (Clark and Mayer, 2016, p. 7). Taking these constraints into account, e-learning can bring many benefits to learners and educational institutions, but also more broadly, to employers and to education systems at the national level and international y. The most significant advantages of e-learning from the learner’s point of view are: • greater flexibility in the time, place, pace and content of learning (just-in- time learning, just-in-place learning); • greater interactivity and faster access to knowledge from different sources (synchronous and asynchronous forms of communication, online resources); • the possibility of adapting learning approaches to individual needs; • transparency of education conditions; • developing new knowledge and competencies. The most significant advantages from the point of view of the educational or- ganisation as a provider of educational services are:2 • reducing certain cost categories (teaching staff costs, rental costs, premises- related costs); • better service options; • transparency and documentation of programme delivery and the consist- ency of delivery; • the possibility of making assessment more objective and comprehensive; • access to quality learning resources; • introducing modern pedagogical models and innovating the teaching process; • better opportunities for marketing education programmes and interna- tionalisation. 2 A systematic overview of the benefits of e-learning from an organisational perspective is provided by Allen (2016, pp. 25–27). It looks at the benefits of e-learning in four groups: strategic benefits, tactical benefits, training modality benefits and infrastructure benefits. 26 Theoretical and Developmental Aspects of E-Learning E-learning enables companies as users of educational services to: • the opportunity to implement the training that would not be possible un- der traditional circumstances (for example, due to space constraints, ab- senteeism, etc.); • cheaper organisation and delivery of training; • faster delivery of training; • making better use of available technology and available internal and exter- nal online information resources; • improving information literacy and developing other digital competencies of employees; • the ability to adapt training content quickly and, as a rule, easily to the needs of the company and, in particular, to the individual (specific) needs of employees. Kenneth Fee (2009, p. 30) cites the ability of e-learning to promote a culture of learning and the learning organisation as a particular potential strength of e-learning. 1.1.5 realising the Benefits of e-Learning There are many difficulties in putting e-learning into practice. The basic pre- requisite for the introduction of e-learning – technological infrastructure, which was initial y the main obstacle to the introduction of e-learning, is now becoming a less and less important limiting factor. The main obstacles and the cause of many unsuccessful attempts lie in untrained professional staff for this form of education, inadequate management, and a superficial and insufficient knowledge and understanding of e-learning in general. Too often we hear that e-learning can be introduced by simply uploading learning materials to the web. What is forgotten here is that the declared ben- efits can only be realised through an integrated approach. This also requires taking into account the pedagogical framework, which must be supported by an appropriate organisational, financial and human resources scheme, regarding the organisation is a provider or a user of educational services. The organisa- tion must be adequately prepared for the introduction of e-learning. CommLab India (a global adult e-learning company) in its e-learning hand- book ”Getting Your Organisation Ready” (2016), states that before imple- menting e-learning, an organisation should review the psychological, social, environmental, technological, financial, contextual, organisational features and human resources as key readiness factors. What is E-Learning 27 A review of the literature on the success of e-learning projects suggests that the main barriers to successful e-learning implementation can be grouped as follows (Medárová et al., 2012, pp. 6–7): • conceptual barriers: misunderstanding of the characteristics of e-learning and mismatch between the e-learning delivery model and the needs of the organisation and target groups; • organisational barriers: inadequate implementation of managerial func- tions, poor communication between stakeholders, insufficient resources; • technical barriers: inadequate technological support (inadequate equip- ment for the software needs, insufficient Internet connections, inconsistent software), inadequate quality of educational media, inadequate mainte- nance and lack of technical support; • human factor barriers: from an individual point of view, the problems are reflected in negative attitudes towards e-learning, while the social aspects are reflected in communication problems between different stakeholder groups, resulting from a lack of motivation, lack of knowledge of e-learn- ing, lack of cooperation or lack of competence. All these factors are interlinked and interact. For example, inadequate organi- sation can lead to technical problems and reduce the motivation of learners, ultimately resulting in the organisation not being able to achieve its objectives. David Miller, one of the e-learning researchers and practitioners who reg- ularly publishes on the e-learning Industry portal (https://elearningindustry. com/elearning-projects-fail-top-5-reasons), points out the typical mistakes that lead to failed attempts to introduce e-learning: • We have neglected the ”big picture” when introducing e-learning. This means that we have not consider the long-term goals that the e-learning project is supposed to lead us towards. • The objectives of an e-learning programme should be defined early enough, before the initiation phase. This is important for the learners themselves and also crucial for the development of the programme. • We do not know the characteristics of the learners. The development of an e-learning programme is not an end in itself, but must be in line with the needs and possibilities of the learners. • Lack of communication between e-learning programme developers, subscribers and participants. • We have not been able to come up with a suitable instructional strategy. This step is decisive for all subsequent stages. Successful instructional design requires continuous innovation with an increasing level of inter- activity and the active participation of the learners in the programme. 28 Theoretical and Developmental Aspects of E-Learning How to avoid these mistakes is discussed in considerable detail in Parts 2 and 3 of the publication. recommended Links Florida National University. The Evolution of Distance Learning: https://www.fnu.edu/evolution-distance-learning/ ICDE International Council for Distance Education: https://www.icde.org/publications-and-resources Learn Upon. Elearning Terms: https://www.learnupon.com/blog/elearning-glossary/ Overview of the State of E-Learning 29 1.2 overview of the state of e-Learning 1.2.1 the global e-Learning market E-learning as a way of education supported by ICT has been in use for less than two decades. The spread of e-learning has been uneven across education sectors and geographic areas, and varied according to implementation models. Analysts in the US estimate that the global e-learning market3 generated $165 billion in turnover in 2015 and is expected to reach $240 billion by 2023 (Do- cebo, 2018, p. 4). Overal , e-learning is expected to grow in the coming years: the e-learning mar- ket for self-learning4 reached 36 billion dol ars in 2011, with the total revenues totalling 46.6 billion by 2016. The highest growth rates were achieved in Asia (17.3%), followed by Eastern Europe (16.9%), Africa (15.2%) and Latin America (14.6%), Western Europe (6%) and North America (4.4%), over the 2011–2016 period (Docebo, 2018, p. 5; 2016, p. 9). The differences in market dynamics are due to the specific functioning and maturity of each regional market. In Asia, e-learning growth has been driven mainly by public projects to increase literacy in rural areas. In the Middle East, government incentives to introduce digital learning material as an educational method suitable for all categories of learners are important. In Africa, mobile telephony and the proliferation of so- cial networks are the main drivers of change in education, and poor infrastruc- ture is a major obstacle. In Eastern Europe, the most important growth drivers for e-learning are public investment and the large number of start-ups. In the Russian market, leading MOOC providers (Coursera and Khan Academy) compete with home-grown initiatives (such as LinguaLeo for learning English). In the Czech Republic and Slovakia, the typical business model is to purchase content, which is then fed by the home organisations into their own learning management systems (LMS). The state and public schools are not very active due to budget constraints and difficulties in implementing European projects (Docebo, 2014, p. 14). The US, Australia and Western Europe exhibit the characteristics of mature markets with the highest levels of e-learning implementation at all levels of education. 3 The e-learning market covers e-learning services offered by the business and higher education sectors. Typical services provided by the enterprise sector include rapid online learning (ROL), LMS, virtual classrooms and various application simulation tools. The higher education sector, however, is more focused on offering mobile learning content, podcasts, LMSs and LCMSs (Docebo, 2018, p. 5). 4 The self-paced learning market comprises LMS, tools for the preparation of learning material and other learning aids, pre-packaged learning content and related services (Docebo, 2018, p.5). 30 Theoretical and Developmental Aspects of E-Learning E-learning is characterised by the constant emergence of new products and services driven by technological innovation. This results in the loss of mar- ket share of products and services that have been on the market for a long time, opening up business opportunities for even better or more commercial- ly interesting solutions. Recent years have been marked by the rise of cloud computing, smartphones and bring your own device (BYOD) policy. Open educational resources (OER), big data and learning analytics have come to the fore. Classical types of LMS, such as Moodle, which has been the leading tool for the development and delivery of e-learning programmes, especial y in for- mal education, for the last decade, are gradual y being phased out (see Section 4.5 Digital Tools for E-Learning). The US has been quick to grasp and exploit the business opportunities offered by e-learning. E-learning has become very popular in the US, especial y as a form of education for employees in large companies, especial y in the corporate universi- ties of large corporations, but also in the higher education sector. The e-learning market in the US is estimated at $27 billion (Docebo, 2018, p. 4), representing a 16% share of the global market. E-learning is therefore of particular interest in the US as a great business op- portunity, which is why the e-learning market for the corporate sector is closely monitored by professional marketing research agencies, mostly for a fee (e.g., Ambient Insight, Docebo, Brandon Hall Group, Technavio). Business analysts predict further growth in the demand for training services and products in the coming years, as surveys confirm managers’ awareness that a skilled workforce is one of the main drivers of productivity growth and profits. At the same time, new, more functional and flexible tools are driving the replacement of existing systems and thus forecasting a growth in demand. In the US, the status of formal e-learning for the higher education sector has been monitored since 2003 using a common methodology. The research is car- ried out by Babson College with support from the Sloan Foundation. Data for 2014 (Allen and Seaman, 2016) shows that of the 20.5 million stu- dents enrolled, 2.9 million or 12.7% studied only online or at a distance. Three million (14.6%) were part-time distance or online learners (enrolled in one or more online courses). In 2014, a quarter of students in the US participat- ed to a greater or lesser extent in online education programmes, compared to almost a third (31.6%) in 2016 (Seaman et al., 2018). Enrolment in online education programmes has been increasing steadily, despite an overall decline in enrolment in higher education programmes since 2013; private non-profit institutions have the highest growth rate, while enrolment in private for-profit institutions has been declining. Interestingly, the proportion of leadership of higher education organisations in the US who consider this format to be strategical y important for the institution Overview of the State of E-Learning 31 decreased in 2015 compared to 2014, from 71% to 63%. However, the decline in the preference for online education is due to a marked drop in organisations that do not offer online education (from 34% to 20%), while the proportion remained unchanged in those that do (77%). In 2015, the majority of university leaders (71%) in the US considered online education to be at least equal to or better than traditional education. The uptake of online education has improved significantly since 2003 (Allen and Seaman, 2016, p. 5). Canada paints a similarly encouraging picture of the uptake of e-learning. In 2015, enrolments in online programmes accounted for 16% of the total enrol- ments. Between 2011 and 2016, the number of higher education institutions with online courses increased by 11%, so online education in the higher educa- tion sector is now also quite widespread in this country. Most Canadian higher education institutions consider e-learning to be a very important strategic com- ponent (Bates, 2017). 1.2.2 e-Learning in the european union and slovenia guidelines and Policies in the european union E-learning or technology-enhanced education5 and training features promi- nently in the European Union’s development documents. The European Com- mission has already clearly underlined the potential of ICT to achieve the European Union’s core strategic objective of becoming the most competitive, knowledge-based society in the Lisbon Strategy (Commission of the European Communities, 2000a). The role of ICTs and the ways to use them to achieve the underlying strategic objective were subsequently outlined in several docu- ments: the eEurope 2002 and eEurope 2005 Action Plans, which were followed up by the strategy document ”i2010 – A European Information Society 2010” (Commission of the European Communities, 2000b). However, in the European Union, despite political support, backed up by fund- ing for a number of projects, the development of e-learning has been much slower, below expectations and accompanied by a number of failed projects. The European Commission report ” The Use of ICT to Support Innovation and Lifelong Learning for Al ” noted that ICT has not yet transformed educational processes in a more visible way than it has in other activities (European Commission, 2008). In May 2009, the Strategic Framework for European Cooperation in Education 5 About ten years ago, the term ”e-learning” disappeared from the vocabulary of the European Commis- sion’s policy and strategy documents. This may be due to the rather low level of sustainability of European e-learning projects in the first stage of e-learning development, or to the prevailing loose definition of e-learning, which causes communication problems between the different stakeholders involved in e-learning. 32 Theoretical and Developmental Aspects of E-Learning and Training up to 2020 – ET 2020 (ET 2020, 2009) was launched to act as a forum for the exchange of best practices, information and advice for policy re- form between Member States, the Commission and educational institutions. The ET 2020 strategic framework covers all forms of learning at all levels of lifelong learning and highlights the education sector as a key enabler for smart, sustain- able and inclusive growth. The 2015 Joint Report of the Council and the Commission on the implemen- tation of the strategic framework for European cooperation in education and training included ”open and innovative education and training, including by ful y embracing the digital era” as one of its new priority areas (Official Journal of the European Union, 2015, p. 27). Actions to address these priorities include the more active use of innovative pedagogical approaches and tools to develop digital competencies, and strong support for teachers and other stakeholders who play a critical role in ensuring student success and implementing educa- tion policy. The document also criticises a number of e-learning initiatives: because they have been scattered and disjointed, investment in infrastructure has not been accompanied by efforts to increase teachers’ and learners’ com- petencies and motivation to use technology. As a result, despite significant investment, few projects have progressed from the pilot phase to the regular implementation of e-learning (European Commission, 2013b, p. 11). In line with the ET 2020 guidelines, several working groups have been in place to help Member States address their education and training challenges, as well as common priority themes. Working Groups have played a prominent role in the formulation of policies and strategy documents. For example, the Digital and Online Education Group contributed to the Communication on Open Education, which was publicly launched in September 2013 ” Opening Up Education; Innovative Teaching and Learning for All Through New Technologies and Open Educational Resources” (European Commission, 2013a). This document has significantly shaped the development of higher education in the European Union. It advocates a change in the fundamental conditions of higher education in order to take advantage of the opportunities offered by technology. The EU should also provide an appropriate policy framework for the introduction of innovative learning and teaching. On this basis, and taking into account the findings of other research projects, in 2017 the European Commission published recommendations for open education in Europe ” Going Open: Policy Recommendations on Open Ed- ucation in Europe – European Commission” (Inamorato dos Santos et al., 2017). The Digital Education Action Plan was published in early 2018 and aims to promote the introduction of innovative approaches and digital technologies in education and the development of digital competencies (along with digital literacy and digital security), as the use of digital technologies in education is Overview of the State of E-Learning 33 noticeably lagging behind their uptake and accessibility. The Action Plan high- lights three priority areas (European Commission, 2018, p. 4): • better use of digital technology for teaching and learning; • developing relevant digital competencies and skil s for digital transformation; • improving education through better data analysis and foresight, analysing, processing and using data more effectively. The Action Plan also recommends concrete actions to promote digital educa- tion, such as setting up a common European platform for the digitalisation of higher education and for enhanced cooperation, digitalised links between uni- versity information systems and the introduction of a European student card. The state of e-learning in the European Union is gradual y improving. Until a few years ago, this education was quite rare in European universities. The Changing Pedagogical Landscape study (European Commission, 2015) shows that in 2014–2015, even in leading universities, only 20% of courses were de- livered in a blended way. The delivery model mostly mimicked traditional teaching, and the quality was usual y poor. Despite good accessibility, the use of technology has mostly been limited to electronic versions of traditional learning materials, with only a few cases of more advanced learning methods. The 2018 Changing Pedagogical Landscape study identified progress. How- ever, the development of online and blended programmes is still largely left to the discretion of individual teachers or small teams, in some cases supported by institutions (Henderikx and Jansen, 2018). strategic orientations and support for e-Learning in slovenia The beginnings of modernising education with ICT in Slovenia date back to the 1990s. In 1994, Slovenia joined the international programme ” Phare Programme Multi-country Cooperation in Distance Education”, which, through a series of educational and promotional activities and pilot projects, has had a significant impact on the development of distance learning and e-learning at all levels of education. The project ended in 2000. In 1994, the Computer Literacy – RO project was launched, aimed at primary and secondary education. The Slovenian Educational Network (SIO) was set up, with a catalogue of materials and events, Trubar software support, promo- tional events at home and abroad, and the annual International Educational Computer Conference (Mednarodna izobraževalna računalniška konferenca – MIRK). In 2006, the Programme Council for the Informatisation of Education of the Ministry of Education and Sport adopted an Action Plan for the Further Leap of 34 Theoretical and Developmental Aspects of E-Learning the Informatization of Education (MIZŠ, 2006) and in 2007 identified the fur- ther development of the SIO as one of the priority areas6. In this document, e- learning is seen as one of the levers of the computerisation of education and is equated with distance learning. The National E-Learning Strategy 2006–2010 was also presented at the conference Education in the Information Society in 2006, but was not adopted as an official document. Several projects were carried out in parallel to the strategic activities in the 2007–2013 period (MIZŠ, n.d.) • E-learning (2009–2013). • E-competencies of teachers in bilingual schools (2012–2013). • Creation of a multimedia and interactive e-learning material (2006–2010). • The first four pilot e-textbooks and basics and recommendations for e-text- books (2011). • E-textbooks with a focus on science courses (2011–2014). • Further development and implementation of the SIO (2007–2015). • Pedagogy 1 : 1 in the light of 21st century competencies (2011–2014). • Infrastructural and technological potential for the inclusion of people with disabilities in the education system (2013–2015). • The e-Schoolbag project (2011–2015). • IR optics (optical connections) (2013–2015). It should not be overlooked that these projects have almost completely bypassed higher education, as well as adult education. Adult education was the focus of the European Social Fund Lifelong Learning Centres project, which developed e-learning materials. The project was co-financed by the Ministry of Education, Science and Sport (MIZŠ). The rather dynamic trend of ICT introduction in primary and secondary edu- cation in the first decade of e-learning development was followed by a general stagnation of the political initiatives for the development of the information society in Slovenia and related project activities in the field of e-learning. The projects initiated in the previous decade were mostly completed in 2011–2013. The re- duction of interest was partly due to the economic crisis. Years of inactivity are reflected in a number of information society indicators and are not ignored in international reviews. The 2014 e-Competencies Report for Slovenia (Empirica, 2014) states that Slovenia was one of the leading European countries in the 1990s (in terms of households equipped with computers, ICT use in schools). However, insufficient funding and poor coordination at the national level have pushed it below the EU average, especial y in the areas of online public services (including e-learning in the public sector) and information society policy (Empirica, 2014, p. 4). This is also confirmed in the field of education with the indicator of the use 6 The first phase of the Slovenian Education Network (SIO) project https://sio.si/ was initiated in 1995 as part of the Computer Literacy Project. Overview of the State of E-Learning 35 of online education courses in the last quarter, where Slovenia is on average at the tail of OECD countries (OECD, 2019, p. 165). The OECD study How’s Life in the Digital Age (OECD, 2019, pp. 38–39) states that, while overall Internet ac- cessibility is solid, Slovenia is characterised by an exceptional y wide divergence in the intensity of Internet activity between different social groups. Activities related to the development of the information society were partial y revived in 2016. In that year, the 2020 Information Society Development Strat- egy ”Digital Slovenia 2020” was adopted. Slovenia’s goals for the digitalisation of education are quite ambitious: ”In the field of education, the whole educa- tion system will work with the aim of adapting education to the needs of the new generations for integration into the digital society, making Slovenia a ref- erence environment for new practices.” (Digitalna Slovenija 2020, 2016, p. 17). In 2016, MIZŠ adopted a strategic document Strategic Orientations for the Fur- ther Introduction of ICT in Slovenian HEIs by 2020. This document is based on the European Union’s core strategic documents7 for the field of e-learning and the introduction of technology in education. It also takes into account national strategic documents relevant to the information society and education (The Strategy for the Development of the Information Society until 2020, The Resolu- tion on the National Programme for Higher Education 2011–2020). The vision for the further introduction of technology in Slovenian educational institutions is to provide individuals with the opportunity to learn in an open, creative and sustainable learning environment, supported by the innovative use of technol- ogy, which will enable them to acquire in an effective and high-quality way the knowledge, skil s and key competencies of the twenty-first century that are nec- essary for successful integration into society and the labour market. The introduction of technology in Slovenian higher education institutions is guided by six strategic objectives: • didactics and e-learning materials, • platforms and cooperation, • e-competencies, • informatization of institutions, • e-learning (higher education, adults), • evaluation (MIZŠ, 2016, p. 6). The first major step in recent years towards providing better opportunities for the modernisation of education in the higher education sector are the projects ”ICT in UL’ s Teaching Degree Programmes” (2017–2018), ”Digital UL – Innova- tive Use of ICT for Excel ence” (2017–2020) and ”Innovative Learning and Teach- ing in Higher Education” (2018–2022). https://www.uni-lj.si/o_univerzi_v_ljubljani/ 7 The Digital Agenda for Europe (https://www.europarl.europa.eu/factsheets/en/sheet/64/digital- agenda-for-europe) and the Education for the 21st Century Resolution (http://unesdoc.unesco.org/ images/0024/002456/245656e.pdf), published by UNESCO in 2014. 36 Theoretical and Developmental Aspects of E-Learning projekti/ . In 2017, more than 20 different types of e-learning material in various fields (electrical engineering, mechanical engineering, ICT, food, etc.) were developed to meet the needs of the labour market in the framework of the pro- ject ”Developing Training for Work 2017” (http://www.konzorcij-sc.si/consortium-of- slovenian-school-centres/). ICT featured notably in the 2004 Resolution on the National Programme for Adult Education in the Republic of Slovenia, saying: ”/.../ the design and delivery of educational programmes is still dominated by the use of traditional forms and methods, lacking flexibility and openness, and un- derdeveloped conditions for experiential learning and alternative models of learning and teaching. We are at the beginning of the development and implementation of modern forms, methods and techniques of adult learning, such as self-directed learning, distance learning, e-learning, module-based learning and other innovations made possible by modern ICT and information society services.” (Uradni list Republike Slovenije, 2004, p. 8591). The Resolution on the National Programme for Adult Education in the Republic of Slovenia for the 2013–2020 estimates that technological de- velopment ”will have a major impact on the conditions in which adult education will operate” and that ”the opportunities for modern forms of learning through information and communication technology (ICT) will increase enormously” (Andragoški center Slovenije, 2014, pp. 33–34). The first priority area (general adult education) also includes increasing digital literacy and inclusion in the information society, while the sec- ond priority area (improving the educational attainment of adults) aims to achieve these objectives through the development of e-learning and various forms of mobile learning by ministries, professional bodies and educational services providers. Different pathways should be made avail- able for participants to reach the primary and secondary school stand- ards – e-learning is mentioned as one of them. The development of new e-learning programmes and e-learning materials should be encouraged as part of the development activities (Andragoški center Slovenije, 2014). In Slovenia, we still have not grasped the importance of technology in mod- ernising education and adapting it to the needs of the digital society. Initiatives and activities remain one-sided, sporadic and unrelated and as such do not lead to sustainable results and more visible progress in education. The lack of understanding of the role of e-learning is also reflected in the Slovenian Devel- opment Strategy 2030 (Šooš, 2017), which includes ”learning for and through- out life” among its strategic orientations, and considers knowledge and skil s for quality life and work as one of Slovenia’s twelve main development goals. Overview of the State of E-Learning 37 However, it only monitors progress towards this target through the level of participation in lifelong learning, the share of the population with tertiary edu- cation, and attainment in maths, reading and science. It neglects the knowl- edge and competencies that are essential for a digital society: information and data literacy, communication and col aboration, digital content creation, se- curity and problem-solving (Carretero et al., 2017). E-learning is the natural environment (ecosystem) for developing these competencies. the spread of e-Learning in the european union and slovenia If we try to shed light on the problem of the spread of e-learning in Europe and Slovenia with empirical data, i.e., the number of persons in e-learning programmes, or data on e-learning courses, we find ourselves in a considerable predicament. There is currently no good quality and methodological y comparable data available on e-learning in general and adult e-learning in particular, at least for Slovenia and the European Union. The European Commission, through its Commissions, has supported quite in- tensive research and development work in recent years to promote and accel- erate the implementation of the Open Education Strategy. The research cited above is mainly qualitative in nature, looking at specific aspects of open and digital education at the EU level. The findings are also of a general nature and do not provide insight into the state of e-learning at the level of the European Union, much less on a country-by-country basis (Bregar and Puhek, 2017). Data on the state of e-learning in the higher education sector at the European level is scarce. The EUA has conducted two surveys: one for 2013 (Gaebel et al., 2014) and one for 2014 (Sursock, 2015). The first survey aimed to gather information on the state and potential for the development of e-learning in the higher education sector in Europe and to collect fairly detailed data on MOOCs. For primary and secondary education, the results of the 2013 and 2017 surveys on ICT use in schools are available (European Commission, 2013b; European Commission, 2019a). International institutions such as the OECD, Eurostat and UNESCO have not yet included data on e-learning in their statistical databases, but as part of their surveys on the information society, they have been collecting vari- ous indicators of the information society that only indicate the potential for the development of e-learning. The World Economic Forum (WEF) calculates the so-called networked readiness index based on official data from international statistical institutions. In 2016, Slovenia ranked 37th out of 139 countries on this indicator (Baller et al., 2016, p. 171). The European Commission calculates the Digital Economy and Society Index (DESI) for EU Member States, which 38 Theoretical and Developmental Aspects of E-Learning ranks Slovenia 16th out of 28 EU Member States in 2019 (European Commis- sion, 2019b). A review of potential data sources on e-learning in Slovenia and the European Union has shown that the most useful data is from the Survey on ICT use in households and among individuals aged 16–74, which is used by national sta- tistical offices in the European Union to collect data on the use of the Internet for education, among other things. The data collected for all European Union countries is published by Eurostat. The following table shows the results on the use of the Internet for the educa- tion of the population aged 16–74 in Slovenia in 2017. In addition to the EU-28, the data for Slovenia was compared with Finland, which is one of the most successful European countries in the development of e-learning. In this respect, it is a standard of excellence and a good starting point for assessing the situation in Slovenia. Table 1: International comparison of Internet use for education, 2017 (Percentage of the total population aged 16–74) Doing Using online Communicating Use of the an learning with instructors Internet online material or students via for any course educational educational websites or activities portals EU 28 7 14 8 19 All SLO 5 15 6 18 individuals FIN 19 28 17 31 Source: Eurostat, Survey on Usage of ICT by Individuals, 2017. The results show that in 2017, the use of the Internet for educational activities in Slovenia was about the same as the EU-28 average. Comparing the use of the Internet for educational activities in Slovenia with Finland, Slovenia’s lag behind is worrying, especial y when it comes to participation in online courses. Due to the lack of relevant data and research, and also due to problems with the very concept and understanding of e-learning, we do not have a true picture of the state of e-learning in Slovenia. This makes it difficult to take a strategic view and to assess and design related actions at the national level, and it also hampers professional communication on the development and modernisation of the education system among the most important stakeholders. Overview of the State of E-Learning 39 In order to fill this information gap in the field of higher education, the Doba Faculty, with the support of the MIZŠ, carried out an analysis of the state of digi- talisation and e-learning in higher education in Slovenia in 2017. The survey, which was methodological y aligned with the EUA surveys, showed the spread of e-learning in Slovenia is significantly lower than in other Euro- pean countries, irrespective of whether these forms of education are delivered at the level of individual courses or entire study programmes. The smallest gap is in the simplest form of e-learning, i.e., the blended delivery of individual courses followed by the blended delivery of study programmes. The biggest differences are in the joint delivery (with other organisations) of online study programmes (Bregar and Puhek, 2017). ACS data on the provision of adult education programmes confirms the Reso- lution’s findings on the scarcity of modern, technology-based adult education. Data available from the ACS on adult education programmes shows that in 2017/18, fewer than one in ten adult education providers offered e-learning programmes, with e-learning programmes accounting for only 3.3% of all adult education programmes (Andragoški center Slovenije, 2018). recommended Links ACS. E-corner: http://tvu.acs.si/paradaucenja/ekoticek/ European Commission. Digital Learning & ICT in Education: https://ec.europa.eu/digital-single-market/en/policies/digital-learning-ict-education. European Commission. Education and Training Monitor, 2019: https://ec.europa.eu/education/policy/strategic-framework/et-monitor_en 2nd Survey of Schools: ICT in Education: https://digital-strategy.ec.europa.eu/en/library/2nd-survey-schools-ict-education EUA. Digital y enhanced learning and teaching in European higher educa- tion institutions: https://www.eua.eu/resources/publications/954:digital y-enhanced-learning-and- teaching-in-european-higher-education-institutions.html 40 Theoretical and Developmental Aspects of E-Learning 1.3 Web development and the future of e-Learning Modern technology, alongside social and political change and the increasing globalisation of the world, is undoubtedly changing education and thus learning. As we have shown in the previous section, change in education is relatively slow, with uneven dynamics and varying degrees of participation by different segments of society. Notwithstanding all these differences, education is cer- tainly very different today than it was two or three decades ago. Education in the twenty-first century is characterised by the following devel- opmental tendencies (Ehlers, 2009, p. 135): • education takes place everywhere, in different places, in different forms, in different contexts, and far from just in the classroom; • learners are increasingly taking on the role of organisers of learning; • learning is a lifelong process that takes place at different times and is not only related to educational institutions; • learning takes place in learning communities, which can be formal or in- formal; • learning is no longer teacher-centred or institution-centred. An important driver of this change, especial y in terms of technology use, is the younger digital natives (n-gens) born in the 1980s and beyond (Prensky, 2001). Members of this generation approach work, learning or acquiring knowledge and other activities differently from previous generations. They are used to re- ceiving information from several sources at the same time, quickly but superfi- cial y. They search for information on demand, not on stock, they communicate constantly and they curate their own ”collections” rather than buying a book or a CD-ROM. Communicating using technology, mostly at a distance, is an indispensable part of their lives and the main way they make social contacts. The OECD study warns that such categorisation may be an unjustified gen- eralisation in a very heterogeneous world. There are significant differences in accessibility and competencies in the use of technology, not only between gen- erations, but also between peers from different developed countries. Studies also warn that the transition from using technology for leisure and entertain- ment to serious use in education is not a given . What matters are competencies that cannot be acquired through play alone (OECD, 2018b). The use of technology in education can only be effective if it is supported by appropriate learning and teaching strategies. Contemporary pedagogical ap- proaches are changing the traditional role of the teacher as teacher-centred and putting the learner at the centre, with an active role and participation in the learning and teaching process. The teacher becomes a facilitator of the learn- Web Development and the Future of E-Learning 41 ing process, encouraging and guiding the learner to acquire knowledge and competencies in a quality way and to use quality diverse resources. These trends are behind new theories of learning (Section 3.2 Theories of Learning). Connectivism is gaining ground . Connectivism starts from the view that knowledge or understanding is present in networks between people and that learning is a process of connecting, growing and navigating these net- works (Siemens and Tittenberger, 2009, p. 11). The implementation of new theoretical approaches in education is supported by the continuous development of the Internet and related online systems, tools and devices. Alongside connectivism theory, new ideas are emerging about contem- porary theories of learning, such as the pedagogy of abundance (Weller, 2011), heutagogy or self-directed learning theory (Blaschke, 2012) and rhizomatic learn- ing8 (Cronje, 2016). 1.3.1 the evolutionary stages of the Web and e-Learning The development of e-learning is often presented in terms of stages or genera- tions in the evolution of the web. Figure 3: the evolutionary stages of the web and e-learning Web 4.0: 2020 E-Learning 4.0 Web 3.0: 2010– E-Learning 3.0 Web 2.0: 2000– E-Learning 2.0 Web 1.0: 1994– E-Learning 1.0 8 As a philosophical concept, rhizome means a web that cannot be given a centre point. The term originates from botany. Similar to the root web, we cannot distinguish the starting point from the end of the plant. Each point is a possible beginning or end (https://en.wikipedia.org/wiki/rhizome (philosophy)). 42 Theoretical and Developmental Aspects of E-Learning Tim Berners-Lee is the pioneer of the web. In 1990, he set up the first web server and the first HTTP connection between a client and a server over the Internet (Dennis, 2019). In its first phase of development (the first generation), the web was an infor- mation space used by the business world to communicate more complex in- formation. It was in fact an e-reader that allowed modest user interaction, as modest as the search engines browsers of the time (Mosaic, Netscape, Internet Explorer) allowed. In this context, the first generation of the web (Web 1.0) is also referred to by some as the static web. An essential element is a system of linked documents on the web. This generation of the web is characterised by the fact that relatively few providers offer information as a system of web-linked documents to a large number of users. The next stage, Web 2.0, called the Social or Dynamic Web, has gone beyond the e-reader to allow users to actively participate in the creation of information. The term Web 2.0 was coined by Tim O’Reil y and Dale Dougherty in 2003.9 Users can add value to the information online by adding their own comments, ratings and blogs. A key feature of Web 2.0 is that it makes it easy and (almost) costless to obtain and add information. For example, guest reviews of hotel services or book reviews by readers are a very welcome addition to the basic online services, which are provided free of charge or at minimal cost to the providers. Web 2.0 is characterised by these development trends (Rosen, 2009): • Easy access to user services. Using modern user services no longer requires software support on the user’s computer. The services are available online. For example, if you want to know the quickest route from A to B, you can simply use Michelin’s route planner or Google Maps. • Focus on uncharacteristic/atypical users. In previous decades, typical users were the focus of business interest, as their abundance and homogeneity made it relatively easy to achieve good business results. Internet technolo- gies are also creating a wide range of peripheral categories of users with specific needs more interesting to business and marketers. Modern web technologies make it relatively easy to tailor services to specific needs, and the multitude of visitors and the globalisation of the Internet ensure that there is a market for a sufficiently large number of these specific users. • The integration of existing technologies or services from different sources to create a new service, called a mash-up or hybrid service. Integration is usual y done with open interfaces (APIs) and data. The use of mapping data with Google Maps to add location data to real estate data creates, for example, a new service or new information, i.e., the identification of real estate segments. Such information, which was not original y planned to be 9 The labelling of the Web, as introduced by Tim O’Reil y and Dale Dougherty for stage 2, has also been applied retrospectively (for Web 1.0) and seems likely to apply to further stages of the Web’s evolution. Web Development and the Future of E-Learning 43 in either of the two databases (the estate agency, database or Google), rep- resents a new and innovative service. • Marketing brands and opening up new uses for technology components that are not directly purchased by the user. For example, Google has patented the use of its search engine as a local search engine on the intranet. • The service can be used on different operating systems. For example, web services accessible via search engines allow the same service to be accessed from a PC or a smartphone. Of course, new ways of using the Internet or new web services 2.0 would not be possible without innovative technologies such as APIs, mobile technology, adaptive learning environments , open access systems, and the development of appropriate tools and standardisation. The main tools of Web 2.0 are (Siemens and Tittenberger 2009, p. 14): • online publishing (blogs, wikis, e-portfolios), • voice over an IP network (for example, Skype), • mobile learning (MP3, mobile phones), • virtual interactive worlds (Second Life, Voice Thread), • integrated classrooms (El uminate), • discussion forums using the LMS or external applications, • chat rooms (IRC, IM), • graphical y supported links (Flickr), • software support for groups (Sharepoint Grove), • social networking tools ( Facebook, Twitter, ELGG, MySpace), • social bookmarking tools ( Delicious). Web 2.0 is characterised by information being broken down into short content units that can be distributed across dozens of different subject areas. The web of documents is transformed into a web of data. Web 2.0 brings new tools that al- low content pieces to be linked (aggregated) and used creatively in new ways and with new utility. 1.3.2 Web 3.0 and trends in the development of e-Learning Today, Web 1.0 is obsolete, but Web 2.0 is still alive and wel , even if the third generation, Web 3.0, was announced almost a decade ago. Web 3.0 is the semantic web. The semantic web is characterised by the fact that it builds on the functionalities of Web 2.0 in dealing with information, using complex, technology-based methods such as machine learning, artificial intelligence, data mining, semantic analysis and network analysis. The main idea of Web 3.0 is to organise or transform online data and links into an on- 44 Theoretical and Developmental Aspects of E-Learning line database so that it can be efficiently searched, linked and used to generate new information and new insights. Web 3.0 is designed to improve online data management and accessibility across devices, to foster creativity and innova- tion, and to encourage participation in social networks. In Web 3.0, the con- cept of a website disappears and data ownership is replaced by the concept of shared services, which provide qualitatively (content-wise) different informa- tion than the initial website. A new generation of the web is coming, and its image is not yet ful y formed. A review of the 2009–2017 literature on the evolution of the web (Almeida, 2017) shows that there is a lack of consensus on the fourth generation of the web; it is characterised by several terms, the most commonly used (usual y as synonyms) being pervasive computing and ubiquitous computing. It is also noticeable that the technical term Web 4.0 itself is still quite rarely used in the literature. Table 2: associated terms with Web 4.0 paradigm in the literature 2009–2017 Concept Number of items % Web 4.0 19 2.1 Symbiotic web 2 0.2 Web of things 158 17.8 Web social computing 36 4.1 Pervasive computing 354 40.0 Ubiquitous computing 317 35.8 Total 886 100.0 Source: Almeida, 2017, p. 7044. Pervasive or ubiquitous computing is the integration of different elements such as the computer desktop, sensors, mobile devices and electronic tools in the workplace, in private life and in other areas. An essential feature of perva- sive computing is a high degree of ”communication” between devices and sen- sors, which enables a synchronised communication infrastructure. A major element of ubiquitous computing is ensuring security. Access to the web is no longer limited to humans, but is also possible for physical objects, devices and vehicles. Web 4.0 is expected to bring smart devices that can read/recognise the content of the web and react to it with actions and decisions (read-write- execution). The symbiotic web, as the coexistence of people and technology and their col aboration in a webOS (multi-tasking operating system for smart devices) environment, is expected to mark a new stage of the web. Web Development and the Future of E-Learning 45 In less than three decades, the web has therefore progressed enormously, from a system that connects information, then people, to a system that connects knowledge and, as some have speculated, ”intelligence beyond the human”. As information becomes more connected, so does the potential for social networking, which is key to educational innovation. The development of the World Wide Web has a major role and potential for modernising education. The transformation of the World Wide Web has also had a major impact on the development of e-learning. The evolutionary stages of the web determine the evolutionary stages of e-learning, albeit with a time lag. The development of the web first made a range of content available to learners online, and this significantly improved the flexibility of the learning process (in terms of time and, to some extent, space). The emergence of the first learn- ing objects was driven by the LMS. They mimic the traditional delivery of the learning process: the teacher is the central figure in the learning and teaching process, preparing the learning material in a media-rich environment and de- livering it to the learners, usual y by publishing it in an LMS (Hussain, 2012, p. 12). However, despite the static nature of the web, the first generation of e-learning al ows for a certain degree of interactivity through the use of differ- ent media in the learning material and the use of other tools outside the LMS (Miranda et al., 2014, p. 97). The term E-Learning 2.0 (i.e., second-generation e-learning) was coined by Stephen Downes as the application of Web 2.0 technologies to learning and teaching (Downes, 2005). It lists the main features of E-Learning 2.0 as: • participants create their own content and col aborate with peers in blogs, wikis, topical discussions, automated notifications, RSS (real y simple syn- dication) and other forms of networks that allow decentralised content production and responsibility sharing; • e-learning takes advantage of the wealth of online resources and integrates them into new learning experiences; • e-learning is based on the combined use of a variety of tools that are other- wise unconnected and available in different places (e.g., online references, learning materials and articles, knowledge management tools, col aboration and search). The learner is at the centre of the learning process; technology enables more active learning and the dynamic adaptation of learning content. The Internet is increasingly a basis for knowledge sharing and less and less a medium for information transfer. Content is created and used; its creators are spatial y independent participants in the learning and training process. Passive acceptance of information prepared by others is taking a back seat. Learning and training become col aborative and interactive. 46 Theoretical and Developmental Aspects of E-Learning Even if E-Learning 2.0 is still dominant in practice and in many places has not even surpassed E-Learning 1.0, the third generation, E-Learning 3.0, has been at the forefront of professional debate and development for almost a decade. This generation of e-learning is characterised by the emergence of cloud computing and modern technologies such as col aborative intelligent filtering, widespread smart mobile technology, increasingly powerful and reliable data storage, touch- based user interfaces and 3D. A key feature of E-Learning 3.0 is the use of a wide range of digital devices, mostly mobile, that al ow learning to take place anytime, anywhere. At the forefront are artificial intelligence techniques, data mining and other learning analytics approaches that can be used to explore massive data to gain deeper insights into the learning process and, on this basis, to adapt the learning process to the real needs of the learner. Web 3.0 as the Semantic Web develops E-Learning 3.0 along these lines (Wheeler, 2009): • implementation: different types of learning material can be easily linked and accessed by semantic queries and by following a valid cognitive cate- gorical apparatus; • responsiveness: the use of intelligent agents to organise and filter informa- tion makes the results of user queries more accurate and faster; • accessibility: semantic queries lead us more easily to relevant content; • personalisation: the ontology allows search and queries to be tailored to the user’s needs; • flexibility: semantic tagging of content allows easy customisation; • symmetry: the integrated platform can be adapted to different learning ac- tivities; • modality: active and fast delivery of content creates a more dynamic learn- ing environment; • sovereignty: as the web decentralises, content management becomes more cooperative. The following table shows how the development of the web and e-learning are linked. Web Development and the Future of E-Learning 47 Table 3: concepts and technologies in the evolutionary stages of the web and e-learning Generation Web E-Learning Concept Tools Concepts Tools 1.0 Read-only, or HTML, HTTP, URL Content CBT, LMS, e-books, publish-only, the management, virtual learning web of documents unilateral environments, LCMS 2.0 Reading and Dynamic tools, Content sharing, LCMS, social net- posting, social ASP, AJAX, multimedia, works, videocon- web podcast, RSS feeds, dynamic ferencing, virtual wikis, blogs ”blind teaching” learning environ- ments, mashups 3.0 Reading, posting, RDF, XML, OWL, 3D Ubiquitous, PLE, social inquiries and collaborative, semantic web, collaboration, semantic virtual worlds, big data, open, avatars, intelligent connected data; agents semantic web Source: Miranda et al., 2014, p. 100. Predictions about the next evolutionary stage of e-learning are even less elabo- rated and less frequent in the literature than predictions about Web 4.0. The reason is that even the third generation of e-learning is still at the stage of expert discussion and testing, and digitalisation is still in its early stages, at least in Europe. E-learning trend researchers predict that the educational technologies that emerged a few years ago, such as mobile learning, microlearning, gamification, social learning and interactive video, will continue to be important in the next few years. However, the emergence of approaches and techniques specific to Web 4.0, such as the Internet of Things (IoT), the new generation of LMSs and robotics, is also expected (Pandey, 2018a; Becker et al., 2018). The Open Uni- versity’s latest report for 2019, Innovating Pedagogy, predicts that technology- led innovations in education will be joined in the coming years by innovations with a strong social and humanistic component. One example is decolonising learning, which moves away from traditional curricula that have typical y been based on a monocultural stereotype (e.g,. the white Western male learner) and introduces largely ignored topics and social groups into the learning process, as well as multiculturalism in education (Ferguson et al., 2019, p. 4). 48 Theoretical and Developmental Aspects of E-Learning recommended Links Gartner. eLearning Hype Curve: https://webcourseworks.com/elearning-predictions-hype-curve/ DOCEBO (2019). E-Learning Trends, 2019: https://www.docebo.com/resource/report-elearning-trends-2019/ EDUCAUSE. Horizon Reports: https://library.educause.edu/resources/2021/2/horizon-reports EPALE. E-Learning: https://epale.ec.europa.eu/en/tags/e-learning Preparation of the E-Learning Development Strategy 49 recommended Links Gartner. eLearning Hype Curve: https://webcourseworks.com/elearning-predictions-hype-curve/ DOCEBO (2019). E-Learning Trends, 2019: BusIness and https://www.docebo.com/resource/report-elearning-trends-2019/ EDUCAUSE. Horizon Reports: organIsatIonaL asPects 2 https://library.educause.edu/resources/2021/2/horizon-reports EPALE. E-Learning: of e-LearnIng PLannIng https://epale.ec.europa.eu/en/tags/e-learning 2.1 Preparation of the e-Learning development strategy 2.1.1 general aspects of strategic Planning Bringing e-learning into the offer of public or private educational organisa- tions requires a lot of money and changes in their operations. Not all the ac- tions and activities required for e-learning can be implemented overnight, but need careful thought and long-term strategic planning. Strategic planning is the process of defining the long-term goals of an organi- sation, formulating policies and plans to achieve them, and planning the re- sources required to implement these policies and plans. The main elements of strategic planning are the same whether an organisation is a public or a busi- ness entity, and regardless of the sector in which it operates.10 Strategic planning gives us a broad picture of the future of the organisation. The typical elements of a strategic plan are: mission, vision, objectives, values and strategies. The development of a strategic plan usual y starts with a reflection on the mis- sion of the organisation, which expresses the overarching expectations and be- liefs that guide the organisation’s performance. The definition of the mission is 10 Pučko defines strategic planning in a school or educational organisation as a sequence of discussions and decisions that relate to the fundamental, vital long-term issues of the organisation. (2005, pp. 29–30). Strategic decisions relate to the organisation’s mission and mandate, the range and level of services, financing, and managerial and organisational matters are in domain of key decision makers of the organisation. 50 Business and Organisational Aspects of E-Learning Planning usual y linked to concepts such as quality, user or customer satisfaction, open- ness, equal opportunities, customer relationships, the environment, employees and profit. At a more concrete level, the mission is expressed through the vision, which is defined by the organisation’s fields of activity and values, aimed at achieving its overall objectives. An organisation’s values determine how it behaves and relates to society, cus- tomers or service users, employees and other stakeholders. You can see how Doba Fakulteta, the largest provider of accredited higher education e-learning programmes in Slovenia, has defined its vision, mis- sion and values on its website https://www.dobabusiness-school.eu/why-doba/ our-vision-mission-and-values. Vision: Doba, the faculty that goes beyond. To become the leading school for online learning in SE Europe and an international champion and a school with the highest number of online students. To use our uniqueness, openness, and flexibility to become the bearer of change in innovative learning, developing current knowledge, and research for sustainable growth. Mission: Transforming challenges into opportunities with new pro- grammes and approaches. We employ innovative approaches and modern programmes to spread knowledge and connect all who believe in success. We are winning new markets and providing an excellent academic experience for the develop- ment of an agile, digital y competent and expert manager for the flexibil- ity of companies. Values: • Agility and change management. • Uniqueness. • Ethical values. • Innovation and development orientation. • Cooperation and col aboration. • Sustainable excellence. To achieve the strategic objectives, we develop long-term plans or strategies that set the overall direction of the organisation’s activities and actions over a longer period of time (ranging from 3 to 10 years). Figure 4 shows the strategic objectives of the UK Open University. Preparation of the E-Learning Development Strategy 51 Figure 4: strategic objectives of the uK open university Gro s w us t t h a i a n Excel ent n ab d ilit teaching and research y Student a success nd i c n u cl l u Technology Dynamic tu si r v e e that enables success Source: Open University, Strategies and Policies. https://www.open.ac.uk/about/main/sites/www.open. ac.uk.about.main/files/files/summary_strategic_Plan_to%202021_22.pdf The setting of objectives is based on thorough preparation, including a detailed analysis of internal and external opportunities and constraints on the imple- mentation of the strategic orientations. Educators in Europe are also increasingly aware of the importance of strategic support for the implementation of e-learning. The 2018 Chang- ing Pedagogical Landscapes study indicates that the number of higher education institutions with some form of technological strategy in educa- tion has increased significantly compared to 2015. Institutional strategies and strong management support for e-learning are typical of large uni- versities, while others are putting implicit strategies into practice with a bottom-up approach. Strategic plans and financial support from the state are important incentives. A lot of educational innovation comes from large open universities. The study provides a number of interesting exam- ples of institutional strategies for introducing technology into education (Henderikx and Jansen , 2018). In Slovenia, only a third of the surveyed higher education institutions had an ICT strategy for education in 2017, while a good fifth was preparing one (Bregar and Puhek, 2017). Technical y, the analysis for strategy development is often carried out as a SWOT analysis, which identifies the strengths and weaknesses of the organisation, as well as the opportunities and threats from the environment. PEST analysis is also a useful tool for analysing the external environment , systematical y examining external factors that are beyond the control of the organisation, but 52 Business and Organisational Aspects of E-Learning Planning that are essential to its performance and therefore relevant for strategic decision- making. The very name of the analysis presupposes that environmental impacts are considered for four domains (political, economic, social, technological). PEST analysis explores opportunities and threats in the external environment. Opportu- nities are positive circumstances that exist independently of the organisation, but that the organisation can turn to its advantage. Threats are understood as unfavour- able conditions or obstacles that may prevent an organisation from implementing its strategy. The results of a PEST analysis often open up new perspectives and op- portunities that could be overlooked by routine, experiential approaches. All areas of PEST analysis are relevant for the strategic planning of e-learning: political trends and government attitudes towards the introduction of modern technology in education and related legislation; economic characteristics, in- cluding competitor analysis, the growth in demand for services; social aspects, including demographic trends such as the ageing population, the purchasing power of the population, employment opportunities and, of course, techno- logical trends and the development of media and infrastructure. Some guidance on how to undertake a PEST analysis can be found on the website http://www.mindtools.com/pages/article/newtmc_09.htm. There are also various tools and examples available online on how to pre- pare a strategy plan, such as http://www.planware.org/strategicsample.htm. An example of a strategy plan for introducing e-learning in a private training company is shown below. 1. SWOT analysis Advantages: Restrictions: Quality management. Lack of cash resources. Support from most Untrained staff for e-learning. important users. Management has a poor Good ICT infrastructure. understanding of e-learning. Threats: Opportunities: Increasing competition. High growth rates in Reduced purchasing power education. Exportability of due to the recession. educational services. Prejudice and ignorance Possibility to enrich about e-learning. service offer. Preparation of the E-Learning Development Strategy 53 2. Vision The company will be a leading, business successful and international y recognised provider of e-learning services for SMEs in the region. 3. Mission The company is a leading innovator, facilitator of development and provider of e-learning services in the region. 4. Entrepreneurial values The company operates to the highest standards of business quality, e-learning and environmental protection. It encourages and rewards the creativity of its employees and implements the principles of the learning organisation. 5. Overall business objectives The company will achieve above-average profit margins in the busi- ness and/or increase its market share. It will become the leading pro- vider of e-learning services for SMEs in the region. 6. Operational business objectives The share of e-services revenues will reach 20% of the total revenues after two years of implementation of the e-learning programme, increasing at 10% per year over the next five years. The rate of profit growth will at least match the rate of revenue growth. Staff growth will lag behind revenue growth by 5 percentage points per year. 7. Activities to implement the strategy: Accelerate the development of e-learning services by strengthening development and research activities. • Strengthen cooperation with development centres at home and abroad. • Getting co-investors to develop e-learning. • Ensure that all staff, especial y leadership and management, are trained in the basics of e-learning. • Systematical y explore markets and introduce appropriate market- ing for new e-learning services. The authors of the article Quality Online Learning: e-learning Strategies for Higher Education (Piña et al., 2018, p. 13) point out that for e-learning to be successful and effective, it is important to: • consider quality as a key priority; • tailor e-learning to the requirements and specificities of the organisation; • invest in the design of e-learning programmes and the professional devel- opment of teaching staff; 54 Business and Organisational Aspects of E-Learning Planning • strategical y address educational technology and its providers; • harness the potential of the e-learning community. It is clear that the constant variability and unpredictability of the factors on which the strategy depends make its implementation quite difficult and unreli- able. The implementation of an organisation’s strategy cannot be definitively determined by an initial strategic plan, but it must be continuously monitored and adapted to the real situation and opportunities. The implementation of the strategy is il ustrated in the following figure. Figure 5: strategy implementation process Defining Developing the vision Setting a strategy Implementing Evaluation and and mission goals with activities the strategy actions Possible Possible Improvements/ Improvements/ revision revision changes changes Round two The strategic plan should therefore not be seen as an inviolable and unchange- able management commitment to which all decisions should be completely subordinated. It must be understood and used as a framework that guides actions and defines the boundaries of the organisation. Let us bear in mind General Eisenhower’s dictum that planning is everything, but the plan itself is nothing. The Strategic Plan is therefore a signpost for management to make the most important decisions; it brings employees together through shared values, expressed in a vision and mission, and it contributes to a common organisa- tional culture. Strategic planning is concretised by other, more short-term forms of planning (tactical and operational planning) and complemented by other management functions, i.e., organising, leading and controlling. More on this in Part 7 Man- agement in the Delivery of E-Learning. 2.1.2 strategic Planning aspects of e-Learning Tony Bates, a renowned expert in the use of technology in education, points out that the barriers to the successful uptake of e-learning are a lack of creative and innovative thinking and an awareness that the introduction of technol- Preparation of the E-Learning Development Strategy 55 ogy can bring about strategic change in education. A major investment in ICT should be a strategic decision, with clearly identified long-term objectives and an appropriate strategy to achieve these objectives. It is by no means sufficient to see the impact of using modern technology for education only in terms of improving particular educational programmes or courses (Bates and Poole 2003, p. 129). The creative and meaningful use of the benefits offered by the integrated use of technology in e-learning creates new opportunities for educators and us- ers. The various improvements that e-learning makes possible in education in principle stem from its main potentials, which are: • the spatial y and temporal y independent delivery of the learning process; • flexibility and diversity in the ways all actors in the education process com- municate and col aborate; • accessibility and openness of knowledge resources and flexibility in the choice of learning approaches and methods. As explained in Section 1.1, the integration of all these features into an e-learn- ing programme defines the notion of comprehensive e-learning. Organisations can use these characteristics as strategic advantages when de- signing their e-learning offer. Strategic advantages can be realised in a variety of ways, through different business strategies. These strategies essential y pur- sue two sets of objectives: • attract new target groups of learners by making education more open and flexible; • attract new audiences for a qualitatively different education. An international study on models for online open flexible technological- ly-enhanced higher education (OOFAT) identified five typical business strategies based on a survey of e-learning characteristics in 69 education- al organisations from all parts of the world (Orr et al., 2018): The fixed core model, where the organisation maintains the educational products and services and does not change its relationship with the users of the educational services (39% of the organisations surveyed). The outreach model, where the organisation maintains its products and services, but its innovations are directed at new target groups and com- munication channels (9% of the organisations surveyed). The service provider model, where providers focus on existing target groups and innovate in in the areas of products, services and communi- cation channels (6% of organisations surveyed). 56 Business and Organisational Aspects of E-Learning Planning The entrepreneurial model, where organisations use innovative strategies to transform products and services and to change audiences and com- munication channels (16% of organisations surveyed). The entrepreneurial model with a fixed core, where the organisation main- tains the traditional approach for the core business but focuses on inno- vation in other areas (30% of organisations surveyed). Business-oriented organisations are also driven by financial motives and aim to maximise their financial performance, such as maximising profits or max- imising the rate of return on investments. In practice, successful organisations tend to exploit the strategic benefits of e- learning simultaneously and in a coherent way, even if their relevance depends on the specific circumstances. The strategic objectives of organisations are also not necessarily offensive and aimed at attracting new target groups, but are often limited to a defensive strategy, i.e., to retain existing ones. For an educational institution or company, the implementation of e-learning is particularly recommended in the following circumstances (Fee, 2009, p. 30; Elkins and Pinder, 2015, p. 19): • a standard programme is needed for many users; • we have little time to implement the programme; • traditional classroom teaching methods are too expensive; • the learning content must be tailored to individual needs to a considerable extent; • coordination of multiple elements such as workplace activities, learning ac- tivities and assessment; • learners are expected to find solutions independently; • learners are more inclined to work and learn in a digital environment. Whatever strategy for the introduction of e-learning is adopted by an organi- sation, it is important to bear in mind at all stages of its implementation the financial, human and organisational implications on the one hand, and the pedagogical and didactical implications for the design and delivery of the pro- grammes on the other. Fee (2009, p. 36) recommends that before making a strategic decision to im- plement e-learning, it should be considered whether e-learning is appropriate in terms of educational needs, the learning and teaching strategies expected by the learners, the cost, the time aspect of delivery, and the pedagogical and other impacts of e-learning. CommLab India (2016) advises companies to explore readiness in terms of psychological, social, financial, human resources, content and environmental Preparation of the E-Learning Development Strategy 57 dimensions before implementing e-learning. If one of these aspects does not support e-learning, or even hinders it, this does not mean that the idea of in- troducing e-learning should be abandoned. This is a reminder to pay special attention to this aspect and try to manage the negative impact. Early experiences with e-learning have already shown that management support is crucial for the successful implementation of e-learning. A common charac- teristic of the organisations featured as examples of e-learning best practice in the study ”Learning Across the Enterprise: The Benchmarking Study of Best Practice” is that they had very high management support. ”In the absence of this support,” say representatives of these organisations, ”the e-learning project gets lost in a mass of shifting priorities and financial commitments. The intro- duction of e-learning needs to be justified to the management in the language and tactics of the business world.” (Brandon Hal , 2001). Once the strategic decision to introduce e-learning has been taken and a stra- tegic plan has been developed, the first step in implementing the strategy is an educational needs analysis (Section 3.3), followed by the instructional de- sign of an e-learning programme (Section 3.4). The programme design defines the essential elements of an e-learning programme. The design also needs to anticipate what equipment and software will be needed to run the planned programme. For the quality and effective planning of an e-learning programme, it is not sufficient to simply draw up a plan, as in a traditional curriculum, which is usual y limited to a description of the programme content, learning require- ments, core and supplementary literature, and the conditions for enrolment in the programme. Developing a plan (design) for how the e-learning pro- gramme will be delivered is much more complex: it needs to roughly define what learning resources or materials the learners will use and how they will access them, what kind of support/assistance will be available to them for self- directed learning and to what extent, what kind of learning activities and tasks they will be expected to complete in the programme, how they will communi- cate in the learning environment, how they will be tested on their knowledge and competences, and what tools and applications they will use. The decision on technological support must be subordinate to the objectives and needs of the e-learning programme and is therefore an integral part of the design. The business aspect of the design is operationalised through the business plan (Section 2.2). The design of the e-learning programme, the technological sup- port and the business plan are closely interlinked and interdependent, so it makes sense that the design and the business plan are developed as coherently as possible. 58 Business and Organisational Aspects of E-Learning Planning Figure 6: Implementing the e-learning strategy Strategic decision-mak ing for e-learning Strategic plan Educational needs analysis Designing an e-learning programme Business plan Development Business and or ganisa Implementation tional suppor t Evaluation The design is the basis for the development and implementation of the e-learning programme. The conducting of these two stages, which are pedagogical in nature, must be closely monitored and supported by the management and the profes- sional, technical and administrative services of the educational organisation. The issues that e-learning planners and practitioners need to deal with after the adoption of the strategy will be explored in more detail in the following sections of the book. Preparation of the E-Learning Development Strategy 59 recommended Links The Open University’s Strategy for 2022–2027: Learn and Live: https://www.open.ac.uk/about/main/sites/www.open.ac.uk.about.main/files/files/ learn-and-live-ou-strategy-2022-2027.pdf Athabasca University. Strategic Plan: https://imagine.athabascau.ca/ Mind Tools: https://www.mindtools.com/pages/main/newmn_str.htm Planware: http://www.planware.org/strategicplan.htm Doba Faculty. Our Vision, Mission and Values: https://www.dobabusiness-school.eu/why-doba/our-vision-mission-and-values 60 Business and Organisational Aspects of E-Learning Planning 2.2 creating a Business Plan 2.2.1 economic aspects of e-Learning types of costs in e-Learning Before we start discussing the economic aspects of e-learning, let us recall what we mean by the term e-learning (Section 1.1). Understanding costs depends on this. To summarise briefly: e-learning is a generic term that encompasses different forms and delivery models, which share the common characteristic that the use of technology is subordinated to pedagogical objectives to improve or modernise education. In order to address the economic aspects of e-learning, it is therefore essential to first define which e-learning delivery model the most important economic categories (costs, revenues, returns) relate to. The delivery model of e-learning has a direct impact on costs, and thus on the profitability and effectiveness of e-learning. To study and evaluate the economic aspects of education, it is first necessary to know the differences between the different types of costs.11 When looking at costs, it is also important to know the results, as the relation- ship between costs and value of output determines the effectiveness of the or- ganisation. Output in education is measured in different ways: by the number of graduates, by the number of credits achieved, or in terms of value (e.g., income generated, value added). Let’s first look at the main types of costs that are relevant to e-learning . Fixed costs are those that do not vary in the short term with the volume of activity and occur even in the absence of activity. E-learning is characterised by fixed costs, such as the cost of computer and software equipment and Internet access, the cost of developing e-learning programmes and the cost of producing learn- ing material. We also need to take into account what the e-learning model is. If we consider the costs of online education, then the costs of buildings and prem- ises for the delivery of the learning process as a category of fixed costs are virtu- al y non-existent (except for the working space for the administrative, technical and management staff). In blended learning, however, the cost of buildings and facilities (classrooms) can be significant. Variable costs vary directly and proportional y with the volume of activity. In e-learning programmes, these are usual y learning support costs, such as tu- tors and administrators. Such costs increase proportionately with the number of participants. 11 The definitions of costs are largely based on Curran (1990). Creating a Business Plan 61 Total costs are all the costs needed to produce a certain result. They are calcu- lated as the sum of fixed and variable costs. The average costs are calculated by dividing the total cost by the number of units of the outcome. Direct costs are those that can be directly linked to the results of an activity, module, programme or to a cost centre or system under study. The direct costs of a specific e-learning programme or course are, for example, royalties for the preparation of learning materials for a specific e-learning programme or course, tutoring support, the purchase of applications or tools for that pro- gramme or course, etc. Indirect costs are those caused by an activity, module, programme, cost centre or system in combination with other activities, modules, programmes, cost centres or systems. Such costs cannot be easily, accurately or cheaply measured for a single activity, module, etc. Indirect costs include the cost of the adminis- trative staff involved in several e-learning programmes or courses, the cost of annual training for all teachers, the cost of educational services marketing, etc. Capital costs are the costs incurred to purchase goods and services that typi- cal y have a lifetime of more than one year. These are costs that are incurred over a short period (for example, one year) but the product or service is used for several years. Typical costs of this type are the costs of computer equip- ment and purchasing licences. They are spread over several years in the form of depreciation. Running costs are incurred for goods and services that are consumed as they are purchased. These include salaries and other expenses of staff, fees of exter- nal col aborators, etc. Opportunity costs are the costs of reduced outputs that arise when employees, with the agreement of their employer, do something else during working hours (for example, receive training), rather than perform tasks related to their job. Opportunity costs are important for the employer. The marginal cost is the cost of one additional unit of output (e.g., an addition- al participant enrolled in an e-learning programme). For example, marginal costs show how much the total cost of running a programme increases if one more person attends the programme. cost of e-Learning in an educational organisation In addition to the basic, economic breakdown by basic cost types, the costs of e-learning can be presented in other ways. Considering the different levels at which e-learning costs are incurred we can distinguish: 62 Business and Organisational Aspects of E-Learning Planning • learners‘ costs (computer equipment and maintenance, Internet access and use, the printing of materials, attendance at live meetings, etc.); • the costs of the educational organisation (costs of developing and delivering e-learning); • employers‘ costs for e-learning for employees (cost of learning or on-the- job training, tuition or registration fees, travel expenses to attend live tutor- ing workshops, etc.); • costs at the national level (subsidising the use of the Internet for educational purposes, developing open educational materials and computer applica- tions, training teachers and others, developing infrastructure). In the remainder of this section, we will limit ourselves to addressing the issue of the cost of e-learning from the perspective of the educational organisation. Different authors present the costs of e-learning in different ways. The costs of e-learning can be shown by activity in the development and implementation phases of e-learning programmes. The ADDIE model (Defelice, 2018; Rac- coon Gang, 2019) can be used as a basis for such a demonstration. The ADDIE model will be presented in more detail in Section 3.1. Instructional Design Models for E-Learning. The presentation and analysis of the costs of e-learning mostly cover only the costs of developing and delivering the e-learning, excluding the costs of evalu- ating programmes. The costs in the development phase are as follows: • analysing educational needs; • designing an e-learning programme; • the preparation of learning and other materials (sets of questions for the assessment and other learning materials, preparing learning activities, etc.); i.e., the development of a programme in the strict sense of the word, in line with the ADDIE model; • preparing and setting up a digital learning environment. In the implementation phase, the costs are as follows: • administration, • pedagogical support and other types of support for learners, • communication, • testing and evaluation. In both phases (development and implementation), the technical infrastruc- ture and management must also be provided. Trained professional staff must be available to develop and implement e-learning programmes. Therefore, the cost of training these staff should not be neglected when planning or assessing or costs. And let’s not forget that once the learning material is ready, it needs to be updated or maintained, which also requires adequate resources. Creating a Business Plan 63 Costs can also be grouped by source of incurrence, depending on the activity that gives rise to the cost (e.g., the training of teachers for a selected e-learning programme, the cost of tutors‘ participation in the programme, etc.) Costs can also be broken down in a combined way, for example by phase and, within that, by source of incurrence. This approach was used in a comparative analysis of the costs and prices of traditional and online higher education in the US and Canada. The cost breakdown starts from the individual phases, and for each phase, more detailed categories are defined according to the source of the cost incurrence (Poulin and Straut, 2017). Table 4: types of costs by source of incurrence for the main phases of e-learning development and delivery Organisational Instructional Assessment Support for and technical design and student learning students and aspects of course delivery of the staff preparation course Accreditation Course Selection, acquisi- Student guidance specification tion and purchase and training of material and tools for assessment Technological support Instructional design Administration/ Staff training (LMS, integrated of course proctoring student databases, assessment teaching tools) Admission, Developing learning Verification of Maintaining the enrolment and materials student identity for library and other verification of assessment collections of student identity learning resources Selection, acquisi- Evaluate / grade Tutoring and tion and purchase assessment academic support of learning material Ensuring accessibil- Activities to ity and compliance reduce drop-out with the accessibility rates legislation for people with disabilities Delivering the Technical programme with assistance teaching staff or by other means Facilitation of group Academic activities counselling Source: Poulin and Straut, 2017, pp. 38–42 (adapted). 64 Business and Organisational Aspects of E-Learning Planning The way the costs are presented, classified and assessed depends on the context or purpose of the cost study, but also on what is meant by e-learning and what delivery model is chosen. In order to address the costs of e-learning in a meaningful way, we therefore first need to define the e-learning model for which we are examining the costs. In the following table, we present the results of the 2017 Association Talent Development survey on the time spent delivering one hour of training for dif- ferent levels of complexity of e-learning models (Defelice, 2018) according to the level of interaction and the presence of activities. The basic model of e- learning was defined in this analysis as asynchronous time-flexible learning. Table 5: average spending (in hours) per hour of training for different training modes, 2017 Type of training Number of Average time spent educational on development of institutions one hour of training surveyed Traditional education 136 38 The basic e-learning model 87 28 E-learning without interaction and with 87 42 limited activities E-learning with limited interaction and some 88 71 animation E-learning with complex interaction and 53 130 complex animation E-learning with real-time (synchronous) 21 143 interaction and complex real-life examples Source: Defelice, 2018. The research confirms that the time spent per hour on training and thus the labour costs increase with the complexity of the e-learning model. The low- est is in the basic model of online education, which offers not only spatial but also temporal flexibility. However, it is almost four times higher for e-learning, which, in addition to spatial and temporal flexibility, involves real-time inter- action and solving complex real-world problems. Having estimated the costs of e-learning, we also need to think about how and from which financial sources we will pay for it. So, we also need to estimate the revenue (usual y separately for the development and delivery of the e-learning programme). Creating a Business Plan 65 In doing so, we need to take into account the external context in which each organisation operates. A country’s education policy may be more or less stim- ulating or supportive of the introduction of e-learning (it may or may not fund the development of e-learning programmes from public funds), the prices of computer equipment and access to Internet services may vary, etc. Also due to the different external circumstances, we usual y start from a specific e-learning model when assessing and comparing the costs of e-learning. costs in e-Learning and traditional education Often, those of us who are looking to develop or deliver e-learning will be faced with the question of how much e-learning costs and how it compares to traditional education in terms of business efficiency. This is an issue of interest to the individual learner, the educational organisation and the employer, but also at the level of the national education system. There is no simple answer to this question. As we have already said, e-learning is a generic concept. This means that in practice, different models for the de- sign and delivery of e-learning programmes are possible, differing in a number of elements. The level of costs therefore depends on the quality of the services we offer to the participants in our programme. Carol Twigg, a US-based expert in transforming education through tech- nology, says that when asked what the cost of e-learning programmes is, the simplest answer is that it can range from $1,000 to $1 million. Different views on e-learning and different technologies among educational institu- tions are the cause of the wide variation in the costs and prices of e-learning products. For some, an e-learning programme is a simple video of a tradi- tional lecture, while for others it is the cost of creating a complex interactive animated video product, which may be just one type of learning material for an e-learning programme. Anyone involved in e-learning cost analysis needs to be aware of these differences (Poulin and Straut, 2017, p. 4). The intensity of the use of modern technology varies between different mod- els. Costs are higher if the learning material includes audio, video, simulations, animations and virtual reality in addition to text. However, it should be borne in mind that with the development of technol- ogy and the increasing availability of software and tools (which can be freely available), the preparation of e-learning programmes is becoming simpler and cheaper (see Section 4.5 Digital Tools for E-Learning). The structure and cost of e-learning programmes is also influenced by the the- oretical underpinnings of their design. If e-learning is based on a constructivist 66 Business and Organisational Aspects of E-Learning Planning concept, there will be more interaction with more tutoring support, and thus higher labour costs. However, if e-learning is based on the theory of connectiv- ism and open education principles, such as cMOOCs, the development costs will be significantly lower and implementation costs will be minimal (see Sec- tion 3.2 for learning theories and Section 6.2 for MOOCs). Due to the variety of starting points and the diversity of options in design and development, as well as the very different experiences involved, opinions on whether e-learning is cheaper than traditional education are quite diverse, often even contradictory. A 2017 survey by the WICHE Cooperative for Educational Technologies (WCET) found that 43% of 197 higher education organisations surveyed in the US and Canada consider the cost of online education to be higher than traditional education, 57% consider there to be no significant cost difference between the two modes of education, and none consider on- line education to be cheaper (Poulin and Straut, 2017, p. 5). The institu- tions surveyed consider online education to entail higher costs, especial y in terms of technology, training the teaching staff, the specification and design of programmes, verification of students’ knowledge and identity, and compliance with disability legislation (Poulin and Straut, 2017, pp. 39–42). The picture of the cost side of e-learning can be significantly altered by consid- ering opportunity costs. For example, the results of the ICT Integration in School Curriculum study showed that e-learning for teachers was more cost-effective than traditional teaching methods. Taking into account opportunity costs, the total cost of e-learning was only 59%, and 43% of traditional training on a per participant basis. If opportunity costs were not taken into account in the analysis, traditional education was more cost-effective (Jung, 2008). When comparing costs, we need to bear in mind that some costs are only in- curred in traditional education, others only in e-learning, and some costs are incurred in both forms of education. For example, training costs for teaching staff are rare or negligible in traditional education. Differences in technology- related costs are blurring as technology becomes more widespread. Creating a Business Plan 67 A similar picture of the relationship between the costs of a short e-learn- ing course (e-training) and traditional training is provided by a cost cal- culation made for a large Slovenian company with 140 branch offices. The calculation was based on the assumptions that the e-training is de- livered in two weeks using shorter digital learning materials with tutor- ing and technical support, while the traditional training is delivered with six classroom teaching hours, that in one year 380 employees from 137 branches attend one-day traditional training at 6 regional centres, and that the course remains unchanged for three years (Beguš, 2015). The total costs were estimated in two versions. Firstly, by taking into account the paid costs (actual costs) of developing and delivering both forms of training, and secondly, by taking into account the total costs, which not only include the paid costs, but also the opportunity costs for the work done during regular working hours. Calculations showed that, taking into account the actual costs, only 47% of e-training funding would be spent on traditional training in the first year. This is due to the high costs of setting up e-learning infrastructure . How- ever, over the three years, the actual costs of the two modes of delivery of the e-course are fairly even (e-learning costs are only 3% higher). The overall picture of the economic viability of e-training can only be obtained by taking into account the opportunity costs. These costs are particularly high for traditional training, accounting for as much as 75% of the cost of such training (mainly due to absenteeism, travel and subsistence costs for participants from off-site offices). Taking into account the total costs (including opportunity costs), the e-training requires 21% less resources over a three-year period compared to a conventional implementation. In addition to the financial aspects, other aspects need to be taken into ac- count when comparing the two training methods. Training in e-learning di- rectly contributes to increasing digital and communication skil s, contributes to a higher level of cooperation and integration between the company’s organisa- tional units (for example, between offices) and promotes the transfer of good practices between them. Such a training is also easier to organise, as there is no need to cover for absent trainees. Of course, these benefits cannot be real- ised without the support of the management and the proper preparation of the participants for the new format of training. It is also important to develop a quality programme and to provide adequate support to participants. 68 Business and Organisational Aspects of E-Learning Planning savings in e-Learning As we have shown in the previous section, e-learning is characterised by a different cost structure by type and importance (share in the total costs) than traditional education, and by the difference in the level of costs, depending on the specific characteristics of the delivery model. Artificial intelligence tech- nologies (AI and e-learning are discussed in Section 6.3) also offer consider- able opportunities for savings. Let’s take a look at the savings that e-learning can provide for an educational organisation. From the point of view of the educational organisation, savings in e-learning can be achieved in administrative matters (computerised administration, on- line enrolment and payment of tuition fees, etc.). In e-learning, most learning material are available online or in the cloud (digi- tal learning material), relieving the educational organisation the costs of print- ing, packaging, storing and shipping learning material (or broadcasting TV and radio programmes) that characterised correspondence education and, later, traditional distance education. In e-learning, printing costs have mainly been passed on to the participants, which increases their costs. The costs of developing digital learning materials are decreasing as more LMS and authoring tools are freely available on the Internet, saving time in develop- ing materials. In the future, we can also expect lower costs for the development of digital learning materials, due to increased standardisation and more quali- fied content creators. When developing digital learning materials, it is necessary to be familiar with the subject of the educational programme, to be able to apply in practice the principles and approaches of developing e-learning materials, and to be able to use an LMS or authoring tools. This is the subject of the fourth part of the book. All of this requires either already trained professional staff or training. The most expensive way is to ”start from scratch”. It is cheaper to have the learning content already written and adapt it. But it may be even cheaper to buy ready-made digital learning materials from another organisation. Due to the relatively expensive development of e-learning programmes and the high fixed costs, it is general y accepted that organisations can be cost-efficient with a small number of programmes and a large number of enrolments in each programme (Jung, 2008, p. 151). Economies of scale are therefore impor- tant for the efficiency of e-learning, reducing the importance of fixed costs in the overall costs. It is important that the technology and software solutions used are ” scalable”, i.e., adaptable in terms of scale. Creating a Business Plan 69 Savings in labour costs can be achieved by employing tutors, software devel- opers and computer experts on a part-time basis or by hiring external staff to carry out various e-learning activities. These solutions are particularly relevant for smaller numbers of programmes and/or enrolments. If the programme is developed entirely by one individual - for example, an enthusiastic univer- sity professor for his or her subject - the costs will be lower than if a group of experts is involved. Of course, we must not neglect quality. Developing e- learning programmes is a complex task, and it is unlikely that a high-quality programme can be developed by one or two individuals. High-quality pro- gramme development requires the highly organised, professional and col abo- rative efforts of a team. Open education, based in principle on free services for the user, is an excellent way to reduce the cost of e-learning. Open education is the subject of Section 6.2. For organisations offering e-learning programmes for formal education or using some form of e-learning for staff training, as well as for individuals in lifelong learning, OER and MOOCs are particularly relevant. However, it is important to be aware that the use of OER or MOOCs by organisations or in- dividuals comes at a cost. MOOC providers charge for additional services such as: certification, tutoring and support during the MOOC, tailoring the MOOC to the specific requirements of the client, the preparation of specific materials, offering refresher programmes, or the formal recognition of programmes, for example through the European Credit Transfer System – ECTS (BizMOOC, 2018). These costs are in addition to translation costs. The service of integrat- ing MOOCs into accredited online programmes is particularly profitable for providers, but still interesting for subscribers or users. Georgia Tech, Arizona State and the University of Illinois are integrat- ing MOOCs from Coursera, Udacity, or edX into their Master of Science and Master of Computer Science programmes. These programmes range in price from $7,000 to $17,000 – relatively affordable compared to tra- ditional comparable master’s programmes. This offer has attracted new audiences (Shah, 2017b). Savings in e-learning are also possible when organisations work together to jointly develop and share digital materials. A range of educational organisations (offering both, formal and non-formal programmes) around the world pro- duce learning materials in digital format every day. As the cost of developing is relatively high, educational organisations are working together to develop the material or are using programmes developed by umbrel a organisations. 70 Business and Organisational Aspects of E-Learning Planning The Food and Agriculture Organisation of the United Nations (FAO), e-learning Academy offers free access to content on a variety of topics of global interest, which is available in a range of formats, including e- learning courses for self-paced learning, blended learning programmes, MOOCs, webinars, online tutored courses, mobile learning, face-to-face training workshops and university master’s and postgraduate degree pro- grammes. FAO e-learning Academy involves partners in all phases of a learning project. The Academy development activities and initiatives are designed, de- veloped, delivered and adapted to different language environments. in col aboration with a wide range of partners. Partner institutions are uni- versities, academic institutions and research centres, non-governmental and civil society organizations, United Nations, development agencies and regional organizations, European Union institutions, and private sector and donors (https://elearning.fao.org/mod/page/view.php?id=4534). When looking for savings in e-learning, it is important to realise that they are not sensible and justified if they compromise the quality of the programme to such an extent that it is no longer acceptable or no longer meets the education- al needs of the learners. For example, costs in e-learning can also be reduced by reducing the number of tutors or increasing the number of learners per tu- tor. Reducing the amount of communication or interaction also reduces costs. However, this usual y means lower quality, lower learner satisfaction, higher drop-out rates (or lower enrolment) and therefore lower cost-efficiency and effectiveness of the educational programme. Organisations have some flexibility in deciding on the types of savings, but not to the extent that the quality of education is compromised. Their decisions regarding the choice of LMS or authoring tools, the type of media to use, the amount of time to develop digital material, etc. are also influenced by the re- source constraints. To manage costs successful y, organisations need to know the economic aspects of e-learning, as well as the factors that contribute to the quality of e-learning and learner satisfaction. Quality in e-learning is discussed in Section 7. 5. Creating a Business Plan 71 2.2.2 Business Plan in e-Learning definition of the Business Plan In a market economy, it is prudent for an organisation to assess its business idea as realistical y as possible by preparing a business plan before putting it into practice. Bringing an idea to life requires a certain investment of work and resources, and a business plan can go some way towards avoiding the risk of making the wrong business decision or identifying new, previously hidden business opportunities. Glas (1995) defines a business plan as a business preparation instrument in which all aspects of a business idea are examined – the product or service it- self, the market, technology, finance, management, financial projections of the business, etc. A business plan is a working document in which the entrepre- neur systematical y captures all the issues essential for successful business. A business plan is essential y the application of general business logic to a spe- cific business, where the entrepreneur needs to know what resources are avail- able, what their business objectives are, how and when they intend to achieve them, and what is important for the success of the business. Entrepreneurs prepare a business plan when they are starting a new business – a new company, a new business activity, restructuring or reorganising a com- pany, expanding existing production or offering new services. Organisations that are already involved in e-learning, or are planning to be, are also advised to prepare a business plan. The core business idea of e-learning can be either development or implementation of a new e-learning programme, or both. It is also possible that the business idea and the related business plan is limited to only one of the e-learning services, for example the development of multimedia interactive learning resources, or the development of self-assess- ment tools, a training and tutor support programme, etc. Business plans in e-learning thus differ mainly according to the scope of the planned activities or services to be included in the programme and the organi- sational aspect - whether the e-learning is to be introduced for a single pro- gramme, for a few programmes, for a department or for the whole organisation. In practice, this decision is usual y related to the e-learning experience that the organisation already has obtained. It makes sense to start introducing e- learning gradual y. If an organisation has no experience, it might first buy a programme developed by another organisation and start delivering it, or it might develop one programme itself and then pilot it. 72 Business and Organisational Aspects of E-Learning Planning reasons for Preparing a Business Plan The preparation of a business plan is an integral part of the planning process, with the ultimate goal of achieving the plan. Often the planning activities (re- flecting, discussing, researching and analysing) are even more useful than the final product, the plan itself. Planning compels us to be clear about what we want to achieve and how, where and when we will achieve it, even if we don’t need a formal plan. A well-developed business plan shows that you know your business and have careful y thought through about its development in terms of products, management, finances, market and competition. It is advisable to prepare a business plan for both existing and new organisa- tions, regardless of the size or type of activity, because of the range of the ben- efits. The reasons for developing a business plan in e-learning are the same as for any other business idea. There are many different risks involved in pursuing a business idea . Develop- ment risk means that it is questionable whether a functional product can be developed from an idea. Production risk arises from the fact that the condi- tions in regular production are different from those under which the prototype product was developed, so that the success of regular production may be ques- tionable. Market risk is related to the sale of certain products; it should be suf- ficient to justify their development and production. Managerial risk is whether we will make a profit from sales, and growth risk is whether the intended busi- ness is such that it will allow the organisation to grow. A business plan can be developed to attract investors to invest in a business idea for a completely new programme, or it could be part of internal entrepreneur- ship, when a group (e.g., technology experts, teachers, enthusiasts) tries to get the support of the leadership to implement an idea in a company or to set up a new organisation unit, for example for e-learning. how to Prepare an e-Learning Business Plan Before we start writing a business plan, we should first ask ourselves who we are writing the business plan for, who will read the business plan and who is expected to finance our business idea. For example, if a business plan is aimed at securing public funding, it will have a different focus than if it is aimed at, for example, a bank as a potential investor. The approach to developing a business plan and the content of the business plan depend on the type of organisation, whether it is developing and/or de- livering e-learning, the design of the e-learning programme, the subject of the educational programme, the purpose and objectives of the training, and exter- nal circumstances. Business plans also reflect the values of the organisation. Creating a Business Plan 73 The business plan of a market oriented educational organisation usual y focus- es more on making a profit, while a public institution’s business plan focuses on achieving excellence, creating new knowledge, reducing barriers to increas- ing access to education, etc. In e-learning, the structure of business plans cannot be prescribed in advance due to the diversity of e-learning programme formats and institutional frame- works. In the following, we will therefore only discuss the general features of developing an e-learning business plan in organisations. When preparing a business plan in e-learning, we follow the general principles of business plan preparation, which we adapt to the specific characteristics of e-learning. In accordance with the general y applicable steps for the preparation of a busi- ness plan, an e-learning business plan first presents the current situation of the organisation, then describes the planned idea or project (development and/or implementation of e-learning programmes) and defines the financial frame- work for e-learning, i.e., the planned costs and revenues. The business plan answers all the important questions about the planned ac- tivities needed to realise the business idea and that have financial or human resources implications. The initial part of the business plan summarises the elements that are typical of strategic plans. Some business plans are broader in scope and similar in structure to strategic plans. University Florida, 2019 – 2024 Comprehensive Business Plan, pub- lished on the website https://ufonline.ufl.edu/_wp/wp-content/uploads/2019/12/ updated_ufonline_BusinessPlan_2019-2024.pdf, also contains elements of a strategic plan (e.g. mission, vision) and analysis of the strategy realisation of the previous period. The preparation of a business plan in e-learning requires strategic orientations and related information, which are also discussed in Section 2.1 Preparation of the E-Learning Development Strategy. The preparation of the business plan is largely based on the information we have already gathered for strategic plan- ning and for designing the e-learning programme. The business plan is prepared in several stages. In each successive stage, we refine the business plan, gradual y eliminating unpromising ideas and adding details. A team is usual y involved in the preparation of a business plan , as an individual rarely has all the knowledge and competencies needed to prepare a business plan. We can also hire external experts for specific specialised do- mains. 74 Business and Organisational Aspects of E-Learning Planning The preparation of the business plan is carried out by outlining the structure of the business plan (the most important elements), identifying the team mem- bers who will write and/or research specific domains (finances, marketing, technical aspects, pedagogical aspects, information aspects, etc.), by estimat- ing the time and resources needed to complete each task and final y, by esti- mating total costs and financial resources. There are no prescribed or uniform rules on the elements of a business plan, only general rules. There are a number of manuals for drawing up a business plan, which also differ according to the type of activity (e.g., manufacturing, trade, catering, education, etc.). However, while there is some flexibility, busi- ness plans should cover all the most important elements that an investor or company’s leadership needs before deciding on a proposed business idea. Several portals are available on the web, offering different types of information and services for preparing business plans, tools for preparing business plans, tools for testing business ideas and business plans, examples of business plans already prepared for different types of activities, and one-to-one advice. Some of the information and services that facilitate entrepreneurs’ work are also available free of charge. The elements of the business plan are described and presented on the website PlanWare (http://planware.org/businessplan.htm). This website of- fers a range of free information, computer tools and guides for preparing business plans. Computer tools for assessing business ideas and evaluat- ing marketing strategies are available for a fee. The website also provides an outline of a short strategic plan, which is the basis for the preparation of the business plan. Useful tools for preparing the elements of a strategic and business plan are available on the Mindtools website https://www.mindtools.com/pages/ main/newmn_str.htm. An example of how to prepare a business plan for the development of an online programme is provided by Minnesota Uni- versity Extension (https://fyi.extension.wisc.edu/ncrvd/files/2015/04/BusinessPlan-develoment-for-online-learning-Kr-2017-002.pdf). Creating a Business Plan 75 Table 6 gives an example of an e-learning business plan with its essential ele- ments. Table 6: a sample example of the elements of a business plan in e-learning12 Summary: • a brief introduction to the organisation and its references, which e- learning programmes the organisation intends to develop or implement, when and why, and who the programmes will be aimed at. Background (starting point) of the project: • a description of the organisation’s activities and credentials (a more de- tailed description of the organisation’s activities and comparative advan- tages and its position in a region); • the organisation’s vision and mission; • the organisation’s e-learning objectives (which can be presented together with strategic objectives12, operational objectives and expected results); • a description of the educational services (a more detailed description of the e-learning programmes, their objectives and positive impacts); • operationalisation of the objectives (a detailed description of how the ob- jectives will be achieved – for example, the number and type of tradition- al educational programmes that will be adapted for delivery as e-learning, the number of employees in the organisation that will need to be trained in the various e-learning activities, the number of external col aborators and their tasks, etc.). Marketing plans: • the market (description of the state of the market for educational services – which educational programmes are offered by competitors and at what price, expected market demand for our e-learning programmes, charac- teristics of our target groups); • cooperation with other organisations (other educational organisations and the business sector); • the promotion of e-learning strategy and methods. 12 For example: strategic objectives – to provide access to education for population groups that cannot attend traditional forms of education; operational objectives – to develop and implement e-learning programmes for specific subject fields; expected results – number of planned or enrolled learners. 76 Business and Organisational Aspects of E-Learning Planning Action plans: • services for learners (for example, tutoring, administrative and computer support, access to these services – where, when and how); • staffing conditions and requirements for e-learning (staff qualifications, types and number of staff needed for e-learning); • expected risks and how to address them (for instance, the risk of lower enrolment due to competition is addressed by better quality and lower price, the risk of unauthorised access to the e-learning programme can be prevented by ensuring that only enrolled participants have access to the e- learning programme based on an assigned username and password); • the risks of technology (for instance, what does it mean to decide to use a particular LMS, what problems might be expected); • management (project management of e-learning development, manage- ment of e-learning delivery). Financial projections: • the estimated e-learning development costs and planned financial re- sources; • the estimated costs of implementing the e-learning programmes, the pro- jected revenues and the break-even point calculation; • financial projections for several years ahead (projected costs and reve- nues by year for a certain period of time, for example three or four years); • the prospects for the sustainability of the e-learning programme (the pro- gramme should have a promising market perspective, so that it can be implemented in the future and pay for its costs). revenue and cost Planning E-learning programmes aimed at adults are often sold on the market (treated as ’educational services’), so the business aspect of these programmes is par- ticularly important, as the survival of the organisation may depend on it. Below we present a simple example of cost and revenue planning for e-learning and break-even point calculations, which can be one of the building blocks for the preparation of a business plan in an educational organisation. A detailed description of the planned activities and cost estimates by activ- ity can be used as a source of information for the preparation of the financial plan, which is an integral part of the business plan. It is important to separate start-up (initial) costs from running costs and to estimate them separately. In the next step, we determine how long it will take to cover the initial investment costs (if we have not received a grant to cover the initial costs), how much Creating a Business Plan 77 funding we need per year for this purpose and the sources of this funding. One way of preparing an e-learning financial plan is to start by preparing a financial plan for one particular module/programme and then repeat the process for other modules/programmes. For a simple explanation of the break-even calculation, where costs equal revenues, see the following video (http://www.youtube.com/ watch?v=sg4YiJZsuI8). Below is a simplified example of a break-even point calculation for an e-learn- ing programme. Let’s say we are preparing an e-learning programme. For the sake of il- lustrative purposes, we assume that in one academic year, we only have the costs of computers and software of €10,000 (as fixed costs) and the cost of tutors (€50 per participant) as variable costs. The planned tuition fee is €100 per participant. We want to know how many participants are needed for the annual costs to equal the revenues, so we want to know the break-even point when we have neither profit nor loss. The break-even point is calculated by dividing the fixed costs by the dif- ference between the tuition fee (i.e., the price of the service) and the vari- able costs per participant. Break-even point = fixed costs / (tuition fees – variable costs per partici- pant) Break-even point = €10,000 / €100 per participant – €50 per participant = €10,000/€50 per participant Break-even point = 200 participants If we estimate that we will not be able to achieve the planned number of participants, and we assume it is realistic to expect, for example, 160 participants, we can easily calculate how much the tuition fee would have to be increased to reach the break-even point with 160 participants. The required tuition fee is calculated by dividing the total cost (fixed and vari- able costs for 160 participants) by the expected number of participants (i.e. 160 participants). Tuition fee = (fixed and total variable costs / number of participants Tuition fee = (€10,000 + €50 per participant × 160 participants) / 160 participants = €18,000 / 160 participants Tuition fee = €112.50 per participant 78 Business and Organisational Aspects of E-Learning Planning Of course, raising tuition fees is the simplest solution. A more viable solu- tion would be not to increase the tuition fees and to find cost reserves (not at the expense of quality, of course) or to increase revenues through appropriate marketing and other activities. The article by Mantzari and Economides at https://www.researchgate.net/ publication/27381325_cost_analysis_for_e-learning_foreign_languages presents a rigorous and systematic model for analysing the costs of e-learning for foreign language courses and shows the calculation of the break-even point considering different options of variable costs. recommended Links University Florida, 2019 – 2024 Comprehensive Business Plan: h t t p s : / / u f o n l i n e . u f l . e d u / _ w p / w p - c o n t e n t / u p l o a d s / 2 0 1 9 / 1 2 / updated_ufonline_BusinessPlan_2019-2024.pdf Planware: http://planware.org/businessplan.htm Mindtools: https://www.mindtools.com/pages/main/newmn_str.htm University of Minnesota. Business Plan Development for Online Learning: https://fyi.extension.wisc.edu/ncrvd/files/2015/04/Business-Plan-develoment-for- online-learning-Kr-2017-002.pdf How to Determine Breakeven Point: http://www.youtube.com/watch?v=sg4YiJZsuI8 Instructional Design Models for E-Learning 79 Of course, raising tuition fees is the simplest solution. A more viable solu- tion would be not to increase the tuition fees and to find cost reserves (not at the expense of quality, of course) or to increase revenues through appropriate marketing and other activities. The article by Mantzari and Economides at PedagogIcaL asPects https://www.researchgate.net/ publication/27381325_cost_analysis_for_e-learning_foreign_languages presents 3 a rigorous and systematic model for analysing the costs of e-learning for of e-LearnIng PLannIng foreign language courses and shows the calculation of the break-even point considering different options of variable costs. recommended Links University Florida, 2019 – 2024 Comprehensive Business Plan: 3.1 Instructional design models for e-Learning h t t p s : / / u f o n l i n e . u f l . e d u / _ w p / w p - c o n t e n t / u p l o a d s / 2 0 1 9 / 1 2 / updated_ufonline_BusinessPlan_2019-2024.pdf Planware: http://planware.org/businessplan.htm 3.1.1 Instructional design Mindtools: https://www.mindtools.com/pages/main/newmn_str.htm In Section 1.1 What is e-learning, we noted that e-learning is manifested in real- University of Minnesota. Business Plan Development for Online Learning: ity through a multitude of different implementation models. What they have in common is the effective use of technology to support better and more innovative https://fyi.extension.wisc.edu/ncrvd/files/2015/04/Business-Plan-develoment-for- education. A number of factors influence the design of e-learning programmes, online-learning-Kr-2017-002.pdf with pedagogical considerations at the forefront. The theoretical framework for How to Determine Breakeven Point: how to approach the development of an educational programme13 from a peda- http://www.youtube.com/watch?v=sg4YiJZsuI8 gogical perspective is provided by the so-called Instructional Design Theory. Instructional design theory addresses questions of how to design educational programmes for different contexts so that the programme objectives and specific learning goals are achieved (Reigeluth and Carr-Chellman, 2009). This includes theoretical reflection on what the programme should look like in terms of the learning and teaching strategies, methods, activities, etc., and also what proce- dures are needed to make it happen. Instructional design theory therefore covers the planning of essential elements of an educational programme. In addition, this theory includes the planning of all the processes necessary for the successful preparation and delivery of the programme, i.e., analysis, conceptual and techni- cal design, development, implementation and evaluation. Instructional design theory provides a general framework for the preparation of e-learning programmes. A further step towards practical application is the so-called Instructional Design Models (IDMs), which help planners concretise 13 We use the term ’educational programme’ as a general term that can refer to, for example, an university course, a non-formal education course, an on-the-job training programme or the preparation of a MOOC, etc. 80 Pedagogical Aspects of E-Learning Planning theory and apply it in practice. The IDMs provide guidance on the content and structure of the programmes. In other words, IDMs guide programme designers in planning the essential features of a programme and all the procedures nec- essary to ensure that the programme will achieve its objectives. Quality IDMs are derived from the postulates of theories of learning. Theories of learning are discussed in the next section. Nowadays, a variety of different models are available for educational program- ming. Donmez and Cagiltay (2016) identified 33 different IDMs based on a literature review for the 2000–2016 period. These models are classified into three basic groups according to their complexity: simpler, classroom-oriented models; more complex, product-oriented models; and complex, system-ori- ented models. Simsek (Simsek in Donmez and Cagiltay, 2016) considered models in terms of their structure and classified them into the following groups: basic models, linear models, flexible models, interactive models and heuristic models. 3.1.2 Instructional models for e-Learning IDMs have made a special mark in e-learning. This is because e-learning pro- grammes are much more complex than traditional programmes. Managing complexity requires a systematic and consistent approach to programme de- velopment and delivery, and IDMs are a tool for this. In e-learning, the most commonly cited IDMs are: the ADDIE model (Analy- sis, Design, Development, Implementation, Evaluation Model), the Dick & Car- ey model, the nine-stage Gagne model, the SAM model and the Agile Planning model. addIe model Among these models, the ADDIE model is undoubtedly the most established and it is considered by some to be a generic model for instructional design (Donmez and Cagiltay, 2016) or as a paradigm for the development of edu- cational products, linking inputs, processes and outputs (Branch, 2009, p. 3). The ADDIE model is underpinned by performance-based learning, which is learner-centred, authentic and motivational (Branch, 2009, p. 2). The ADDIE model was developed in 1975 at Florida State University, Cen- tre for Educational Technology, for the educational needs of the military. The model was partly based on the five-stage model developed in 1970 for the de- sign of training for military aviators in the USA (Branson et al., 1981). Instructional Design Models for E-Learning 81 The ADDIE model consists of five interrelated steps or stages (Bates, 2016, p. 134; Branch, 2009, p. 3): • analyse: analyse the characteristics of the learners, their current knowledge and identify their learning needs, the resources available, the overall objec- tive and the framework of the content; • design: select an effective learning and teaching strategy and define the learning objectives of the programme, for example, what the content will be, what media will be used for each piece of content, what technologi- cal support will be used, decision on the type of digital learning environ- ment, videos and social networks; specify the elements of the designed programme, organise these elements into a coherent whole; • develop: provide appropriate learning materials according to the content and learning objectives, including the preparation of other elements of the educational programme (e.g., learning activities, assessment tasks and questions, etc.). We can develop learning materials in-house or have them produced elsewhere; design a digital learning environment; clarify copy- right issues; embed content on a website or in a LMS; • implementation: the actual delivery of an e-learning programme, including training or information for staff to support learners and evaluation; • evaluate: collect feedback to identify areas for improvement and use it in the next iteration of programme planning based on the ADDIE model. Figure 7: Basic stages of the addIe model Analysis: analysis of the educational needs and characteristics of learners; definition of the basic objective of the programme Evaluation: with Design: recommendations defining learning for improvement and teaching strategies and specific learning objectives Implementation: Development: with assessment, developing learning staff training material and digital learning environments Source: Adapted from Branch, 2009. 82 Pedagogical Aspects of E-Learning Planning The ADDIE14 model is a very good management tool and has become almost standard in the development of e-learning programmes. It has been and contin- ues to be used by many open universities with many students, as well as by large corporations. The use of this model is suitable for managing projects of varying sizes, and it is particularly recommended for large and complex projects. One of the reasons for the outstanding success and popularity of the ADDIE model is the systematic and consistent application of the principles of quality programme design. This means careful design planning with clearly defined learning objec- tives on the one hand, and on the other hand, programme elements subordinate to these objectives: careful y structured content, appropriate learning activities and assessment methods, control of the workload of learners, teachers and other learners, the use of various media and technological support. The quality and detail of the educational programme envisaged by the ADDIE model has also been the subject of considerable criticism. Constructivists criticise the model for focusing too much on design and devel- opment and not enough on the interaction between teachers and learners, as well as for promoting a behaviourist approach to teaching. Improvements can only be made on the basis of iterations after the cycle has finished, i.e., after evaluation. The model is unidirectional and acts like a wa- terfall. Some authors consider that the phases of the model are too detailed, which can stifle creativity (CommLab India, 2018; Allen, 2012). The model also does not provide guidance on how to make decisions for in- dividual stages, for example how to choose between different technologies or which assessment strategies to use. In practice, the application of the model can lead to overly complex projects requiring different profiles of skilled staff and extensive division of labour. Such projects take a long time from approval to programme implementation (up to two years). The more complex the approaches, the greater the chance of the initial y planned costs being exceeded and the lower the chance of errors being corrected. Despite much criticism, ADDIE is still alive and wel , and new versions of the model are being developed. One of these is PADDIE, where planning and preparation are added to the beginning of the model (Bates, 2016, p. 136). 14 Similar to the ADDIE model, the phases of the andragogical cycle are defined (Andrilović et al., 1985, p. 106). In adult education, the so-called interactive model of instructional design has recently gained ground (Caffarel a and Ratcliff Daffron, 2013). Instructional Design Models for E-Learning 83 A detailed video tutorial on how to create a simple course using the AD- DIE model can be found at https://www.youtube.com/watch?v=vsLv2KdYz9a. dick & carey model A fairly well-established version of the ADDIE model is the Dick & Carey model. This model is based on the assumption that all the elements involved in the teaching process are interconnected and work together to achieve the desired results. It is about the teacher, the learners, the learning materials and resourc- es, the activities and the delivery system. The model is characterised by the fact that learning objectives are defined for more narrowly defined units and that the design seeks to provide appropriate opportunities for the achievement of these individual objectives. The model contains ten steps: • identify the learning objectives; • identify the skil s, knowledge and attitudes to be achieved; • carry out an analysis to identify what needs to be learnt in order to com- plete a task; • analyse the learners and the context, the circumstances of the learning; • develop assessment tools; • develop an instruction strategy; • develop and select; • prepare and conduct ongoing (formative) evaluations; • revise elements of teaching; • prepare and conduct final (summative) evaluations. 84 Pedagogical Aspects of E-Learning Planning Figure 8: Instructional design with the dick & carey model Revise Instruction Conduct Instructional Analysis Identify Develop Develop Write Develop Develop Instructional & Select & Conduct Performance Criterion Instruction Goals Instructional Formative Objectives Referenced Strategy Tests Materials Evaluation Identify Entry Develop Behaviors & Conduct Summative Evaluation Source: Educational Technology (https://educationaltechnology.net/dick-and-carey-instructional-model/ ). However, compared to the ADDIE model, the Dick & Carey model provides for a certain degree of interactivity with ongoing evaluations in each planning cycle, and also describes the individual phases and their associated procedures more concretely, especial y the initial analysis phase. the nine-stage gagné model Even more detailed planning is foreseen in the model developed by Robert Gagne. This model is narrower and focuses mainly on the effectiveness of the interaction between the teacher and the learner in a concrete lesson, which consists of nine sequential steps. Some also call it the nine events of instruction model. At its heart are lesson planning and the learner. These learning events provoke a response in the learner that leads to learning. Using this model, the teacher can systematical y prepare a teaching plan. However, there is a fear that teachers might find this model too rigid. The levels of the Gagné model, with responses from learners, are: • gain the attention of the participants – reception, • inform the learners about the objective – expectation, • stimulate recall of previous knowledge –retrieval, • present information – selective perception, • providing guidance – semantic coding, • elicit performance – responding, • giving feedback – reinforcement, Instructional Design Models for E-Learning 85 • assessing learning performance – activate retrieval, • enhancing retention and learning transfer – providing cues (CommLab In- dia, 2018; Branch, 2009). The model was quite useful in the past, but it is too inflexible and predeter- mined for today, as the environment changes very fast: new content is being developed rapidly, new technologies or new applications emerge, etc. (Bates, 2016). successive approximation model The Successive Approximation Model (SAM) and the Agile Planning Model are better suited to today’s needs. The SAM model has three phases: a preparation phase, an iterative design phase and an iterative development phase. It enables the rapid development of e-learning programmes through a series of iterations, which are carried out in iterative reviews of the design, development, implementation and evaluation phases. Any change can be implemented immediately, saving time and money. The model differs from others in that the preparation phase only involves the collection of relevant material, but not a sophisticated analysis of the content of the material collected. Another characteristic of this model is that several phases can run simultaneously. Iterations or improvements to each phase are implemented as soon as necessary (CommLab India, 2018). the agile Planning model In the last few years, new approaches such as agile design models have emerged, spurred by new technological opportunities. The English term ”agile” is also the acronym used to describe the essential steps of the model: align, get set, iterate & implement, leverage, evaluate. 86 Pedagogical Aspects of E-Learning Planning Figure 9: elements of an agile planning model AALIGN G I L E GET SET ITERATE & LEVERAGE EVALUATE IMPLEMENT Source: Adapted from Dick et al., 2005. A key feature of the agile planning model is its flexibility and adaptability to circumstances. Teachers today face rapid changes, such as the rapid develop- ment of new content, technology or applications, changing target groups, and the demands of developing the knowledge and competencies needed for the 21st century. They have to cope with the circumstances of today’s world, which are volatile, uncertain, complex and ambiguous (VUCA). The agile design model should be distinguished from rapid instructional design or rapid pro- totyping, both of which follow the processes of the ADDIE model in a more condensed form. In an agile planning model, usual y only a small number of people (one or two) are involved. When using these models, we try to make the most of the potential of new tools or software. For example, the sub-objectives of such pro- grammes may change or new capabilities may be developed in learners each year as technology changes to enable the development of new products or ser- vices. The focus is not on doing the same thing with new technology, but on trying to achieve new and different results with technology that is more rel- evant to the digital world. Bates (2016, p. 64) compares the ADDIE model to a 100-piece orchestra and the agile planning model to a jazz trio gathered for a single concert. The main advantage of the agile design model is that it focuses directly on prepar- ing learners for a rapidly changing, uncertain, complex and ambiguous (VUCA) world. This approach al ows programmes to be developed and implemented much faster and at a much lower initial cost compared to the ADDIE model (Al- len, 2012; Bates, 2016). Instructional Design Models for E-Learning 87 The following table shows the main differences between the ADDIE model and the Agile Planning model. Table 7: comparison of the addIe model and the agile planning model ADDIE Model The agile planning model Stages Analysis Align Design Get set Planning Iterate and implement Implementation and evaluation Leverage and evaluate Features of the models Focus on content (with measurable objectives Learner-centred (learner-centred active and related achievements and activities at the engagement and interaction with the learning forefront). content). Linearity (progression in stages with no Non-linearity (flexible approach, allowing evaluations in each cycle). Less cost-efficient for cooperation, flexibility and ongoing because corrections can only be made after programme updating). More cost-efficient due the fact. to the possibility of replenishment on the fly. Source: Morgan, 2016. The ADDIE models have clearly proved to be an appropriate basis for the de- sign of e-learning programmes in the early stages of development character- ised by the use of LMS and Web 2.0. However, the personalisation of learning as a twenty-first century imperative cannot be achieved with the rather rigid models of the ADDIE family. It is to be expected that planning will continue to be driven by the quality requirements of the programmes, but the implementa- tion pathways will be quite different. Learning analytics, artificial intelligence, intelligent learning planning and intelligent learning content design are meth- ods that will make significant inroads into traditional programme planning in the future (Vipond, 2017). recommended Links Instructional Design.org: https://www.instructionaldesign.org/ Educational Technology: https://educationaltechnology.net/teaching-and-learning/ eLearning Industry. Instructional Design Models and Theories: https://elearningindustry.com/instructional-design-models-and-theories 88 Pedagogical Aspects of E-Learning Planning 3.2 theories of Learning Learning theories are an indispensable basis for designing e-learning pro- grammes, as they help us to understand the learning process and guide us in the selection of appropriate learning methods, learning materials, learning activities, pedagogical support, assessment and evaluation, according to the programme’s objectives, taking into account the specific circumstances. MOOC Instructional Design Foundations and Applications, published on Coursera https://www.coursera.org/learn/instructional-design-foundations- applications, justifies the use of learning theories in the design of e-learning programmes with the following. Learning theories help to answer the essential questions that an instructional designer faces before starting a programme. How do we learn? What triggers the learning process? What hinders the learning process? What is the role of memory in learning? How is knowledge transferred from short-term memory to long-term memory, or from one context to another? Which practical questions, im- portant for designers, are not answered by learning theories? Looking at how people learn, or understanding the learning process, should be the starting point for how to design e-learning programmes, and also for how to use different types of technology in learning and teaching. There are several theo- ries of learning. In this handbook, we will focus first on the basic theories of learn- ing that are most often mentioned in the literature in relation to e-learning. These are behaviourism, cognitivism and constructivism (Bates, 2016). Next, we intro- duce two theories that were stimulated by the rise of e-learning less than twenty years ago: the theory of connectivism and the theory of Community of Inquiry. Let us also briefly recall the theory of heutagogy and the pedagogy of abundance, which have emerged in the last decade as a result of the further development of technology and the changed social circumstances and educational needs. Greg Kearsley and Richard Culatta from Instructional Design.org (https:// www.instructionaldesign.org) give an overview of different learning concepts and teaching methods, including more than 50 different theories of learning or teaching. 3.2.1 theory of Behaviourism Behavioural theory is based on the assumption that a certain stimulus will elicit a certain response (such as: when light increases, the pupil constricts), and rejects mental activity as a basis for learning. In behavioural theory, an indi- Theories of Learning 89 vidual is born with a relatively limited pattern of behaviour, and through con- ditioning and imitation, new and increasingly complex patterns of behaviour emerge. In this way, the individual’s personality is gradual y shaped as a result of conditioning, learning and experience. An individual’s personality is there- fore shaped not so much by instinctive, motivational and dispositional fac- tors, but rather by the environment and learning. The most prominent authors of this theory (Watson, Thorndike and Skinner) have argued that learning is determined by external factors that help to reinforce an individual’s behav- iour in the expected way (Bates, 2016). A reward or punishment can be used to reinforce the link between a stimulus and a response or reaction. Skinner developed this fundamental law of learning into the concept of operant condi- tioning. Unlike classical conditioning, operant conditioning is more complex, explaining how an individual learns more complex, intentional actions rather than just reflexes. As Bates (2016) points out, this theory is based on the as- sumption that human behaviour is predictable and can be controlled. We can see the reflection of behavioural theory in the development of learn- ing machines, measurable learning objectives and computer-assisted instruc- tion (CAI). Until recently, computers were thought to be closely linked to behavioural learning. Although behavioural theory has been discarded in ed- ucational psychology and didactics due to its simplistic view of individual de- velopment, recent approaches in e-learning, such as personalised learning and intelligent tutoring systems (ITS), have, according to some authors, brought it back to prominence in e-learning (Emerich, 2018). In the Skinner’ s Box video you can watch Skinner’s research approach to the be- havioural theory of learning. https://www.youtube.com/watch?v=mogowry2Wc0. In the video https://www.youtube.com/watch?v=jth3ob1Irfo Skinner presents his visionary teaching machine and its benefits. 3.2.2 cognitive theory The individual is not just a mechanical product of the environment, but an ac- tive participant in the learning process, deliberately trying to process informa- tion from the outside world. Cognitive theorists believe that mental processes – internal and conscious representations of the world – are essential for learn- ing. According to this theory, human beings are first and foremost creatures who learn about, explore and interpret the world in which they live. According to Carver and Scheire (2012), the cognitive conception of personality is based on three assumptions: 90 Pedagogical Aspects of E-Learning Planning • Human behaviour can be understood by understanding how humans pro- cess (receive, store and integrate) the information they receive about them- selves and their environment. • An individual’s life is a complex web of decisions that are shaped by the processing of information. • Our behaviour is implicitly goal-directed, self-directing (self-regulating) – the decisions we make lead us in a programmed direction, fulfilling the goals we have set for ourselves. Cognitive psychological theories are therefore primarily concerned with the processes of decision-making, goal selection and behaviour in which people are willing to invest their energy, curiosity and activity. The concept of cognition is extremely broad, encompassing the various beliefs, expectations, goal prob- abilities, attributions, etc., that influence an individual’s behaviour. In contrast to some earlier motivational theories (e.g., psychoanalysis and behaviourism), which sought to explain the totality of human action, cognitive motivational theories only focus on a specific aspect of human motivation (self-efficacy, per- formance motivation, beliefs, etc.). 3.2.3 theory of constructivism The most important point in constructivist theory is that learning is conceived primarily as a social process, requiring communication not only between the learner and the teacher, but also among learners, colleagues, friends, etc. This social process cannot be replaced by technology, but it can be facilitated by it. The social context of education is the most important aspect for many educa- tors. Knowledge is acquired through social processes or organisations created by society. Even knowledge that is labelled ”valid” is a product of social con- struction. Knowledge is therefore not only content but also an expression of values, and it is therefore necessary to continuously explore it and be critical of the knowledge provided (Bates and Poole, 2003). The individual makes a conscious effort to understand his/her environment in the light of past experi- ences and his/her current situation. This process is done through reflection. Learning based on constructivist theories is characterised by the teacher as a kind of assistant or mentor, helping the learner to learn, and communication between the two. In the constructivist approach, one of the most important forms of pedagogical support is group discussion. Over the last century, understanding of the learning process has advanced considerably. The early twentieth century was characterised by a behavioural theory that corresponded to the industrial age. In the mid-twentieth century, the shortcomings of behavioural theory led to the development of two new Theories of Learning 91 theories of learning – cognitive and constructivist. Much research has been conducted and the findings can be used to guide decisions on how to design learning activities and approaches to effectively engage learners (Siemens and Tittenberger, 2009). The three main theories of learning – behavioural, cognitive and construc- tivist – are all based on the idea that knowledge is a goal or a state that can be achieved through thought, experience and how one acquires knowledge or learns. They are concerned with the learning process, not with the value of what is learned. However, with the increasing volume of knowledge and information in modern society, it has become important to evaluate knowledge quickly, as we need to act (and act quickly) on information that is outside our primary knowledge. Theorists try to modify theories when circumstances change. At a certain point, however, when the situation has changed significantly, further modification of these theories is no longer meaningful (Siemens, 2005). Sie- mens’ response to the changed circumstances in which learning takes place today is the theory of connectivism. 3.2.4 theory of connectivism George Siemens and Stephen Downes (Siemens, 2005; Downes, 2007) have developed a theory of learning that is suitable for the digital age called con- nectivism, which aims to overcome the limitations of behavioural, cognitive and constructivist theories. Downes described it as: ”the thesis that knowledge is distributed across a network of connections, and therefore learning consists of the ability to construct and integrate these networks” (Downes, 2007). Connectivism is therefore based on the assumption that knowledge is distrib- uted across human, social and technological networks, and that learning is about connecting, growing and managing these networks. Learning can be de- scribed as ” networked learning” at three distinct levels, namely: • the nervous system level: The formation of connections in the nervous sys- tem triggered by new stimuli and experiences. Research shows that connec- tions and networks are central to the formation and activation of memory; knowledge and learning are not located at a particular point in the brain, but are distributed across many areas of the brain; • conceptual level: In a discipline or field, there are networked concepts that are crucial to the knowledge of a discipline or field; • external level: The development of participatory web technologies has con- tributed significantly to networking; blogs and wikis increase the possibili- ties for individuals to connect with others, with experts and with content; RSS as a means of collecting information and hybrid services as a means 92 Pedagogical Aspects of E-Learning Planning of combining information from different places have also contributed. The very high participation in social networks – especial y of young learners – points to new ways of thinking about the role of education. Networks are characterised by ”nodes”, which are different at each level. At the level of neural connections, it is a neuron, at the conceptual level it is an idea or a collection of ideas, and at the external level it is a person or an information source (Siemens and Tittenberger, 2009). Connectivism is based on chaos theory and the importance of networks or inter- connectedness in modern society. Chaos theory assumes that everyone is con- nected to everyone. Identifying patterns that appear hidden is a challenge for the learner. Learning is a process that takes place in variable environments that are usual y beyond the control of individuals. Knowledge is present in various net- works (computer and social), but in modern society, the most important thing is to be able to connect different sources of information. This integration enables us to learn more and is more important than our current state of knowledge (Siemens, 2007). The role of the teacher in connectivism is to lead, guide and improve the qual- ity of the networks formed by learners (Siemens and Tittenberger, 2009). From the outset, the theory of connectivism has sparked debate about wheth- er it is a learning theory, a theory of teaching, or a purely pedagogical view. However, some authors point to the conditions without which the theory of connectivism cannot flourish. These are digital literacy, autonomy and social interaction between learners (Kop, 2011). 3.2.5 the theory of community of Inquiry The theory of Community of Inquiry (CoI), developed in 1999 by Garrison, Anderson and Archer (2000), has been for many years one of the most im- portant models for understanding how learning takes place in e-learning, the essential feature of which is the spatial separation of the learner and the teacher in the process. The authors define the CoI model as a model of edu- cation in which the e-learning experience is derived from the interaction of three dimensions – the social, cognitive and teaching presence (ibid.). The CoI model therefore assumes that effective e-learning is not only the result of cog- nitive factors and teacher intervention, but that the social aspect is equal y important. This means that effective e-learning requires the coherence of the participants in the learning process, i.e., the existence of a community. They are rooted in the constructivist paradigm of education and place the learning experience at the centre of learning, which is the result of the interaction of Theories of Learning 93 three independent components of learning: the social, the cognitive and the teaching presence (ibid.). Figure 10: community of Inquiry framework Community of Inquiry Supporting Discourse Social Cognitive Presence Presence Educational Experience Setting Selecting Climate Content Teaching Presence (Structure/Process) Communication Medium Source: Garrison, Anderson and Archer (2000, p. 88). The authors of the CoI theory understand all three components of a commu- nity of inquiry as a prerequisite for successful learning (e-learning or tradi- tional). However, the model was designed primarily to explore how to set up and maintain these components in e-learning. The indicators defining each CoI component allow an analysis of the extent these components are present and how they are developing in the learners. cognitive Presence Cognitive presence is defined as the ability of learners to form and validate their own conceptions through reflection and discussion (Garrison, Anderson and Archer, 2001). It partly depends on the possibilities of the medium that encourages or restricts this communication. The authors argue that the very nature of the communication facilitated by a virtual learning environment or 94 Pedagogical Aspects of E-Learning Planning face-to-face communication in the classroom determines the development of learning strategies, learning styles and, ultimately, learning outcomes. They re- fer to various studies that have looked at the differences between learning in virtual and ”traditional” communities of learners, and where the results have shown that there are differences in the development of critical thinking, the generation of new ideas, etc. More specifical y, students in online classrooms were more likely to relate their learning to materials they found online and to link ideas to solutions, whereas students in a traditional classroom were slightly better at generating new ideas. They also found that while students in online classrooms were less interactive (interacted less with each other), they had higher levels of critical thinking. The indicators used to monitor the presence and development of the cognitive element in learners are: • triggering event, which shows the learner’s reactions and feelings (e.g., feel- ings of embarrassment/discomfort); • exploration, for example by sharing information; • integration as the bringing together of different ideas; • resolution, applying new ideas. social Presence As the online classroom is primarily a medium for communication and learn- ing, this feature could be a barrier to developing a sense of belonging to the group (other learners and teachers) and to the study programme. Social pres- ence is defined primarily as the ability of learners in a community of inquiry to present themselves social y and emotional y in that community, as they would in person, within the contexts that the ’medium of communication’ in the digital learning environment allows (Garrison et al., 2001). However, as the development of a sense of belonging is highly dependent on the individual’s perception of the learning environment and the other learners in it, the lack of this belonging and connectedness to the group can have a marked impact on e-learning achievement and, consequently, on drop-out (Swan et al., 2009). The direction of development of the social component is manifested in three ways: • affective expression, which refers primarily to the opportunity for learners to express their feelings and values; • open communication, which aims to develop and maintain a sense of be- longing to a group; • group cohesion, which refers to the ability of learners to participate in group tasks, to interact frequently in different activities, etc. Theories of Learning 95 The social presence of e-learning has received the most attention from the au- thors of CoI theory, as well as from other researchers, especial y because of the limitations of online media in developing a sense of belonging to a group and expressing emotions in e-learning. Since the theory was developed, the technological possibilities for communicating in a digital environment have improved significantly, and communication in such an environment has be- come fairly established in the meantime. We can assume that these trends have partly changed the problem of the social presence of education in the digital environment. But the question of how learning takes place in such an environ- ment and what the links are between the components of the CoI model raises a number of research challenges. teaching Presence The teaching component is ”the design, facilitation and direction of cognitive and social processes for the purpose of realizing personal y meaningful and educational y worthwhile learning outcomes” (Anderson et al., 2001, p. 5). The dimensions of the teaching component are defined according to the re- sponsibilities of the teacher in e-learning (Anderson et al., 2001): • planning and organisation refer to the activities carried out by the teacher or other competent person before the learning activities take place, i.e., the selection of the learning strategy, the preparation of the learning materials, learning activities, learner support, etc.; • stimulating discussion to keep learners interested, motivated and engaged; • teaching as ”providing intellectual guidance to learners and sharing one’s own knowledge in a particular field with learners” (ibid., p. 8). The teaching aspect is as important as the cognitive and social aspects for the development of a community of inquiry. Many studies have confirmed the positive impact of the teacher on the learning activities and achievements of e-learners, but at the same time, the teacher is a key factor in creating and maintaining communication and interaction – a particularly important goal of e-learning. In e-learning, a tutor usual y takes on part of the teacher’s activity. 3.2.6 Pedagogy of abundance Weller, the founder of the pedagogy of abundance, points out that the digitalisa- tion of content and the ability to share content online have led to enormous changes in many activities, including education (Weller, 2011). In his view, many industries have traditional y been based on ”scarcity”, e.g., the content was not accessible to al , schools/education programmes had limited places, etc. How- 96 Pedagogical Aspects of E-Learning Planning ever, with digital and open modes, we are witnessing a shift towards a multi- plicity (abundance) of content, which is influencing the development of new economic models and approaches. In the author’s view, this transition also has consequences for the development of education and the didactic approaches that characterise such circumstances. In the digital age, access to professional or scientific content is much easier. Jour- nal articles are often freely available, as are videos, podcasts, blogs and so on. Content is available that was only accessible to a select few in the pre-Internet era. In addition to static content, individuals can also engage in dialogue with others in social media discussions. They also have access to social networks of peers and other learners. These tools also allow access to and dialogue with sub- ject-matter experts. These opportunities and the increased volume of content that can be used as a resource for learning are, in the author’s view, a trigger for developing appropriate teaching and learning approaches to make the most of them (Weller, 2011). This view is largely reflected in more recent theories of learning (e.g., connectivism and heutagogy), which emphasise the shift from teacher- and teaching-centred learning to learner-controlled learning. According to Weller (2011, p.7), a pedagogy of abundance is based on these elements: • free access to content – not all content is free, but increasingly a learner can find free content in whole or in part; • content is abundant – the amount of content is increasing with digitalisa- tion and ease of publishing on the web; • diversity of content – content is no longer primarily text-based; • ease of sharing – using tools such as social bookmarking, tagging and link- ing, the ”cost” of sharing has been reduced or has disappeared; • social based – seeking and sharing as by-products of a social approach to learning; • ease of connection – it is easy to make and preserve connections within a network; • low-cost organisation of learning – online technologies make it easy and in- expensive to organise and run groups, so informal groups are more likely to form and be successful; • the generativity of the system – some authors, such as Zittrain (2008 in Weller, 2011), argue that unpredictability and freedom are essential features of the Internet and the reasons why it has enabled so much innovation; • user-generated content – the ease of content creation allows content to be produced in a variety of formats, as well as more and more educational material produced by the learners’ themselves. In this context, Weller (2011) sees two main challenges for educators: first, how to make the best use of the abundance of content and information in their Theories of Learning 97 teaching practice and, second, how to empower learners to find and use these resources themselves. The last challenge is perhaps the most important, as it highlights the importance of technology and digital literacy as a prerequisite for the successful use and creation of content online. 3.2.7 heutagogy Heutagogy is a theory of learning in a digital society developed by Hase and Kenyon (2010). This theory is rooted in autonomous or self-directed learning and unequivocal y renounces the dependence of learning on the teacher. In this, it also departs from the pedagogical and andragogical orientations, which rely learning primarily on instruction under the guidance of a teacher, tutor or other authority. Heutagogy therefore ful y extends and transfers control and responsibility for learning to the individual and sees the individual as an im- portant agent of learning (Hase and Kenyon, 2010). It is important to note that in heutagogy, learning does not take place in complete ’isolation’, but, as in an- dragogy, the teacher is sometimes important to the learner, participating pri- marily by providing guidance and resources, but leaving the approaches and methods of learning entirely up to the learner (Hase and Kenyon, 2010). In this respect, heutagogy is similar to the theory of self-directed learning as defined by Knowles (1986), who foresaw three steps in self-directed learning: identify- ing learning needs, identifying learning resources and strategies, and identify- ing learning achievements and evaluating learning outcomes. From this point of view, we can agree with Lisa Blaschke (2012), who says that heutagogy goes one step further than andragogy and complements it. E-learning is uniquely positioned to create learning environments that can sup- port heutagogical teaching and learning, thanks to the use of technology. In par- ticular, second-generation online technologies (Web 2.0) allow learners to direct and define their own learning paths and empower them to take an active role in their learning. Current online technologies promote interaction, dialogue, col aboration and information sharing (Lee and McLoughlin, 2007). Blaschke (2012) also highlights the benefits of social media, such as connectivity with oth- ers, discovering and sharing information (individual y or as a group), and per- sonal information-gathering and personalisation. It is expected that heutagogy will gain further relevance as a theory of learning in the context of eLearning 3.0. Open Educational Resources, MOOCs, ubiquitous computing, Intelligent Tutoring systems (ITS), mobile and microlearning, all approaches and tools of the semantic and symbiotic web, support self-directed learning perfectly, thus confirming the validity of the theory of heutagogy. In any case, the prerequisite for the realisation of self-directed learning is adequate digital literacy, which is closely linked to a pedagogy of abundance. 98 Pedagogical Aspects of E-Learning Planning 3.2.8 the relevance of Learning theories for the e-Learning Instructional design Different theories of learning express different views on what knowledge and learning are. By knowing the different theoretical approaches instructional de- signers decide how to design educational programmes, which learning strate- gies and methods best suit the content, the needs of the learners, the objectives of the educational programme and the learning goals. E-learning designers also need to know which tools and approaches are best suited for translating theoretical underpinnings into e-learning delivery models. Dron and Anderson (2014) note that different technological possibilities have given rise to or, better, facilitated the emergence of particular theories of learn- ing. Behavioural and cognitive theories were established in the era before the advent of online technologies – characterised by one-to-one or one-to-many communication. Constructivism and the theory of the community of inquiry flourished with the rise of Web 1.0, followed by the era of e-learning 1.0. Con- nectivism is the child of Web 2.0 and social tools. Web 3.0 as a semantic web with MOOCs, virtual and augmented reality, mobile learning, etc., suggests a link with the theory of heutagogy and the pedagogy of abundance. In practice, e-learning programmes are based on several different learning the- ories, especial y when they are more complex and developed by experienced and knowledgeable instructional designers. Sink (2014) gives the example of a short e-learning training programme for IT professionals on specifying user requirements, his approach was based on constructivism. The topic of the first day of the course was how to clearly define and verify user requirements. Behavioural learning methods (various tests) were used, with immediate and thorough feed- back. In the following days, the constructivist approach was again at the forefront: participants ran simulations, reviewed relevant written mate- rial and took part in interactive online lectures, which included practical activities and feedback. Each day, they were given a thought pattern for the topics covered that day, so that they could better understand and mas- ter the topic and the links with other topics. Here, a cognitive approach was used. Brief games were also included in the programme to stimulate the participants’ reflection and active participation. It is obvious that such a well-designed e-learning programme would not be possible without the intelligent use of educational technology. The emergence of new theories of learning does not mean that other, existing theories are no longer useful. A professional y competent instructional design- Theories of Learning 99 er will be able to apply different approaches, grounded in different learning theories, to different parts of an educational programme or groups of learners, perhaps even to individuals, in a meaningful way. Knowledge, understanding and the ability to apply learning theories in a meaningful way is just one of the competencies needed for quality e-learning instructional design. Before we start to find a suitable pedagogical solution in the design stage, we need to know what are the educational needs and capacities of the individual or group for whom we are designing the programme. This is the task of educational needs analysis, which is the first stage of instructional design in the ADDIE model. The analysis of educational needs is discussed in the next section.15 recommended Links Instructional Design: https://www.instructionaldesign.org/ Learning Theories and Online Learning: https://www.tonyBates.ca/2014/07/29/learning-theories-and-online-learning/ How People Learn II: http://nap.edu/24783 Theories of Online Learning: http://technologyandlearning.weebly.com/theories-of-online-learning.html 15 The ADDIE model, and models like it, do not deal specifical y with theories of learning and the impact on instructional design. 100 Pedagogical Aspects of E-Learning Planning 3.3 educational needs analysis Educational needs analysis is the first step in most instructional design mod- els. It identifies knowledge or competency requirements and suggests possible solutions to acquire that knowledge or competencies. The solutions are then fleshed out in the next stages of instructional design. The literature on IDMs does not give a clear picture of exactly what this level should cover. The boundary with the next level, i.e., with programme design, is also quite blurred. The most useful starting point for a more detailed definition of educational needs analysis is provided by the Dick & Carey model, where the first three stages of the model refer to needs analysis (Dick et al., 2005). In this model, the educational needs analysis involves: • identifying the programme's instructional objectives, • instructional analysis, • learner analysis. 3.3.1 objectives of the educational Programme The analysis of the programme’s objectives defines what the programme aims to achieve overal . The focus of the needs analysis is on the learners in the pro- gramme and the central educational objective, i.e., to bridge the gap between the existing and the desired level of knowledge or competencies of the participants. To il ustrate the basic elements of an educational needs analysis, we will use an example from the FAO e-learning Methodologies manual. The ex- ample relates to an E-Learning programme on food security assessment skil s and competencies. The programme aims to improve the analytical processes of food safety assessments and to promote and encourage the use of these assessments in decision-making (FAO, 2011, p. 28). The nature of the gap between the existing and desired levels of knowledge or skil s can vary widely. Most often, the knowledge or capability gap is associ- ated with inadequate performance compared to the expected results (Branch, 2009, p. 26) – this is usual y interpreted mainly as a problem of employee train- ing. Training programmes therefore aim to fill this gap. However, it should be borne in mind that a failure to achieve the desired results may also be due to other factors, such as the inadequate organisation of work, technical short- comings, lack of motivation, etc. Before starting to plan training programmes due to the non-achievement of results, it is necessary to check whether lack of abilities and insufficient knowledge are the real cause of the results gap (FAO, 2011, p. 28). Educational Needs Analysis 101 For example, productivity in the accounting department has decreased with the use of new computer software. First of al , we might consider that the reason is that the staff are not properly trained to use the new software. The real problem may be inadequate organisation, which is not adapted to the data processes required by the new software. The causes of educational needs vary. The gap between existing and desired levels of knowledge is caused by the rapid development of a particular field, the mastery of which requires new knowledge and new capabilities. This aspect of the emergence of educational needs is particularly characteristic of highly pro- fessional and specialised fields, such as medicine. Programme objectives may also be geared toward training to meet regulatory requirements or the needs of the environment, or simply towards updating programmes to maintain the quality and competitiveness of institutions. Successful education programmes need to be able to identify gaps in future education needs, not just in the cur- rent offerings of competitors. 3.3.2 Programme analysis In addition to the identified educational needs, the content the programme re- lates to is a key factor in determining the overall objectives of the educational programme. The educational needs analysis therefore also includes an analysis of the pro- gramme itself, which includes an outline of the content of the programme and the identification of all the activities needed to achieve the programme’s objective. The content is not yet defined in detail in the educational needs analysis phase. This is the subject of the programme design. A description of the basic themes and how they are linked is sufficient. This can be done using a variety of tools such as thought patterns, process diagrams and other tools to visualise concepts and the links between them. There are a number of freely available tools on the web for this purpose (https://www.mindvectorweb.com/blog/free-mind- mapping-software-for-visualizing-ideas/). The outline of the content of the programme also guides us in identifying the main tasks that need to be fulfilled in order to achieve the programme’s objec- tives. 102 Pedagogical Aspects of E-Learning Planning FAO (2011, p. 31) lists the following tasks for the Food Security Assess- ment Model Programme: • selecting the most appropriate method for assessing food security in the specific circumstances; • selection of indicators for the different components of food security; • analysis of the results of food security assessments using standardised methods; • producing effective reports for decision-makers. We use a variety of curricula and teaching and learning materials from the same or comparable programmes, research literature, etc. to help us set the tasks. Websites of related educational programmes, OERs and MOOCs are ex- tremely useful resources. The Food Security Assessment programme can be supported by MOOCs on food security/ safety. Class Central, the central portal for MOOCs, offers us 136 hits on this topic (https://www.classcentral.com/search?q=food+safety). For the topic Instructional Design, which mostly includes the topic Ed- ucational Needs Analysis, there are 337 MOOCs available (https://www. classcentral.com/search?q=instructional+design). 3.3.3 analysis of Learners When specifying the tasks that will frame the design of the next phase of the programme, we need to keep in mind the results of the learner analysis. It is essential to analyse the educational needs for an e-learning programme, whether the analysis relates to an individual or to a group of learners. In e-learning, which is based on the learner-centred paradigm, personalised learning is one of the main concepts used to implement this paradigm. Even if the concept of personalised learning is not a 21st-century invention, the development of technology has significantly increased the possibilities for its practical application. New technologies make it possible to adapt the content, time and space, learning activities and all other elements of an educational programme to the needs of the individual in a meaningful and relevant way (U.S. Department of Education, 2017, p. 9). The analysis of learner character- istics is based on learning analytics and artificial intelligence methods. (See Sections 6.3. in 6.4). Educational Needs Analysis 103 Personalised learning is increasingly entering the educational practice of both e-learning and traditional education. Despite these changes, the traditional approach still prevails, regardless of the level and mode of education; in this approach, educational programmes are planned and delivered for a group of participants, not for the individual. Which characteristics of the participants should be taken into account in the analysis of the educational needs for the group? The analysis of the character- istics should provide us with all the information that will help us to decide how to design the educational programme so that it is suitable for the participants, so that the participants can best achieve the basic objective of the educational programme, as well as the specific learning objectives (these are discussed in the next section on the design of educational programmes). Elkins and Pinder (2015) recommend looking at the demographic character- istics, environmental characteristics, computer familiarity, subject knowledge, motivation and special needs of participants. In the participant analysis, it is useful to include a description of the circumstances that directly affect the par- ticipants’ chances of successful y participating in the planned programme. This is an essential element that should not be overlooked during programme planning. The FAO manual proposes the following characteristics and circum- stances for the analysis of the participants in the Food Security Assess- ment Programme. Table 8: analysis of the characteristics of participants in the food security assessment Programme Characteristics of the participants and Implications for the programme circumstances Geographical area of residence of participants Language, cultural differences, time zones (important for synchronous communication) The type of organisation in which the To be taken into account when setting specific participants are employed and their position in learning objectives by participant group the organisation Computer and technical capabilities of To be taken into account in the design of participants technology-enhanced forms of interaction and other activities Time available for learning and the learning To be taken into account in the scope of the context content included in the programme and in breaking the content down into smaller parts 104 Pedagogical Aspects of E-Learning Planning Characteristics of the participants and Implications for the programme circumstances Internet access location It affects the time participants can be active in the digital learning environment and also their ability to access certain content or tools Quality of Internet connections It affects the effectiveness of the participants’ work. If participants do not have good Internet connections, this should be taken into account when selecting applications Characteristics of the technical infrastructure of To be taken into account when integrating the participants media and related tools Source: FAO, 2011, p. 29. The example shown in this table describes the characteristics and circumstanc- es typical of an international non-formal education programme for groups of participants coming from technological y, cultural y and geographical y very different backgrounds. An analysis of the characteristics of the participants, for example, prepared by a vocational secondary school for the development of a blended e-learning programme to develop skil s for Industry 4.0, would of course look very different. The focus would be on analysing the personal char- acteristics and motivation of the students, their subject knowledge and abili- ties, especial y digital ones, their equipment with various electronic devices, knowledge of their learning style, their workload, special needs, etc. 3.3.4 conducting an educational needs analysis Having researched and defined the basic elements of an educational needs analysis, the analysis needs to be carried out. In principle, the analysis of edu- cational needs is carried out in the same way as any other area of research. First, the data must be collected, organised for analysis and then processed using appropriate analytical methods. Recently, learning analytics, which are typical y based on the integration of databases of different types of data and the combined use of different statisti- cal methods, have also been making inroads into the field of educational data analysis. For the most part, however, traditional approaches to data acquisition and analysis are still used in conducting educational needs analysis. These ap- proaches are briefly described in Bregar et al. (2010, p. 49–53). We also rec- ommend Paul F. McCawley’s Methods for Conducting Educational Needs Educational Needs Analysis 105 Assessment, which clearly and concisely describes the methods for obtaining data for educational needs analysis. Otherwise, the acquisition and analysis of educational needs data follows the same methodological procedures and rules as quantitative research in the social sciences. Any good textbook on the use of statistical or quan- titative methods in the social sciences can help us design a methodol- ogy for eliciting and analysing educational needs. The PDF Drive portal contains a huge collection of e-books (more than 80 million), includ- ing many high-quality textbooks on statistics and quantitative methods (https://pdfdrive. com/research-methods-and-statistics-e20154756.html). recommended Links Class Central. Instructional Design: https://www.classcentral.com/search?q=instructional+design PDF Drive: https://pdfdrive.com/research-methods-and-statistics-e20154756.html 106 Pedagogical Aspects of E-Learning Planning 3.4 Instructional design of an e-Learning Programme 3.4.1 framework of an Instructional design for e-Learning The educational needs analysis outlined the basic determinants of the pro- gramme: what the programme aims to achieve, the characteristics of the par- ticipants and the educational content. The next step in planning educational programmes is to prepare a design, which outlines the pedagogical image of the planned programme. The design makes the main decisions about the characteristics of the elements that make up an e-learning programme. The elements of the design and the link to the education needs analysis and theories of learning are il ustrated in the following figure. Figure 11: framework of an instructional design for e-learning Educational needs Theories analysis of learning Learning objectives Learning and teaching strategy Content Learning Learning structure resources and activities Assessment Support material Delivery method of implementation Media Technology Before embarking on the design, we need to careful y and thoroughly inves- tigate which circumstances are most important for the implementation of the Instructional Design of an E-Learning Programme 107 programme. We need to know what the status of the planned programme will be: whether it will be a stand-alone programme or part of a wider whole. We can de- velop an educational programme that is just one course in a formal educational programme, or a shorter in-company training programme, or a MOOC, etc. We also need to decide whether we will develop a complete e-learning programme with all the elements, or whether we will limit ourselves to a few elements and acquire others in other ways (perhaps free, from project partners, through pur- chase, etc.). We need to think about what kind of certificate the participant will receive after successful y completing the programme. This issue is particularly relevant for formal education programmes. We may also encourage interest in shorter pro- grammes by offering recognition in the form of micro-accreditations or badges. Closely linked to the status of a programme are questions about the level of complexity of the planned programme. E-learning programmes therefore need to define the level of complexity of the programme from the outset. The expected level of difficulty also determines the conditions for participation in the programme (for example, prior knowledge of the subject area or certain competencies). For formal education programmes, we usual y need to set out the expected workload for participants in advance (for example, based on ECTS), as well as the requirements for formal qualifications already achieved. Modern e-learning is evolving towards flexible, open and personalised education. From this point of view, pre-determining the entry conditions, the expected loads and the necessary background knowledge becomes meaningless. Artificial intelligence and learning analytics are expected to play a prominent role in adapting programmes to individual needs in the future. More on this in Part 6. 3.4.2 Learning objectives The learning objectives provide the basic orientation for the design of the e- learning programme. Learning objectives define what participants should be able to do or do better after the completed education. Learning objectives define what participants should achieve after the completed education. For e-learners who are mostly self-learners, well-defined learning objectives are very helpful. They make it clear what participants can expect in concrete terms from a successful y accomplished programme. The learning objectives 108 Pedagogical Aspects of E-Learning Planning define the relevance of each content in the programme and thus guide the learner’s learning. Comparing achievements and learning outcomes with learning objectives allows the participant to assess their learning effectiveness. Clear learning objectives also help the authors of the learning materials to adapt the structure of the programme to the learning objectives. Well-defined objectives guide all the subsequent stages of the educational programme: the development of learning materials, learning activities, assessment questions, the delivery of the programme and the evaluation of the programme. Learning objectives are a particularly important starting point for the develop- ment of self-assessment questions and the design of various activities to promote interaction in the learning process and to replace face-to-face contact between the learner and the teacher in e-learning. Well-defined objectives guide learning efforts, reducing the chance of misdirection and unnecessary work. Well-defined objectives are clear, precise and relevant. The importance of knowing your objectives is well il ustrated by a pas- sage from Lewis Carrol ’s famous literary work Alice in Wonderland. ”Alice has arrived at the crossroads of three paths. She had just come down one, and had two more routes to choose from. Which route should she follow? ”Cheshire Puss,” she began, rather timidly, as she did not at all know wheth- er it would like the name: however, it only grinned a little wider. ”Come, it’s pleased so far,” thought Alice, and she went on. ”Would you tell me, please, which way I ought to go from here?”. ”That depends a good deal on where you want to get to,” said the Cat. ”I don’t much care where—” said Alice. ”Then it doesn’t matter which way you go,” said the Cat. The learning objectives are based on the purpose and primary objective of the programme, as defined in the educational needs analysis. However, we also need to take into account other components of the educational needs analysis: the content area of the programme, the characteristics of the programme par- ticipants and the specific context in which the programme will be developed and implemented. Let us clarify the difference between the purpose, the primary objective of a pro- gramme and the learning objectives, as these terms are often used imprecisely. The purpose is a general definition to justify why a programme is needed in education in the first place. The primary objective of the programme defines what the programme aims to achieve overal . The learning objectives set out in concrete terms what the participant will gain from successful y completing the programme. The primary objective refers to the programme as a whole, while the learning objectives refer to the individual components of the programme. Instructional Design of an E-Learning Programme 109 Learning objectives can be further broken down to the level of individual learning activities. In Section 3.3 Educational Needs Analysis, we have listed as an example of the Food Security Assessment programme the primary objective ”to im- prove the analytical procedures of food safety assessment and to promote and encourage the use of these assessments in decision-making”. The aim of this programme is to improve professional support for food safety. The learning objectives are, for example, to identify the main factors influenc- ing safety for the main food categories, to be able to select the appropriate analytical procedures for assessing the quality of a specific food category, to prepare a written report on the results of the food assessment, etc. Rowntree (1994a, p. 52) classifies the objectives according to their effects on the participant into four groups. These include: • memorisation objectives (concepts, terms and procedures to be remem- bered, etc.); • attitudinal objectives (how the programme will influence the participant’s attitudes and values, etc.); • objectives of understanding (e.g., the new aspects of understanding of cer- tain phenomena and processes that the participant will learn); • the objectives of the activity (mastering new physical skil s or communica- tion skil s, etc.). When defining learning objectives, we usual y use a version of Bloom’s taxonomy of cognitive levels (see Section 4.3 Assessment Methods in E-Learning for more on this topic). Table 9 shows Bloom’s taxonomy of cognitive levels with a description of the learning objectives corresponding to each level. Table 9: Bloom’s taxonomy of cognitive levels with a description of the learning objectives Cognitive level Learner’s learning objectives Remember Able to recognise or remember information. Understand Able to reformulate concept. Apply Able to use information in a new way. Analyse Able to break down a concept and identify the links between its elements. Evaluate Able to justify a decision against a criterion or standard. Create Able to realize a new product or develop a new approach. Source: FAO, 2011, p. 35. 110 Pedagogical Aspects of E-Learning Planning 3.4.3 Learning theories and Learning objectives When defining learning objectives, it is important to take into account how learning takes place. This is explained by theories of learning (Section 3.2). We need to know which theories of learning support the learning goals that we want to achieve. For example, if the learning objectives aim at knowledge and understanding of certain concepts related to food safety, we will draw on cogni- tive theory and choose appropriate methods and tools, learning activities and assessment methods in line with this theory. If we want to train participants to effectively detect practices that do not respect the principles of food safety, constructivist theory and related pedagogical approaches, etc. will be at the forefront. The following table summarises, following Kinte’s (2013) article, the main fea- tures, strengths and weaknesses of behaviouristic, cognitive and constructivist theories, and the types of goals each theory supports. The table is supplement- ed by our description of the elements for the theory of connectivism. Table 10: theoretical background for designing learning objectives Theories behaviouristic cognitive constructivist connectivist Main Acquiring simple Discussions, Acquiring more Knowledge features knowledge and problem situations, complex and acquisition in an skills (e.g. motor information demanding open, unstructured skills) by applying processing knowledge, environment what has been using experience learnt unchanged/ and perceptions directly already acquired Advantages Clear learning Clear learning Broadly defined Learning objectives objectives, rapid objectives, objectives, are loosely defined, acquisition of cognitive ability to the ability to learners acquire knowledge and perform specific understand and and build on the skills against procedures interpret different competencies they predefined criteria situations, learners need to function acquire the effectively in a capacity to act in digital society real situations Restrictions Individual Learners acquire Not suitable for A certain level differences are some skills in situations where of digital and not taken into connecting the exchange of independent account, nor is and structuring different views is working skills is the influence of information, which not appropriate required cognitive processes are not necessarily on behaviour effective Instructional Design of an E-Learning Programme 111 Theories behaviouristic cognitive constructivist connectivist Learning Monitoring learner Ability to perform a Active participation Creating new objectives behaviour variety of cognitive of learners in knowledge at processes, solving solving real-life different levels of problem situations problem situations. integration and at a cognitive level building capacities for the digital society Source: Adapted from Kinta, 2013. The learning objectives of an education programme are usual y not uniform, as they aim to achieve different levels, types of knowledge and competencies. Accordingly, we take into account different theories of learning, rather than a single one, when designing the implementation model of an e-learning pro- gramme (see 3.2.8 The Relevance of Learning Theories for the E-Learning In- structional Design). 3.4.4 Learning and teaching strategies and e-Learning Theories of learning provide us with basis for the formulation of learning ob- jectives, but they do not provide clear and concrete guidance on what peda- gogical approaches and methods can be used to achieve the general aim of the programme and the specific learning objectives. As Bates (2016) points out, educators have yet to figure out how to transform theoretical assumptions into practical implementation. For each learning objective, it is necessary to identi- fy the content the objective relates to, select the sources of information on that content, define the learning activities and the means of assessing knowledge and competencies, and identify the pedagogical support. The use of media, technological support and the way the programme is delivered should also be considered (Figure 11: Framework of an instructional design for e-learning). The specific definitions of these elements are fleshed out during the programme development phase. This is the subject of Part 4 of the book. In the design phase, we remain at the level of strategic orientations, defining the learning and teaching strategy in this design phase. 112 Pedagogical Aspects of E-Learning Planning The English-language literature refers to ”instructional strategy”. In the concept of modern e-learning, which focuses on the learner and his/her educational needs, these strategies are defined in such a way as to en- able independent and active learning by the learners, usual y in a situation of spatial separation between the learners and the teacher. The teacher’s role has changed significantly; teaching methods are giving way to ac- tive and independent learning methods.16 Teaching (instruction) in the traditional sense is virtual y non-existent in e-learning. In view of this, we believe that the translation of the term ”instructional strategy” as ”teaching strategies” or ”instructional strategies” would be inappropriate for e-learning. We have therefore decided to use the term ”learning and teaching strategy” (LTS) in the Slovenian version of the book which is retained in English one. The LTS must outline the basic guidelines on how to develop and implement an e-learning programme for all the elements that define its pedagogical image.16 Below are some of the dilemmas and issues for which the LTS for e-learning programmes should provide indicative guidance. Programme content and structure Will the content of the programme be ful y predefined, or will it be adapted to the needs, interests and abilities of the participants? How will the content of the programme be structured – how many rounded units, such as modules, sections and learning units? It may not be necessary to disaggregate content (due to flexibility of content, small size, etc.). Will the sequence of these units be predetermined? What classification criteria will we use? Learning material Will we prepare the learning materials ourselves, buy them or perhaps use OERs and MOOCs? We may prepare our own learning material for some topics, or use material already prepared by other authors. Will we allow par- ticipants to access the material on different devices? What will the role of par- ticipants be in selecting other materials and preparing their own? What about copyright? Which media will be used in the learning material? 16 A paper from the University of Alberta (2002) states that teaching strategies become learning strategies when learners independently select learning approaches and methods and use them effectively to achieve learning goals. Instructional Design of an E-Learning Programme 113 Learning activities Which learning activities will be used when delivering the programme? The nature of the learning objectives will be a fundamental guide for the choice of learning activities. Nevertheless, we wil have to weigh up a number of dilemmas: should we include the learning activities in the learning material or carry them out independently? What will the balance be between traditional and newer, more technological y sophisticated learning activities? Will learners do the ac- tivities individual y or in teams, at the same time or with a delay, flexibly or with predetermined deadlines? assessment of Knowledge How will we assess the learners’ knowledge and skil s: on an ongoing (forma- tive) basis, at the end of the programme (summative), or both combined? The types of obligations of learners and the way in which their knowledge will be examined must be defined. How will we assess col aborative work, e.g., in dis- cussion forums, blog, and complex learning activities (e.g., projects, portfolios, teamwork)? What is the role of teacher or tutor feedback and is it mandatory or voluntary? What is the relationship between automated and teacher-led as- sessment and examination? support for Learners What will be the tutoring and administrative and technical support? Which forms of assistance will be available to participants, to what extent, and through which media? If participants will be divided into groups, how many partici- pants will be in each group and how many forms of support will be available as part of the programme at no extra cost? Can we provide all forms of assistance on a 24/7 basis, i.e., without interruption 24 hours a day, every day of the week? technological support Will we use an LMS for e-learning or a combination of tools? If we are going to use an LMS, we need to decide whether to go for a commercial or an open system. Will we use additional tools alongside the LMS? What technical ser- vices will be available to participants? What technology will participants use to learn? Will they use desktop computers, laptops, tablets, smartphones, etc.? How will we ensure data security and data protection? media Which media will be included in the learning material and in what propor- tion? Which criteria will we take into account when selecting media and which 114 Pedagogical Aspects of E-Learning Planning criteria will we give priority to? Will it be pedagogical, cost, organisational or other aspects? What are the teachers’ skil s in using different media? What is the competence of the participants to learn using different media? How limited are we in our choice of media by existing technological solutions? Programme delivery How the programme is delivered depends on the characteristics of the learners, the institutional and organisational features of the educational organisation and the technical possibilities. If it is an educational organisation that is known for offering online education, it is natural to expect the planned programme to also be delivered in this way, but of course there are different options de- pending on the mode of communication. You can run a programme with with minimum or maximum synchronous communication, or you can limit your- self to asynchronous communication (this is mostly typical of MOOCs). There are even more options if you opt for blended learning or hybrid learning. 3.4.5 Linking the Pedagogical and Business- organisational components In the design, we identify the essential features of the e-learning programme from a pedagogical perspective. In the planning and implementation of e-learning programmes, it is not enough to consider the pedagogical specifi- cities and the meaningful support of the technology, but a number of other factors, mainly in the field of management in the educational organisation, need to be taken into account at the same time. In Figure 6 Implementing the e-learning strategy (Section 2.1), we pointed out the link between the peda- gogical and the business-organisational components, which needs to be re- spected in the planning phase of the programmes and in their development and delivery. So far (in Part 2 and Part 3 of the book) we have dealt with the business-organisational and pedagogical aspects of e-learning in the planning phase, but in the following sections we will consider both aspects in relation to the development and implementation phases. recommended Links eLearning Design and Development: https://elearningindustry.com/subjects/elearning-design-development Carnegie Mellon University. Eberly Center: https://www.cmu.edu/teaching/designteach/design/index.html Instructional Design for eLearning: https://www.udemy.com/course/instructional-design-for-elearning/ Learning Material 115 criteria will we give priority to? Will it be pedagogical, cost, organisational or other aspects? What are the teachers’ skil s in using different media? What is the competence of the participants to learn using different media? How limited are we in our choice of media by existing technological solutions? Programme delivery the deveLoPment of 4 How the programme is delivered depends on the characteristics of the learners, the institutional and organisational features of the educational organisation and the technical possibilities. If it is an educational organisation that is known e-LearnIng Programmes for offering online education, it is natural to expect the planned programme to also be delivered in this way, but of course there are different options de- pending on the mode of communication. You can run a programme with with minimum or maximum synchronous communication, or you can limit your- self to asynchronous communication (this is mostly typical of MOOCs). There are even more options if you opt for blended learning or hybrid learning. 4.1 Learning material 3.4.5 Linking the Pedagogical and Business- organisational components 4.1.1 starting Points for the development of In the design, we identify the essential features of the e-learning programme Learning material from a pedagogical perspective. In the planning and implementation of e-learning programmes, it is not enough to consider the pedagogical specifi- Developing learning material for an e-learning programme requires a different cities and the meaningful support of the technology, but a number of other approach and thinking than writing learning material for an educational pro- factors, mainly in the field of management in the educational organisation, gramme that will mostly be delivered face-to-face. At all stages of the design of need to be taken into account at the same time. In Figure 6 Implementing the an e-learning programme, as well as in the development of the learning content, e-learning strategy (Section 2.1), we pointed out the link between the peda- we need to keep in mind the learner, who will be going through the programme gogical and the business-organisational components, which needs to be re- largely on their own. That’s why it’s important to make learning material inter- spected in the planning phase of the programmes and in their development esting, stimulating, fun and proactive. E-learners can use various types of learn- and delivery. So far (in Part 2 and Part 3 of the book) we have dealt with the ing material: textbooks, manuals, lecture notes, computer slideshows, audio and business-organisational and pedagogical aspects of e-learning in the planning video recordings of lectures, FAQs, quizzes for self-assessment, databases, im- phase, but in the following sections we will consider both aspects in relation to ages and graphics, games, simulations, etc. A common feature of the different the development and implementation phases. types of learning material in e-learning is that it is produced in digital format. Therefore, in the following discussion of the learning material, we do not specifi- recommended Links cal y mention that we are referring to digital learning material. eLearning Design and Development: In the design stage, we decide whether to develop the learning material our- selves, buy it or use OER. We usual y use several different options, so only https://elearningindustry.com/subjects/elearning-design-development some of the materials are prepared by us. Carnegie Mellon University. Eberly Center: In this section, we look at the issues that arise when learning materials are https://www.cmu.edu/teaching/designteach/design/index.html developed independently, either in their entirety or for specific content strands. Instructional Design for eLearning: Developing learning material requires specific decisions on content, design https://www.udemy.com/course/instructional-design-for-elearning/ and technological support. When looking for solutions, we need to bear in 116 The Development of E-Learning Programmes mind the conclusions reached in the previous two stages of programme design, i.e., the analysis of educational needs (Section 3.3) and the instructional design for an e-learning programme (Section 3.4). We need to take into account the basic orientations of the educational needs analysis, who the learners in the programme are, which content areas should be covered by the programme or the learning material, the basic objective and the specific learning objectives, which resources are available, etc. We also need to take into account the deci- sions we have made when designing the programme on the elements of the LTS and the way the programme will be delivered, as well as on the technologi- cal environment. An important decision factor is the resources available and the constraints as identified in the business plan. Today, creating an e-learning programme or learning material is much sim- pler, easier and cheaper than it was a decade or so ago, thanks to the wide range of digital tools available for creating, editing and customising learning mate- rial. These tools can be installed on your computer or used on the Internet in the cloud. Often, these tools are also part of the LMS. Popular and widely used paid tools for developing learning materials are Articulate Adobe, Easy Generator and iSpring. In addition to general tools, there are a number of specialised tools available for specific parts of the learning material, such as recording and editing audio and video, cre- ating 2D and 3D simulations, educational games, mobile apps and quiz- zes (Berking, 2016). For more information, see Section 4.5 Digital Tools for E-Learning. On the iSpring website https://www.ispringsolutions.com/blog/adobe-captivate- 9-vs-articulate-storyline-2-vs-ispring-suite-8-1, a comparison of the usability of three tools is available: iSpring, Adobe Captivate and Articulate 360. In evaluating this assessment, we must keep in mind that it was prepared by a provider of one of these three tools. However, simply choosing a good tool does not guarantee quality learning ma- terial. E-learning programme and learning material development tools are just tools to make some of the complex activities of developing learning mate- rial routine, simple and transparent (e.g., using animation, using various me- dia, creating graphics, designing web pages, etc.). Notwithstanding the changed and simpler way of designing the learning mate- rial, it is still necessary to think careful y and describe in detail all the elements that we have included in the programme design, and to outline only roughly in the LTS their main features, as would be appropriate in terms of learning objec- tives. It is useful to keep the following guidelines in mind (Fee, 2009, p. 102): Learning Material 117 • careful management of all activities related to the development of learning materials; • the material must enable an effective learning experience; • the material should contain a variety of learning activities, not just reading material; • the material must make the most of the technological affordances. Even when it comes to selecting learning content, tools are now available that allow for automated content preparation. However, the development of learn- ing material is still the domain of content experts or teachers. 4.1.2 content of the Learning material scope of the content Included For most teachers, content is a central issue in the development of learning ma- terials (Bates, 2016). The content is essential y determined by the objectives of the programme, which are sometimes pre-determined (external y). In the case of training for specific purposes or areas, for example as defined by legislation, regulations and other official documents, there is no dilemma as to the content of the programme and the learning material. There are also no particular prob- lems with the content of a fairly well-established and well-known curriculum (e.g., the course Fundamentals of Statistics). Of course, even for programmes whose content is well-known, we need to pay particular attention to keeping up with new developments in the profession and continuously refresh the con- tent of the programme. Bates (2016) points out that teachers and educators are often burdened with having to cover everything that fal s within the content area of a programme. The issue of competencies’ building is side-lined. He recommends that two issues should be born in mind when deciding what content to include in the programme and whether to acquire or develop appropriate materials for it: • what is the contribution of the content to the main objective of the educa- tional programme; • what content is essential to achieve the learning objectives of the programme and what content would be useful but not essential? In a digital society characterised by the abundance and accessibility of infor- mation, it is particularly important to consider careful y whether learners re- al y need to know certain facts, ideas, principles or procedures. Is memorising them necessary to achieve other learning objectives or is it an end in itself? For example, is it necessary to memorise the form for calculating the arithmetic mean in the Fundamentals of Statistics course, or is it necessary to know and understand it in order to understand the concept of statistical averages and to 118 The Development of E-Learning Programmes be able to interpret different statistics? The rationale for addressing and know- ing certain content is also important for the learner and his/her motivation to learn, and should be clearly reflected in the learning objectives. The abundance of information can only be managed by developing compe- tencies such as knowledge management, problem-solving, digital literacy and decision-making, but these competencies are also related to content. In order to make decisions, it is not sufficient to know certain rules, principles and pro- cedures on how to make decisions, but we need information about the spe- cific problem situation to which we will apply decision-making competencies (Bates, 2016). When we design the content of the programme, we must not be seduced by our professional interest in the field to expand the content uncritical y, because it is ’all interesting and important’. The role of the teacher is to support and facilitate the learning process of the learner, not to prepare learning material according to his/her own professional interest. The right measure of the scope of the content to be included must be found in the triangle of the programme’s objectives, the resources available (human, time and financial) and the acceptable burden on the learners. structure of the content The structure of the content is an important element in the design of learning materials. Traditional y, its content is divided into several thematical y round- ed sections, which are then subdivided into more detailed units. The learning material follows this breakdown.17 Web 2.0 allows for more flexibility in the way the content of the learning mate- rial is broken down. LMSs such as Blackboard allow learners to access learning content in any order. Web 2.0 tools open up opportunities for learners to find and organise their own learning content using discussion forums, blogs and e-portfolios, as well as to share it with peers and others in the learning process. In recent years, a new way of curating learning content (content curation) has been developed using artificial intelligence methods. This approach is based on the automated research of online resources (blogs, discussion forums, OER, etc.) that are relevant to a given topic, selection of the most appropriate resources, and the preparation of the selected resources for e-learning (eLearning Industry, 2019). 17 Traditional approaches to content parsing have been described in some detail in the handbook ”The elearning Essentials” (Bregar et al., 2010, pp. 104–108). Learning Material 119 4.1.3 storyboard for Learning material The learning material we develop for e-learning programmes is infinitely more complex than traditional printed learning material. The process of planning the development of this material should take into account a number of ele- ments: content, breakdown, media, interactions, activities, links and feedback. An effective tool for preparing a detailed plan for the development of learning materials or complete programmes is a storyboard for learning material or a sto- ryboard. The idea of preparing a detailed plan for realising the design of a complex product or service with a story description listed in tabular form, i.e., ”storyboard”, originated in the film industry. Walt Disney used a special record book as a tool for preparing cartoons, calling it a ”storyboard”. Each cartoon scene was sketched and explained in advance. Today, this tool is used in many other areas, such as project management, industrial process planning, advertising and e-learning. A storyboard is a document consisting of a series of images and descriptions of a computer screen (screenshot) that detail the expected implementation of an e-learning programme or learning material. It contains guidance on the use of video, text, audio, photos, graphics and interactive elements. The basic text, which is an integral part of the learning content, is complemented by guidance notes and il ustrations for each of the elements that will be included in the learning material. The information that can be contained in a single display is (Elkins and Pinder, 2015; Georgieva, 2019): • the position of each screen image in the structure of the programme or learning material and the title; • the text that the user sees on the screen (learning content, directions and guidance for the learner, descriptions of icons and other graphical ele- ments); • description or links to the media included on the screen; • an audio recording with instructions; • video description; • a description of the graphical elements; • instructions to the software developer on navigation; • instructions for adding links, documents and other instructions; • instructions for interactions; • assessment questions with instructions. 120 The Development of E-Learning Programmes Of course, each screenshot will only contain some of this information, but information on the position of the screen, the text for the learner and basic guidance for the developer are indispensable. The storyboard can be prepared in different ways: as text in a spreadsheet, on a computer slideshow or as a prototype using an appropriate authoring tool. This can be done with the help of various authoring tools. Figure 12: example of a spreadsheet storyboard Source: Colman, 2019. Figure 13: example of a slideshow storyboard Source: Colman, 2019. Learning Material 121 A comparison of spreadsheet and computer slideshow displays shows that a spreadsheet can contain much more information, but not as clearly presented as a slideshow. Developing a storyboard is a professional y demanding and complex task: it is necessary to take into account the already developed design of the educa- tional programme, but also to be familiar with the usability of the media from a pedagogical point of view, to be familiar with the basic principles of design in a digital environment, and also with the technical possibilities. It is usual y a team effort involving subject, pedagogy and educational technology experts, a designer, a computer technician and a manager. The way the screen storyboard is developed requires close col aboration and communication between all the team members before we start developing the learning material. This avoids any later misunderstandings and inconsisten- cies in the final product and unnecessary time wastage. The storyboard gives the subject expert a clear enough picture of what the learning material should actual y look like. This also gives subscribers and users a clear idea of how the planned idea of learning material will be realised. Of course, we cannot ignore the fact that the concept of a storyboard requires a significant time investment, which is all the greater the more complex the learn- ing material. Corrections to the storyboard are rather inconvenient and time- consuming. These constraints are particularly disadvantageous in the business world, where rapid solutions are needed to update employees’ skil s. The concept of rapid learning development is being developed in this direction. The concept of ’rapid development’ is based on the use of programs such as Word or PowerPoint (PPT), to which the developer then adds interactive elements and other add-ons using specific authoring tools, without requiring any special programming skil s, to create a simple e-learning programme (Berking, 2016). Some further information on rapid e-learning can be found at https:// elearningindustry.com/training-managers-guide-to-what-rapid-learning-is-and-not. recommended Links Elearning Learning. Storyboard: https://www.elearninglearning.com/storyboards/ CommLab India. Rapid eLearning Solutions: https://blog.commlabindia.com/elearning-development/types-of-elearning-rapid- authoring-tools CommLab India. Storyboard Elements: https://blog.commlabindia.com/elearning-design/storyboard-elements 122 The Development of E-Learning Programmes eLearning Industry – how to Create Effective Storyboards for eLearning? https://elearningindustry.com/create-storyboards-effective-elearning-tips Virtual College. Content Curation: https://www.virtual-college.co.uk/news/virtual-college/2017/08/content-creation- vs-content-curation-in-e-learning Learning Activities 123 4.2 Learning activities 4.2.1 the role of Learning activities in e-Learning Learning activities are one of the essential features of e-learning programmes. A learning activity can be defined as the interaction of a learner in a learn- ing process, using a variety of resources, directed toward the achievement of specific learning objectives (Beetham and Sharpe, 2007, p. 28). A learning ac- tivity is an activity in which the learner performs specific work (task) related to, for example, certain basic skil s, thought processes, attitudes or behaviour. This can be done on your own, in a group or with the help of a teacher or tu- tor. By learning activities, we mean any kind of work (reading, writing essays and seminars, self-assessment, discussions, experiments and excursions) that is included in the educational programme in order to achieve specific learn- ing objectives. However, the learner’s work itself is not necessarily a learning activity. We don’t learn much if we just click through different web addresses or chat in an e-café about our New Year plans, for example. We learn by reflect- ing, researching, organising, judging, summarising, discussing, deciding, ap- plying ideas, etc. While learning activities can be done by clicking or chatting in a chat room, the goal must be clearly defined, because only in this way can we stimulate a learning experience that will lead to knowledge. The learning activities completed are usual y an element of the evaluation of the learners’ performance. 4.2.2 types of Learning activities Three different types of learning activities are needed to achieve the learning objectives (Horton, 2012): • absorb-type activities (absorption activities), • do-type activities (’do’ activities), • connect-type activities (’connect’ activities). Absorption activities are usual y done by reading, observing and listening. Examples of such activities include presentations and demonstrations, story- telling by the teacher or visits (to museums, places of interest, etc.). In these activities, the learner is mostly not physical y active, but mental y active. They are particularly suitable for motivated participants who want to refresh or up- grade their knowledge, as these activities provide them with information and the opportunity to obtain new knowledge and competencies. 124 The Development of E-Learning Programmes In ’do’ activities, learners are asked to do or make something related to what they have been learning about. The learner can practice a procedure, play a game or answer questions. Compared to absorption activities, the preparation and implementation of ’do’ activities are more complex and costly. We use ’do’ activities whenever we want: • to give participants safe and interesting practice so they can apply what they learn in real life; • improve the motivation of participants, especial y those who find learning theory and concepts boring; • encourage participants to engage in absorptive activities by demonstrating the usefulness of knowledge through performance activities. Through ’connect’ activities, participants combine what they have learned with what they already know or have experienced at work and in life. We develop more complex skil s and competencies through ’connect’ activities. The main purpose of the ’connect’ activities is to facilitate the use of knowledge later on. Examples of ’connect’ activities are in-depth questions, participant storytel - ing, exploratory activities and original work. ’Connect’ activities are used by the teacher in the learning process when: • the use of knowledge or competencies is essential; • the current use of knowledge and competencies is inadequate; • an educational programme is of a general nature and the ability to apply general principles is important; • learners do not recognise how to apply new knowledge or competencies; • learners are not able to independently integrate knowledge and competencies. Typical y, learning takes place by first acquiring basic information, testing it in practice, and then combining the newly acquired information with other knowledge or competencies, starting with absorption activities, moving on to ’do’ activities, and finishing with ’connect’ activities. However, the learning ac- tivities can also be used in a different order. The sequence is influenced by the content of the programme itself and the LTS on which the training programme is based. Less demanding absorption activities, as well as ’do’ activities, are particularly suitable for behaviouristic theory-based programmes. The orientation (philoso- phy) of cognitive theories requires the completion of more extensive tasks that allow the analysis and synthesis of the knowledge acquired and its practical ap- plication (’connect’ activities). Programmes, based on the theory of construc- tivism, on the other hand, focus on activities that lead to the creation of new knowledge. These are mainly ’do’ activities. The implementation of pedagogical approaches based on connectivism presupposes the use of web 2.0 technologies Learning Activities 125 that enable ’connect’ activities such as social networking, various forms of on- line publishing and discussion, participation in virtual interactive worlds, etc. The following table shows the most commonly used learning activities by ac- tivity type. Table 11: absorb-type, do-type and connect-type learning activities Learning activities absorb-type do-type connect-type Presentations (live Practical exercises (drills, testing, In-depth activities (rhetorical demonstrations, guided and team exercises) questions, reflections, giving slideshows, films, examples, summarising, interviews, event evaluating, flipped learning) descriptions) Reading (different Discovery (virtual labs, case In-depth questions documents from different studies, role-playing) sources) Teacher’s storytelling (live, Games and simulations (quizzes, Learner storytelling (live or virtual) word games, programmed virtual) simulations, device simulations, mathematical simulations, event simulations, etc.) Excursions (live, guided and Use of tools (glossaries, virtual self-guided, virtual) assistants, statistical packages, etc.) Research activities (finding relevant sources, processing information, reporting) Original activities (decision- making, document production, blogs, wikis, etc.) Source: Adapted from Horton, 2012. Learning activities can be carried out at different levels of difficulty. Participants can read a simple text from a primary school textbook or a challenging article from a scientific journal. The same learning activity may present different chal- lenges for different categories of learners: preparing the exact same seminar as- signment may be a tough nut to crack for an adult re-entering education after a long interruption, but a routine matter for a postgraduate learner. It is essential to choose and combine learning activities in such a way that the learning objective is achieved. 126 The Development of E-Learning Programmes For example, suppose the aim of a learning activity is for participants to acquire knowledge that will help them decide whether or not to use an ingredient (for example, honey) in the preparation of cosmetic products. We will try to achieve this by using all three types of learning activities: • absorption activity: a representation of the chemical and other prop- erties of this ingredient; • ’do’ activity: the use of an ingredient for a specific practical purpose; • ’connect’ activity: identification of possible uses of this ingredient in cosmetics. 4.2.3 Learning activities in e-Learning The support of modern technology in e-learning increases the range of dif- ferent learning activities and the possible ways of delivering them.18 Learning methods and approaches enabled by digital technologies open up new, inno- vative possibilities for designing and combining learning activities. This is the subject of Part 6 of this book. Learners can learn from microlearning, OERs, MOOCs and other online re- sources, listen to audios and watch videos, and familiarise themselves with various simulations. These are all examples of absorption activities that are suitable for e-learning programmes. The technology also enables the creation of a wide range of games, virtual labs, virtual and augmented reality, mobile learning as forms of ’do’ activities. These provide the learner with il ustrative information about the behaviour of a phe- nomenon or person under different circumstances, which helps the learner to transfer the acquired knowledge or competencies more effectively into prac- tice. Various discussion forums, weblogs and other forms of virtual communica- tion can be classified as ’connect’ activities. Modern technology also plays an important role in providing feedback on the correctness of activities. Feedback can be given in different ways: • by giving the correct answers to each question; • by advising relevant literature; • by preparing sample responses; 18 More on e-learning activities in the works: John G. Hendron (2008): RSS for Educators and William Horton (2012): e-learning by Design, pp. 67–214. Interesting examples of learning activities can be found on William Horton’s website (https://horton.com/). Learning Activities 127 • showing and analysing the responses of participants who have been includ- ed in previous iterations of the programme; • showing advice, encouragement and suggestions from previous participants. Feedback is provided through various forms of asynchronous and synchro- nous communication. Preparing and monitoring learning activities in a learner-centred e-learning programme undoubtedly involves more work for teachers or tutors and re- quires specific types of knowledge and. In practice, this aspect is often over- looked or at least underestimated. Teachers can be partial y relieved of some of the burden by using appropriate tools that allow them to provide feedback in different ways and through different media. In addition to written feedback, the Blackboard LMS also al ows for audio or video feedback (https://www.youtube.com/watch?v=nqoBqubLaQe). This option is particularly welcome if you want to add a personal touch to the information you send, but also if you are pressed for time and the information is more extensive. 4.2.4 Learning activities as an Integral Part of the Learning material The learning activities included in an e-learning programme for self-directed learning are broadly understood as any work of the learners (reading, writing, group discussions, etc.) that has been planned by the teachers participating in the programme as authors or tutors, in accordance with the set learning objectives. However, only some of these activities are directly included and highlighted in the learning material. The activities included in the material must therefore ensure that the attention, engagement and concentration are maintained at all times through a variety of tasks (checking knowledge, giving examples, sharing own experiences, etc.) and through active participation. The frequency of the activities in the material varies, but the activities should not be too sparsely interspersed. Rowntree (1994b, p. 131) recommends that a learner should complete an activity at least after reading or reviewing learn- ing material that is roughly equivalent to three pages of printed text. However, he warns that the participant’s concentration and interest can be expected to wane after five pages of reading without any other activity. The activities included in the learning material itself can also be considered from different perspectives: in terms of the time needed to complete them and 128 The Development of E-Learning Programmes the objectives they achieve, in terms of their complexity, the way in which the learner provides information on the activities completed, the manner and con- tent of the feedback, and the pedagogical approaches. Considering activities as learners’ commitments, the activities in the learning material can be grouped into a few fairly homogeneous groups of tasks: • ongoing tasks, • self-assessment tasks, • larger assignments, • examples. The ongoing tasks are embedded in the text itself. They are usual y less demand- ing and are mainly designed to keep the reader’s attention and to check their comprehension of the text. The questions and tasks are related to the topic at hand. Ongoing tasks only achieve their purpose if feedback on correct, possible or expected answers is provided. Ongoing tasks are mostly less demanding ab- sorption activities. The LOLA (Learning about Open Learning) programme, which was im- plemented in Slovenia in 1999 as part of the PHARE Multicountry Co- operation in Distance Education project, included the following types of activities: exercises, questions in the text and reflection points. Exercise 5: Activities Specify Activities (Exercises, in Text Questions and Reflection Points) to complement the aims, objectives and SAQs you have written previously. Note your ideas for activities down in your personal workbook. Don’t spend too much time on this, but remember activities must be relevant to the objectives of the course. Self-assessment activities are usual y added at the end of a learning unit or broader content topic. A learning unit is a conceptual y and didactical y coher- ent piece of learning content or material that has an independent pedagogical function and learning objective (Rebolj, 2008, p. 209). The learning activities for the self-assessment are in the form of simple ques- tions that require, for example, choosing the correct answer from several op- tions, finding connections between different concepts, completing the text or inserting a missing word, etc. The simplicity of the questions allows correct answers to be given automatical y, usual y accompanied by explanations. These Learning Activities 129 are ’do’ activities that can be designed with various levels of complexity. For more information, see Section 4.3 Assessment Methods in E-Learning. Larger assignments or exercises usual y involve the tutor’s participation, as they are usual y more complex, and the use of different forms of synchronous and asynchronous communication. They are mostly ’connect’ activities. In the 2004–2007 period, an online e-learning programme was developed at the ACS with the support of the European Social Fund, with the aim of providing adult educators in Slovenia with basic training in e-learning (Bregar, 2007). In addition to the online activities and self-assessment questions, a num- ber of more extensive tasks were included in the learning material. One of them was a seminar paper. Participants were given guidance on how to complete this task: ”In your seminar assignment, you will develop a plan for tutoring sup- port and the use of online material for your (actual or imaginary) online educational programme. Describe the programme as precisely as possi- ble (its objectives, purpose, level of knowledge, target groups, etc.). Please justify your proposals.” Examples are a specific, broadly defined, widely used and popular form of learning activity. They can be very simple or very complex and, depending on the learning objectives, are designed as any of the three types of activity (ab- sorption activities, ’do’ activities and ’connect’ activities. Examples can be prepared in a number of ways, such as: • references to what is already known (for example, the meaning of graphical representations is il ustrated by the famous Chinese saying that a picture tel s a thousand words) and to other authors, testimonies and statements, • analogies, anecdotes and other interesting stories, • situational examples, • case studies, • graphic presentations, audio and video material, • specimens and samples, • computational examples and simulations. When preparing examples, as with other activities, it is crucial to think in ad- vance and determine what function the chosen example will have in learning and what learning objectives you want it to achieve. 130 The Development of E-Learning Programmes Rowntree (1994b, p. 149) distinguishes two basic approaches to the use of ex- amples in explaining fundamental concepts or ideas: • we explain the rule with examples, • examples lead to the rule. Rowntree cal s the first approach RUL-EG and the second EG-RUL. The EG-RUL rule allows learners to arrive at new knowledge on their own and is the starting point for constructivism theory-based learning. Such learning can be quite time-consuming and may be too demanding for learners. This is why we often use the opposite approach (RUL-EG), where we first set out the rule or definition and then allow learners to explore it more thoroughly and in more depth with examples. recommended Links ALLENCOMM: https://www.allencomm.com/blog/2018/07/types-elearning-activities-include- elearning-course-design/ eLearning Industry: https://elearningindustry.com/interactive-elearning-activities-enhance-learner- engagement-4-examples William Horton Consulting: https://horton.com/ Carnegie Mellon University. Eberly Center. Teaching Excellence & Educa- tion Innovation: https://www.cmu.edu/teaching/technology/index.html Assessment Methods in E-Learning 131 4.3 assessment methods in e-Learning 4.3.1 general aspects of assessment Assessment and testing are an integral part of evaluation, which is the sys- tematic collection of data on the quality of a process, product or service. Most often, evaluation processes are intended to provide information to make deci- sions that will lead to improvements in the process, product or service. When we are referring primarily to the assessment of learners’ knowledge or learning achievements, we are talking about the testing and assessment of knowledge, skil s and competencies. The literature provides different definitions of these terms; in general, assessment is the systematic collection of data on how well someone is achieving the learning objectives, and assessment processes eval- uate and usual y assess this learning (Bates, 2016; Farrell and Rushby, 2016; Horton, 2012; Redecker and Johannessen, 2013). As Bates (2016, p. 463) points out, the manner and purpose of assessment will always be influenced by the teacher’s expectations and beliefs about what he or she considers to consti- tute knowledge and what learners need to demonstrate as their knowledge and abilities. The skil s and competencies needed in the digital world also have an impact on the way assessments are carried out. In an e-learning programme, the assessment of content knowledge is equal y important, as are the assessment of different competencies. This is made possible by different technology-based methods for formative and summative assessment. In this section, we will first introduce the taxonomies of learning objectives. The learning objectives guide the choice of methods and ways of assessing knowledge and competencies. We will then introduce the traditional forms of assessment and look at some of the most established alternative methods, which are primarily designed to check and assess the extent to which more challenging learning objectives have been achieved. We will pay particular at- tention to reviewing the benefits and potentials of testing and assessment tech- nology. The sections will conclude with a brief description of the specificities of assessment and testing in e-learning. The first thing that probably comes to mind is that the assessment is primarily a check of the learners’ performance. Have they learnt the content? Is the quality of their knowledge sufficient to allow them to progress in the programme? As- sessment has several functions. On the one hand, the results of the assessment give feedback to the learner, letting him/her know which parts of the material or which objectives he/she has mastered or not. This information guides their further learning. On the other hand, the results of the assessment are useful for 132 The Development of E-Learning Programmes the teacher or tutor19, as they give feedback on the performance of individual learners and the group as a whole. The teacher tries to identify the gaps in the learners’ knowledge and to find and eliminate their causes. Assessment not only measures the quality and extent of the knowledge and competencies acquired during the course, but also indirectly measures the teacher’s performance and the quality of the educational programme. After al , one of the important functions of assessment is to motivate and stimulate learning. For learners , grades are not only information about their achieve- ments, but also about their own abilities. In this way, the evaluation becomes one of the factors in self-confirmation and a positive self-image. Before we start developing assessment tools (tests, tasks, etc.), we need to think careful y about the rationale behind the assessment and what we want to achieve. Horton (2012, p. 216) points out that the reasons for testing knowl- edge and competencies are not always particularly well-founded. Table 12: reasons for testing knowledge and skills Suitable reasons Less suitable reasons To measure the learner’ progress against set To follow the stereotype that tests are an learning objectives unavoidable and unpleasant part of e-learning programmes To highlight important topics and motivate To seemingly increase the authority of the tutor learners to focus on them or teacher To encourage learners to apply what they have Tests to show how learning is difficult and learnt to deepen their knowledge challenging To monitor the performance of individual parts To seemingly boost the learners’ confidence of the programme as a basis for improvement with light questions and encouraging feedback Assessment of knowledge or competencies To use the tools, we paid dearly for that are part of formal or certified programmes Identify learners’ existing skills and capabilities Assessment is the only way to make the to avoid redundant work programme interactive Source: Horton, 2012, p. 216. 19 The tutor may also assess or check the work of the learners. In formal education, a tutor must meet the relevant legal conditions to be allowed to assess. In the following text, we use the generic term ’teacher’ and only use ’tutor’ when describing the specific role and tasks of the tutor in the assessment of knowledge and competencies. Assessment Methods in E-Learning 133 4.3.2 Bloom’s taxonomies of Learning objectives One of the main purposes of assessment is to monitor the achievement of learn- ing objectives. Taxonomies of learning objectives are helpful both in defining the learning objectives themselves and in developing the instruments to monitor their achievement. original Bloom’s taxonomy The first taxonomy of learning objectives was developed by Benjamin Bloom in the 1950s (Bloom, 1956). The taxonomy covers cognitive processes and as- sociated mental abilities, which Bloom classifies from less complex to more complex: knowledge (knowing), comprehension, application, analysis, syn- thesis and evaluation. The categories following ’knowledge’ were defined as competencies, with the assumption that knowledge is a necessary precondition for the realisation of these skil s and competencies. Although each category included sub-categories, we know the taxonomy mainly by the six main cat- egories. Learning objectives can be defined as the mastery of mental abilities of a certain level of complexity, and assessment can be used to determine to what extent these abilities have actual y been acquired. Table 13: Bloom’s original taxonomy of cognitive learning objectives Level of cognitive Description ability Knowledge • recognition, recall, reconstruction of facts, data, technical terms, sym- bols, definitions, rules, procedures and interpretations; Comprehension • describing, summarising, explaining in your own words, summarising the gist in your own words; • giving examples, explaining graphs, maps, results; • translation from one symbolic notation to another; Application • explaining and solving a problem situation with a known principle; • predicting impacts and consequences based on given data; • identifying and justifying exceptions; Analysis • identifying the individual elements in a message; • analysis of the relationships between elements (hypotheses and evidence, assumptions and arguments, establishing associations and cause and effect); • analysis of organisational principles; 134 The Development of E-Learning Programmes Level of cognitive Description ability Synthesis • developing and shaping ideas and messages; • formulating hypotheses and ways to test them, designing experi- ments; • deriving generalisations, classifications, models and theoretical con- clusions; • recommending and planning ideas, justifying decisions; • taking different opinions into account, taking part in discussions, co- ordinating; Evaluation • assessing the adequacy and completeness of data, the reliability of ob- servations, procedures and instrumentation; • assessing a work or document in the light of arguments, evidence; • comparing one piece of work against another according to the criteria; • recognising biases and emotional factors. Source: Rutar Ilc, 2004, pp. 68–73. revised Bloom’s taxonomy In the 2001 revised edition of Bloom’s Taxonomy, the levels are slightly differ- ent: remembering, understanding, applying, analysing, evaluating and creat- ing. In 2001, a group of cognitive psychologists, experts in curriculum design, assessment and psychometrics, published a revision of Bloom’s taxonomy entitled A Taxonomy for Teaching, Learning and Assessment (Anderson and Krathwohl, 2001). The taxonomy moves away from the more static notion of ”educational objectives” and points to a more dynamic notion of classifica- tion. The authors of the revised taxonomy emphasise this dynamic by using verbs to denote categories and subcategories (instead of nouns in the original taxonomy). These ”action words” describe the cognitive processes that an in- dividual uses to learn. Assessment Methods in E-Learning 135 Table 14: Bloom’s revised taxonomy of cognitive learning objectives, with a description of the characteristic categories Level of cognitive ability Method of implementation Remember • identify • retrieve from Understand • interpret • compare • sort • summarise • conclude • clarify Apply • realise • implement Analyse • differentiate • organise • attribute to Evaluatie • check • critically evaluate Create • make • plan • produce Source: Anderson and Krathwohl, 2001, p. 31. In the revised taxonomy, knowledge is the basis of all six cognitive processes, and the authors have also developed a taxonomy of types of knowledge, which includes factual, conceptual, procedural and metacognitive knowledge (An- derson and Krathwohl, 2001, p. 28). In education, we conduct oral and written tests, and we may also test the per- formance of a task or product. Understandably, in e-learning, where commu- nication is mostly asynchronous and computer-mediated, most assessment is written. In addition to established or traditional methods of assessment, new methods are increasingly being introduced that include more authentic, real- istic and practical methods of assessment of knowledge and skil s. Bloom’s digital taxonomy The characteristics of assessment and evaluation in e-learning presented so far are based on approaches that make limited use of the technical possibilities of- fered by technology for designing and implementing these procedures. But the fact is that the use of technology in education is changing educational practice, 136 The Development of E-Learning Programmes bringing new forms of delivery and causing knowledge to be produced dif- ferently. Mastery of mental capabilities is still important for knowledge and capability creation in the information society, and the methods and tools used also have an impact on knowledge creation and capability acquisition. The importance of the methods and tools used to create knowledge and build capabilities has already been demonstrated by the emergence of Web 2.0 tech- nologies, and even more clearly by the emergence of Web 3.0 and 4.0 technolo- gies (see Section 1.3 for more on this). Based on the characteristics of Web 2.0, Churches (2008) developed a new version of Bloom’s taxonomy, called the re- vised digital taxonomy. In the revised digital taxonomy, Churches adds meth- ods and tools enabled by technology to cognitive processes of different levels of complexity. One particular element that he adds at all levels is collaboration, because he considers that the ability to col aborate is the most important skill in the twenty-first century, both in general and for learning in particular. Tech- nologies allow for an extremely diverse range of ways to col aborate, such as discussion forums, wikis, blogs, chat rooms, email, Skype and webinars. In the following table, we present Bloom’s revised digital taxonomy, indicating the basic levels of cognitive abilities, typical learning activities for each level and il ustrative examples of tools that can be used at each level. Table 15: Bloom’s revised digital taxonomy Levels of cognitive ability and Tools related learning activities Creating: conceiving, designing, Movie making (Movie Maker), animation making, planning, making, inventing, consulting, blog and videoblog making (Blogger, Worldpress, ”mixing” and ”remixing” Edublogs), wiki making, multimedia production (DTP; Movie Maker, GIMP, Corel) Evaluating: supervising, experimenting, Panels and focus groups (chat rooms, email, judging, discovering, commenting on discussion forums, audio conferences), research (WP; blogs, reviewing, moderating, linking GIS, Google Maps, Flickr); collaboration (discussion boards, forums, blogs, wikis, microblogs or tweets) Analysing: collating, organising, breaking Online surveys, the use of databases (MySQL, GIS; down, drafting, structuring, grouping Google Earth, Google Maps), graphical displays (various graphical web tools from SPSS and other statistical packages, e.g. Tableau), spreadsheets (Libre Office Calc, Microsoft Excel) Applying: implementation, usage, Illustrations (Corel, Paint), simulations (Google operation, sharing, installation Sketchup, graphic tools), presentations (PPT, Skype, interactive virtual whiteboard, audio and video conferencing) Assessment Methods in E-Learning 137 Levels of cognitive ability and Tools related learning activities Understanding: explaining, summarising, Drafting (thought patterns), editing (blogs), sorting inferring, classifying, comparing, (tagging) clarifying Remembering: recognising, listening, Social networking, social bookmarking, search, using identifying, finding, naming, locating, browsers, quiz indenting Source: Adapted from Branch, 2008. For each of the methods used to achieve a certain level of ability, it is possible to assess the level of quality or complexity of the participants’ performance. Churches (2008) distinguishes four levels, with the first being the lowest and the fourth the highest. To il ustrate, here is an example of an assessment of the knowledge and skil s needed for participants to master the ’advanced search’ function in a web browser (Churches, 2008, p. 25): • Level 1. Participants choose a relevant search engine; they use keywords to reach the starting pages. • Level 2. Participants choose a relevant search engine; they use keywords and phrases to reach the starting pages. Participants are able to navigate between the pages they find and know how to assess the quality of websites (e.g. identifying sponsored sites). • Level 3. Participants select the appropriate search engine; they use advanced search tools, using options that allow more precise searches (domain, coun- try, language, document type, location, page). They get to their landing pages by using keywords, which they modify. Participants are able to navi- gate between the pages they find, understand how a search engine works and know how to assess the quality of websites (e.g. identifying sponsored sites). • Level 4. Participants select the appropriate search engine and are able to justify their choice; they use advanced search tools with options that allow more precise searches (domain, country, language, document type, loca- tion, page). They can justify the options they have chosen. They know how to use the ”exact match” option, phrases and field exclusions. They get to their landing pages by using keywords, which they modify. Participants are able to navigate between the pages they find, understand how a search en- gine works and know how to assess the quality of websites (e.g., identifying sponsored sites). 138 The Development of E-Learning Programmes 4.3.3 traditional and alternative methods of assessment types of classic Questions In written assessments, we distinguish between standardised tests of knowl- edge and competencies and tests devised by teachers. Standardised tests are those that have detailed procedures for use, resolution times and instructions. They also contain validated questions where the difficulty of each question is known, and norms for each group of participants, which can then be used to compare individual performance. Traditional education usual y uses tests that are made up by the teachers themselves. Knowledge can be tested with different types of questions, including: • multiple-choice questions, • alternative-type questions, • short-answer or follow-up questions, • essay questions. The first three types of questions are objective questions, the essay questions are subjective questions. Multiple-choice questions consist of a question and several (at least three) pos- sible answers. The possible answers should be as short as possible, but it is also important that they are the same length. It is often the case that the cor- rect answer is slightly longer and can therefore be identified more quickly. An- swers must not overlap, of course. Short and closed questions are mainly used in e-learning in connection with ”self-assessment” and feedback on progress through online content. Alternative-type questions are most often used for content checking where there are only two possible answers (for example: Yes/No, Correct/Incorrect, etc.). Such questions are easy to construct and the evaluation of the answers is also objective. They are suitable for example for pre-testing. It is important to realise that such tasks can only test lower levels of knowledge (remembering, understanding). Another disadvantage is that the few possible answers can en- courage guessing. Short-answer or follow-up questions are mainly designed to test knowledge of facts, concepts, rules and understanding. Such questions have the advantage of being relatively simple to construct and they also eliminate guesswork. The bad news is that they mainly promote the reproduction of knowledge rather than analysis and evaluation. When formulating questions of this kind, care must be taken to make them as specific, unambiguous and brief as possible. In addition, keep the answer as short as possible, e.g., just a word, a number. Assessment Methods in E-Learning 139 To avoid confusion, it is sometimes better to ask a short question rather than a fill-in-the-blank exercise. If more than one answer is possible, a note should be made of how many answers are to be ticked or answered. In an era when self-directed learning is becoming more and more popular, quiz- zes with so-called objective questions for the self-assessment of knowledge are becoming a common addition to instructional material. In digital learning en- vironments such as Moodle or Blackboard, tools are already available for teach- ers to prepare quizzes, as well as tools for easy learning analytics of the answers (more on this in Section 6.4). There are also a range of stand-alone tools for pre- paring test questions, such as Hot Potatoes, Perception’s Questionmark, (https:// www.questionmark.com/), Articulate Quizmaker (https://articulate.com/360), iSpring Quiz Maker (https://www.ispringsolutions.com/ispring-quizmaker) and others. The Blackboard LMS allows you to assess the quality of the questions ( item analysis) for each test in terms of difficulty and discrimination, both at the level of the test as a whole and at the level of the individual ques- tions. The teacher can use the information from the question analysis to improve or revise the test questions or to correct the distribution of scores in the marking of individual parts of the test. The analysis of the questions also informs the appropriate design of the test according to the learning objectives. For example, if the test is designed to motivate stu- dents initial y and to determine their level of knowledge before starting the programme, it will not include very difficult questions that require in-depth knowledge. Figure 14: summary report of the analysis of the questions for the pilot test Online test (**Webcam**) – Requires Respondus LockDown Browser Analysis Last Run June 7, 2018, 9:06 AM. Run Item Analysis again to display the latest question data Test Summary Discrimination Difficulty 3 Good Questions i 19 Easy Questions i 48.0 41 0 8 34.5 00 hr 45 min 0 Fair Questions i 16 Medium Questions i Possible Possible In Progress Completed Average Average 4 Poor Questions i 7 Hard questions i Points Questions Attempts Attempts Score Time i i i i i i 34 Cannot Calculate i Source: Bregar and Puhek, 2018, p. 52. essay Questions Essay questions are quite common in e-learning. Essential y, these questions are designed to test the higher learning objectives: analysis, synthesis and evaluation. They contain questions learners answer freely, in several sentences. It is pointless to ask for facts and data with essay questions, because these can be objectively 140 The Development of E-Learning Programmes verified in other ways. Essay questions also have the advantage of being tech- nical y simple and time-saving. However, it takes a lot of practice and careful preparation to compose real y good essay questions. It is important that the essay question is posed in a clear way so that the learner knows what his/her task is. We can make it easier for learners by asking more easy-to-understand and simple sub-questions. Essay questions that are not clearly worded make the assessment more difficult and reduce the validity of the assessment. McCormick and Pressley (1997) suggest the following word descriptions for each learning objective. Table 16: verbal descriptions of learning objectives Assessment objectives Verbal description Use ”Describe an example in which...” Comparison ”Compare two methods, theories, models...” Analysis ”Explain how...” Integration ”Connect the theories...” Evaluation ”Evaluate the position advocated...” Source: McCormick and Pressley, 1997. A fundamental weakness of essay questions is that it is much more difficult to ensure objectivity when scoring answers than in closed-ended tasks. Before we start marking the essay questions, it is advisable to work out the correct an- swers in advance, as broken down as possible, and give them a possible score. It is also useful to assess the same task for all participants at the same time, if possible, with two or even more readings (from a first, rough screening, in which we rank the answers in order of quality and identify potential assess- ment problems, to a detailed reading and scoring of each answer separately). However, we can save ourselves a lot of work by using rubrics. alternative methods of assessment Assessment formats that encourage learners to learn more deeply and to in- tegrate theory and practice are gaining ground in educational practice. Tradi- tional methods of assessment are being joined by more varied and authentic modes of assessment to see how learners understand, apply and analyse con- tent in new contexts and from different perspectives. Alternative methods of assessment, such as practical assessment and so-called authentic questions, are particularly important when integrating higher-level learning objectives (analysis, synthesis, evaluation). They are characterised by the fact that the knowledge acquired is not tested using traditional test questions, Assessment Methods in E-Learning 141 but by tasks that are as similar as possible to life or work situations. Authentic tasks and problems are those in which participants can face real-life challenges and show how they respond to them. Authentic methods of learning have the advantage of not only providing relevant knowledge and competencies, but also stimulating interest in learning. The methods that make such learning possible are discussed in Part 6. According to B. Marentič Požarnik (2003) alternative methods of assessment include: • compose conceptual networks and professional text independently, • self-assessment against agreed criteria, • peer or mutual assessment, • group evaluation of the results of teamwork or col aborative learning, • practical problem-solving (for example, open-book testing), • performance evaluation (e.g., performance, concert, activity), • product evaluation (artistic or technical products), • assessing diaries and records (practice reports), • assessment of seminar and project assignments, • assessment of portfolios. Below, we will briefly outline some alternative methods of assessment that are particularly suitable for e-learning: • e-portfolio assessment, • evaluating project assignments, • problem-based learning. Portfolio assessment is increasingly used as a form of authentic assessment – in- cluding in e-learning. Conrad and Openo (2018) consider portfolios to be one of the most authentic and e-learning-friendly methods of assessment. Bates (2016, p. 464) also mentions the ePortfolio as one of the fundamental ways of assessing capabilities of participants in e-learning. A portfolio is a personal collection of information that represents and describes an individual’s learn- ing, career, experiences and achievements. As a form of assessment, a portfolio allows an individual to use products to showcase his or her achievements in a particular field in an organised way. It is evidence that allows an authentic as- sessment of what the learner has achieved in the training. It allows the learner to self-assess his/her own performance, but it is also a means for the teacher to assess the progress and achievements of the learning. The portfolio as a means of assessment guides the learner towards goal-oriented planning, as in order to produce it, the learner has to keep the end-goal in mind: what he/she wants to prove with the portfolio, what skil s, competencies and abilities he/she wants to demonstrate with it. 142 The Development of E-Learning Programmes The development of e-learning has led to the development of digital port- folios or e-portfolios. The e-portfolio can be created on the e-Nefiks web- site, developed by Nefiks (www.nefiks.si/). It allows participants to build and complete their portfolios in the educational, professional and private spheres. They can draw up a plan for their education and share their work experience, goals and achievements with careers advisers, employers, etc. Figure 15: enefiks user interface Source: Nefix (https://app.nefiks.si/ ). The description and usefulness of e-portfolios are also presented on the University of Queensland website (https://elearning.uq.edu.au/guides/eportfolio). Project-based learning is a common way of working in e-learning programmes. It can involve individual project tasks and group activities, usual y leading to the production of a final product. Projects are usual y designed to mimic real-world challenges and can lead to the development of plans, products, research projects, multimedia presentations and more. The advantage of group projects is that they promote cooperation and cohesion among the working group. Project-based learning allows the integration of several learning modalities into a coherent whole and is therefore one of the most appropriate e-learning methods. Assessment Methods in E-Learning 143 On the PBLWorks website (https://www.pblworks.org/) there are many resources on project-based learning that are useful for planning such learning in traditional and e-learning contexts. It is also recommended to watch the video by John Spencer (YouTube: https://youtu.be/crmm4z3oKmQ). Problem-based learning is a similar approach to project-based learning. In this kind of learning, participants are confronted with a problem for which they have to find a suitable solution. They usual y work in smal discussion groups, de- fining the problem and objectives, then planning the way forward. They then apply their learning to solve the problem. The problem-based learning process ends with a reflection on learning, on the problem-solving process and on the characteristics of group work. Although such learning explores some predefined aspects of a problem, not all learning objectives need to be predefined. Problems should be unstructured and amenable to different approaches and solutions. The main difference between project-based learning and problem-based learn- ing is that in problem-based learning, participants’ activities focus more on solving pre-defined problems, rather than solving problems that participants encounter on their own while working on their project. A further description of this method with background material is also available on the Maastricht University website: https://www.maastrichtuniversity.nl/education/why-um/problem-based-learning. rubrics A useful tool for assessing complex tasks, such as project or problem-based tasks are rubrics. They are almost unknown in Slovenia, but have long been known in the world (Jonsson and Svingby, 2007). Their use is widespread, especial y in universities in the USA, and partly in Europe. They are particularly suited to the assess- ment of complex tasks, whether traditional or e-learning, and can be imple- mented by digitalised tools or paper-based format. Rubrics are a useful tool for assessing tasks that aim to reach higher cognitive levels, such as: • individual research assignments, problem-based assignments, reflections, essays, evaluations; • col aborative tasks: team tasks, discussion forums, projects, peer evalua- tions, wikis, blogs; • multimedia projects (preparing video, podcast, website, e-portfolio); 144 The Development of E-Learning Programmes • games and simulations. Rubrics are usual y formatted as spreadsheets. The components of a rubric are: • criteria (rows of the table) • the level of achievement of each criterion (columns in the table), • descriptors of the level of achievement of each criterion (table cell). The assessment of the level of achievement of a given criterion within a task (each cell of the table) is determined by the different criteria (in rows) and the possible levels of achievement of each criterion (in columns). The rating can be expressed in points, ranks, intervals, etc. It is important to base the rubric for a particular task or activity on the learning objectives of the specific educational programme. Table 17: example of a rubric Criterion Levels of achievement of the criteria Exceptional In line with Improvements Unsatisfactory expectations are needed Quality of (20 points) (15 points) (10 points) (5 points) content The content is The content is Some of the The content clearly related mostly clearly content is not covered is largely to the topic. It related to the related to the topic. unrelated to the includes more topic at hand. It The task does topic at hand, or relevant details includes some not contain any only to a modest and examples. relevant details and relevant details or extent. examples. examples. Comprehen- (20 points) (15 points) (10 points) (5 points) siveness of The assignment The assignment Some aspects The task is treatment is compre- is quite of the thesis are incomplete, and hensive in its comprehensive in omitted and some some essential content, cover- its content, covering questions are not questions are not ing all aspects most aspects and answered. answered. and answering answering most the questions of the questions posed. raised. Structure and (20 points) (15 points) (10 points) (5 points) transparency The task is well The assignment is In some parts, the The assignment structured and well-structured and thesis is not well is inadequately transparent. transparent. structured and structured and not transparent. transparent. Total 60 points 45 points 30 points 15 points Source: Adapted from University of Florida: Center for Instructional Technology and Training (https://undergrad.aa.ufl.edu/general-education/gen-ed-courses/rubrics/ ). Assessment Methods in E-Learning 145 Table 17 is a simplified example of a rubric. If the rubric is to provide quality feedback to learners, it needs to be more thorough and detailed. It usual y ex- ceeds one page. More examples of rubrics use are available on the Carnegie Mellon University website: https://www.cmu.edu/teaching/assessment/assesslearning/rubrics.html. The use of rubrics has important advantages for teachers: • the evaluation of participants’ work is (more) objective, based on prede- fined criteria; • the assessment is standardised; • using rubrics, participants get immediate and clear feedback; • time savings (short- and long-term); • analysis of the rubrics assessment results provides a basis for evaluating participants’ achievements; it provides guidance for improving learning and teaching processes; • having more evaluators makes the evaluation more consistent and trans- parent. The use of rubrics is also beneficial for learners: if they know in advance exactly and specifical y what is expected of them, their learning will be more effective and they will be more likely to achieve their learning objectives. Learners also gain a better understanding of the learning process, their achievements and their progress. The use of rubrics contributes to the credibility of the assess- ment in their eyes and increases their trust in the teacher and the institution. These advantages were confirmed by a pilot study with a survey on the use of rubrics at the Doba Faculty (Bregar and Puhek, 2018). One student surveyed added in the survey: ”Using Rubric, I knew exactly where I was losing points and which skil s I needed to improve. Comments alone (the traditional way) do not give as good an idea of the ratings as the new way.” However, one of the evaluators ”initial y had some concerns about how she would be able to transfer or apply her previous (years of) experience to rubrics, but her fears proved to be unfounded as she was able to transition to rubric evaluation without any problems”. The use of rubrics is particularly effective when they are integrated into a digi- tal learning environment, such as the Blackboard LMS. The integration of ru- brics into the digital learning environment allows learners’ achievements to be checked in detail and on an ongoing basis with learning analytics. Using learning analytics, we gain an in-depth analysis of the quality of assessment and information on how to improve the work of teachers, the quality of the educational programme and the performance of students (Section 6.4). 146 The Development of E-Learning Programmes 4.3.4 the Benefits of technology-enhanced testing and assessment The area of assessment is one of those elements of educational programmes where technology is penetrating quite rapidly, even in traditional education. Technology in assessment is still mostly at an early stage of development: the focus is on efficiency and effectiveness in administering knowledge tests, im- proving the validity and reliability of the test results, and developing tests that allow for automated assessment. This approach contributes mainly to streamlin- ing the work of teachers and administrative staff and to reducing costs, but does not bring about significant changes in the quality of the assessment. Despite the diversity of computer-based assessment, assessment strategies are still largely based on the traditional assessment paradigm that has dominated formal educa- tion and training for centuries, based on explicit and clear benchmarks. Redecker and Johannessen (2013) argue that a paradigm shift is needed in tech- nology-enhanced assessment to reflect the needs of modern times. Educators are increasingly aware that curricula and the accompanying assessment strategies, need to shift towards the development of more holistic transversal and generic competencies (Redecker and Johannessen, 2013). Technology offers a solution to such requirements, as it allows for the creation of new assessment modalities that can also cover complex capabilities that are otherwise difficult to assess. The figure below il ustrates the main differences between traditional and next generation (technology-enhanced) assessment, and we briefly describe these differences. Figure 16: differences between traditional and technology-assisted assessment Characteristics of Traditional Next generation assessment assessment assessment Assessment time after learning embedded in learning Accessibility limited universally designed Learning pathway fixed adaptive Assessment Methods in E-Learning 147 Characteristics of Traditional Next generation assessment assessment assessment Feedback delayed In real-time Types of questions generic enhanced Source: US Department of Education, 2017, p. 56. assessment time: assessment embedded in the Learning Process Embedded assessment is integrated directly into the learning activities of e- learning. Such assessments are usual y embedded in digital learning resources or games. It is usual y invisible in the learning process because it is part of the regular activities of the online classroom. Assessment in the learning process can be useful for diagnostic and support purposes as it provides insight into why learners are having difficulty mastering concepts and facilitates tailoring feedback to address these challenges (Shute et al., 2013). Assessment embed- ded in the learning process is also often a part of didactic approaches based on the gamification of learning. Recent research also shows promising ways of using formative assessment practices in e-learning (Toppo, 2015), for example when the scores achieved in the games give a more accurate indication of the knowledge and competencies acquired by the participants. accessibility Advances in technology have made assessment more accessible to a greater number of learners. Specific adaptations for learners with dyslexia and other reading disabilities include the option to increase the font size and change the colour contrast, text-to-speech, bilingual dictionaries, glossaries and more. These features can be used in the assessment and made available to learners, depending on its purpose and the identified needs of the learners. Built-in ac- cessibility features reduce the need for additional support for individual learn- ers, which is beneficial for learners, teachers and tutors. Assistive technologies such as text-to-speech, ’alternate response systems’ and refreshable brail e dis- plays make learning accessible to learners with disabilities. 148 The Development of E-Learning Programmes A good example of the possibilities offered by modern technology to in- crease accessibility is Ghotit (https://www.ghotit.com/), which develops various applications for people with dyslexia, dysgraphia and other writing difficul- ties. Ghotit offers a range of spelling and grammar-checking software on its website. Figure 17: screen image for people with dyslexia and dysgraphia Source: Ghotit (https://www.ghotit.com/dyslexia-software-real-writer-for-windows/ ). adapting Learning Pathway to the abilities and Knowledge of the Participants In technology-enhanced assessment, algorithms can be used to adjust the dif- ficulty of the questions in the assessment based on the answers given by the learners. For example, if a learner answers a question correctly, the next ques- tion or task is slightly more challenging; if they answer incorrectly, he/she may get another chance to demonstrate their knowledge in a different way, etc. The assumption behind adaptive testing is that both the content and the testing can be much more (accurately) matched to the level of knowledge of each learn- er. Adaptation thus leads to a more accurate knowledge assessment for each learner in a much shorter time than could be achieved with traditional testing (Shute et al., 2016). Assessment Methods in E-Learning 149 In 2007, Professor Gangadhara Prusty from the University of New South Wales in Sydney started using the Smart Sparrow tools (https://www. smartsparrow.com/) and created a set of different types of learning material for first- and second-year mechanics students. The project later became known as Adaptive Mechanics. The main feature of this material is that it dynamical y adapts to the feedback and progress of each individual student, offering a personalised learning experience (Prusty and Russel , 2011). Figure 18: smart sparrow toolbox Source: Smart Sparrow (https://www.smartsparrow.com/2018/05/02/creating-more-interactiv e-lessons-with-smart-sparrow ). An analysis of learning achievement shows that student performance has improved over the years. Among the main consequences are a significant reduction in the student failure rate – from 31% to 7% – and an increase in the proportion of students achieving the highest grades – from 5% to 18% (https://www.smartsparrow.com/research/). real-time feedback Technology-enhanced online assessment provides the real-time reporting of results, improving the teacher’s understanding of learners’ achievements and their additional learning needs. Such information helps teachers assess and respond to the learners’ work more quickly than with traditional assessment approaches. Similarly, learners can access this information almost simultane- ously. The technology-enabled final (summative) evaluation also allows the faster communication of results. 150 The Development of E-Learning Programmes Technology-enhanced assessment offers a wider choice of feedback approaches than traditional education. Some formative assessment platforms allow teach- ers to send feedback to learners with comments (video, audio or text), engage in online chats, email feedback directly to learners, and link them to additional resources for learning. the choice of different types of Questions for assessment Technologies allow for different types of questions that go beyond the limited possibilities of ”traditional” questions, such as multiple choice, alternative ques- tions, etc. Questions can be answered: • with a graphic activity: participants ”answer” by drawing, moving, editing or selecting areas of the image; • simulated situations, where participants test their knowledge and competen- cies in simulated environments that closely resemble real-world situations; • by entering equations; • by carrying out various complex activities. Technology-enhanced questions allow participants to demonstrate more com- plex knowledge and share their understanding of the content in a way that would be difficult to assess using traditional means of testing and assessment. The following figure shows an example of two problems from the Maths 4 in- teractive textbook (i-učbenik). Figure 19: maths 4 homework Source: I-učbeniki (https://eucbeniki.sio.si/mat4/index.html). Assessment Methods in E-Learning 151 4.3.5 specific features of assessment in e-Learning Assessment in e-learning is much more complex than in traditional education. If we ignore the differences in the educational paradigm and assume that the concept of student-cantered learning is becoming more and more prevalent also in traditional education, the main reason for the complexity of assessment in e-learning is the spatial separation of the teacher and the learners, especial y if the assessment itself is conducted in a spatial y separated (online) environ- ment.20 In such a situation, it is essential that the e-learning programme is de- signed to compensate for the absence of face-to-face communication through appropriate pedagogical approaches and to set up effective interaction mecha- nisms between teachers, learners and other stakeholders. Gikandi et al. (2011) write that the effective integration of online assessment in e-learning can improve learner-teacher interaction and foster the development of learning communities that promote effective learning and assessment. Up- to-date feedback information is particularly important as there is less chance of live communication between the participants and the teacher. Horton (2012) also emphasises that feedback should be clear, detailed and encouraging. Partici- pants expect and need it, so it is also important that it is delivered on time. If too much time passes between submitting an assignment and receiving feedback, its impact will fade. Ongoing assessment activities are embedded in the learning process itself, so that the continuous assessment of progress is not a threat or a means of control for participants, just part of everyday learning activities. One of the advantages of this approach is that gaps in knowledge or misunderstand- ings of the material can be identified early enough, before they become a major obstacle to the participant’s progress in the programme. Ongoing feedback also reduces the feeling of isolation among participants. While the online assessment is based on an agreed sequence of submissions, this sequence allows participants to use their time relatively freely. The opportunity to test knowledge at your own pace has many advantages for e-learners. Such as- sessments enable adults with many work and other commitments to complete projects and assignments that give them a good demonstration of their learning. The diversity dimension should not be forgotten when designing assessment and evaluation as e-learning involves very different groups of adults. These differ- ences (in age, education, experience, etc.) enrich the learning groups and allow for a lively exchange of experiences, opinions and expectations. But they can also cause problems if the selected method of assessment is more appropriate for one group than for another (some tasks or answer choices are more appropriate for young people, or for adults with a social science background, or for certain occupational groups, etc.). 20 It should be remembered that in this book, e-learning is treated as a generic concept, characterised by the subordination of the use of technology to the objectives of the educational programme. E-learning is manifested in practice in different implementation models, and online learning is a form in which the learning process takes place (entirely or at least predominantly) in a situation of spatial separation. 152 The Development of E-Learning Programmes Learning analytics (LA) and adaptive learning offer new possibilities for adapting assessment to the abilities and characteristics of learners. The digitalisation of the learning process provides the educational institution delivering e-learning with large amounts of data (so-called big data) that can be used to tailor the learning experience to the characteristics of the learners. Solutions based on LA can improve the motivation and learning achievements of learners. More on LA in Section 6.4. The Master’ programme on e-learning and social networks at the Uni- versidad Internacional de La Rioja in Spain uses the iLime personalised learning system. iLime takes into account student interaction in for- mal and informal settings and offers tutoring and assessment features for each individual student (http://research.unir.net/blog/ilime-operational- implementation-of-a-recommendation-model-for-informal-and-formal-learning). Clearly defined assessment rules are particularly important for e-learning, as the assessment process must be understandable to the learner. Instructions are most often given in writing, at the beginning of a unit or module. Therefore, all the requirements – both subject and technical – must be written in a clear and unambiguous manner, otherwise they could be misunderstood by the partici- pants and render the assessment process invalid and useless. In e-learning, there is usual y little or no face-to-face contact, so some classic subjective errors are less likely to occur (e.g., the sympathy-antipathy error). Personal relationships are established through communication in different media. The way we communicate using different media also creates an image of us, just as we get an image of the person we are communicating with. In the online environment, we must not forget the rules of ethical behaviour. Online opens the door to abuse if the assessment itself is not properly organised and if security and plagiarism prevention are not ensured. recommended Links Eberly Center. Carnegie Mellon University: https://www.cmu.edu/teaching/assessment/index.html teAchnology:. https://www.teachnology.com/ Coursera. Assessment for Learning: https://www.coursera.org/learn/assessmentforlearning JISC: Transforming Assessment and Feedback with Technology: https://www.jisc.ac.uk/full-guide/transforming-assessment-and-feedback Selecting and Integrating Media into the Programme 153 4.4 selecting and Integrating media into the Programme 4.4.1 media and technologies definitions and conceptual delimitations By media, we usual y mean activities that produce and transmit information, such as television and newspaper companies, publishing or the Internet. In everyday language, we also use the term ”media” as a synonym for technology. The use of modern media is one of the fundamental features of e-learning. In order to properly understand the role of media in education, the term ”media” needs to be more precisely defined and delineated from the term ”technology”. The media are a means of communication, each in its own way presenting knowledge, information and learning material (Gerlič et al., 2002, p. 20). The elements of the media operation are the source of information, the means of transmitting information and the recipient of information. Bates, one of the leading researchers on the use of technology in education, classifies speech, written text and drama as media. The classroom, the book and the theatre are the technologies that make media work. Technologies do not communicate, they are physical means (Bates and Poole, 2003, p. 48). This understanding of media allows us to consider direct spoken word teach- ing as a medium, even if it is not technological y supported. When considering media in education, it makes sense to also consider direct instruction as one of the important educational media. The role of the media in education can be explained using the communication model. Figure 20: the role of the media in the communication process Communication process Communicator’s experience Receiver’s experience Communicator Encoding Message Decoding Receiver Communication Source: Adapted from Zikmund, 1994, p. 601. 154 The Development of E-Learning Programmes Communication is the process of exchanging messages or information between two or more participants. The typical communication process is learning: the communicator- the teacher – shapes (encodes) the learning message or infor- mation into a form that can be transmitted (e.g., spoken word, picture or writ- ten text). The learning message is then sent using technology to the receiver – the learner – who decodes the message. If the message is delivered in spoken words, the receiver must hear the words and understand their content, and if it is delivered in pictures, the receiver must see and understand the picture. Communication is successful if the receiver – the learner – has the same un- derstanding of the message as the communicator – the teacher. Effective com- munication requires the information given to take into account the experience, competencies and situation of the receiver – the learner. An effective communication process also provides for feedback from the re- ceiver to the message communicator to check that the information received has been correctly understood. The use of different media implies different learning processes with different out- comes and different types or kinds of knowledge acquired. For example, learn- ing about e-learning from the professional literature provides a different type of knowledge than learning about e-learning through direct participation in an e-learning programme. Different media and different technologies allow us to learn about real phenomena from different perspectives (Bates, 2019, p. 558). Different media do not mean that certain media can be labelled as absolute- ly better or worse. Using different media in education is therefore not about choosing a better or a worse option. The fundamental question of media use in education is what mix and combination of media can be used to achieve optimal learning outcomes. It is important to bear in mind that there is no one-size-fits-all medium, whatever its technological appeal, that will suit all educational requirements and circumstances. The usefulness of media will be discussed later in this section. classification of media and features of the communication Process Bates (2019, p. 557) classifies technology-enabled media into these groups: • a text (textbook, novel, poem, research report, etc.); • graphics (diagram, photography, drawing, poster, graffiti, etc.); • audio-media (speech, sound, podcast, radio programme, etc.); • video media and film media (TV programme, film, YouTube clip, video of talk shows); • computer-based media (animation, simulation, online discussion forum, virtual world, etc.). Selecting and Integrating Media into the Programme 155 Media (except live speech) cannot function without adequate technological support. Text can be delivered by a printed publication, by computer or by television. Video can be streamed on TV, computer, smartphone or tablet. For educational applications, it is important to distinguish whether the media allow one-way or two-way communication. Communication can be simultane- ous (synchronous) or delayed (asynchronous). The communication possibilities of different media and their associated tech- nologies vary considerably. The main sources of differences are whether or not feedback from the teacher to the learner (interaction) is possible, and whether contact between them is only possible at the same time or with a certain time delay. Media also differ in whether or not they allow for the spatial separation of teacher and learner. The face-to-face, spoken word contact between teacher and learner requires their simultaneous physical presence in the same place, while technology-enabled media allow the learning process to take place in a situation of spatial separation, an essential feature of e-learning. This classification therefore il ustrates the fundamental strengths of modern media and technologies in the learning process. These technologies make the following possible: • two-way communication, thus ensuring interaction as an essential compo- nent of education, which is what traditional education (with face-to-face contact between teacher and learner) is all about; • to carry out the learning process in a situation of spatial separation, where communication can be simultaneous or delayed: this provides the spatial and temporal flexibility that is an essential advantage of e-learning com- pared to traditional education; • integrating text, audio and video in multimedia form into the online en- vironment and the possibility of accessing a variety of online information resources; this means enriching the possibilities for acquiring knowledge and competencies compared to traditional education. 4.4.2 criteria for media selection In recent decades, dynamic technological developments, accompanied by a rap- id proliferation of new technologies, have enriched and transformed the range of technological options available in education day by day. The availability, ac- cessibility and diversity of media and educational technology offerings are also increasingly improving, thanks to lower prices, continuous technical improve- ments and increasingly reliable and powerful technological infrastructure. In such a context, decisions on the use of media and technology in education require careful professional judgement, whether in traditional or e-learning con- 156 The Development of E-Learning Programmes texts. While the basic orientations on the use of media and related technologies in an e-learning programme are already defined in the strategic and business plan and the programme design, the development of an e-learning programme itself requires a clear definition of the criteria on which concrete decisions will be based. As a general rule, the selected criteria should not be limited to individual aspects of the use of a particular medium for a particular purpose, but should, in line with the strategic and business plan and the programme design, cover broader aspects and respect the pedagogical and didactic component of the pro- gramme, as well as the organisational, human, technical and financial capacities. The assessment selection criteria must be such that they can identify significant differences between the characteristics of individual media and technologies in terms of educational and technical usability. The Bates SECTIONS model for assessing the usability of educational media and related technologies can be used as a starting point for defining criteria for selecting media and technology for education. It has become well established in practice (Bates and Poole, 2003; Bates, 2019, p. 758). The criteria for select- ing educational media in the SECTIONS model are: • S (students). The characteristics of the students (learners) and the appropri- ateness of the media and technology to their characteristics, including in terms of the accessibility of technologies. • E (ease of use). How easy is it for teachers and learners to use technology that supports a particular medium? • C (costs). What is the cost structure for a given technology and what is the cost per learner? • T (teaching and media selection). What educational approaches are needed when using a particular medium or technology? Which media and tech- nologies are best suited to certain educational approaches? • I (interaction). What kind of interaction does a particular technology en- able? • O (organisational issues). What are the organisational requirements and constraints imposed by the use of a particular medium or technology? Are organisational changes needed? • N (novelty)/ (networking). How long has the new technology been in use? / How important is it to enable learners to network beyond a course, with others such as subject specialists, professionals in the field, and relevant people in the community? • S (speed)/ (security and privacy). How quickly can educational programmes be adapted to the use of a particular medium or technology? How quickly can learning material be adapted to new technologies?21 21 In his latest work (2019), Bates uses the ”S” as a measure of security and privacy. The criterion for the speed of adaptation of programmes to a given medium, which in previous works was labelled ”S” (speed), is now included in the criterion labelled ”E” (ease), which deals with the ease of use and reliability of the technology. Similarly, N currently indicates ’networking’, previously N was used for novelty. Selecting and Integrating Media into the Programme 157 Decisions on the use of media and technology in education are taken in differ- ent ways, at the level of the individual, the small group or the whole organisa- tion, depending on the institutional environment and the organisation of the education system in general, the type of educational organisation and, ulti- mately, the importance of the decision itself. The financial resources needed must also be taken into account. The role of the individual is rather marginal when it comes to the strategic decisions of the educational organisation, but then the organisation’s management can be counted on to support the intro- duction of technology in education. If the decision to modernise education using technology is taken by an indi- vidual or a small group, they are in principle more free to do so, with less support from the organisation and therefore more limited possibilities of securing the necessary money. Regardless of whether, or at what level, decisions are taken, this must take into account: • the purpose, the basic aim of the programme and the learning objectives; what the participants need to be able to do, demonstrate and produce at the end of the learning? • which media are available, what each media enables, considering the learn- ing context; do they support the learning objectives; what are the specific requirements of a particular learning activity, as well as the characteristics of the learners; • the costs of using each medium, how much time we have, whether we have the right skil s, what the technological constraints are. Sometimes we may not be able to choose the most appropriate medium be- cause of constraints. Nevertheless, the media and technologies chosen must be capable of delivering quality education. We need to be aware that text is the most appropriate medium in certain circumstances and that there is no ideal medium that is suitable for all circumstances. 4.4.3 usability and Limitations of media in education One of the most important features of e-learning is the use of technology- supported media. This can take many forms and combinations of media use, from the use of a single medium, such as sound, to the simultaneous use of several different media. The simultaneous use of several different media has led to the term ”multimedia”. Historical y, the term ”multimedia” first meant the simultaneous use of several media supported by different technologies. Today, ”multimedia” means the integration of video, sound, graphics and recorded 158 The Development of E-Learning Programmes data, supported by the same technology, most often a computer. We also talk about ”media-rich technologies” . Website Multemedia https://sites.google.com/site/multeamedia/7-1/examples-of- multimedia-systems offers plenty of useful advices and examples how to create educational and other multimedia. What are the characteristics of education using technology-enhanced media compared to education without it, i.e., education based on face-to-face, spo- ken-word teaching? The following table summarises the main differences. Table 18: comparison of human and technology-based education Education Education without technology- Education with technology- element based media enhanced media Planning and More flexibility to align delivery with Prior systematic coordination of the preparation programme design design with the plan is required Expertise The opportunity to provide up-to-date Risk of outdated knowledge in knowledge and high professionalism case of the inadequate selection of with the involvement of experts from experts and insufficient updating of practice content Interactivity Education focused on the group, not The programme can be tailored to the individual the individual in terms of content, pace, type of activity, etc. Learning Variable Can be much bigger than traditional retention education Consistency Adaptation to the group, hence loss of In principle, strict compliance with consistency the standards set, but may also adapt to learner’s performance or preferences, if designed to do so possible deviations if foreseen in advance Feedback, Particularly effective for ongoing Easy recording and documentation performance evaluation and feedback on of outcomes; possibility to design tracking performance systems with automatic adaptation to the evaluation findings. Source: Anderson and El oumi, 2004, p. 151. Media in e-learning are transforming the educational process and, if used ap- propriately, can greatly improve the quality of education by allowing greater flexibility in the design and delivery of learning programmes (by adapting to different learning approaches; flexibility in time, pace, space and content), encouraging the development of more complex thought processes, allowing Selecting and Integrating Media into the Programme 159 greater learner autonomy in learning, allowing greater authenticity of the learning experience and interaction between learners despite spatial separa- tion. While the availability of technologies that enable the integration of media in education has improved significantly in recent years due to their increasing ease of use and affordability, a wide range of factors make it difficult to use media more widely and more quickly in education in general. The accelerated integration of modern media is hampered by the inability of educators to inno- vate, by fear and mistrust of technology, by ignorance and lack of control over the changes brought about by the use of multimedia, by lack of money, and by unsuccessful attempts. The introduction of media in education is based on the design of the e-learn- ing programme and its content features. In this context, we take into account the strengths and limitations of each medium when deciding how to combine and integrate different media in e-learning programmes.22 recommended Links Media & Learning Association: https://association.media-and-learning.eu/portal/resources Mediasite. Education: https://mediasite.com/education/ 22 The most significant limitations and advantages of each medium are described in the handbook ”The E-Learning Essentials” (Bregar et al., 2010, pp. 139-147). For more detailed guidance on integrating the media into e-learning programmes, see Bates, 2019: Chapter 8: Pedagogical differences between media. 160 The Development of E-Learning Programmes 4.5 digital tools for e-Learning 4.5.1 digital tools for e-Learning Today, a wide range of digital learning tools are available to support the devel- opment and delivery of e-learning programmes, from the simplest to complex systems supported by large databases. Individual tools can be specialised and used only for individual tasks (content preparation, assessment, etc.) that are relevant for e-learning, or they can be integrated systems with the possibility of carrying out a large number of different tasks. By digital tools for education, we mean LMSs and authoring tools. LMSs are digital tools that enable the preparation and presentation of online materials, various communication systems, testing, assessment of learners’ knowledge, monitoring of learners’ learning and progress, and the manage- ment of the e-learning programme, all in one package in some standard for- mats. LCMS used to be different from LMS. Now these two concepts are inter- changeable. Like an LMS, an LCMS allows the creation, storage and de- livery of learning content in e-learning programmes (Lawless, 2018). As LMSs also allow content creation, this is the term that has gained ground in practice and is used in this book. An authoring tool23 is a software for developing content for web-based e-learning programmes, or applications used to develop eLearning products (Berking, 2016). 23 In the e-learning literature and glossaries, we come across different names, definitions and classifications of authoring tools. Here are some other definitions of an authoring tool. An authoring tool is ”a user program for creating documents, web content and learning materials, e.g., wikis” (I-slovar, 2019). An authoring tool… ”is a software application or program that allows people to create their own e-learning courseware” ... (Kaplan Leiserson, n.d.). The authoring tool is the software used to complete the e-learning programme. It is a tool used to combine all the elements of a programme (such as text, graphics and, questions) and turn individual screens into a complete programme (with pages, navigation, menus and buttons), (Elkins and Pinder, 2015). Digital Tools for E-Learning 161 Since 2007, an annual list on Top Tools for Learning has been compiling from the results of an open survey carried out by the Centre for Learn- ing & Performance Technologies (C4LPT) – one of the world’s leading websites on learning trends, technologies and tools. The list24 includes online resources and applications (e.g., YouTube, Wikipedia, TEDTalks/ Ed, Vimeo), personal tools (e.g., Kindle), social networks (e.g., Facebook, Twitter, LinkedIn), platforms (e.g., Coursera, Udacity), content produc- tion tools (e.g., PPT, Photoshop), interactive content production tools (e.g., H5P, HiHaHo), team col aboration tools (e.g., Slack, Yammer), LMS (e.g., Moodle), video conferencing tools (e.g., Zoom), quiz and test tools (e.g., Kahoot), etc. Figure 21 shows the list on Top Tools for Learning for 2022. The digital tools are grouped into three categories: • the tools for personal learning; • the workplace learning tools (digital tools for designing, implement- ing and supporting the workplace learning); • the tools for education (digital tools used by educators and learners in schools, universities and adult education). The top 10 digital tools are shown in the middle of Figure 21, as the area of the intersection of the three circles of tool categories. Figure 21: 100 top-rated digital tools for learning in 2022 Source: Top 100 Tools for Learning, 2022 (https://www.toptools4learning.com/about/ ). 24 24 The understanding of the term ”digital tool” in this list is broader than the one used in the book. 162 The Development of E-Learning Programmes On the website of the Didakt.UM project at the University of Maribor (https://didakt.um.si/stolpic/strani/default.aspx), a list of tools (with short descriptions) suitable for modernising the higher education teaching process is available. A wide range of tools are included – for example, tools for sub- titling, audio recording and editing, video editing, interactive col aboration, online virtual whiteboard col aboration, various web applications, etc. In addition to the LMSs, other digital tools can be used to deliver e-learning programmes. Implementation models of online education from a technologi- cal perspective according to Siemens and Tittenberg (2009, p. 37) are: • use of LMS such as Blackboard, Moodle; • using a combination of digital tools, e.g., blogs, wikis, Skype, discussion forums; • use of online lectures (synchronous) with virtual classrooms supported by an LMS or a combination of digital tools; • use of podcasts, video lectures and OER, either with an LMS or a combina- tion of digital tools. The 2009 classification refers to e-learning in the context of Web 2.0. Since then, many new digital tools have emerged, some have been updated and oth- ers have fallen into disuse. This is also true for many LMSs. These have been further developed and up- dated, and a number of new ones25 have appeared. LMSs are becoming an in- tegral part of the wider digital learning environment. At the core of a digital learning environment are LMSs, around which other digital tools are grouped, like Lego blocks. The Lego bricks approach leads to the next-generation digital learning environment (NGDLE), ( Baule, 2019). 4.5.2 Learning management systems Lms – general aspects LMSs offer a wide variety of implementation options. Because LMSs include the use of different media in one package (text, audio, video, etc.), they allow for different learning experiences and support different learning approaches by the learners. The decision on which elements will be made available to participants in a particular e-learning programme is based on the anticipa- tion of which learning approaches and methods will best help the participants to achieve their learning objectives, taking into account their characteristics 25 According to some estimates, there were around 700 LMS available in 2019 (Godsey, 2019). Digital Tools for E-Learning 163 and needs (Section 3.3 Educational Needs Analysis and Section 3.4 Instruc- tional Design of an E-Learning Programme). However, we have to take into account certain constraints in terms of money, staff and time to develop the programme, which we have considered in the business plan (Section 2.2). As the LMSs have evolved and been updated, and as new tools have emerged, the expectations of the features that the LMSs should provide have changed. The primary role of an LMS, according to Allen (2016), is to support the learn- er by: • ensuring enrolment and access to the e-learning programme for partici- pants; • automatical y collecting and analysing data and reporting on the partici- pants‘ learning progress; • facilitating optimal learning by guiding the individual learner towards the learning events that are most appropriate for them, based on an analysis of their learning needs, achievements and activities; • providing feedback. Everyone involved in the development or delivery of an e-learning programme has predefined rights to the features in the programme they can access or use. The allocation of rights to individual actors depends on the implementation model of the e-learning programme. These rights, together with the username and password for access, shal be assigned by the computer administrator as agreed according to their role and responsibilities in the development or delivery of the programme. This ensures the security and privacy of Internet services (no unauthorised access and no tasks for which you have no pre-assigned rights). experience to date with the Lms Recent literature on digital tools for e-learning shows that less attention has been paid to LMSs and more to authoring tools. The use of LMSs in e-learning has become more widespread in recent years, especial y in formal education and higher education. LMSs are also increas- ingly present in the training of company employees (Origin Learning, 2018). These are mostly second-generation LMSs that already include Web 2.0 tools. The current predominant use of LMSs in e-learning largely follows the model of traditional education. The teacher selects the learning material. Learning process is structured by modules or organised by weeks. Learners can take part in discussion forums and learn from the material, usual y all at the same pace (on a weekly basis). At the end of the learning process, there is an assessment in the form of tests or written assignments. The main differences to traditional education are: 164 The Development of E-Learning Programmes • content is delivered primarily as text rather than oral y (although the inclu- sion of audio and video in LMSs is increasing); • communication is mostly asynchronous (synchronous in traditional edu- cation); • online content is accessible at all times from anywhere with an Internet connection. However, LMSs have the potential to be used in a way that represents a departure from replicating the traditional model of education. How much, if at al , their different options will be used depends mainly on the teacher (Bates, 2016). E-learning using an LMS, when making even modest use of its potentials, represents an advance over some programmes that are presented as e-learning programmes and offer nothing more than PPT presentations in pdf format or video recordings of lectures online (Bates, 2016). The 2017 analysis of the state of digitalisation and e-learning in higher education in Slovenia shows that Slovenia is lagging behind in terms of the level of LMS uptake compared to European countries, as well as in terms of the way in which LMSs are used. 70% of the surveyed higher education institutions in Slovenia have an LMS. All the institutions sur- veyed use them for publishing learning materials online, many of them for the administration of the educational process, and only a few for communication (blogs, discussion forums and chat rooms), (Bregar and Puhek, 2017). Similar findings are also presented in the analysis of the state of the di- dactic use of ICT at the University of Ljubljana. Almost all members us- ing the LMS use it for various exercises, the submission of student work, communication with students and for the educational process. However, the vast majority of members use the LMS to publish scripts, past exami- nations and other documents. The most frequently mentioned LMS is Moodle (Radovan et al., 2018). Moodle is one of the most widely used LMS in the world. According to the Moodle website (https://moodle.net/sites/), there are currently (18 Sep- tember 2022) 168,195 registered websites in the world, 306 of which are in Slovenia (in Slovenia, these are mostly primary school websites). Some other e-learning experts are also critical of the current practice in the use of LMSs. Allen (2016) advocates the use of LMSs to support e-learning, but is not impressed with the way many LMSs are used in practice. He points Digital Tools for E-Learning 165 out that LMSs provide good administrative support in terms of learner enrol- ment and data collection. A similar view can be found in the article ”7 Things You Should Know About NGDLE” (Educause, 2015). It states that LMSs have proven to be very useful for administrative tasks in e-learning, but less for ef- fective learning support. Lms Perspectives The main value of an LMS today is not to administer the educational process, but to enable assessment of the learner’s progress and needs, select the optimal learning experience for him/her from those available by LMS, measure the learners’ performance at the level of individual and of group, evaluate it using learning analytics (LA) and according to the evaluation results define amend- ed learning experience in the frame of LMS. LMSs are expected to provide more LA than ever before to improve the learners’ progression performance (Allen, 2016). Many LMSs (including Moodle and Blackboard, for example) integrate quite complex LA modules, but they are not widely used in practice, mostly only at the level of individual projects or for research purposes (see Section 6.4 for more on this). The use of LMSs also does not exploit the technological poten- tial of Web 2.0, which is widely used in other areas outside education. Some LMSs are declining in popularity. On the website https://www. toptools4learning.com/about/ an alphabetical list of digital tools from 2007 to 2022 is available with their ranking based on international voting. The survey results show that the LMSs are getting worse and worse on the list of popularity. For example, Moodle went from 12th place in 2007 to 39th place in 2022, Canvas from 57th place in 2021 to 72nd place in 2022 and Blackboard is even not included on the top 100 tools list for 2022. Today’s learners expect to be able to access learning content according to their needs at any time and from anywhere. If LMSs are to survive, they should follow this trend and support different modes of learning. The move away from tradi- tional LMSs is already evident in the fact that many organisations offer content as a separate service from the LMS (for example, in the cloud). The survival of LMSs could be ensured by two functions of LMS (for which the need will not disappear), namely – the assessment and monitoring of the learners’ progress (Berking and Gal agher, 2016). 166 The Development of E-Learning Programmes What are the characteristics of a good LMS? A post on the Origin Learning website https://blog.originlearning.com/what-defines-an-ideal-learning-management- system/) lists the following: • modern, user-friendly, intuitively designed, • with diverse features, • interoperable with other ICT systems in the organisation, integration ca- pabilities, • enables learning to be tracked, enables real-time analytics, enables social and personalised learning, enables social and personalised learning on LMSs, • includes the ability to market and sell educational programmes, includes the ability to market and sell courses using the LMS, works as an app on your smartphone, a full-function LMS experience on the mobile app, • enables customer support & pricing. LMSs should adapt to development trends or be replaced by NGDLEs, as noted at the beginning of this section. They will be powered by artificial intelligence (Aldridge and Powel , 2018; Educause, 2015). The NGDLE is expected to in- clude a traditional LMS as only one of its components (Brown et al., 2015 in Berking and Gal agher, 2016). The NGDLE is intended to be a kind of federation of IT systems and appli- cation components that would conform to common standards. Compared to traditional LMSs, the NGDLE is intended to follow the concept of component- based design, allowing teachers and learners to customise their digital learning environments to support their needs and goals. New environments should be characterised by: • interoperability and integration (the ability to interconnect and operate and to combine different systems or digital tools), • personalisation (digital learning environments and activities should be tailored to the needs of individuals and educational department or pro- grammes), • learning analytics, counselling and assessment, • participation, • accessibility and universal/uniform design. In this way, NGDLEs would work similarly to smartphones by combining con- tent elements and functionalities that would not be the same for any individual participants, teachers, departments or organisations (Educause, 2015). Digital Tools for E-Learning 167 In the article Modern E-Learning Technologies (Klobučar, 2013), experi- ences with the practical use of a personal learning environment in the iCamp project and in three online courses at the Doba Faculty are pre- sented. This is an innovative approach in e-learning, where the student builds their personal learning environment during the learning process (choosing the tools, learning material and learning resources, the layout of the digital environment, etc.). This raises a number of issues (the in- terconnectivity of different tools, digital literacy of students and teach- ers, student autonomy, etc.). For more information, see this article on the website https://journal.doba.si/oJs/index.php/jimb/article/view/154/167. 4.5.3 authoring tools A few years ago, you had to know how to code to create e-learning programmes. Now, however, there are more and more tools that are quite easy to learn to use, even for non-computer experts. These tools include for instance Trivantis Lec- tora, Articulate Studio and Storyline and Adobe Captivate. Such tools allow the rapid development of e-learning programmes. Their disadvantage is that they already have certain functions or features that are part of the software, so they cannot provide all the desired functions or elements that we had in mind when we designed the e-learning model. There are also tools available on the market that allow you to prepare 100% of the material in PPT and con- vert it into a web version that can then be placed in an LMS. Some tools allow you to add e-learning elements or features, such as interactions, quizzes and tracking learners’ progress in PPT. These include Articulate Studio and Adobe Presenter. Such tools allow for the rapid production of e-learning content, but this requires caution and the thoughtful use of technological possibilities in the design of the software itself, as there is a risk that the final product will look more like an online presentation of content rather than e-learning (Elkins and Pinder, 2015). E-learning tools can also be used to deliver e-learning content in a variety of ways. Many of the authoring tools are relatively easy to use, as they work on the principle of ”what you see is what you get” . This reduces the need for skilled computer professionals with programming knowledge (Berking, 2016). In practice, developers rarely use only one authoring tool. Most use a combi- nation of several authoring tools, many even four or more. They usual y use one authoring tool as the primary tool to be used as a shell or base screen, and combine the others into an eLearning product. These are secondary authoring tools that are optimised for specific functions. These tools are different from 168 The Development of E-Learning Programmes ancil ary software tools (such as word processors). These are not stand-alone authoring tools and can only be used in conjunction with authoring tools (Berking, 2016, p. 7). Authoring tools allow us to build an e-learning programme as a whole or to develop a single element of the programme, which we then integrate into the primary tool or LMS. However, some tools or LMSs already allow the prepara- tion of these individual elements, so we do not need any other or additional tools. Authoring tools can be classified according to the following aspects: • complexity (they can be simple or more complex – these require computer programming skil s); • payability (they can be commercial or free); • purpose (specialised in e-learning curricula or more broadly applicable), (Haghshenas, Khademi and Kabir, 2012). 4.5.4 the choice of digital tools Rapid technological developments are also reflected in many new or updated digital tools used for the development and implementation of e-learning, and in their diversity and variety of applications. The authoring tool (or LMS) can be selected in different ways, based on differ- ent procedures. Here is an example of the process of selecting a digital tool (LMS and/or au- thoring tool). The LMS and authoring tool selection processes are similar, so we will not show them separately. The choice of an LMS is a bit more complex than the choice of an authoring tool, due to its greater complexity and the size of the system. The individual steps common to both processes (when choosing an LMS or authoring tool) are: 1. identification of the main requirements (in terms of functionality and usability, which are different for each user group (learners, computer ad- ministrators, teachers, tutors, etc.)); 2. cost-benefit analysis (it is not sufficient to estimate the funds available to buy the tool, you also need to consider the costs of any adaptations, sup- port and training, etc.); Digital Tools for E-Learning 169 3. identify the categories of tools needed (the complete LMS or just one component, the type of authoring tools depending on the type of training to be delivered); 4. identification of specific tools to support the type of educational pro- gramme you want to deliver; 5. preparation of a table with the most important requirements for compar- ing the different tools; 6. filtering the list of potential tools (removing those that do not meet the mini- mum requirements or that exceed the organisation’s financial capacity); 7. preparing a list of detailed and complete requirements for the remaining potential tools and sending the request to the providers; 8. preparation of a more detailed table to compare the features/functions of the remaining individual tools and to assess the suitability of the provid- ers of the selected tools; 9. contacting the providers (three to a maximum of five) that have been rat- ed the highest; asking them to allow a trial of the tool; asking about other elements of the providers’ quality (quality of support, reputation, respon- siveness, customisability of individual features, etc.); 10. extend the spreadsheet by adding the new information obtained in step 9; 11. making a decision based on all the information gathered, together with that on training, the provision of tool updates, maintenance of the necessary software, user support, etc. (Berking and Gal agher, 2016; Berking, 2016). Vendors of paid LMSs should allow interested buyers to try the LMS free of charge for a certain period. The eLearning Industry website https:// elearningindustry.com/the-best-learning-management-systems-top-list presents infor- mation on the 10 top-rated LMSs (along with short descriptions of the LMSs). Most of them offer a free demo version. The following table shows an example of an evaluation of authoring tools. 170 The Development of E-Learning Programmes Table 19: example of comparing the characteristics of authoring tools Feature/ Weights Authoring Authoring Authoring Authoring function tool Tool tool tool of the A B C D authoring tool Price 15 4 0.60 1 0.15 3 0.45 2 0.30 Ease of use 10 1 0.10 2 0.20 4 0.40 3 0.30 Graphical 5 1 0.05 2 0.10 4 0.20 3 0.15 capabilities Animations 10 3 0.30 1 0.10 4 0.40 2 0.20 Interactivity 15 2 0.30 3 0.45 1 0.15 4 0.60 Quizzes 10 2 0.20 4 0.40 1 0.10 3 0.30 Power capacity/ 10 3 0.30 4 0.40 1 0.10 2 0.20 flexibility ADA26/ 5 3 0.15 4 0.20 1 0.05 2 0.10 accessibility Use on mobile 15 4 0.60 3 0.45 2 0.30 1 0.15 devices Simulation 5 4 0.20 2 0.10 1 0.05 3 0.15 software TOTAL 100% 2.80 2.55 2.20 2.30 Source: Adapted from Elkins, 2016 (https://www.td.org/insights/finding-the-right-authoring-tool-for-you ). 26 On the left side of the table, we enter the features or functions of the author- ing tool that are relevant for our e-learning programme. Some of them can be elaborated further and additional functions can be added to the spreadsheet as you wish. Then, we prioritise each individual function by re-prioritising it. In this case, the weights are expressed as a percentage and the sum of all the values is 100%. The features of the selected authoring tools (shortlist) are then scored according to how well they meet our criteria. The maximum possible score, in this case, is 4 and the minimum is 1. The number of points scored for the individual characteristic assessment is entered in the left-hand column under each authoring tool and then multiplied by the weighting. The decision 26 ADA stands for the Americans with Disabilities Act (ADA) Standards. These are standards issued by the US Department of Justice that require all electronic and information technology to be accessible to people with disabilities. Digital Tools for E-Learning 171 on which authoring tool to use is based on the sum of the weighted scores. We choose the authoring tool that scores the highest weighted points. There is a lot of information available online about authoring tools to help you choose: • The Capterra website, Course Authoring Software, provides a range of information on authoring tools, including their ratings (https://www. capterra.com/course-authoring-software/). • The PCMag website provides more general advice on choosing au- thoring tools and also allows you to compare them in terms of price and features (https://www.pcmag.com/roundup/348022/the-best-elearning- authoring-tools). • The eLearning Industry website https://elearningindustry.com/directory/ software-categories/elearning-authoring-tools provides a list of authoring tools for e-learning, which can also be compared with each other. The Learning GUILD has published its 2021 report on the use of author- ing tools for eLearning, based on responses from its 808 members. Most respondents (69.1%) used Storyline (Articulate), 53.2% Rise 360 (Ar- ticulate), 47.0% Camtasia (TechSmith), and 26% Captivate (Adobe). All other authoring tools were used by less than 20% of respondents. recommended Links Capterra: https://www.capterra.com eLearning Industry: https://elearningindustry.com The eLearning GUILD: https://www.elearningguild.com/ Učni stolpič – IT orodja za sodoben pedagoški pristop: https://didakt.um.si/stolpic/strani/default.aspx General Features of Pedagogical Support in E-Learning 173 Pedagogical SuPPort 5 5.1 general Features of Pedagogical Support in e-learning One of the fundamental features of e-learning is integrated learner support systems. Their main purpose is to alleviate the problems caused by the spatial separation of the learner and the teacher in the process of such education. The e-learner, who learns mostly on his/her own and independently, needs cer- tain competencies and attributes that are not so prominent in traditional educa- tion in order to successful y achieve the learning objectives. These are (Simpson, 2015, p. 2): • intellectual ability, • good study habits, • motivation to learn, • self-confidence, a positive attitude, • ability to deal with stress, • time management, • the ability to balance family, work and other commitments. In addition to personal qualities and abilities, the e-learner needs different types of support and assistance. Pedagogical support systems for learners in- clude learning material and other elements of the e-learning programme (learning activities, various forms of assessment, etc.) designed to enable the achievement of the learning objectives in a situation of the spatial separation of learners and teachers. To achieve these goals, learners also need support in the form of a range of services, organised by the training organisations and provid- ed by tutors, counsellors, administrative and technical staff. In general, tutors provide learning-related support to learners, which, together with the learning 174 Pedagogical Suppor P t material and other elements of the programme, constitutes pedagogical sup- port; while counsellors, administrative and technical staff are involved in per- sonal counselling and other types of counselling that are not directly related to the pedagogical process ( non-pedagogical support). In practice, tutoring support is often intertwined with forms of non-teaching support, depending mainly on the organisational model and staffing capacity of the organisation. Figure 22: Basic types of support in e-learning Support in e-learning Pedagogical Non-pedagogical support support Learning material Tutor and aids support Non-pedagogical support is discussed in Section 7.2. The learning material and other elements of an e-learning programme are examined from different perspectives in Part 4 of the book and in Section 6.227. In this part, we focus on issues related to more narrowly defined pedagogical support, i.e., tutoring support systems. Tutoring support in e-learning is conceptual y rooted in the traditional dis- tance education model from a period before ICT was widely used. The dif- ferences lie in the way tutoring support is delivered. In traditional distance education, tutorial support took the form of face-to-face group tutorials or face-to-face individual contact between the learner and the tutor (form of blended learning), telephone contact, written messages, and the use of (mainly printed) teaching material prepared in accordance with the pedagogical spe- cificities of distance education. Modern technologies offer many new possi- bilities for the implementation of pedagogical support in e-learning, both in terms of the use of media and technology and in terms of the characteristics of communication. 27 In ”The E-Learning Essentials” handbook (Bregar et al., 2010), we have discussed in detail the different types of online learning materials (sub-section 5.2.1). General F Gener eatur al F es of P eatur edagogical Suppor es of P t in E-Learning 175 Figure 23: Pedagogical support in e-learning Pedagogical support in e-learning Media: Technologies: Communication: written text, print, asynchronous, audio recordings, broadcasting services, synchronous video recordings, telephony, mobile telephony, multimedia computer and internet technology When deciding which media and technologies and which forms of pedagogical support to use in an e-learning programme, we need to consider the charac- teristics of the learner, the learning material, the content and the characteris- tics of the digital learning environment and, of course, the resources available. This was discussed in Section 4.4. It is particularly important to anticipate the expectations of pedagogical and non-pedagogical support that the learner has when starting e-learning. In particular, the learner expects: • accurate and valid information; • flexible learning, adapted to different learning approaches; • advice on what and how to learn (content and methods); • the opportunity to exploit one’s own experiences efficiently for learning; • feedback on learning performance; • assistance with administrative and other problems related to the pro- gramme. When it comes to the way communication is used, the main distinguishing fac- tor between the communication tools is whether the communication is asyn- chronous or synchronous. In traditional education, communication between teacher and learner takes place simultaneously, synchronously and in the same place, while in e-learning, the digital learning environment allows communi- cation to take place either synchronously or asynchronously, i.e., in real-time, or independently of time and, as a rule, independently of space28. The following table shows some typical didactic approaches and methods in terms of the adaptability of education to time and place. 28 We have described the synchronous and asynchronous communication methods in detail in ”The E- learning Essentials” handbook on pages 235–253. 176 Pedagogical Support Table 20: Flexibility of education in time and place 5.2 the changing role of teachers and Place learners Same Different Live (face-to face) Webinars, In addition to appropriate learning material and other elements of the educa- tional programme adapted to the specifics and possibilities of e-learning, an Same media: lectures, video-conferencing, Synchronous seminars, tutorial, virtual worlds, important factor for quality pedagogical support is qualified and quality tutors labs, workshops remote labs who are able to organise and deliver this education effectively and profession- Time Recorded media: al y from a pedagogical point of view. books, cassettes, Self-managed labs/ LMSs, online In e-learning, tutors are a kind of interface between the organisation organis- Different workshops/ discussion forums, Asynchronous ing the learning process and the learners. Therefore, the tutor needs to have studios, lecture capture/ a good knowledge of the tools, techniques and learning and teaching meth- library/learning centres streamed video, ods that are suitable for e-learning. It is useful to have teaching experience blogs, wikis and traditional pedagogical and andragogical knowledge. That’s why tutors Source: Bates, 2019, p. 586. are usual y selected from among teachers in traditional education, and their specific knowledge and comptencies are acquired through additional training. The specific features of e-learning, which are reflected in the adaptability of ed- The characteristics of the tutor’s work and will be discussed in more detail later ucation to space and time, also require specific forms of pedagogical support. in this part of the book. The tutor is involved in several activities, such as supporting learners, lead- ing discussions, col aborating and moderating. This is done mainly through written communication (discussions in forums, via email, blogs, chat rooms and other means). Interactions can be simultaneous, ”live” (synchronous com- munication) or not time-sensitive (asynchronous communication). E-learning also allows the use of various educational media and the use of multimedia, audio and video, graphics and the ability to exchange materials and opinions in a shared virtual environment. All of this has an impact on the characteristics of learning process, as well as on the role of the tutor and the learners. Table 21 shows the differences between the role of a teacher in traditional edu- cation and that of a tutor in e-learning. Table 21: comparison of the teacher’s role in traditional education and the tutor’s role in e-learning Teacher in traditional education Tutor in e-learning Omniscient and a lecturer. A consultant and guide, advising on the choice of learning resources. Offers answers. An expert who asks questions. Is the only source of knowledge. Provides learning experiences. Dictates the structure of the participants’ work. Encourages and facilitates independence and initiative in participants. The Changing Role of T The Changing R eacher ole of T s and Lear eacher ners ner 177 5.2 the changing role of teachers and learners In addition to appropriate learning material and other elements of the educa- tional programme adapted to the specifics and possibilities of e-learning, an important factor for quality pedagogical support is qualified and quality tutors who are able to organise and deliver this education effectively and profession- al y from a pedagogical point of view. In e-learning, tutors are a kind of interface between the organisation organis- ing the learning process and the learners. Therefore, the tutor needs to have a good knowledge of the tools, techniques and learning and teaching meth- ods that are suitable for e-learning. It is useful to have teaching experience and traditional pedagogical and andragogical knowledge. That’s why tutors are usual y selected from among teachers in traditional education, and their specific knowledge and comptencies are acquired through additional training. The characteristics of the tutor’s work and will be discussed in more detail later in this part of the book. The tutor is involved in several activities, such as supporting learners, lead- ing discussions, col aborating and moderating. This is done mainly through written communication (discussions in forums, via email, blogs, chat rooms and other means). Interactions can be simultaneous, ”live” (synchronous com- munication) or not time-sensitive (asynchronous communication). E-learning also allows the use of various educational media and the use of multimedia, audio and video, graphics and the ability to exchange materials and opinions in a shared virtual environment. All of this has an impact on the characteristics of learning process, as well as on the role of the tutor and the learners. Table 21 shows the differences between the role of a teacher in traditional edu- cation and that of a tutor in e-learning. Table 21: comparison of the teacher’s role in traditional education and the tutor’s role in e-learning Teacher in traditional education Tutor in e-learning Omniscient and a lecturer. A consultant and guide, advising on the choice of learning resources. Offers answers. An expert who asks questions. Is the only source of knowledge. Provides learning experiences. Dictates the structure of the participants’ work. Encourages and facilitates independence and initiative in participants. 178 Pedagogical Support Teacher in traditional education Tutor in e-learning Mainly draws attention to just one aspect of the Highlights the different aspects of the content content. and how they are linked. The only one who teaches. The tutor is part of the learning team. Has full control over learning environment. The tutor works on an equal basis with the participants. Treats all participants equally. The tutor adapts to the participants’ learning approaches where possible. The role of the teacher is authoritative. Participants and tutors are collaborators in a shared learning process. Source: Adapted from Goodyear, 2001, pp. 89–91. The teacher and the tutor provide professional guidance and leadership throughout the learning process, motivating, engaging, assessing and dealing with many of the problems that arise during the programme. The tutor’s role differs from the teacher’s mainly in the scope and the way they communicate. In e-learning, there is usual y more interaction between the tutor and the learner than in traditional education, where the emphasis is on group communication. Often, the role of the teacher or lecturer is limited to imparting knowledge and instruction on the subject to the whole group, and contact with the learners is often limited to lecture time. Even though traditional education is becoming increasingly computerised and the staff in educational organisations are more connected to the learners (by email, the school web portal or online classroom) than they used to be, the distribution of roles has not changed much. The teacher tel s the group exactly what needs to be learned and then lectures or explains it. In e-learning, the tutor is most often the one who guides, moderates and interacts with the learners. If they want forums and chat rooms to flourish, they need to stimulate, guide and evaluate discussions. Their tasks, therefore, include organis- ing and moderating learning, in addition to contributing to the debate. Collison et al. (2000) also understand the role of the tutor (facilitator) accordingly. The role of the tutor is divided into three categories: advisor/mentor29, trainer/project leader and facilitator of group processes. All three roles can change with different activities during education. One of the tutor’s abilities (which will be mentioned later in the text) is to be able to adapt to the group. Tutors, who come into (live or virtual) contact with learners, represent the core of e-learning staffing for an 29 In practice, the terms ”tutor” and ”mentor” are often used interchangeably. However, the role of mentor is broader and involves providing technical and personal support to learners, while tutor is more specific and focuses on providing academic support to help learners achieve their specific learning goals. The latter understanding of the two terms is also used in this book, except where referenced. The Changing Role of T The Changing R eacher ole of T s and Lear eacher ners ner 179 educational organisation. All other professionals involved (administrators, au- thors of learning materials, etc.) in the development and delivery of e-learning general y have less contact with the learners. Modern educational technology can complement and often replace the work of a tutor. With the development of learning analytics systems, the tutor can closely monitor the dynamics and performance of a learning group or an individual and adjust the activities in the online classroom on the fly (see Section 6.4 for more on learning analytics). Tutor work will be partly replaced in the future by ITS, which allows teaching and learn- ing to be personalised to the learner and feedback to be given without the need for tutor intervention (more on this in Section 6.5). The differences between the teacher’s role and the tutor’s role also have an impact on the role of the e-learners. Greater equality and activity in learning process requires much more time and effort on the part of the learners, as well as more self-discipline and motivation due to the objectives that dictate the pace of learning. Table 22: roles of learners in traditional education and e-learning Traditional education learner Learner in e-learning Passive knowledge receiver Creator of your own knowledge Learning facts by heart Solving complex problems One-sided treatment of content Addressing content from different perspectives Learning in isolation from others (no contact Group learning, cooperation with other participants), working on your own task Subject to the teacher’s assessment Formulating your own questions and finding your own answers Active in one cultural context Increased multicultural awareness No impact on the pace and schedule of Autonomous and independent management learning of time and learning process Getting feedback only from the teacher Discussing learner’s work and progress with the tutor and other participants Learning emphasises the reproduction of the Focus on knowledge creation and application teacher’s knowledge Use only prescribed teaching materials Access to multiple learning resources Source: Adapted from Goodyear, 2001. 180 Pedagogical Support Both tables show that the roles of the tutor and the learner are quite differ- 5.3 the tutor’s competencies in e-learning ent in e-learning compared to traditional education. A successful tutor who wants to offer an effective learning experience to e-learners needs to know and understand the principles of e-learning, the possibilities offered by the digital The tutor acts as an intermediary between the institution and the e-learner. learning environment, the tasks and roles of the learners and the tutor, and They need to have a good understanding of the digital learning environment modern pedagogical approaches. in which e-learning takes place, and of the teaching methods used in such an environment. The tutor is also often involved in preparing learning material, Of course, the learners themselves have to take an essential part of the respon- leading discussions, providing technical and contextual support to partici- sibility for their own learning. They need to be able to create the right condi- pants, etc. The tutor’s role is particularly important from the point of view of tions for their own learning and to plan how to spend their time and carry out building learning communities, which are an important factor in the success activities. It is important that they are active and that they consistently fulfil of individuals’ work in e-learning. their obligations and tasks and achieve quality interaction with the tutor and other e-learners. Carril et al. (2013) identified eight different roles (tasks) of a teacher or tutor in a virtual environment and the associated competencies. In addi- tion to their main roles – pedagogical, social and evaluative – a teacher or tutor plays an administrative, technological, advisory, personal and research role. Horton suggests that tutors should have the following attitudes: • Selflessness: ”I want to help people, not just be a sage on a stage.” • Validity: ”E-learning is a valuable form of instruction.” • Self-confidence: ”I can do this.” (Horton, 2012, p. 564). The following table summarises the essential differences in the competencies needed to teach in a traditional classroom and a virtual classroom. Table 23: comparison of the competencies needed to teach in a traditional and virtual classroom In a traditional classroom In a virtual classroom Subject-matter knowledge. Subject-matter knowledge. Authoritativeness tone of voice. Well-modulated tone of voice Non-verbal communication Communication tailored to the specificities of the medium Classical writing Writing and typing PPT presentation basics Advanced PPT skills, such as animations Source: Horton, 2012, p. 564. The Tut The T or’s Com or’ pe s Com t pe encies in E-Learning 181 5.3 the tutor’s competencies in e-learning The tutor acts as an intermediary between the institution and the e-learner. They need to have a good understanding of the digital learning environment in which e-learning takes place, and of the teaching methods used in such an environment. The tutor is also often involved in preparing learning material, leading discussions, providing technical and contextual support to partici- pants, etc. The tutor’s role is particularly important from the point of view of building learning communities, which are an important factor in the success of individuals’ work in e-learning. Carril et al. (2013) identified eight different roles (tasks) of a teacher or tutor in a virtual environment and the associated competencies. In addi- tion to their main roles – pedagogical, social and evaluative – a teacher or tutor plays an administrative, technological, advisory, personal and research role. Horton suggests that tutors should have the following attitudes: • Selflessness: ”I want to help people, not just be a sage on a stage.” • Validity: ”E-learning is a valuable form of instruction.” • Self-confidence: ”I can do this.” (Horton, 2012, p. 564). The following table summarises the essential differences in the competencies needed to teach in a traditional classroom and a virtual classroom. Table 23: comparison of the competencies needed to teach in a traditional and virtual classroom In a traditional classroom In a virtual classroom Subject-matter knowledge. Subject-matter knowledge. Authoritativeness tone of voice. Well-modulated tone of voice Non-verbal communication Communication tailored to the specificities of the medium Classical writing Writing and typing PPT presentation basics Advanced PPT skills, such as animations Source: Horton, 2012, p. 564. 182 Pedagogical Support 5.4 gilly Salmon’s e-Moderating Model The description of the different types of tutor and learner activities can be facilitated by the five-stage model of online teaching and learning developed by Gil y Salmon (2004). Each level requires specific tutor competencies and a certain amount of interaction with the learners. These stages should be taken into account when preparing tutors for e-learning, before planning and im- plementing tutor support for education. They can also be used as a basis for planning activities for your chosen e-learning programme. Figure 24: e-Moderating Model 5. Development Supporting, Providing responding links outside closed conferences 4. Knowledge construction Facilitating process y Conferencing tivit 3. Information exchange acter LEARNING Facilitating tasks and supporting Searching, use of learning personalising software materials t of in 2. Online socialisation amoun Familiarising and providing bridges between Sending and cultural, social and learning receiving messages environments 1. Access and motivation Welcoming and Setting up system encouraging and accessing E-Moderating Technical support Source: Salmon, 2004, p. 29. The stages of the model include access to the learning environment, getting to know each other in a group, information sharing, knowledge creation and development. The individual stages are briefly outlined below. Stage 1: Access to the Digital Learning Environment and Motivation At this stage, the most important thing is for learners to understand how to quickly and easily enter the digital learning environment. Attention should Gilly Salmon’s E-Moder Gilly Salmon’ ating Model s E-Moder 183 also be paid to any prejudices, fears and negative attitudes learners may have towards technology, and to informing them about the different forms of sup- port they can count on during their learning. The most important parts of this stage are therefore the training to access and motivate the use of the digital learning environment. This is not to say that learners should receive training specifical y designed to familiarise them with the digital learning environment. Above al , we need to enable them to: • learn about the accessibility and features of digital learning environments; • obtain a username and password to access the digital learning environment; • log in to a digital learning environment. The entry level is also an opportunity for a friendly welcome and first indi- vidual contacts by phone or email. Stage 2: Online Socialisation As in traditional forms of education, such as workshops and seminars, e-learn- ing also needs to be about the familiarisation and socialisation of the group. At this stage, we need to get learners used to the new digital learning environ- ment, so we need to create the right atmosphere and ensure the well-being of all learners, especial y those who are not used to computer and Internet tech- nology. Psychological theory suggests that exclusion or alienation in a group can reduce the motivation to participate in it and can also reduce interest in the learning content. Feelings of exclusion can ultimately lead to dropping out of education. The tutor’s activities in this phase include: • getting to know the group; • welcome new participants and latecomers; • knowledge of the way, forms and rules of work (including in terms of time); • encouraging ”quieter” learners to take part; • addressing the most common problems encountered when participating in e-learning. Stage 3: The Exchange of Information In step 3, participants are already aware of the importance of a digital learning environment that makes a variety of information quickly and easily accessible to al . At this level, the tutor helps all the learners to develop independence, confidence and enthusiasm for working in e-learning. It is important that tu- tors then ensure the active participation of all learners; they should be active not only in learning but also in discussions with other learners when necessary. 184 Pedagogical Support When an online group real y takes off, the many messages and files in attach- ments can make the discussions opaque. The tutor must therefore make sure that they focus only on the content of the learning programme. Learners may react differently to a large volume of information, which can sometimes seem chaotic. Some people choose only the messages that they are interested in, others respond to all messages, and still others withdraw. The tutor must pay particular attention to the latter. At this stage, learners expect to be guided through a multitude of messages and encouraged to use more relevant mate- rial relating to the content. The tutor must: • clearly define the activities of the participants, their purpose and objectives, the timetable, the tasks of the learners related to these activities; • encourage the active participation of all stakeholders; • ask questions and stimulate debate; • encourage learners to send messages; • assign roles to individual learners (for example: one learner presents the content of an article; another summarises the discussions on a particular forum topic); • to draw conclusions from thematic discussions in the forum when neces- sary; • encourage discussion groups to work independently, to be autonomous and to develop a sense of belonging to the group (own discussions, shared expressions, metaphors, jokes created by the group, etc.). Stage 4: Knowledge Construction At stage 4, learners start to communicate and collaborate more actively and deeply. What does this mean? In the discussion forums, they share and test their ideas and understanding of the learning content. They respond to the substantive contributions of other learners and contribute their own. Learn- ers critical y evaluate the concepts and theories that they learn about and post their reflections in discussion forums. We should also bear in mind that learners can learn from each other through interaction and discussion just as effectively as when they interact primarily with a tutor. In this type of learning, the process by which learners share their experiences, views, knowledge and thus contribute to the formation of the group is important, in addition to the learning outcome (knowledge and com- petencies) itself. In this case, learning is not only active, but also interactive. The tutor’s role at this stage is mainly to form, develop and maintain the group. The tutor must be able to bring together different views, debates and perspec- tives, as well as to weave them into a common conclusion, which must relate to the concepts and theories that emerge from the content of the course. Gilly Salmon’s E-Moder Gilly Salmon’ ating Model s E-Moder 185 At stage 4, the tutor’s tasks are mainly related to: • creating divergent activities (for example, activities where several correct solutions are possible); • promoting learning; • encouraging learners to reflect and think creatively; • encouraging learners to reflect on existing theories and practices (e.g., the relation between theory and practice). Stage 5: Development At the next level, learners use discussion groups as a learning tool to achieve their learning objectives. At this stage, they reflect critical y on their learning and their experience of e-learning. They become independent and need little support and help from a tutor. The tutor’s main task is to respond to sugges- tions and questions. 186 Pedagogical Support 5.5 Planning the tutor’s Work When planning their work, tutors need to take into account several factors: the specificities of e-learning, which takes place in several stages, the organisation of the work and the size of the group, asynchronous communication skil s, time management, networking, etc. The tutor’s work in the different stages of learning was presented in more detail with the e-moderating model in the previous section. Below, we will outline other factors that a tutor should take into account when planning their work. Work organisation and group Size What is the appropriate group size for e-learning to maximise learning effec- tiveness and efficiency? The optimal number of participants depends mainly on the purpose and objectives of the e-learning programme. In small groups, participants can work directly with each other and with the tutor, actively en- gaging with the tasks or problems set. The atmosphere can be more personal, sometimes even more so than in traditional education. The tutor must also encourage the participation of the learners, which must be aligned with the learning objectives. Even if the tutor’s work in e-learning is less visible external y, the tutor must be proactive in the group and not leave the learning process to spontaneous flow. The tutor achieves active regulation of the group through the following activities (Horton, 2012): • individual contact with learners: it is useful to contact learners individual y (by email or phone) to let them know that you are treating them individu- al y and that you are genuinely available and willing to help them with their learning difficulties; • helping to connect peers (learners) in the group: encouraging learners to in- troduce themselves (i.e., through a blog), allowing them to get to know each other better and find common interests; • strict adherence to the published programme of activities: it must take into account that the learners have already adapted their commitments to the activities announced in the programme and that any deviation from the programme would cause them considerable difficulties and inconvenience. The tutor should continuously monitor the learners’ participation in discus- sions and other activities and, if necessary, respond in time to prevent (if pos- sible) drop-outs. We have to bear in mind that e-learning communication in general takes much more time than traditional face-to-face communication. As a general rule, it is worth bearing in mind that, compared to a ten-minute face-to-face meeting, we need for discussions in the online space: Planning the Tut Planning the T or’s W or’ or s W k or 187 • 20 minutes for the audioconference, • 30 minutes for the chat room, • 1 to 2 days for a discussion forum (Horton, 2012, p. 485). Horton also points out that it takes about a week to 10 days for a discussion to take place in a discussion forum, compared to a one-hour live discussion in a classroom. For these reasons, facilitating groups in e-learning requires a smaller number of learners compared to facilitating discussions in traditional learning, where there are around 20 participants. Experience shows that a group size of 10 to 15 learners is still acceptable in e-learning. Horton (2012) states that most experts recommend a group size of 7 to 10 learners. Of course, when deciding on the size of the group, the economic aspect must also be taken into account, as the cost per participant is much higher in a small group. If for economic reasons, we cannot afford to run the programme in small groups, then we try to provide the best quality e-learning by breaking the group into small learning groups of up to 10 learners to run just some of the activities. Successful y organising and moderating discussion forums is particularly im- portant for the tutor as one of the most effective and appropriate forms of e- learning. Proper organisation of the discussion forums means that learners are greeted on their first visit to the discussion forum with all the information they need to participate smoothly and actively. Such information includes: • presentation of the discussion forum topic and learning objectives; • information on the administrative and technical aspects of the forum and on relevant support; • contextual information of a general nature that cannot be classified under a specific topic (useful links, glossaries, etc.); • information on opportunities for the exchange of views between learners outside the discussion forum; • guidance on how to use the different features of the discussion forum (how to reply to messages, make new posts and open new topics, edit and make posts transparent, etc.). By moderating the discussion forums, the tutor must first of all ensure that the discussion between the learners is constructive and contributes to the deepening of the knowledge already acquired and to the acquisition of new knowledge. It does this by imaginatively opening up new topics and asking interesting questions, by appropriately encouraging responses to any questions that go unanswered for a long time, by pointing out mistakes or misunder- standings, by removing unnecessary or inappropriate messages, by including interesting ”guests” in the forum (such as practitioners), and final y by evaluat- ing and summarising the discussion. 188 Pedagogical Support the importance of the tutor for Building a learning community and collaborative learning The main problem with the first generation of e-learning (1.0) was the limited interaction between learners and teachers or tutors. In those days, interaction was mainly through learning material, and only to a significant extent with tutors if face-to-face meetings with the learners were guaranteed and the e- learning took the form of blended learning. The development of the Inter- net, which has enabled the development of second-generation e-learning, has brought many technological opportunities for effective and efficient ’distance’ tutoring. Tutors today have a wide range of synchronous and asynchronous online communication tools at their disposal to encourage, guide and support learners in developing their abilities and acquiring knowledge. Artificial intel- ligence, with the emergence of intelligent tutoring systems (Section 6.5), opens up new perspectives for tutoring. The theoretical starting point for tutoring is the theory of the ”Community of Inquiry” . This theory, which was presented in more detail in the section on learning theories (Section 3.2), assumes that in order to create a successful learning group in e-learning, it is necessary to ensure the presence of a social component, a cognitive component and a teaching component. The role of the tutor is crucial in establishing the social component. Closely related to this theory is the concept of online collaborative learning. In online col aborative learning, the tutor encourages and helps learners to create knowledge them- selves: to explore, to work, to make connections, to find ways to solve prob- lems instead of just repeating what they have learned in their answers. In such learning, the tutor is an expert authority and a guide to sources of knowledge (Harasim, 2017). recommended links Gil y Salmon: The Five Stage Model: https://www.gillySalmon.com/five-stage-model.html WikiHow: How to Tutor Online: https://m.wikihow.com/tutor-online The Community of Inquiry: http://www.thecommunityofinquiry.org/ the importance of the tutor for Building a learning community and collaborative learning The main problem with the first generation of e-learning (1.0) was the limited interaction between learners and teachers or tutors. In those days, interaction was mainly through learning material, and only to a significant extent with learning aPProacheS and tutors if face-to-face meetings with the learners were guaranteed and the e- 6 learning took the form of blended learning. The development of the Inter- net, which has enabled the development of second-generation e-learning, has MethodS in a digital Society brought many technological opportunities for effective and efficient ’distance’ tutoring. Tutors today have a wide range of synchronous and asynchronous online communication tools at their disposal to encourage, guide and support learners in developing their abilities and acquiring knowledge. Artificial intel- ligence, with the emergence of intelligent tutoring systems (Section 6.5), opens up new perspectives for tutoring. 6.1 introduction The theoretical starting point for tutoring is the theory of the ”Community of Inquiry” . This theory, which was presented in more detail in the section on learning theories (Section 3.2), assumes that in order to create a successful learning group in e-learning, it is necessary to ensure the presence of a social In Part 1 of this book, we examined the status and development trends in the component, a cognitive component and a teaching component. The role of the field of e-learning. tutor is crucial in establishing the social component. Closely related to this International studies and our own research on e-learning show (Henderikx and theory is the concept of online collaborative learning. In online col aborative Jansen, 2018; Gaebel and Zhang, 2018; Bregar and Puhek, 2017) that e-learning learning, the tutor encourages and helps learners to create knowledge them- practice, especial y in formal education, is only slowly progressing along the e- selves: to explore, to work, to make connections, to find ways to solve prob- learning development arc. It is still largely driven by an educational paradigm lems instead of just repeating what they have learned in their answers. In such that puts the teacher at the centre of education, transferring ”his/her knowledge” learning, the tutor is an expert authority and a guide to sources of knowledge to ”his/her students”. Educators in Europe general y agree that this paradigm (Harasim, 2017). is outdated and that it is time for it to be replaced in the digital society by a so- recommended links called learner-centred paradigm. This is dictated by changing educational needs, and the development and diffusion of technology is a genuine basis for such a Gil y Salmon: The Five Stage Model: transformation of education. Of course, adequate technological support is not https://www.gillySalmon.com/five-stage-model.html the only prerequisite. There is a need for a meaningful, pedagogical y-oriented WikiHow: How to Tutor Online: integration of technology into learning and teaching. Learning and teaching must https://m.wikihow.com/tutor-online be designed and delivered in such a way that the learner is an active creator of knowledge and the teacher is a guide and facilitator in the process. The concepts The Community of Inquiry: that support the realisation of this educational paradigm are: http://www.thecommunityofinquiry.org/ • personalised learning and adaptive learning, • knowledge creation / creative learning, • active, independent and authentic learning, • col aborative and open learning, • ubiquitous learning. 190 Learning Approaches and Methods in a Digital Society Table 24 shows innovative teaching and learning approaches and methods that 6.2 open education can be used to implement the learner-centred paradigm. Most of these ap- proaches and methods are regarding development stage part of e-learning 3.0 and partly 4.0, while the social tools and methods typical of e-learning 2.0 play 6.2.1 opening up education and open education an important role in col aborative and open learning. The table is based on the annual Innovating Pedagogy Reports, prepared since 2012 by experts at the Open University in the UK, and some other reports. Open education has a long tradition. The foundations are rooted in the value that education is a public good that should be accessible to everyone (Weller, Table 24: learning and teaching concepts and innovative teaching and learning ap- 2014). The concept of open education has changed with the development of proaches and methods to realise the learner-centred paradigm30 society and its prevailing values. Technological development has always played an important role in opening up education, since its early beginnings in the late Personalised learning Learning analytics, artificial intelligence, stealth assessment, rapid Middle Ages with the emergence of education outside church and monastery / adaptive learning course development tools, microlearning, digital badges as certificates wal s and the invention of Gutenberg’s printing press in the sixteenth century Individual research, rhizomatic learning, flipped learning, learning through to the first open universities in the mid-1960s. These made studying Knowledge creation / by reasoning, learning by doing, computational thinking, big at university without entry conditions a possibility. The spread of mass media, creative learning data research, remote labs, visualisation, microlearning, artificial such as radio and television, has brought about new development opportu- intelligence nities for distance education, i.e., ”spatial y independent studying” (EADTU, Virtual reality, augmented reality, gamification, simulations, 2014). This phase in the development of open education in the second half of Active and authentic makerspace, robotics, storytelling or event-driven learning, geo- learning learning, digital learning environments fused with immersive the twentieth century, which is characterised by distance education is referred learning, design thinking, computational thinking, drone learning to by Mulder and Jansen (2013) as classical open education, which grew into Collaborative and Social media, repositories, MOOCs, curated content, big data learning, digital open education at the turn of the millennium. open learning OER, ”decolonised learning”, empathic learning The rise of digital open education has been encouraged by open universities, Ubiquitous learning New generation LMS, BYOD, robotics, IoT, natural user interfaces, freely available software, and Web 2.0 (Weller, 2014). An important milestone mobile learning in the development of digital open education was in 2001, when the prestig- Source: Sharples et al, 2012; 2013; 2014; 2015; 2016; Ferguson et al, 2017; 2019; Becker et al, 2017; ious Massachusetts Institute of Technology (MIT) published the first Open-2018; Pandey, 2018a; Aldridge and Powell, 2018. CourseWare and announced that it would provide free access to all of their From the wide range of possible approaches and techniques for learning and courses on the world wide web in ten years. The prediction came true and it teaching in a digital society, the remainder of the Part 6 presents in more detail also spurred a large-scale movement for open education. those that have received the most attention in the professional literature and In 2002, UNESCO introduced the concept of open educational resources (OER), are increasingly gaining ground in educational practice. Below we discuss: and the Cape Town Declaration in 2008 set up the concept of modern (digital) • open education, open education. • artificial intelligence, The Cape Town Declaration not only includes OER in open education, but also • learning analytics, technological y supported flexible and col aborative learning and the sharing • intelligent tutoring systems, of learning resources in teaching, assessment, accreditation and col aborative • mobile learning, learning ( • http://www.capetowndeclaration.org/read-the-declaration). microlearning, • gamification, The Cape Town Declaration was followed by many other views31 on what open • simulations, education meant in the twenty-first century. These views differ in one way or • virtual and augmented reality, another in their nuances with regard to the meaning and role of open educa- • digital storytelling. tion, the areas it covers and the very understanding of openness. The common 30 The classification of learning and teaching approaches and methods according to learning and teaching concepts is subjective and based on our rough assessment of the relative suitability of each approach and 31 The Year of Open website https://www.yearofopen.org/open-education-definitions/ contains a dozen of method for a particular learning and teaching concept. different definitions of open education. Open Education 191 6.2 open education 6.2.1 opening up education and open education Open education has a long tradition. The foundations are rooted in the value that education is a public good that should be accessible to everyone (Weller, 2014). The concept of open education has changed with the development of society and its prevailing values. Technological development has always played an important role in opening up education, since its early beginnings in the late Middle Ages with the emergence of education outside church and monastery wal s and the invention of Gutenberg’s printing press in the sixteenth century through to the first open universities in the mid-1960s. These made studying at university without entry conditions a possibility. The spread of mass media, such as radio and television, has brought about new development opportu- nities for distance education, i.e., ”spatial y independent studying” (EADTU, 2014). This phase in the development of open education in the second half of the twentieth century, which is characterised by distance education is referred to by Mulder and Jansen (2013) as classical open education, which grew into digital open education at the turn of the millennium. The rise of digital open education has been encouraged by open universities, freely available software, and Web 2.0 (Weller, 2014). An important milestone in the development of digital open education was in 2001, when the prestig- ious Massachusetts Institute of Technology (MIT) published the first Open- CourseWare and announced that it would provide free access to all of their courses on the world wide web in ten years. The prediction came true and it also spurred a large-scale movement for open education. In 2002, UNESCO introduced the concept of open educational resources (OER), and the Cape Town Declaration in 2008 set up the concept of modern (digital) open education. The Cape Town Declaration not only includes OER in open education, but also technological y supported flexible and col aborative learning and the sharing of learning resources in teaching, assessment, accreditation and col aborative learning (http://www.capetowndeclaration.org/read-the-declaration). The Cape Town Declaration was followed by many other views31 on what open education meant in the twenty-first century. These views differ in one way or another in their nuances with regard to the meaning and role of open educa- tion, the areas it covers and the very understanding of openness. The common 31 The Year of Open website https://www.yearofopen.org/open-education-definitions/ contains a dozen of different definitions of open education. 192 Learning Approaches and Methods in a Digital Society denominator is the concept of openness as a way to reduce or remove barriers to accessing research, teaching and learning at al levels of education. The idea of open education can be put into practice in very different ways and in different areas. Here we talk about open courses, open pedagogy, open educational resources, open data, open access and open knowledge (Weller, 2014). In specific circumstances or in a specific context, open education is de- fined by different dimensions with different levels of openness. Open educa- tion, distance education and online study are rarely found in their purest form in practice (Bates, 2016). Open education, therefore, means different things to different people. Despite the diversity of manifestations of open education, these are connected by some common principles (Weller, 2014, p. 42): • freedom to reuse, • open access, • free cost, • easy use, • digital, networked content, • social community-based approaches, • ethical arguments for openness, • openness as an efficient model. Mulder and Jansen (2013, pp. 36–37) defined open education using five components. These include: • OER – Open Educational Resources; • OLS – Open Learning Services: A variety of online and virtual services and tools that are available free of charge and intended for tutoring, counselling, meetings, teamwork, presentations, etc.; • OTE – Open Teaching Efforts: The overall contribution of the engaged staff to education that is the result of the activities of all staff members (pedagogical and non-pedagogical) involved in education with differ- ent roles in a professional, open and flexible environment and culture; • OLN – Open to Learner s’ Needs: Participants in the learning process want achievable, feasible education of appropriate quality that will be interesting and useful for them. Open education should also not be limited by requirements for access, time, space, pace and program of- ferings. Courses must be based on lifelong learning, the recognition of practical experience and easy transitions between formal and non- formal education; Open Education 193 • OEC – Open to Employability and Capabilities Development: Educa- tion must be open in the light of changing social and labour market needs; whereby knowledge and innovation play a crucial role. At the same time, it must offer opportunities for the acquisition of new skil s, critical thinking, ethics, creativity, personal growth and citizenship. The first three components relate to supply and the last two to the de- mand for open education. Figure 25: components of open education OER: Open Educational Resources OLS: Open Learning Services OTE. Open Teaching Efforts OLN: Open to Learners' Needs OEC: Open to Employability & Capabilities Development open 100% 100% closed Source: Mulder and Jansen, 2013, p. 41. As the il ustration in Figure 25 shows, the degree of openness in the hypothetical model varies considerably between individual components. It is highest for OERs that are available without restrictions, and much lower for OTEs and OECs. Therefore, the openness of education has to be understood as a generic um- brel a concept, at the heart of which is the transparency of education and the removal of restrictions at all its levels. It is a flexible concept that can, in prac- tice, encompass different forms and degrees of openness (Inamorato dos San- tos et al., 2016) and it more or less differs from the pure concept. In accordance with this, we see open education as a social y desirable goal of democratising education in the digital society and, as such, is a suitable starting point for the orientations and policies of education in the twenty-first century. Open education stepped into the spotlight of education policy in the European Union during the first years of the previous decade. A 2013 initiative that is significant for the development of open education in the European Union was entitled ”Opening up Education; Innovative Teaching 194 Learning Approaches and Methods in a Digital Society and Learning for All Through New Technologies and Open Educational Re- sources” (European Commission, 2013). Based on this document and taking into account the findings of the large-scale OpenEdu research project, in 2017, the European Commission published recommendations for open education in Europe entitled ”Going Open: Policy Recommendations on Open Education in Europe – European Commission” (Inamorato dos Santos et al., 2017). Recommendations based on the OpenEdu project arise from an all-inclusive (holistic) understanding of open education. Open education is a way of con- ducting education, mainly using digital technologies. Open education aims to improve the accessibility of education by removing obstacles and through the flex- ibility and availability of education. Open learning encompasses different types of teaching and learning, as well as the creation and sharing of knowledge, offering a wide range of pathways for access to formal and non-formal educa- tion, while also connecting both types of education. 6.2.2 categories of open education Broad definitions are of little use in implementing policies and recommen- dations. Therefore, the Joint Research Centre of the European Commission (JRC), within the context of the OpenEdu project, developed a methodology for studying openness in higher education institutions and for removing ob- stacles in education (in terms of access, costs, technology and pedagogy), (In- amorato dos Santos et al., 2016). The methodology has been called the Open Education Framework (OEF). Open Education 195 Figure 26: open education Framework STRATEGY TEHNOLOGY CONTENT ACCESS PEDAGOGY 10 DIMENSIONS of OPEN EDUCATION RESEARCH RECOGNITION COLLABORATION LEADERSHIP QUALITY Source: Inamorato dos Santos et al., 2016, p. 24. The OEF consists of ten dimensions that determine the openness of education from different perspectives. The basic dimensions are: access, content, peda- gogy, recognition, col aboration and research. These dimensions define what the area of openness is. Transversal dimensions (leadership, strategy, technol- ogy and quality) are the basis for the implementation of the basic dimensions and define how to implement open education. Each of these dimensions is defined and described in the OEF handbook and broken down into components. The components are presented in detail using so-called descriptors that describe each component in terms of implementa- tion according to the characteristics of the transversal dimensions. The OEF handbook is a good basis for the introduction of open education in educa- tional organizations and its evaluation. 196 Learning Approaches and Methods in a Digital Society The following table shows the definitions of the basic dimensions of the OEF and their components. Table 25: general open education framework: definitions and components Dimension: Definition Rationale Components Basic dimensions Access Removal or reduction Expanding access to Costs of access to of economic, knowledge is a core resources, courses and technological, value of openness and a programmes. geographical and major factor in formal and The accessibility and institutional obstacles non-formal education. flexibility of educational that impede access to From the point of view content (including knowledge. of the educational channels and modes of organisation, accessibility communication) according can be promoted at three to specific requirements interrelated levels: access and needs. to programmes (which Flexible learning via the lead to a degree or full Internet and mobile qualification), access to technology. courses and access to Entrance requirements to educational content. courses and programmes. Participants in the learning process must be able to access and interact with teachers and peers. Content Learning and teaching This includes texts of OERs, including all material material and research all kinds, textbooks and useful for learning and results that are free of course materials, pictures, teaching that are openly- charge and available games, podcasts, video licensed content and free to all. lectures, software, data, of charge, ranging from research papers and teaching components to outputs, as well as all other MOOCs. material that is useful in Free-of-charge content. the learning process. It Freely accessible content may be available on the subject to copyright. basis of open licences or copyright, but free of charge and without restrictions for anyone. Open Education 197 Dimension: Definition Rationale Components Basic dimensions Pedagogy The use of technology Opening up pedagogical Supported open learning. to expand accessibility practices is about The learner is independent and make the range of developing the design in the learning process, teaching and learning for learning so that it assisted by a mix of media, practices more widens participation and resources and pedagogical transparent, sharable collaboration between practices. and visible. all involved. Pedagogical Personalised teaching and approaches with an learning. emphasis on the learner Collaborative and are very suitable to open networked learning. education. Use of authentic sources. Sharing educational resources and pedagogical practices. Recognition Two aspects: a) The recognition of open Assessment. recognition by education enables the Identity validation. an accreditation transition from non-formal Trust and transparency. institution that to formal education and Recognition of prior learning formally recognises the completion of tertiary (RPL). a particular type education programs in a of open education more flexible way, or to get Mutual recognition (Fast with a certificate, recruited/ promoted at the track recognition). diploma or title; b) the workplace. Qualifications. process of the formal Social recognition. acknowledging and accepting credentials in the form of badges, certificates and diplomas issued by third-party institutions. 198 Learning Approaches and Methods in a Digital Society Dimension: Definition Rationale Components Basic dimensions Cooperation Cooperation is Collaboration in open Intra-institutional about to removing education means realising collaboration (connecting barriers to education the idea of eliminating departments and faculties, by facilitating the obstacles by creating students, staff, alumni exchange of practices networks between members, etc.). and resources with individuals and institutions. Inter-institutional a view to improving cooperation. education. National/regional Open cooperation cooperation. based on open Cross-border cooperation. educational practices goes beyond traditional university cooperation and enables building a bridge between formal, non-formal and informal education. Research Openness in research Openness in research Open access (free of charge means eliminating means a paradigm shift to the end-user) . obstacles to accessing in the way research and Open research collaboration data and research scientific activity is carried (largescale, remote outputs, as well as out and it affects the collaboration between expanding research entire process of these scientists through the use participation. activities. The traditional of Internet-based Tools orientation of ”publish similar to open-source first” is counter to the idea software collaboration). of the fastest possible Open data (freely development of science accessible and can be used on the basis of sharing and distributed in various scientific achievements ways, subject to the and cooperation. principle of open licences). Citizens’ science (research collaboration in which professional scientists engage with members of the public, who then contribute to the research). Source: Inamorato dos Santos et al., 2016, pp. 35 –58. The performance features of each of the six basic components depend on the characteristics of the transversal dimensions (strategy, technology, quality and leadership) and their components. Open Education 199 Table 26 shows the components of the transversal dimensions of the OEF. Table 26: components of the transversal dimensions of the open education Framework Dimension: Components Strategy Integrated institutional policy Funding Technology Software and platforms Development and maintenance Training The vision and audience Quality The quality of the open education offer The quality of institutional staff support The quality of services to open learners Leadership Institution’s stakeholders Personnel Learners Community Source: Inamorato dos Santos et al., 2016, pp. 62–70. 6.2.3 open educational resources (oer) learning objects and oer OERs, together with MOOCs, are nowadays the most recognisable and perva- sive component of the open education movement. They represent a component of open education content. The Hewlett Foundation, a U.S. foundation that sup- ported the development of the first OERs, including those of MIT, identified OERs in the initial stage of their development as teaching, learning, and research materials that are either (a) in the public domain or (b) licensed with an open licence in a manner that provides everyone with free use or for other purposes, depending on the type of licence used. The use of OERs for commercial purposes has not been ruled out in this definition (Atkins et al., 2007, p. 4). The beginnings of OERs were already indicated by learning objects. The Ka- plan-Leiserson E-Learning Glossary (n.d.) defines the learning object as reus- able, technological y (of the media) independent information that can be used as a modular component of the learning content of an e-learning course. At the turn of the twentieth century, it seemed that learning objects were an impor- tant innovation for education, particularly effective in e-learning. 200 Learning Approaches and Methods in a Digital Society Stephen Downes (2001) il ustrated the economic eligibility of learning objects using the example of trigonometric functions. The explanation and learning material on trigonometric functions cannot differ significantly among schools, as the content and meaning of this mathematical concept are clear and un- ambiguous. Why would each school bother to spend money developing their own materials? The cost of preparing and distributing materials is drastical y reduced if millions of different but similar materials are replaced by one or just a few well-prepared collections of learning objects that are available online and in different languages. These great expectations did not come to fruition. Weller (2014) explains the failure of learning objects stating three reasons: the reusability paradox, the problem of standards, and obstacles arising from different social and cultural circumstances. The reusability paradox (Wiley et al., 2004) means that the usefulness of a learning object is inversely proportional to its pedagogical value. The useful- ness of a learning object depends in principle on how many different contexts it is appropriate for. The transfer of a learning objects, for example, of trigono- metric functions, into different contexts is quite easy due to the clearly defined content and its demarcation from other, related concepts. More difficult and problematic would be the transfer of a learning object from the field of social sciences, such as a learning object about slavery. However, if we tried to en- hance the usability of the learning object by narrowing its treatment to a dry abstract conceptual definition, its pedagogical value would decrease. In addition, the use of uniform standards is a condition for the re-use of learn- ing objects. With the spread of learning objects, these standards became more complex and demanding. It is this technical complexity that has deterred many a user from using learning objects. The use of learning objects was also hampered by prejudices, incompatibility with existing institutional frameworks, ignorance, etc. For example, the prepa- ration of learning objects has not been recognised in academic circles as a rel- evant activity in terms of professionalism and has not been taken into account in academic advancements, as is the case with the publication of articles. In addition, the use of learning objects was hampered by language barriers and the unwillingness of teachers to use the material of others in their work. For a number of reasons, the learning objects have not reached a sufficiently large scale of application to allow their penetration and sustainable use in education. The development of OERs was encouraged by the Hewlett Foundation project. The declared goal of this project was to explore the possibilities of equalising access to knowledge and educational opportunities using modern information technology. The project involved several reputable universities, mostly from Open Education 201 the USA, with the leading being MIT. The project also provided training to institutions from developing countries in the use of OERs (Atkins et al., 2007). The pioneering Hewlett Foundation project acted as a catalyst for the spread of the OER movement, initial y involving mainly individual education enthu- siasts, usual y with the modest support of their home educational institutions (Plotkin, 2010). The term OER was first used formal y at the 2002 UNESCO conference entitled Forum on the Impact of OpenCourseWare for Higher Ed- ucation in Developing Countries. The definition adopted at the time already emphasised the non-commercial nature of OERs: ” the open provision of educa- tional resources, enabled by ICT, for consultation, use and adaptation by a com- munity of users for non-commercial purposes” (UNESCO, 2002, p. 24). definition and types of oers The more successful spread of OERs compared to learning objects is mainly due to free access and a more flexible, open manner of use that is available to the user through pre-defined rights. These rights are crucial for educational material to acquire OER status. They are briefly designated by 5R, respectively (Wiley and Hilton, 2018, p. 134): • Reuse: the right to use the content in a wide range of ways (e.g., in a class, in a study group, on a website, in a video). • Revise: the right to adapt, adjust, modify, or alter the content itself (i.e., translate the content into another language). • Remix: the right to combine the original or revised content with other ma- terial to create something new (i.e., incorporate the content into a mashup). • Redistribut e: the right to share copies of the original content, your revisions, or your remixes with others (i.e., give a copy of the content to a friend). • Retain: the right to make, own, and control copies of the content (e.g., download, duplicate, store, and manage). 5R implementation is only possible with an open licence. This means that the author defines the licence or the conditions under which the users will be able to access and use the material without the need for special permits or incur- ring costs. The OECD definition of OER is based on the 5R concept (Orr et al., 2015, p. 19), which is widely used due to its clarity and integrity: ”Open educa- tional resources (OER) are digital learning resources offered online (although sometimes in print) freely and openly to teachers, educators, students and in- dependent learners in order to be used, shared, group, combined, adapted, and expanded in teaching, learning and research”. OERs are available in different forms: • complete courses, • textbooks, 202 Learning Approaches and Methods in a Digital Society • individual course study units or modules, • lesson plans, • syl abi, • lessons, • assignments, • games and simulations, • quizzes and tests, • podcasts, • videos. The Open Education Handbook (EADTU, 2014) groups OERs into three main categories: open learning material, open textbooks and MOOCs. The condition for classifying learning material as an OER is that it meets all five conditions that determine the user rights. Thus, MOOCs are only OERs if the 5R conditions are met. OERs usual y combine the use of different media, of which one is the main one. Depending on the medium used, OERs can be classified as text, video, animation or multimedia OERs (EADTU, 2014). advantages and limitations of oers OERs are useful for individuals who are in the role of teachers or learners in the educational process, for educational organisations and for the educational system in general (EADTU, 2014; Bates, 2015; Weller, 2014). Using free learning materials that are already available, regardless of how they are used, general y saves resources and time; this can be put to good use by teachers to improve other elements of the learning process. The advantages of OERs are manifested differently in their use in higher edu- cation compared to their use in primary and secondary school education, as well as in on-the-job training (EADTU, 2014). Within the context of higher education (including research), there is a greater emphasis on the active use of OERs. Higher education professors can use dif- ferent OERs as examples of good practice for learning material and, as such, can be the basis for assessing the quality of their material. They can be included in the learning process in different ways: they can be revised or supplemented and then shared through online publishing. This contributes to the better rec- ognition of both authors and educational institutions. In addition, OER rights provide an opportunity for other authors to improve and further develop the original material. At lower education levels, OERs are usual y used unchanged, especial y if they are used in a standard content course. Open Education 203 For learners, OERs are an opportunity to find information on any content and to connect with peers in learning. Using OERs, they also get advanced information about the study programmes and specific courses offered by an individual educational organisation, making it easier to decide on enrolment. For those who have difficulties in learning, OERs can provide additional help in overcoming learning difficulties and frustrations and a valuable learning tool, especial y if it is quality material with a lot of interaction. OERs can also contribute to a better and more in-depth learning experience in non-formal education (Weller, 2014). The development of OERs increases the visibility of an educational organisa- tion, since by publishing OERs online and listing them in various search en- gines increases the likelihood that a wide range of potential users will become familiar with their study programmes and courses. OERs are an opportunity to provide information about the study orientations and achievements of an individual organisation and are a good basis for establishing various partner- ships and cooperation with domestic and foreign organisations. At the national and global levels, OERs contribute to more equal opportunities for education (Weller, 2014). Even though almost two decades have passed since the formal adoption of the idea at the UNESCO Conference (UNESCO, 2002) on how to democratise education with free resources available to anyone anywhere and anytime, and despite the undoubtedly great idea of how to make it happen with the 5Rs, OERs have still not been involved in education to a predominant extent. The main reason that OERs are still mostly used only by their authors is their poor quality. Bates states (2016, p. 207) that OERs are mostly texts without interaction, available in PDF format (unsuitable for making revisions), simula- tions are approximate, and graphics are poor and unclear. One of the reasons for the less frequent use of OERs lies in the prejudice that free stuff cannot be of good quality, which is exploited for advantage by commercial publishing houses. In addition, the question is how the quality of OERs varies between authors (Falconer et al., 2013). One of the reasons why teachers do not accept OERs stems from the self-image of teachers, particularly in higher education. They see themselves not only as mediators of content, but as creators of new knowledge that they want to offer to learners (”not invented here syndrome”) (Bates, 2013). Problems are also caused by the fact that OERs often lack a certain contex- tual framework, which is bad from a pedagogical point of view. Like learning objects, OERs are threatened by the paradox of usefulness, but in the case of OERs, however, due to the 5Rs idea, it can be successful y mastered as long as there is the knowledge and will to further develop the original material. 204 Learning Approaches and Methods in a Digital Society Pedagogical y meaningful use requires that developed OERs are assessed by per- sonal reflection, feedback from the teachers or their assistant and, more impor- tantly, feedback and interaction from colleagues, family and friends (Bates, 2016). Where to Find and how to use oers The volume of available OERs is increasing on a daily basis. In order to use those among the multitude of OERs that are most suitable for us and of ap- propriate quality, it is necessary to follow some basic principles of strategies for finding the right sources and also to consider quality criteria. There are several ways to search for OERs: • We use specialised search engines for OERs, such as: – Openverse (Creative Commons Search): https://search.creativecommons.org/; – Open Education Consortium: https://www.oeconsortium.org/courses/search/; – MERLOT: https://www.merlot.org/merlot/; – The Mason OER Metafinder (MOM): https://oer.deepwebaccess.com/oer/ desktop/en/search.html; – The Open Education Handbook contains a range of information on search engines, portals, manuals and software tools (https://education. okfn.org/handbook/index.html); • The ClassCentral search engine is intended for MOOCs (https://www. classcentral.com/). • The search can begin by reviewing the specialised portals of individual in- stitutions or associations (for example, the Commonwealth of Learning – https://www.col.org/, or UNESCO – https://en.unesco.org/). Freely available information is usual y classified in the Resources section. OERs are also available on a number of other portals and repositories, such as: • OER Hub: Researching Open Education: http://oerhub.net/, • OER Commons: www.oercommons.org, • Open Education Global: https://www.oeglobal.org/, • Open Stax: https://openstax.org/, • CCCOER Community College Consortium for OER: https://www.cccoer.org/ learn/find-oer/, • OASIS: https://oasis.geneseo.edu/, • OER Knowledge Cloud: https://www.oerknowledgecloud.org/, • Find OER: https://open4us.org/find-oer/, • Open Science Repository: http://www.open-science-repository.com/journal-of- education.html, • EdReNe – Educational Repositories Network: http://www.edrene.org/. Open Education 205 We certainly need to take into account that links to web addresses are quite unstable and tend to change, meaning that many web addresses will no longer be active during our research. Prior registration is often required for access. When choosing OERs, we can pursue different adoption strategies (Bates, 2016): • Based on predefined criteria, we select the OER and include or adapt it to the needs of our course. • We can prepare digital learning material ourselves and, on the basis of open licenses, grant others access to them. • The OER is taken as a basis that is supplemented with additional assign- ments, activities and projects for students. • The course is based on OERs in its entirety and we only add knowledge as- sessment assignments and provide additional feedback to students. When choosing an OER, keep in mind that a quality OER has the following characteristics (RMIT University, n.d.): • It is easy to identify and locate (at one or several locations). • It is clearly described with key metadata. • The licence is evident (usual y Creative Commons). • Its source is credible. • It is easily modified. • For its use, we do not need to know or use other sources. • It was used or recommended by colleagues or recognised experts. It is advisable to develop an OER gradual y by adding individual components (images, videos, text) and always having a clear picture of what our material is like in relation to the original material. Accurate documentation is also re- quired because each piece can be a stand-alone OER that can be published and redistributed (EADTU, 2014). In the development of OERs, it is essential from a formal point of view to check the licence and to include in our material appropriate indications of the attribution of the OER used (EADTU, 2014). The substantive aspect of OER development, however, requires the right relationship between simplicity, dic- tated by the requirement for the widest possible usability, and the complexity and authenticity that contribute to the pedagogical quality of the material. 6.2.4 Massive open online courses (Moocs) the Beginnings of Moocs The last decade of e-learning development has been significantly marked by the development of massive open online courses – MOOCs. MOOCs are ac- 206 Learning Approaches and Methods in a Digital Society tual y the most widespread form of open education, involving a combination of OERs, various pedagogical approaches and methods of e-learning, and the application of col aborative learning principles (EADTU, 2014, p. 38). An es- sential feature of MOOCs compared to other online courses is that there are no entry restrictions, therefore there are no restrictions in terms of the number of entrants, entry conditions or costs. MOOCs must therefore be designed to have unlimited demand scalability (OpenupEd, 2015). However, in an eco- nomic sense, this is only possible if the costs of an additional y enrolled par- ticipant ( marginal costs) are equal to or at least close to zero. The naming of the MOOC with initials of four quite vague and imprecise at- tributes suggests that this is a complex phenomenon for which no unambigu- ous definition can be expected. In Europe, the definition developed jointly in several European projects (HOME, ECO and OpenupEd) is often used. This definition defines MOOCs as online courses designed for a large number of participants that are accessible to anyone anywhere. If an Internet connection is provided, they are accessible without entry restrictions and provide a comprehensive online learning expe- rience free of charge (OpenupEd, 2015). Several sources also refer to the definition posted in Wikipedia, which, by de- scribing the individual components, limits the applicability of the definition to certain types of MOOCs (that is, to the xMOOCs that are discussed in the next section). ”MOOC is an online course aimed at unlimited participation and open access via the Web. In addition to traditional course materials, such as filmed lectures, readings, and problem sets, many MOOCs provide interac- tive courses with user forums or social media discussions to support commu- nity interactions among students, professors, and teaching assistants, as well as immediate feedback to quick quizzes and assignments.” (https://en.wikipedia.org/ wiki/Massive_open_online_course). The emergence of the first MOOCs, developed by the elite US universities, was accompanied by comments that they were a revolutionary novelty (a dis- ruptive innovation) and a major step towards democratisation, particularly in higher education, though many viewed them primarily as one of the short- lived ideas of how to use technology in education (Bates, 2016; Daniel, 2012). Stephen Downes, George Siemens and Dave Cormier, who prepared the Con- nectivism and Connective Knowledge course at the University of Manitoba in Canada in 2008, are usual y32 cited as the initiators of MOOCs. The course was available in two versions: as a fee-paying course in the classic version, in which 32 Weller (2014, p. 93) says that the dating of the first MOOCs depends on what we understand a MOOC to be. Thus, as early as 2007, David Wiley and Alec Couros provided open access on the Internet to the traditional university course. Even earlier beginnings of MOOCs are cited by Littlejohn and Hood (2018, p. 5), who see the beginnings as early as the first attempts at distance education in the nineteenth century. Open Education 207 27 students enrolled, and as a free online course (Bates, 2016). To the great surprise of the authors, as many as 2,200 students enrolled in the free online course. Depending on the method of implementation, Downes (2012) marked this course as a connectivity Massive Open Online Course (cMOOC). The second development direction of the MOOCs was indicated by the Intro- duction to Artificial Intelligence course, developed in 2011 by Sebastian Thrun and Peter Norvig of Stanford University. The success of this project, which saw 160,000 enrolments, spurred the development of other, similarly designed MOOCs and the authors founded the companies Udacity and Coursera. These companies developed their own software support that allows mass enrolment. Other universities can publish their MOOCs on these platforms for a fee. MIT and Harvard University developed the edX platform for MOOCs in 2012. In addition, edX provides pedagogical support (Bates, 2016; Weller, 2014). In Europe, the learning environment for MOOCs was first set up in the same year by the Open University of the United Kingdom (FutureLearn). Unlike OERs, which are being implemented gradual y, the growth of MOOCs has been remarkable in just over five years. Figure 27: number of Moocs in the period between 2012 and 2018 8k Growth of MOOCs 6k ourses 4k 2k Number of c 0 2012 2013 2014 2015 2016 2017 2018 Source: Shah, 2018. MOOCs have therefore been expanding unstoppably over the last five years. According to the Class Central33 portal on innovations and trends in online education, in 2017, there were already around 900 universities that developed one form of MOOC or another; about 81 million enrolments in MOOCs were registered and 9,400 educational programs declared as MOOCs were already available (Shah, 2017a). 33 Class Central (https://www.class-central.com/report/) is a search engine for MOOCs, but also provides a range of up-to-date information about the field (e.g., the most popular MOOCs that will be available for enrolment shortly, professional news, various tools for MOOC users, etc.). 208 Learning Approaches and Methods in a Digital Society common Features of Moocs The question is to what extent, in this multitude of educational opportunities labelled as MOOCs, it is possible to talk about MOOCs in terms of the original idea of massive open online courses. The concept of a MOOC is usual y explained by the meaning of the individual terms that make up the acronym: • M – massive, • O – open, • O – online, • C – course. So, what are the characteristics of an educational programme or course that can be abbreviated to MOOC? Which Courses are Massive? The main feature of a massive course is that it is designed for a huge number of participants. The massive feature is only confirmed in practice. Thus, more than 12 million participants enrolled in Coursera courses in the period from 2011 to 2015, and 240,000 were enrolled in the largest course. More important than the enrolment numbers themselves is how flexible the MOOCs are in terms of demand (Bates, 2016). The pedagogical model needs to be set in such a way that the scope of pedagogical work does not increase significantly if the number of participants increases (scalability of the course). In practice, this means that the costs should not include the so-called vari- able costs, such as costs related to pedagogical support (i.e., the tutor’s help and feedback to the enrolled). Therefore, one of the essential characteristics of MOOCs is that, as a rule, they do not include any direct pedagogical support. OpenupEd (2015) sets the massive criterion more loosely and in relation to the size of a normal course. The number of MOOC participants exceeds the number of participants in a typical university classroom. In addition, Dunbar’s number is used as a criterion.34 The importance of unlimited demand scalabil- ity is also emphasised. Ensuring massiveness at the expense of absent or limited pedagogical support is an essential feature of MOOCs that has received the most criticism. Littlejohn 34 Dunbar’s number is the upper cognitive limit of the number of people with whom one can main- tain stable social relationships. This means that an individual knows a certain person and knows how they are connected to another person. This definition was provided by the anthropologist Robin Dun- bar in 1990 and he estimated that this number totals around 150 people (https://en.wikipedia.org/wiki/ dunbar%27s_number). Open Education 209 and Hood (2018) sum up the views of various authors that MOOCs lean to- wards the dehumanisation of education. When is the Course Open? A feature of the open course is that it is free, without entry conditions, freely accessible to al , anytime and anywhere, there is no pre-schedule of learning activities, and no cultural and linguistic barriers, and it uses open licenses (OpenupEd, 2015). In any event, MOOCs are not completely without restric- tions and requirements, as the participants must have access to appropriate computer equipment (e.g., desk computer, laptop, tablet, mobile phone, etc.) and access to the Internet in order to participate in the program. In practice, there are various obstacles to openness. For example, Coursera retains the copyright to the materials and therefore they cannot easily be used for any other purposes. Teaching and learning materials can be removed from the web after the end of the course. The edX uses a different approach: an in- stitution that joins edX can develop its MOOC and freely chooses whether to copyright it or not. The open question is the status of copyrights in MOOCs whose content and materials are created by the learners themselves (Bates, 2016). What is an Online Course? As an online course, at least 80 percent of the learning content is considered to be delivered online35. As a rule, there are no face-to-face meetings. In addi- tion, all the learning activities are conducted online and there are no one-on- one synchronous tutorials. If 100% online implementation is not guaranteed, Jansen (2017) believes that it is not possible to talk about a MOOC, but rather about a blended or hybrid course. Bates points out (2016) that with the development of MOOCs, it is increas- ingly common for material from MOOCs to be used in blended courses. When is a MOOC Considered a Course? Downes (2013) proposes three criteria for a MOOC to be considered a course: • restrictions regarding the course’s start and end dates, • it is framed by a specific topic or discussion, • it is designed as an advancement in a series of successive events. The criteria proposed by Downes are typical of a traditional teaching model, rath- er than a model that should aim to maximise the flexibility of learning and teach- 35 This criterion was used by the Sloan Foundation, led by Babson College, for annual data collection on e-learning in the United States (Allen, 2016, p. 7). 210 Learning Approaches and Methods in a Digital Society ing. Nowadays, there is an increasing number of MOOCs that are not defined in terms of time (the beginning and end of implementation), as this achieves greater autonomy and independence in choosing the pace of the learning. OpenupEd (2015) defines a course with two requirements: • the study load must be at least 1 ECTS, corresponding to approximately 25 to 30 hours of study load; • the whole course must contain the following elements: learning content, provision of interaction (usual y between peers and at least some with edu- cational staff), it must contain learning activities (assignments, tests and feedback) and opportunities for recognition, especial y informal, in the form of various certificates; a study guide or curriculum. Bates (2016) defines MOOCs in relation to OERs and points out that, unlike OERs, MOOCs include all the components of an educational course. However, the question of their recognition remains unresolved. The above descriptions of individual characteristics express the substantive essence of MOOCs. Each of these MOOC attributes can be defined and ad- dressed differently, resulting in a great variety of MOOCs in practice in terms of the main purpose, underlying pedagogy, mode of delivery and related ser- vices, technological support, etc. An explanation of the basic dimensions (M, O, O, C) is not sufficient to understand MOOCs and to recognise their role and importance in modern education systems. It is necessary to get acquainted with the different types of MOOCs and the relations between them. This, how- ever, is the task of classifications. types of Moocs MOOCs are basical y divided into so-called cMOOCs and xMOOCs. This di- vision was introduced in 2012 by Siemens to highlight the difference between connectivist MOOCs and xMOOCs, i.e., the so-called extended MOOCs on the Coursera, Udemy and edX portals (Siemens, 2013). xMOOCs The MOOCs, first developed at Stanford and later at MIT and Harvard, are based on the learning theory of behaviourism. As a rule, they contain online recordings of short lessons (lectures) supplemented by computer-assisted au- tomated tests that sometimes include peer assessment (Bates, 2016). Knox (2018) describes the pedagogical approach of this group of MOOCs as a broadcast method, i.e., the direct transfer of information to a large number of passive participants (a teacher-centred approach). Open Education 211 Nowadays, the xMOOC model predominates, particularly with commer- cial providers. Although the authors have a fairly free hand in designing the xMOOC, the following is general y typical of them (Bates, 2016, pp. 184-185): • special y designed platform software that enables the registration (enrol- ment) of a large number of participants, the storage and transfer of digital material on request, automated assessment procedures and monitoring of participants’ activities, including learning analytics; • video lectures: xMOOCs use standard lectures, recordings of which are available to participants online, which can also be saved. The lectures ini- tial y lasted about 50 minutes, though now they are shorter. Even the cours- es are now shorter and last about five weeks; • computer-marked assignments: The participant receives feedback as soon as they complete the test. Tests can just provide feedback to participants, but they can also be a formal way of testing knowledge; • peer assessment: In some MOOCs, the possibility of participating in ran- domly formed groups was tested, especial y for more demanding assign- ments. This option usual y did not work due to differences in the level of knowledge and motivation of the participants; • supporting material: MOOCs usual y also contain a variety of audio or vid- eo recordings, links to other sources, freely available articles, etc.; • a shared comment/discussion space. In the learning environment, partici- pants are usual y provided with a virtual space where they can ask ques- tions, ask for help, send comments, etc. It is characteristic of xMOOCs that discussion is usual y limited or non-existent. The leader’s communication is focused on the group as a whole and not on the individual. The leadership of the group is carried out differently, this assignment can be performed by a teacher, hired tutors or by the participants themselves; • badges and certificates: In most MOOCs, participants receive some recog- nition (a badge) that is general y not formal y recognised; • monitoring the activities of the participants is possible using learning analytics. 212 Learning Approaches and Methods in a Digital Society Figure 28 shows an example of an xMOOC from the Coursera platform. Figure 28: coursera, learning how to learn, McMaster university, university of california San diego Source: Coursera (https://www.coursera.org/learn/learning-how-to-learn?ranMid=40328&raneai d=SayystvligQ&ranSiteid=SayystvligQ-7hhvu8tyntnyy1Zahdw5y ). cMOOCs From a pedagogical point of view, cMOOCs are the exact opposite of xMOOCs. They are derived from learning theory of connectivism, which was originated by Stephen Downes, co-author of the first cMOOC from the University of Manitoba. According to theory of connectivism, knowledge in cMOOCs is created through networking and discussion among participants related to the use of social media. Technological support is diverse and almost non-standard, as is typical of xMOOCs. Participants use a wide range of web applications and tools, from blogs, tweets and discussion forums to a variety of freely avail- able tools. The dynamics of cMOOCs are mostly determined by the interests and activity of the participants, and there is usual y no formal knowledge test (Bates, 2016). As a rule, there is no official y appointed teacher, though teach- ers can occasional y participate in the performance of cMOOCs (for example, through a blog or webinar). Open Education 213 Figure 29: links in a cMooc; provider’s view COURSE PROVIDER PERSPECTIVE Subscribed Student students content cMOOC Course content Live online Event events recordings Source: Downes, 2014a. The planning of cMOOCs is free and left to the initiative of the participants. Downes (2014a) sets out four basic principles that guide the emergence of cMOOCs: • the autonomy of the participant (in terms of the content or capabilities he or she wishes to acquire); learning is tailored to the individual, but the cen- tral theme and the most important organisational guidelines are identified; • diversity; this is reflected in the diversity of tools used, the structure of the participants and their level of knowledge, the topics covered, etc.; • openness; in terms of access, content, activities and knowledge testing; • interaction; col aborative learning, communication between participants and the resulting knowledge generation. In practice, the implementation of these principles deviates from the pure model (Bates, 2016); there is almost no MOOC without some form of support. For the most part, at least the key content topics are defined in advance. 214 Learning Approaches and Methods in a Digital Society Figure 30 exposes the essential feature of a cMOOC, that is networking. Figure 30: cMooc ”creativity & Multicultural communication”: a visualization of the participants’ activities Source: Yaeger et al., 2013, p. 139. Figure 30 shows the connections that occurred over the first four months of delivery of the cMOOC ”Creativity & Multicultural Communication” in the period September–December 2011. Participants are represented by a node, and a connection (an interaction as evidenced by replies) is depicted as a link. The size of the nodes is proportional to the number of posts made during the observed period, and the thickness of the links is proportional to the number of replies sent from one user to another. The colours indicate the relative in- fluence or importance of a participant in its position within a network. The largest node depicts course facilitator’s activities (Yeager et al., 2013, p. 138). A comparison of xMOOCs and cMOOCs shows that there are so many dif- ferences between them that it is almost impossible to talk about the same edu- cational concept. The starting point of cMOOCs is the creation of a learning community and connections through which new knowledge is created, while in the case of xMOOCs, it is the design of a massive performance course that is not cost-sensitive to the number of enrolled. Open Education 215 Figure 31: determinants of xMoocs and cMoocs What is Local massive? cohorts? X • 100? Self-paced? • 1000? Open • 10,000? registration? • l00,000? Start/end dates? College credits? FOCUS ON SCALABILITY M O O C Badges?Role of theinstructor? C MASSIVE OPEN ONLINE COURSE Real-time Learning Open content? interaction? community? Free of charge? Scripted assessments FOCUS ON and feedback? COMMUNITY Affordable? AND CONNECTIONS Source: Mathieu Plourde – (https://www.flickr.com/photos/mathplourde/8620174342/sizes/o/in/ photostream/ ). As Figure 31 shows, given the large number of determinants, MOOCs can be designed and carried out in countless ways. The classification of MOOCs is possible on the basis of one or more determinants, as well as other criteria (e.g., type of educational organisation, field of education). Classifications of MOOCs The multiplicity of manifestations has thus yielded numerous classifications of MOOCs. Brown (in Jansen and Konings, 2018, p.7) classifies MOOCs into three devel- opmental phases (waves) that partial y overlap and intertwine. These phases reflect the main objectives to be achieved with the MOOCs: Marketing MOOCs: Promoting the institution through MOOCs was the main motive of the elite universities that were the first to offer xMOOCs. Brown cites data from a study on the prevalence of online studies in the USA (Allen and Seaman, 2014) showing that the main reasons for introducing MOOCs are to increase an organisation’s visibility and encourage student enrolment. MOOCs for lifelong learning: Lifelong learning has stimulated the development of MOOCs, particularly in Europe, where their development was lagging. With some European projects (for example EMMA, HOME, MOONLITE & SCORE 2020, the OpenupEd portal and the FutureLearn portal), as well as the growing 216 Learning Approaches and Methods in a Digital Society popularity of US portals, interest in MOOCs has increased substantial y. How- ever, the reasons for the introduction of MOOCs in Europe are quite different from those in the USA. In primary place is the possibility of more flexible learn- ing opportunities, while business reasons are less important. MOOCs for Continuous Professional Development – (CPD): In recent years, MOOCs for continuous professional development have been on the rise in Europe. In 2017, FutureLearn of the UK offered the first set of online accredited programs in col aboration with Deakin University and Coventry Univer- sity. They planned to prepare a total of 50 courses over the next five years and are also in talks with other universities. In 2022, only for the field Business & Management, 21 degree courses and 28 microcredentials are offered by FutureLearn (https://www.futurelearn.com/degrees). The SURF (2014) survey lists a number of typical forms of MOOCs but the criteria for defining these forms are different (e.g., pedagogical design, scope, accessibility according to geographical area). Table 27: an overview of the main features of some typical Moocs Acronym Title/name Description MOOC Massive Open Online Course Massive open online courses mOOC Micro Open Online Course Micro open online courses SPOC Small Private Online Course Online courses of a mostly closed type SOOC Selective Open Online Course Selectivity in terms of enrolment, and not the number of enrolled DOCC Distributed Open Collaborative Course Participants collaborate through networking (similar to cMOOCs) LOOC Local Open Online Course Access is restricted to members of a particular community MOOR Massive Open Online Research Emphasis on the research component ROOC Regional Open Online Course Area restrictions (language barriers) HOOC Hybrid Open Online Course A combination with the traditional (face to face) teaching COOC Classically Offered Online Course Use of the fully online approach Source: SURF (2014), p. 67. Open Education 217 Clark (2014) cites eight different groups of MOOCs; the main criterion for their classification is functionality in the learning process. As can be observed, other criteria also seem to be taken into account, such as authorship and the pedagogical framework: • transferred MOOCs, • made MOOCs, • synch MOOCs, • asynch MOOCs, • adaptive MOOCs, • group MOOCs, • connectivist MOOCs , • mini MOOCs . A similar array of MOOCs classified on the basis of rather inconsistent crite- ria is also offered by other authors (Bates, 2016; Littlejohn and Hood, 2018). Numerous classifications of MOOCs primarily indicate their direction of de- velopment and also point out that assigning a variety of online educational courses a single MOOC name is actual y unnecessary and inappropriate. These classifications do not have the properties of classifications that would allow a consistent and unambiguous listing of a MOOC into a single category. advantages and limitations of Moocs A thorough analysis of the advantages and limitations of MOOCs would cer- tainly require separate treatment by individual groups of MOOCs. At this point, we limit ourselves to showing the advantages and limitations of the basic, origi- nal concept of MOOCs (Bates, 2016; Littlejohn and Hood, 2018; Weller, 2014). The main advantages of MOOCs are as fol ows: • MOOCs, particularly xMOOCs, allow anyone with a computer and Inter- net connection free access to high-quality educational content produced by world-renowned universities. • MOOCs can be a way to increase access to quality educational content even in the underdeveloped world, though they should be partial y adapted and resources should be provided for local support and partnerships. • MOOCs are a very useful tool for learning the basic concepts of a field and creating large learning communities with common interests or experiences. • In addition, MOOCs are suitable for lifelong learning and for continuous professional development. • MOOCs have encouraged traditional and, above al , elite universities to rethink online and open education strategies. Many institutions are now asserting their brand and status through MOOCs as an all-accessible way of acquiring knowledge at a high-quality level. 218 Learning Approaches and Methods in a Digital Society • MOOCs are also affordable for providers, particularly in case of a large enrolment, as there are no variable costs or they are minimal. However, it should not be overlooked that the low costs are mainly due to the fact that MOOCs general y do not include pedagogical support. MOOCs have aroused great interest from the public and the media, leading to a discussion of a number of issues relevant to education (Weller, 2014). MOOCs have received their share of criticism. Many of them focus on the un- favourable ratio between the number of enrolled in MOOCs and the number of those who successful y complete them. Data on the high enrolment numbers can be misleading at first glance, as less than half of those enrolled actively participate in the courses and only a small proportion of them successful y complete the course; nevertheless, speaking in absolute numbers, MOOCs go beyond traditional courses. In highlighting the problem of the small proportion of enrolled who successful y complete the MOOCs, we need to keep in mind that the motivation of those who enrol in a MOOC is very different. Many people who formal y enrol in the MOOC never planned to complete all the activities and to finish it. Ho et al. (2014, p. 13) classified persons enrolled in MOOCs into four groups according to the reasons for enrolment: registered persons, observers, researchers and persons enrolled for the purpose of obtaining a certificate. Despite savings on the cost of pedagogical support, the development of MOOCs requires considerable resources. Commercial providers with appropriate busi- ness strategies and models find solutions for their sustainability, while educa- tional institutions from the public sector have more limited opportunities. In ten years, MOOCs have gone from being a tool for experimenting on how to modernise education with technology, to a way of education mastered by just a handful of institutions (Weller, 2014, p. 114). The use of MOOCs to date shows: • that MOOCs are of particular interest to the better educated, while the less educated use them much less; • that MOOCs have, at least so far, been quite limited in developing the more demanding academic learning and competences required by the knowl- edge society; • that it is also problematic to assess more complex knowledge and compe- tencies obtained through MOOCs. The use of MOOCs may also be restricted by copyright. Open Education 219 information on Moocs The search for MOOCs can be performed using the specialised portals and websites of the most important providers. We can also use the normal tech- niques of online searching. MOOC web portals: • Class Central: https://www.class-central.com/ • Wikipedia: an up-to-date overview of MOOC platforms, https://en.wikipedia. org/wiki/list_of_Moocs • European MOOC Consortium: https://emc.eadtu.eu/ • MOOCs for the world of business: http://bizmooc.eu/ The Most Important MOOC providers Table 28: the most important Mooc portals and their characteristics Provider Sector Types of courses Platform/ Open environment licence Coursera MOOCs: offered by Commercial Closed No (www.coursera.org) universities worldwide edX (www.edx.org) Yes, MOOCs: offered by various Non-profit Open universities worldwide open licences Udacity Commercial Corporate training courses Closed No (www.udacity.com) FutureLearn MOOCs: offered by Non-profit Closed Unknown (www.futurelearn.com) universities worldwide OpenupEd (www. MOOCs and online openuped.eu) Non-profit courses: offered by Closed Unknown universities worldwide Iversity MOOCs: offered by Commercial Closed Yes (http://iversity.org) individual professors Blackboard course sites MOOC platform based on Commercial Closed Yes (www.coursesites.com) LMS Blackboard Learn Source: SURF, 2014, p. 23. 220 Learning Approaches and Methods in a Digital Society 6.2.5 the State of open education in the World, the european union and Slovenia. In the almost half-century of development of open education that was heralded by the establishment of Open University in 1970 in the United Kingdom, to the milestones set at the end of the 2010s, i.e., the adoption of the Ljubljana Action Plan at the Second World OER Congress in 2017 and the UNESCO recommenda- tions for OER development in 2019 (UNESCO 2017, 2019), the field of education in general has changed significantly. The reasons do not only lie in the realisation of political y supported ideas about the democratisation of education by opening it up. More educational opportunities, the development of new knowledge and competenies, and more flexibility in education are a necessity of the social reality of the twenty-first century. Technologies make it possible for these requirements to become a reality and for new, more innovative solutions to be constantly of- fered in the field of education as wel . But the process of opening up education is taking place at different paces and in different ways in different parts of the world. The global Commonwealth of Learning’s report on the state of OERs for 2017 also highlights the uneven development of OERs across the world’s regions and also that there is no real cooperation. OER stakeholders are still most concerned about how to secure resources and less about OER development and use. The re- port also reveals that education decision-makers often do not understand what OER means, what digital material is and what a MOOC is, what the differences are between them and the relationship between these categories and e-learning (Commonwealth of Learning, 2017). Large market-oriented universities are more successful than countries in recog- nising the opportunities offered by open education. In particular, large US uni- versities have already secured a monopoly or at least an oligopolistic position in the supply of MOOCs. According to data for 2017 (Shah, 2018), out of 81 mil ion enrolments in MOOCs, as many as 68.4 million or 84.4% belong to the five largest providers. Table 29: the world’s largest providers of Moocs Provider Number of users Coursera 30 million users edX 14 million users XuetangX 9.3 million users Udacity 8 million users FutureLearn 7.1 million users Source: Shah, 2018. Open Education 221 Large providers upgraded initial y free xMOOCs with successful business models; Udacity with nanostages, Coursera with specialisations, and edX with the so-called X-series (Shah, 2017b)36. Corporate MOOCs are also on the rise. According to the Class Central portal, 35 million people enrolled in MOOCs in 2017, compared to 16 to 18 million the year before (Docebo, 2018, p. 27). In the study on trends in e-learning, Docebo predicts the following for MOOCs in 2018 (ibid.): • An increase in time-flexible MOOCs. The participants are completely free to decide when they will enrol in the course, at what pace they will be edu- cated and when they will complete the course. • End of free certificates. In order to ensure revenue, many MOOC providers have abolished free certificates. • Higher education – a new target group. MOOCs are increasingly used as learning aids in higher education, especial y for the acquisition and con- solidation of knowledge by students when entering university. In the field of corporate training, MOOCs are increasingly being upgraded to online accredited programmes, which is proving a very successful business move. The business world has thus successful y transformed the original idea, which was first presented as a great opportunity to open up education with free, all- access quality courses, and subdued it to its profit-oriented interests. In the field of open education, the European Union is taking a path that is en- couraged by business interests, but to a lesser extent. Open and innovative ed- ucation and training suitable for a digital society is one of the priority strategic directions of education in the European Union (ET 2020, 2009). An important impetus for the promotion of open education (at least in political terms) was provided by the 2013 initiative ”Opening up Education: Innovative Teaching and Learning for All through New Technologies and Open Educational Re- sources”, in which open education is defined as an important goal of formal and non-formal education (European Commission, 2013). The adoption of open education as a mechanism for modernising education in the countries of the European Union in accordance with the needs of the society of the twenty-first century is also confirmed by the results of several surveys that have been conducted in recent years. These results suggest that revenue generation, high enrolment and cost reduction are among the less im- portant goals of MOOCs in the EU, with increasing educational opportunities at the forefront (Jansen and Konings, 2017). 36 X-series are prepared by world-renowned experts and the best-of-class universities. They consist of a series of educational courses with in-depth treatment of a specific field (What-is-an-XSeries-program: https://support.edx.org/hc/en-us/articles/207206427). 222 Learning Approaches and Methods in a Digital Society The results of these surveys show that the use of MOOCs is increasing, though they are quite unevenly distributed across Europe. The use of OERs is much more widespread (Muñoz et al., 2016). The relatively rapid introduction of MOOCs in some European countries was probably due to the traditional view that is characteristic of ”continental” Eu- rope, that higher education is also a public good. However, other factors should not be overlooked, such as the high youth unemployment rate in some Eu- ropean countries and the inadequate training of employees in some lines of business. In some countries, national policies and guidelines (e.g., Germany, Ireland and France) have also played an important role in promoting and cre- ating opportunities for the implementation of MOOCs and open education (Brown, 2018). On a global scale, Slovenia is considered a reference country in the field of the opening up of educational space with the introduction of the OER system. Compared to other countries, Slovenia has considerable experience in the de- velopment of technologies for open education that enable the creation of an educational environment that is personalised to the individual. These experi- ences come mostly from the VideoLectures.Net project that was started more than fifteen years ago by the Jožef Stefan Institute, where they launched record- ing and publishing lectures on their portal. The portal has become internation- al y recognised due to its connections with reputable European and American universities, as well as due to the impact of the World Summit Award that was presented to the creators of the portal in 2009 by the United Nations Summit on Information Society. VideoLectures.Net (http://videolectures.net/), which at the end of March 2019 hosted 16,164 authors, 22,020 lectures and 25,550 videos, is the larg- est portal of its kind in the world. Since 2014, the Jožef Stefan Institute has been the holder of the Chair on Open Technologies for OERs at UNESCO. The work of the chair takes place under the auspices and in synergy with the initiative Opening up Slovenia and the Knowledge 4 All Foundation Ltd., which brings together more than 60 mem- bers, including the most important research and development centres in the field of artificial intelligence in Europe (https://ouslovenia.net/). Thus, Slovenia has the opportunity not only to cooperate with the most advanced in this field, but also to encourage countries or regions that have not yet ful y embraced open education and OERs. At the forefront is the Opening up Slovenia ini- tiative; its purpose is to introduce a variety of practices for a more open and accessible learning environment, notably by introducing resources that are publicly available to all under a Creative Commons licence. With the Opening up Slovenia initiative, Slovenia has become one of the world’s leading coun- Open Education 223 tries in the field of OERs. In 2016, with the support of UNESCO, it launched a joint document at the global level with recommendations that countries should formulate policies and measures to open up education. These activities were rounded off by the Second World OER Congress 2017 and the adoption of the Ljubljana Action Plan that includes activities to empower teachers and preserve the cultural and linguistic diversity of OERs (UNESCO, 2017). UN- ESCO formal y adopted a document with recommendations for the use of OERs at the 40th General Assembly in November 2019 (UNESCO, 2019). The University of Nova Gorica and the Jožef Stefan Institute are the initia- tors and coordinators of the innovative international project Open Edu- cation for a Better World (OE4BW). The basic objective of this project is to use the potentials of open education to achieve the objectives of sus- tainable development. The core innovation of the course is in develop- ing OERs by connecting interested OER authors and reputable experts who mentor the authors on a voluntary basis. The results to date confirm the usefulness of the project, particularly for the development of capaci- ties for open education and also for the development of directly useful and social y important open learning materials (Urbančič et al., 2019). In three years, from 2018 onwards, 35 countries from six continents participated in the OE4BW project https://oe4bw.org/ with more than 120 projects. This project is the recipient of the OE Award for Excellence of the international non-profit association Open Education Global for 2020. (https://awards.oeglobal.org/awards/). Slovenia is therefore extremely active and successful beyond its borders in the field of opening education, but how many of these opportunities are made use of at home? A dozen MOOCs (acronym MOST in Slovenian language) are available on the Slovenian Education Network portal, which are primarily intended for the devel- opment of digital competencies. They were developed by ARNES (Academic and Research Network of Slovenia) within the context of the SIO-2020 programme as part of the e-services and e-content project (https://www.arnes.si/en/). 224 Learning Approaches and Methods in a Digital Society A survey on the situation in the field of e-learning in the higher educa- tion sector in Slovenia in 2017 (Bregar and Puhek, 2017) showed that MOOCs are used by higher education in Slovenia to very modest extent (less than 10% of higher education institutions). A slightly greater num- ber of higher education institutions (16%) plan to use them. The main reasons for the planned introduction are the acquisition of new target groups, promotion and greater visibility, as well as the expanding of the educational offer. Three-quarters of higher education institutions show no interest in MOOCs (as they neither use them nor plan to use them). Most of them stated that they did not have enough money to develop their own MOOCs, as well as a lack of properly trained staff. Many of them did not consider MOOCs to correspond to their pedagogical ap- proaches or statements. The most common reasons for not using MOOCs in Slovenia are different from the usual y stated shortcomings and indicate a lack of knowledge and misunderstanding of the topic. In fact, this is not surprising: MOOCs are gen- eral y poorly known in Slovenia. Among the few higher education institutions that use them, most are private, medium-sized and in the field of social sci- ences. There is no systematic provision of information and training of teachers in Slovenia, regardless of the level of education on the notion and opportuni- ties of open education in general, or about MOOCs. On the other hand, the Opening up Slovenia portal is intended primarily to present the activities of the initiators of this initiative in Slovenia and to raise awareness of various international events in this field. recommended links Class Central: https://www.classcentral.com/ Open Education Handbook: https://education.okfn.org/handbook/index.html EdReNe: http://www.edrene.org/ Open Education Global: https://www.oeglobal.org/ VideoLectures Net: http://videolectures.net/ Opening up Slovenia: https://ouslovenia.net/ OPEN EDUCATION for a BETTER WORLD: https://oe4bw.org/ Open Education 225 UNESCO. Open Educational Resources (OER): https://en.unesco.org/themes/building-knowledge-societies/oer/resources EU Science Hub. Open Education: https://ec.europa.eu/jrc/en/open-education Encore Project. European Network for Catalysing Open Resources in Education: https://encoreproject.eu/ TL National Forum Ireland. Using OER and OEP for Teaching and Learning: https://open.teachingandlearning.ie/all-resources/#reps 226 Learning Approaches and Methods in a Digital Society 6.3 artificial intelligence 6.3.1 What is artificial intelligence Artificial intelligence (AI) is a field of computer science that has fascinated humanity for more than six decades. The event that established AI as a new scientific discipline is usual y cited as the 1956 Dartmouth Conference in the USA, and the term is attributed to John McCarthy, who was then a mathemat- ics teacher at Dartmouth College (Moor, 2006, p. 87). Figure 32: Plaque to the pioneers of artificial intelligence at dartmouth Source: Moor, 2006. In the decades since then, AI has been the main inspiration for many works of science fiction. Films such as 2001: A Space Odyssey, The Matrix, Star Wars, The Exorcist, Ex Machina and many others are credited with popularising the idea of AI (List of Artificial Intelligence Films, n.d.). In recent years, AI has experienced a remarkable rise as a scientific discipline, with results and achievements such as travel navigation systems, spam filtering, purchase recommendations for various products and services which are inexo- rably becoming part of our everyday lives and changing them. The breakthrough of AI is attributed to the increasing capacity of comput- ing systems and devices, improved algorithms, the tremendous growth in the Artificial Intelligence 227 amount of data available and the increasing investments (Craglia, 2018, p. 16). Today, machines and devices are already using AI to perform a variety of human tasks, from driving cars to playing chess and other games, detecting abuse, translating in real-time and giving advice and assistance. Robotics is now a reality in many industrial activities. The growth of AI is expected to be enormous in the coming years. The global market for AI software is expected to more than double between 2018 and 2025, from $10 billion in 2018 to near- ly $120 billion in 2025 (Statista, 2019). The AI market consists of robotics, autonomous vehicles, big data, virtual agents/assistants, voice recognition services and message processing. AI has also been high on political agendas recently, mainly due to its expected im- pact on the labour market and many other areas, including education (Tuomi, 2018). It is seen as the electricity of the modern age, the impact of which will only become apparent over time, just as the widespread consequences of the discovery of electricity did not become apparent until decades later. When we talk about AI, we usual y think of humanoid robots, i.e., human-like robots that carry out tasks that humans normal y do. AI is often defined as a computer system capable of performing tasks that normal y require human abilities, such as visual perception, speech recognition, decision-making and translation (Lexico Dictionary, 2019). As a generic term, AI is any machine or algorithm that is capable of observ- ing its surroundings, learning, and based on the knowledge and experience gained, performing intelligent (rational) actions or suggesting appropriate ac- tions (Craglia, 2019, p. 19). AI is usual y divided into two basic groups (UN- ESCO, 2019b, p. 9): • knowledge-based AI about a domain and its associated rules; • data-driven AI. AI tools and methods make the following possible (Deloitte, 2017, p. 4): • recognition and understanding (AI enable devices to recognise and under- stand handwriting, text, voice, images and videos using machine transla- tion, semantic computing, predictive algorithms and machine learning.) • identifying the meaning of semantics (With AI, devices can use semantics to understand the meaning of words in relation to contextual information.); • the use of context and interactivity (AI generates information depending on the context, taking into account the results of the computer processing of data from different sources and using different methods); • imitating thought processes and decision-making (AI allows standard thought processes to be adapted to the user and appropriate decisions to be made depending on the circumstances.); • learning and improving based on what you learn (AI allows the application to continuously) learn and always improve based on the results and feedback). 228 Learning Approaches and Methods in a Digital Society Figure 33 shows the 2016 ranking of the most common AI methods by the research organisation Deloitte (Mil s, 2016). Figure 33: overview of artificial intelligence methods deep learning supervised machine learning (ML) unsupervised content extraction classification machine translation natural language processing NLP Artificial question answering Intelligence (Al) text generation expert systems image recognition vision machine vision speech to text speech text to speech planning robotics Source: Mil s, 2016, p. 3. The methodological tools of AI are very diverse. They are made up of (Corea, 2018): • logic-based tools, • knowledge-based tools, • probabilistic methods, • machine learning, • embodied intelligence, • search and optimisation tools. MOOCs offer the opportunity to learn more about AI. One of the most popular is the Machine Learning Specialisation course, developed by Stanford University in the US and available on Coursera. It is an updated version of Andrew Ng’s pioneering Machine Learning course taken by over 4.8 million learners since it launched in 2012 (https://www.coursera. org/specializations/machine-learning-introduction). Artificial Intelligence 229 6.3.2 the impact of artificial intelligence on education Expectations about the role of AI in transforming education are high to- day. Certainly, some of this enthusiasm is attributable to the AI score on the Gartner innovation popularity curve in 2019 (Tuomi, 2018). Educause’s 2019 Technology Trends Forecast promises a fairly rapid uptake of AI: within two to three years, it should already be used in higher education in developed coun- tries (Alexander et al., 2019). The European Commission notes that trusted AI technologies could help predict more accurately which jobs and occupations will be completely trans- formed by technology, which new roles will be created and which skil s and ca- pabilities will be needed. In addition, AI can be a great tool to fight educational inequalities and to design personalised and flexible education courses. AI tools can help people acquire new qualifications, knowledge, skil s and abilities in line with their learning abilities. AI could accelerate learning and improve the quality of education from primary school to university (European Commis- sion, 2019c). AI is important for education not only as an opportunity for more effective and higher quality learning and teaching, but also as a means to better under- stand the processes of thinking, knowledge creation and intelligent behaviour (Woolf et al., 2013). The development of AI tools to-date predicts that it will have a particular im- pact on the following aspects of education in the future (Woolf et al., 2013): • individualised pedagogical support, • learning twenty-first-century competencies, • interaction data to support learning, • universal access to the global classroom, • lifelong learning and lifewide learning. Individualised pedagogical support. ITSs were the first examples of the use of AI in education. They usual y consist of a domain model (of the domain of ex- pertise that is the object of learning) and a learner model, which describes the learner’s state and learning characteristics. The expert or pedagogical model manages learning materials, tasks and activities through a flexible and inter- active user interface. The trend is towards machine learning using big data. Current ITSs already allow for the personalisation of learning according to the learner’s characteristics (e.g., personality, preferences, motivation, etc.), (Tuomi, 2018). More on ITSs in Section 6.5. 230 Learning Approaches and Methods in a Digital Society IBM Watson is one of the best-known AI systems. It offers a number of tools, mainly for the business sector, but also useful for education. IBM Watson for Education enables the creation of personalised learning content based on users’ knowledge, personalised tutoring (1 : 1), moni- toring learning progress, communication and analytics, and special ap- plications for children’s vocabulary development (IBM Watson, n.d.). Learning twenty-first-century competencies. This type of learning requires changes in the way we learn and teach, which must take into account of an un- predictable future and be able to respond to constant change. Tomorrow’s job market will require the ability to solve interdisciplinary problems in a cross- cultural environment in a team and to think scientifical y. The Edulai Intelligent Programme (https://www.edulai.com/) helps learn- ers and teachers to monitor and measure the development of competen- cies such as critical thinking, communication, col aboration, leadership, problem-solving and interculturality. The programme can simultane- ously check the level of these competencies and adjust the recommended learning content according to the level achieved. Achievements are dem- onstrated by badges, which are entered into a public portfolio of learners that is also accessible to potential employers. Interaction data to support learning. An important aspect of AI is the use of digital data, which is generated in various forms of interaction in the learning process. Two methodologies support this aspect of use: LA and educational data mining. Both methodologies share the common goal of supporting learn- ers by keeping them informed about their learning achievements, planning appropriate help and providing additional support. The information provided by LA and data mining is also useful for educational organisations and teach- ers to plan the learning process and tailor its delivery to the characteristics of individuals. More on LA in the next section. The review article Systematic Review on Educational Data Mining (Dutt et al., 2017) provides an overview of 316 articles for the 1983 to 2016 period on the use of clustering methods to analyse data on student char- acteristics. Most of the articles are about e-learning, thanks to the avail- ability of digital data. Universal access to the global classroom. The idea of open access to the global classroom is linked to the concepts of OERs and open education. AI methods Artificial Intelligence 231 offer solutions to the two main pedagogical problems of MOOCs: no person- alisation of learning and no pedagogical support. AI has the tools to use OERs effectively. The aim of the X5gon project, coordinated by the Jožef Stefan Institute, is to develop a technology for linking open repositories and other forms of OER that will allow users to select relevant resources in real-time through interactive processes according to user characteristics. Osnabruck Uni- versity, Universitat Politecnica de Valencia and the Knowledge 4 All Foundation are developing pilot solutions for the integration of open re- sources in the framework of this project (https://www.x5gon.org/). Lifelong learning and lifewide learning. One of the fundamental orientations of lifelong learning is to connect with reality and real-world problems. AI offers a range of opportunities to implement the principles of authentic and active learning, from the ability to tailor learning resources to the interests of the learner, to different mobile learning formats, to the use of virtual and augmented reality methods. The aspects of the use of AI mentioned in the previous sections (personalised pedagogical support, training for 21st century capabilities, etc.) are also relevant for lifelong and lifewide learning. Both types of AI-assisted learning can take place both within and outside formal educa- tion. Technological developments are bringing new opportunities for the use of AI. Blockchain, the technology that underpinned the development and use of cryptocurrencies, is now knocking on the door of education. Blockchain technology is a decentralised, transparent way of transferring (transacting) data. Educational institutions can use blockchain technol- ogy for the cloud storage of various information related to the educational process. This technology can provide a consistent, transparent and reli- able record of an individual’s learning achievements in an open learning environment (i.e., the successful completion of MOOCs and other open learning modalities), (Jarman, 2019). 6.3.3 artificial intelligence and e-learning AI will undoubtedly be a considerable factor in changing education – both traditional and e-learning – in the years to come. We believe that e-learning is better equipped to take advantage of the opportunities offered by AI more quickly and efficiently. The reasons are as follows. 232 Learning Approaches and Methods in a Digital Society Firstly, educational institutions and systems that are already implementing e- learning courses and programmes have a significant advantage over traditional education when it comes to introducing AI into education, as their organisa- tional and staffing structure is already more or less adapted to the digitalisation of learning and support processes. Experience shows that implementing these changes is not a simple process that can be done quickly. Secondly, AI offers e-learning more advanced and higher quality technical so- lutions to the main problems that e-learning has been dealing with since its inception (like its predecessor distance education). It is a question of how to compensate for the absence of direct social interaction in the learning process at the expense of its greater spatial and, to some extent, temporal flexibility. These problems have been partly alleviated by improved communication using web 2.0 tools. AI has the potential for many innovative possibilities, especial y towards the greater personalisation of learning and even greater openness. 6.3.4 the limits and challenges of artificial intelligence Experts believe that AI is still in its early stages of development. However, de- velopments to-date have already highlighted the serious dilemmas posed by its further expansion. The reach of AI is limited firstly by the quality of the input data and by the fact that the algorithms are based on historical data, the use of which can only pro- ject the world as a reflection of the past (Tuomi, 2018). The study Artificial In- telligence, European Perspective points out that the performance of algorithms in machine learning, which is one of the most important areas of artificial intelligence, often has the characteristics of a black box: we know the inputs and the outputs, but we don’t ful y understand what happens in between, so we don’t have a good understanding of how the outputs are generated or the decisions made (Craglia, 2018). The use of personal and other individual data, which is the main source for much of the AI methods, raises a number of seri- ous issues of data security, protection and misuse, including in education. The Financial Times (in OECD, 2018a, p. 3) warns of the danger that the edu- cation system will produce individuals who will be no more than second-class computers if education remains focused on the transmission of explicit knowl- edge from one generation to the next. The challenge for AI is not only how to train more AI and computer scientists. Above al , it is important to develop com- petencies that cannot be replicated or even surpassed by AI. These are essential human capabilities, such as teamwork, leadership, listening, positive orientation and empathy, working with people, crisis and conflict management. Artificial Intelligence 233 These and other issues are, of course, a serious professional challenge for inter- disciplinary teams of experts (in AI, IT, ethics, law, philosophy, etc.). The Eu- ropean Commission’s experts have grouped potential threats and constraints into these groups (European Commission, 2019c, pp. 33–35): • using AI to track and identify individuals, • covert AI systems, • scoring citizens using AI in violation of fundamental rights, • lethal autonomous weapon systems, • potential longer-term concerns, such as super-intelligence and the poten- tial danger of subjugating humanity in the future. The European Commission has been quite active in addressing the ethical and other issues raised by AI. The European AI Alliance website provides informa- tion on various AI activities in the European Union. In April 2019, the Euro- pean Commission published the Ethical Guidelines on Trustworthy AI, which were drafted by an independent expert group of 52 experts from various fields related to AI (European Commission, 2019c). The following figure shows the guidelines for trustworthy AI. Figure 34: the guidelines as a framework for trustworthy ai Trustworthy AI Lawful AI Ethical AI Robust AI (not dealt with in this document) Foundations of Trustworthy Al 4 Ethical Principles Respect for human autonomy Prevention of harm Adhere to ethical principles based on Adknowledge and address tensions Fairness fundamental rights between them Explicability Realisation of Trustworthy AI 7 key Requirenments Human agency and oversight Technical robustness and safety Implement the key requirements Evaluate and address these continuously Privacy and data governance throughout the Al system's life cycle Transparency via Diversity, non-discrimination and fairness Societal and environmental wellbeing Technical Non-Technical Accountability Methods Methods Assessment of Trustworthy AI Trustworthy AI Assessment List Operationalise the key requirements Tailor this to the specific AI application Source: European Commission, 2019c, p. 8. 234 Learning Approaches and Methods in a Digital Society 6.3.5 artificial intelligence in Slovenia Research on AI in Slovenia began around 1972 at the Jožef Stefan Institute in Ljubljana, and a little later at Faculty of Computer and Information Science at the University of Ljubljana. The Artificial Intelligence Group was founded at the Jožef Stefan Institute in 1979, renamed the Artificial Intelligence Labora- tory in 1985, and the Artificial Intelligence Laboratory was established at the Faculty of Computer and Information Science in 1981. The two laboratories have gradual y grown into several research sections and laboratories within both institutions. Similar laboratories and centres have also been developed at most Slovenian universities, some research institutes and some companies, especial y those in the field of computer science and ICT. The first research on AI in Slovenia dealt with heuristic investigation algorithms, mainly in connec- tion with computer chess. According to Slovenian Current Research Informa- tion System, there are now more than 30 research groups in Slovenia whose research is related to AI (Slovenian Artificial Intelligence Society, n.d.). The work has subsequently expanded into areas such as machine learning, knowledge representation, expert systems, computational decision support, qualitative reasoning and combinatorial optimisation. Algorithms, methods and computer software were developed that soon proved useful in fields ranging from mechanical engineering to medicine. In the 1990s, as everywhere else in the world, research into AI in Slovenia expanded and diversified into many other areas. The most significant developments have been made in the following areas: machine learning, expert systems and knowledge representation, inductive logic programming, intelligent systems and intelligent agents, evolutionary computa- tion and genetic algorithms, decision support systems and decision modelling, human language technologies and speech synthesis, cognitive modelling, data analysis and data mining, semantic web technologies, knowledge management (Artificial Intelligence in Slovenia, n.d.). All these methods and approaches are also applicable to education. The achievements of Slovenian AI experts are international y renowned and im- portant. This is also evidenced by UNESCO’s agreement in April 2019 to sup- port the proposal of MIZŠ to establish an international research centre for AI in Slovenia under the auspices of UNESCO (MIZŠ, 2019). The UNESCO Assembly adopted a decision to establish the Centre at the end of November 2019. Its mis- sion is to be the first global centre for AI under the auspices of UNESCO and a model for other similar centres to be established around the world (SiolNET, 2019). Artificial Intelligence 235 recommended links Jožef Stefan Institute, Department of Artificial Intelligence: http://ailab.ijs.si/si/ Jožef Stefan Institute, Department of Knowledge Technologies: http://kt.ijs.si/ Jožef Stefan Institute, Department of Intelligent Systems: https://dis.ijs.si/ Towards AI: https://towardsai.net/ Educational Data Mining: http://educationaldatamining.org/edm2019/ Edulai: https://www.edulai.com/ The European AI Alliance: https://ec.europa.eu/digital-single-market/en/european-ai-alliance 236 Learning Approaches and Methods in a Digital Society 6.4 learning analytics 6.4.1 the concept and importance of learning analytics The development of learning analytics (LA) as a distinct discipline of an inter- disciplinary nature dates back to 2011, when the Society for Learning Analyt- ics Research (SoLAR) was founded, and provided a definition of LA that is still widely used and quoted today: ”Learning analytics is the measurement, collection, analysis and reporting of data about learners and their contexts, for purposes of understanding and optimising learning and the environments in which it occurs.” (Long and Siemens, 2011, p. 34). In less than a decade, LA has become one of the most promising and rapidly developing areas for modernising education through the use of modern tech- nologies. LA is one of the central themes in the literature and at conferences on contemporary trends in education. Educational equipment providers are following suit with new tools. Development is particularly dynamic in the US and Australia. The main purpose of LA is to transform learning and teaching data through analytical methods into useful information that enables activities and actions that can improve learning and teaching. The LA is therefore made up of three core components: data, analytics and ac- tions or measures based on the analytical results. LA differs from traditional analytical approaches in that it is largely based on the exploitation of big data generated by technology-enhanced education, so-called digital footprints. It also highlights the importance of acting and acting on the information derived from data (Chatti et al., 2012). LA is a suitable tool for improving the quality of learning and teaching, espe- cial y in organisations where online or blended learning is the predominant mode of education, or is used at least at the level of educational programmes. These methods assume a high degree of digitalisation of educational processes – and the consequent automatic (real-time) generation of large amounts of data. The LA concept is based on the availability of big data. The Joint Information System Committee – JISC lists the following areas of education where the use of LA can be most effective (Sclater et al., 2016): • Improving quality. Educators can improve their practice based on the infor- mation they get from LA. LA can be given on an ongoing and continuous basis Learning Analytics 237 • Feedback to teaching staff on the quality of their teaching, learning materials and assessment, which also allows for continuous revision and improve- ment. LA also allows the real-time monitoring of learners and identifica- tion of the main problems they face in the learning process. • Keeping learners informed about their learning progress and guiding them on how to best achieve their learning goals is also one possible use of LA. LA empowers learners to monitor the learning process and make decisions about it. • Preventing drop-outs (increasing throughput). LA allows the identification, at an early stage of the learning process, of learners who are more likely to fail to complete their education. This is the basis for immediate action (ad- ditional aid, incentives, etc.). • Introducing adaptive learning. Adaptive learning helps learners to develop knowledge and competencies in a more personalised and customised dy- namic. The learning content is also adapted to the individual’s abilities. Papamitsiou and Economides (2014), based on a systematic literature review of empirical research in the field of LA and data mining for the 2008–2013 period, extracted these advantages of LA that directly contribute to improving the learning process in an educational organisation: • Large volume of data allows for the greater precision of results. • Using powerful, pre-defined and valid algorithmic methods. • Multiple and effective contextual visualisations of analytical results are pos- sible. • More precise user models for the personalisation and adaptation of learn- ing systems. • Identification of critical moments and learning patterns. • Insights into learners‘ learning strategies and behaviour. LA is original y about the individual in the learning process at the level of the course, or educational department and is primarily aimed at improving learn- ing and teaching. In parallel to LA, so-called academic analytics are being de- veloped, focusing on the institutional and national levels. 6.4.2 the design and development of learning analytics Studies on the development and use of LA point out that educational institutions and the education sector cannot realise the great opportunities and comparative advantages offered by big data analytics unless they undertake the implementa- tion of LA as a multifaceted and complex project (Siemens et al., 2013; Gašević et al., 2018; Papamitsiou and Economides, 2014). In practice, it is often paid too 238 Learning Approaches and Methods in a Digital Society little attention to organisational and other ”non-technical” factors. The conse- quence of neglecting these factors is – as it holds also for e-learning in general – that the project results fade away quickly after the project is over. To help organisations implementing LA, researchers from the Open Univer- sity in the Netherlands, W. Greller and H. Drachsler (2012), have developed an integral model of LA factors, which they call the Generic Framework for Learn- ing Analytics. This model is a good starting point to prepare an educational organisation for the introduction of LA, as it connects the essential factors for a successful LA introduction in a simple and transparent way at a generic level. Figure 35: general framework for learning analytics Internal Limitations External Constraints Competences Acceptance Conventions Norms Institution Technology Instrumen Teachers Algorithm LEARNING ANALYTICS Learners Theories ts Stakeholders Other Other Reflection Prediction Open Protected Objectives Data Source: Grel er and Drachsler, 2012, p. 44. The overall LA framework consists of six critical dimensions: • objectives, • data, • internal limitations, • external constraints, • stakeholders, • instruments. Learning Analytics 239 Each of these dimensions requires appropriate consideration or review in the development and implementation of any version of the LA. This framework can also be used as a starting point for introducing LA in specific areas or for introducing the use of specific LA tools. The following table il ustrates the application of this generic model to the SNAPP tool (Social Networks Adapting Pedagogical Practice); this tool is widely used for discussion forum analytics. Table 30: example of the definition of the critical dimensions of la framework for the SnaPP tool Dimension Definitions Stakeholders Data holders: student group. Data users: tutor, discussion moderator. Objectives Reflection/assessment of the situation: Analysis of student interactions in the discussion forum, the identification of connections (networks) between students and isolated students. Data: Protected dataset: accessibility: student interactions and posts in the LMS discussion forum. Relevant indicators: forum posts, replies to posts. Time scale: the period of analysis. Instruments Pedagogic theory: social constructivism with the hypothesis that more active participants in the discussion achieve better learning results. Technology: social network analysis (SNA), statistics. Presentation: SNA graph, statistical tables. External constraints Legislation and legal frameworks: 1) Privacy: does the analysis comply with data protection provisions, are students informed about the use of the data, is this use of the data legally mandated? 2) Ethics: What are the risks of the misuse or improper use of data? Timescale: Will students be able to use the results of the analysis? Are the results available in real-time or post festum? Internal limitations Required competencies: 1) Interpretation of results. Do users have the capacity to interpret the results correctly? Do they understand the results shown (graphically or in a table)? 2) Critical thinking: Can they evaluate the quality of the results? Do they know what the analytical limitations are and how far the results can be used? Source: Grel er and Drachsler, 2012, p. 45. 240 Learning Approaches and Methods in a Digital Society 6.4.3 learning analytics implementation circle In terms of implementation, LA is in fact a particular application of the analyti- cal (scientific) research approach to education in a digitalised society. Analytics is broadly defined as the methodological discipline of the systematic process of transforming data into decision-useful information. LA is its application in the field of education and as such can be seen as a systematic process of trans- forming data into useful information for decision-making related to learning and teaching. Like any analytical process, it roughly comprises three phases: data collection and processing, analysis of the results, decisions based on the find- ings of the analysis37; these findings can be elaborated in more detail in the operational phase (Figure 36). Figure 36: the steps (stages) of the learning analytics process model Improving the elements of the learning process Data Proposals specification for action and collection Data Methods analysis and and data interpretation processing Source: Bregar and Puhek, 2018. The implementation of LA follows the following steps: data specification, se- lection and collection, the selection of methods and data processing, analysis and interpretation of results, proposals for measures, the redesign/refinement of elements of the learning process and their improvement. The steps of the LA process model largely follow the steps of e-learning pro- gramme design (mainly the ADDIE model), (Section 3.1). They are specific to issues related to data and analytical methods, which are briefly discussed below. 37 This is how they define the so-called LA process models – Detrick (2016); Dron and Anderson (2009); Chatti et al. (2012). Learning Analytics 241 data Specification and collection The LA model can be data-driven or questions-driven. Some theorists point out (Chatti et al., 2012; Fergusson, 2012) that the development of both business analytics and LA is data-driven. However, we believe that when developing user models of LA, it makes sense to start from the objectives of LA, which should be grounded in the substantive issues of the learning process that the edu- cational organisation wants to highlight or explore. Of course, in the second step of developing the user model, we also need to consider the possibilities and limitations of the data collected. Depending on the availability of data, it is useful to distinguish between data from traditional and digitalised sources. The data from traditional sources, on which traditional analytical models are based, are characterised by being cap- tured sporadical y and in that sense are static. Such data includes, for example, student records, personnel records, satisfaction surveys of different stakehold- ers, financial data and curriculum data. Even if this data is stored electronical- ly, they cannot be accessed continuously and can only be retrieved for certain periods of time. Data from digitalised sources (so-called fluid data) is characterised by the fact that it is created by the activity of a subject (i.e., a student in a digital environ- ment – a so-called digital footprint), (Detrick, 2016). Sources of such data are student IDs, LMSs, online library, use of MOOCs, etc. The design and development of the developed (complex) LA model requires an integrated database that captures data (static and fluid) on essential impact factors of students’ outcomes, on the learners’ learning activities and achieve- ments and on the features of virtual learning environment (VLE), (Kuzilek et al., 2017). As an example, the student database developed by the Open Uni- versity in the UK is presented in Figure 37. The anonymised dataset is freely available at https://analyse.kmi.open.ac.uk/open_dataset. Figure 37: components of the integrated student database for the learning analytics Student database Demographics Registrations Assessments VLE Data Courses Source: Open University Open Dataset (https://analyse.kmi.open.ac.uk/open_dataset ). However, when dealing with data, we should not forget about the quality of the data from two perspectives: the way it is captured and any problems in the 242 Learning Approaches and Methods in a Digital Society system or application itself that affect the quality of the data. Take for example the average time a student is logged into the LMS, which is usual y used as an indicator of activity. A number of circumstances affect the analytical useful- ness of this data. For example, how is a single or multiple inactivity of a logged- in LMS handled? Is it detected by the system? Furthermore, the intensity of, for example, a student’s work can vary greatly when an activity is formal y presented, learning strategies vary greatly among learners, etc. In general, the quality of data used in LA has received little research attention (Kovanović et al., 2015; Kovanović et al., 2016). In order to improve the qual- ity of the fluid data, it is first necessary to know and understand the algorithm used to generate the data (for example, for the number of active learners in each course in the LMS) and also the specific circumstances under which the data is generated. But special tools and approaches are also needed to ”clean” (edit) large data sets. All this only makes it possible to isolate the impact of these anomalies and to improve the quality of the data. Methods and data Processing LA studies the performance of learning and teaching processes in relation to learners’ achievements, using a variety of methodological approaches, from the less sophisticated, traditional ones using mostly simple methods of de- scriptive statistics, to the more complex statistical methods of multivariate analysis, and the innovative approaches enabled and encouraged by ”big data”. These include quantitative data and text mining methods, semantic and lin- guistic analysis methods, social network analysis, AI methods. In the literature on methods used in LA, methods and approaches are most often classified according to analytical focus (Detrick, 2016; Fergusson, 2012). Social network analytics examine the situation in social groups in terms of in- terpersonal relationships, either as egocentric network analysis (from the per- spective of the individual) or whole network analysis (the group as a whole). Discourse analysis (dialogue analytics and debate analytics) examines the inter- actions between learners to identify their characteristics. Dialogue and debate re- flect the cognitive processes of the individual. The focus is on attention analysis, rhetorical features, the choice of discussion topics and social interaction. Content analysis encompasses a range of methods for browsing, indexing, sort- ing and selecting online resources in order to guide learners through the ”mass” of potential y useful resources available. They are derived from classification tags and metadata generated by learners (users) using online information. Disposition analytics – LA examines the complex combination of experience, motivation and intelligence that influence an individual’s attitude to learning Learning Analytics 243 and his/her learning performance. Learning dispositions are influenced by a willingness to learn and change, critical curiosity, the ability to make sense of information, sensitivity, creativity, relationships and connectedness, and stra- tegic awareness. Context analytics – All the categories of social LA discussed here can be used in a wide variety of contexts, such as all levels of formal education, informal education and mobile learning. Learning can take place synchronously or asynchronously, in a group or alone, etc. Context analytics comprises tools that allow the exploration and understanding of the context of the learning process in terms of individual characteristics, timeframe, location, activities and connections (between learners, teachers and resources). These methods and approaches are mostly used only at the research and ac- ademic level, while traditional statistical methods stil dominate the practical application of LA (Dawson et al., 2018). Of these, the most commonly used are simple descriptive statistics methods (measures of means, frequency dis- tributions, various coefficients and simple indices, etc.) presented in tables or graphs. Here and there, we also encounter statistical hypothesis testing (para- metric and non-parametric tests). Predictive models are based on the use of various statistical methods of correlation and dependence (various regression models), while more sophisticated statistical methods that are also suitable for LA (multivariate methods of variance and covariance, factor analysis, cluster- ing, etc.) are less commonly used. 6.4.4 learning analytics Models by level of complexity LA models can be designed and implemented with different levels of sophistica- tion, depending on the available data and IT support, and consequently on the approaches and methods used in data processing and analysis. As the follow- ing figure and the examples below show, the use of LA models with different levels of complexity also provides different types of information, ranging from simple to complex, which thus form the basis for different types of actions in the learning process. In 2014, Gartner published a paper on why advanced business analytics is a top business opportunity, including a classification of business analysts according to their analytical reach, which is closely related to the complexity or sophisti- cation of the methods used (Gartner, 2014). The Gartner classification is also widely used in the LA field. 244 Learning Approaches and Methods in a Digital Society Figure 38: gartner classification of analytics complexity Analytics Human Input Descriptive (What happened?) Diagnosti (Why did it happen?) Data Predictive Decision Action (What will happen?) Decision Support Prescriptive( What should I do? ) Decision Automation Source: Gartner, 2014. The simplest form of LA is descriptive. With descriptive LA, we use historical data to describe what happened. Descriptive LA indicators can be calculated using simple arithmetic procedures. Typical indicators are the number of stu- dents, the number of successful students, the grade point average (GPA) and the number of hours spent per course. The biggest drawback of descriptive LA is that it is discussed out of context (without comparisons). Traditional evalu- ations often stop at this level. Course Signals Alert System (Purdue University, USA) Purdue University is cited in the literature as one of the first successful examples of the use of learning analytics (Clow, 2013). Using as much relevant data as possible, the university wanted to identify students at risk of failing a course in a timely manner. The model is based on the use of four sets of individual data on students in each course, namely: demographic characteristics, student characteris- tics based on historical data, student course activities in the LMS, current activities and achievements. They use statistical regression analysis to assess the risk of failure. The ap- plication identifies students according to their risk of failing the course into three groups (high risk, medium risk and low risk). The results are auto- matical y converted into graphical warnings with traffic light symbols. Learning Analytics 245 Figure 39: Screenshot of the Signals warning system Source: Clow, 2013. Teachers have the autonomy to decide when and how to contact a student at risk and what action to take. Evaluations show that student performance has increased significantly since the introduction of the alerts: the pass rate has risen from 67% to 87%, and the GPA have also improved. The authors also cite positive feedback from teachers and students (increased student activity, teachers rating the tool as a useful tool for the course). However, the limitation to three categories of students according to the level of risk does not allow an in-depth analysis of the reasons that led the student to the ”at risk” situation. Diagnostic LA takes the context into account for the interpretation of descrip- tive statistics. For example, LA contains information on how a phenomenon is important and how it compares to a related phenomenon. With diagnostic LA, we are trying to explain not only what happened, but also why it happened. Diagnostic LA is more data-intensive. These methods provide answers to ques- tions such as which of the (several) teachers of a course have helped learners achieve better results, which material in a course is best visited, what is the ac- tual learning path of learners compared to the planned one, what is the impact of certain measures on learners’ learning achievement, etc. 246 Learning Approaches and Methods in a Digital Society Check My Activity tool (University of Maryland, Baltimore County – UMBC) UMBC is a public university with about 12,000 students using an LMS (Blackboard). It offers both online and face-to-face courses. Due to the increasing use of LMSs, they wanted to find out how students’ LMS use is related to their exam grades. A survey of 131 courses showed that stu- dents with lower grades (D and F) use the LMS 39% less than those with higher grades (C and above). This prompted them to develop a special tool for students called Check My Activity (CMA), which gives the student immediate information about his/her behaviour during the course itself in relation to the ”class average” (formative self-evaluation). Students can also compare the average be- haviour of those who have achieved the same grade as them (summative evaluation). Research shows that students who used CMA were almost twice as likely (1.92) probability to achieve at least a grade C (Fritz, 2013). To produce a CMA, all that is needed is data from the LMS, specifical y Blackboard in this case. However, upgrading the CMA as a communica- tion tool with the teacher (tutor) requires the student’s prior consent to use the collected data for immediate teacher interventions. They use de- scriptive statistics for analysis (frequencies and averages for registrations, activities and grades), which are displayed in simple analytical tables. Learning Analytics 247 Figure 40: Screenshot of the check My activity tool showing statistics Source: Fritz, 2013. Predictive LA are used to estimate, using past (historical) data, likely events or states, i.e., what will happen. These predictions are usual y represented by hypotheses, which are validated by statistical tests of significance; confidence intervals are calculated to show the interval over which the event will occur given a certain probability level. For example, predictive LA predicts that a student has a 95% probability of achieving a grade between 65 and 72, or of passing 6 to 7 out of the required 9 exams. 248 Learning Approaches and Methods in a Digital Society Developing an analytical data hierarchy (Lewis & Clark Community Col- lege; USA) The Lewis Clark Community College is a public college of over 20,000 students with a dual program delivery (traditional and online). First, a strategy for monitoring student activity was developed and then, based on the LA, the importance or hierarchy of individual parameters was de- termined to facilitate the monitoring of students at higher risk of failure. Teachers have access to a student performance matrix, which they can use to predict students’ performance in online courses. They can then use the numerical data to identify the most vulnerable students and then take appropriate measures to maintain or improve the pass rate. They found a high correlation between GPA and student performance in online courses. The analysis was based on five years of data (demograph- ics, learning activity data, cumulative average grades, etc.). There was a high degree of correlation between average grades and performance in online courses: students with lower average grades were on average less successful in online courses. Based on these results, they have introduced a rule that, for example, only students with a GPA above 2.7 can register for online courses. They have also identified five core criteria for identifying at-risk students, with critical values in four of them already providing a reason for action (Blackboard, 2016). The prescriptive LA (guidance) are an extension of the predictive LA. Based on the results of the predictive LA, the prescriptive LA will identify the most relevant events or situations (in relation to the defined objectives) and rec- ommend appropriate actions based on their characteristics. Prescriptive LA therefore communicates what needs to be done. It is based on detailed analyses of predictive models and complement them with specific decisions that are expected to deliver positive impacts. For example, students who watch at least 10 (recommended) videos perform better. Action: the compulsory viewing of a certain number of videos. Learning Analytics 249 Designing a personalised learning pathway PASS (Open Universities Aus- tralia) Open Universities Australia (OUA) https://www.open.edu.au/how-oua-works is a consortium of several Australian universities that offer distance edu- cation, either as an online study or as a blended study. The OUA has developed the Personalised Adaptive Study Success (PASS) tool, which allows students to choose a study path for a particular course that suits their needs and abilities, and is supported by appropriate peda- gogical support (Ifenthaler, 2014). Figure (41) shows a flexible and personalised learning pathway that dy- namical y adapts to the needs and abilities of the individual. This allows students to reach their destination who would otherwise stop at certain points in the course and therefore not be able to complete it successful y. Such personalised and adaptive learning models can be created with the PASS tool. Figure 41: Personalised and adaptive learning STUDENT MODULE MODULE MODULE MODULE MODULE PASS EXAM MODULE MODULE MODULE Move MODULE MODULE forward MODULE MODULE MODULE LEARNER MODULE OUA HELP CONTENT MODULE OUTCOMES Preparatory MODULE MODULE ACHIEVED Units MODULE DEFINED MODULE MODULE MODULE Move forward MODULE MODULE MODULE MODULE MODULE MODULE STUDENT PASS EXAM Week Week 1 12 Start Study Demonstrated Failed to End Study period competencies demonstrate period competencies Source: Ifenthaler, 2014. The PASS application uses a wide variety of data sources (student da- tabases from all partner universities, LMS, information on study pro- grammes, etc.). The most important data is as follows: • student data (location, socio-demographic characteristics, previous knowledge, etc.); 250 Learning Approaches and Methods in a Digital Society • data on the programme implementation (online content, grades, fo- rums); • information on the programme’s curriculum (entry requirements, as- sessment). PASS uses LA to process the quantitative and qualitative data, produce a report and then generate recommendations on personalising (adapt- ing) the course for the individual student. By combining qualitative and quantitative data, it is possible to build up a holistic picture of the student (qualitative data on the student’s activities in the forum, open questions, etc. are also processed). 6.4.5 the State of learning analytics in Practice In 2014, SoLAR published a report (Dawson et al., 2014) on the introduc- tion of LA in higher education organisations in Australia, the US and the UK, which focused mainly on the technical-organisational and strategic aspects of introduction. On this basis, a LA model has been developed that classifies the effects of the introduction of LA in terms of organisation. The complexity of the model increases from an initial level of awareness and experimentation to more holistic approaches, which are then built upon and manifested in organi- sational transformation and, ultimately, sector transformation. Organisational and IT support must also adapt to more complex and sophisticated LA models. However, it is evident that practice has not kept pace with the intensive research and technological opportunities for LA. The research done on the state of the art in this field (Dawson et al., 2018; Colvin et al., 2015; Fergusson et al., 2016; Tsai and Gašević, 2017) and also study of LA in Slovene higher education (Bregar and Puhek, 2018) show that practice is still dominated by models that mostly use traditional statistical methods and are at the level of awareness raising and ex- perimentation. For example, a study by Educause (Arroway et al., 2016) on LA uptake in the US points out that higher education institutions are at a stage in their uptake where LA is only of interest but largely not considered a strategic priority. This is also reflected in the relatively modest inputs and investments. However, the JRC study (Ferguson et al., 2016) cites a number of encouraging examples of LA use, not only in the US and Australia as leading countries in this field, but also in Europe (UK, Denmark, Netherlands and Norway). Slovenia is not one of the countries that would recognise LA as a way of mod- ernising and improving the quality of learning and teaching. A survey on the state of e-learning (Bregar and Puhek, 2017) showed that the surveyed higher education institutions that stated that they implement e-learning do not use Learning Analytics 251 LA in practice (to a very limited extent at the level of individual courses in only two out of twenty-three faculties or higher education institutions). The Doba Faculty started to systematical y develop and implement LA in 2018 through awareness raising and experimentation activities. At this stage of development, the focus is on making the most efficient use of the data and tools available in the Blackboard learning environment, current- ly used at the Doba Faculty, for drop-out prevention, better pedagogical support, higher quality of online study programmes and a better assess- ment system. Particular attention is paid to the ethical and legal aspects of the use of LA (Bregar and Puhek, 2018). recommended links Learning Analytics 2018 – An Updated Perspective: https://www.iadlearning.com/learning-analytics-2018/ Handbook of Learning Analytics, Second Edition: https://solaresearch.org/wp-content/uploads/hla22/hla22.pdf SoLAR: https://solaresearch.org/ Open University Institute of Educational Technology: https://iet.open.ac.uk/themes/learning-analytics-and-learning-design NYU Teaching and Learning Services, Learning Analytics: https://wp.nyu.edu/tlt/learning-analytics/ 252 Learning Approaches and Methods in a Digital Society 6.5 intelligent tutoring Systems 6.5.1 What are itSs Intelligent Tutoring Systems (ITSs) are computer-based tools that allow per- sonalised teaching and learning and the provision of feedback to learners, mostly without the need for teacher intervention (Kulik and Fletcher, 2016). The common purpose or goal of all ITS is to enable meaningful and effective learning using a variety of computer technology and software. There are many examples of ITS being used in formal education and in workplace learning and training, which have shown their advantages but also their limitations. The use of ITS is closely related to findings in the field of learning, in particular cognitive learning theory (Section 3.2) and instructional design models (Sec- tion 3.1), as well as approaches to learning and teaching. ITS seeks to replicate the benefits of personalised tutoring in a learning environment where learners would otherwise be part of traditional teaching (lectures or live explanations) or self-directed learning in e-learning. The basic premise of ITS is to ensure that every participant has access to the highest quality education possible (Van Lehn, 2011). However, it is important to acknowledge the point made by Nye (2015) that the uptake of these sys- tems depends on various factors, such as the level of technological and infra- structural sophistication, and that there are currently significant differences between developed and underdeveloped countries. ITSs are software that simulate tutor behaviour 38 and autonomously assist the learner. ITSs provide personalised support and help with learning by asking learners questions, correcting errors in their answers and offering personalised guidance and feedback. ITSs are made up of four components (Nkambou, Mizoguchi and Bourdeau, 2010): • the domain component, which includes all the knowledge, concepts, rules and strategies in a domain that a person needs to learn; • the learner component is about knowing the learner, their knowledge and emotional states as well as possible; • the teacher component, as a person with knowledge of pedagogy and di- dactics; • the communication component refers to the communication environment (user interface) in which the learner interacts with the system. 38 The role and importance of the tutor in e-learning is discussed in detail in Part 5: Pedagogical Support. Intelligent Tut elligent T oring Systems 253 The main advantage of ITS over other computer assisted instruction (com- puter assisted instruction – CAL; computer assisted learning – CAL) is that with ITS, the teacher does not have to define the correct solutions beforehand, which are compared with the learners’ answers. The system generates them itself based on data from the domain module (Conati, 2009). An ITS is there- fore characterised by its ability to learn and adapt to the learning process and the learner. The feedback given by ITS is therefore real-time and personalised. Unlike ”statistical” systems (i.e., CAI), these answers are not programmed or generic, but are offered to learners according to their level of demonstrated knowledge or ability. Programmed teaching In the 1950s and 1960s, behavioural psychologist B.F. Skinner of Harvard Uni- versity also addressed the problem of automated teaching machines. Skinner built a learning machine that rewarded learners for answering questions cor- rectly. To watch a clip of Skinner presenting his visionary learning machine, visit http://www.youtube.com/watch?v=jth3ob1irFo. In the 1970s and 1980s, newer computer-based tutoring systems based on concepts from AI and cognitive theory began to appear in schools. The first generation of computer tutors was called computer-assisted instruction tutors (CAI tutors), (Van Lehn, 2011). 6.5.2 Virtual assistants In parallel with the development of ITS, the importance of virtual assistants in education is also evolving, and the 2019 EDUCAUSE report identified, them as one of the most important trends in education technology (Alexander et al., 2019). Advances in machine learning technology, which have greatly increased the accuracy of automatic speech recognition and related natural language pro- cessing, have spurred the development of virtual assistants such as Siri, Alexa or Google Assistant. Virtual assistants are available on most smartphones, tab- lets and PCs. Virtual assistants are most commonly used by universities to provide informa- tion and support services on campuses. 254 Learning Approaches and Methods in a Digital Society The North-Eastern University in the USA has developed the Husky Helper virtual assistant based on Alexa. It can answer the 20 most com- mon questions that students have asked the call centre over the past three years. Husky Helper will use AI and machine learning to identify stu- dents’ learning and other needs. Saint Louis University has gone one step further and developed the AskSLU app using Alexa, which provides an- swers to more than 130 questions related to studying at the university (https://huskyhelper.northeastern.edu/). It should be noted that the rapid progress in the development of virtual as- sistants is particularly visible in English-speaking areas. The functionality of these assistants in other languages is relatively limited. 6.5.3 using itS in education E-learning, with the introduction of new technology-based teaching para- digms, is an excellent starting point for ITS ideas and possibilities. In addition, other assistive technologies such as multimedia systems, simulations and LA are also being integrated with ITSs. ITSs offer many advantages for learners, such as personalised learning with instant feedback and flexibility in time and place. ITSs have evolved from research in cognitive psychology and AI, and are now commonly found in formal education and e-learning. ITSs are available in digital learning environments and are used at all levels of education. There are several courses focusing on teaching maths, but they are also used in health, language education and other areas of organised learning. Reports of improve- ments in learners’ understanding, engagement, attitudes, motivation and learning achievement have contributed to the further research and develop- ment of these systems. In the field of education, there are a number of ITSs. The full list of ITSs is not available. Some of the most commonly used courses are listed below. Intelligent Tut elligent T oring Systems 255 AutoTutor AutoTutor is the system that is being developed most actively. It commu- nicates with users in natural language and is used in many areas (com- puter literacy, physics, critical thinking development, etc.). The main research areas for the development of AutoTutor are tutoring strategies and pedagogical factors, technology to support tutoring, and communica- tion that is as human-like as possible. Figure 42: example of an autotutor user window Source. Auto Tutor (http://ace.AutoTutor.org/IISAutoTutor). The AutoTutor algorithm incorporates human teaching protocols, as well as the most effective strategies, based on insights from some of the most fundamental research on learning processes. 256 Learning Approaches and Methods in a Digital Society MATHiaU Carnegie Learning’s MATHiaU software enables maths learning and is based on personalised learning, teaching and instant feedback to stu- dents. The following picture shows a virtual assistant explaining the correctness or incorrectness of an answer given in a maths test. Figure 43: Mathiau Source: Carnegie Learning (https://www.carnegielearning.com/services/ ). Intelligent Tut elligent T oring Systems 257 PAL3 – Personal Assistant for Lifelong Learning (PAL3) The University of Southern California’s Institute for Creative Technolo- gies is pioneering the creation of smart virtual environments and applica- tions based on AI, 3D gaming and computer animation to develop virtual characters that enable real social interactions. The Institute is developing e-learning technologies that can be used in online classrooms and will include virtual teachers and tutors. A personal assistant (PAL3) is currently being developed to work in lifelong learning courses. Figure 44: Personal assistant prototype for lifelong learning Source: USC Institute for Creative Technologies (http://ict.usc.edu). The picture above shows a typical situation in which a participant is faced with a problem or in a situation in which a virtual helper is helping them. 258 Learning Approaches and Methods in a Digital Society 6.5.4 Benefits and limitations of itSs 6.6 Mobile learning Determining the effectiveness of ITSs is extremely difficult because of their diversity, the context in which they are used and the objectives they pursue. 6.6.1 What is M-learning Nevertheless, many studies have attempted to measure the overall effectiveness of ITSs, often by comparing them with human tutors. A review of the effective- The development of e-learning over the last decade has been marked by mo- ness of modern ITSs by Kurt Van Lehn (2011) found no statistical y signifi- bile learning, or m-learning. The rise of mobile telephony, and especial y the cant differences in effectiveness between experienced (human) tutors and ITS. rapid spread of smartphones, has given education new opportunities to choose However, a recent meta-analysis has shown that ITSs can even outperform when, where and how to learn. Increasingly powerful devices, tools and appli- CAIs and human tutors (Kulik and Fletcher, 2016). The study showed that cations are making m-learning more than just another form of first-generation students who received intel igent tutoring outperformed students in traditional e-learning, characterised by access to learning content and communication classes in 46 (or 92%) out of 50 control ed evaluations. However, the improve- through mobile devices. Functionalities such as powerful video cameras, voice ment in learning achievement was large enough to be considered significant in 39 recorders, GPS systems, interfacing with other devices (e.g., Internet of Things (or 78%) of the 50 studies (Kulik and Fletcher, 2016, p. 26). (IOT) and Bring Your Own Device (BYOD)), open up new dimensions for tech- ITS are expensive to develop and to implement in the education process. nology-enhanced education, whether it is delivered in the classroom, blended Through various research and projects, ITSs are evolving and making their or online, or for non-formal education, such as on-the-job training. The break- way into more general use. However, development research is usual y expen- through of m-learning is not only the result of the extremely rapid techno- sive, as it requires the involvement of many experts. logical development of mobile devices, but also of their high level of broad dissemination. According to Eurostat (Eurostat, 2019), in 2018, three-quarters From a pedagogical point of view, the criticism of the current ITS is mainly of the adult population in the European Union (EU-28) had a smartphone, directed at teaching methods based mainly on the automation of instant feed- compared to only slightly less in Slovenia (71%). Smartphones have become back and recommendations that are supposed to make such a system ”more an inseparable tool in our daily work and personal lives, and a natural step is intelligent”. These approaches have also been criticised for not proving success- to enter the field of education. ful in developing more in-depth learning strategies (Gowda, Baker, Corbett and Rossi, 2013). The benefits of m-learning, especial y in relation to microlearning (Section 6.7), were first recognised in the business world, as learning shorter, focused content ITSs are a promising technology, but in some domains, they are less capable than tailored to the learner’s circumstances (context) improves knowledge retention human tutors (D’Mello and Graesser, 2012; Madaio, Ogan and Cassel , 2016). and increases employee activity and initiative (Simon, 1974; Cowan, 2001). For example, while human tutors can easily perceive and interpret an individ- The rise of m-learning is expected to continue. Docebo (2018) states that the ual’s emotional state and can potential y adapt the course accordingly, this is mobile learning market will reach nearly $38 billion in 2020, with an impres- still too complex a task for a ”virtual” tutor. The development of ITS is therefore sive 36% annual growth rate. North America dominates the m-learning mar- primarily aimed at removing these constraints and making ITS more efficient. ket, followed by Europe and Asia Pacific. recommended links Educause’s 2019 report on technology trends relevant to higher education (Alexander et al., 2019) highlights m-learning alongside LA tools as a tech- Science Direct: Intelligent Tutoring System: nology option that is suitable for immediate deployment in education (in one https://www.sciencedirect.com/topics/psychology/intelligent-tutoring-system year or less). EI Design (Pandey, 2018a) states that the further development 100 Best Intelligent Tutoring System Videos: of m-learning will be characterised by: wider use of BYOD, transformation https://meta-guide.com/videography/100-best-intelligent-tutoring-system-videos from adapting technical solutions to mobile devices to mobile-first design, Examples of Artificial Intelligence in Education: increased use of m-learning in non-formal education, the personalisation of https://emerj.com/ai-sector-overviews/examples-of-artificial-intelligence-in- learning through the design of individual learning pathways and the develop- education/ ment of content tailored to the needs of individuals, engaging learning, in- creased use of video and interactive video in the learning process, and the wider use of gamification. Mobile Learning 259 6.6 Mobile learning 6.6.1 What is M-learning The development of e-learning over the last decade has been marked by mo- bile learning, or m-learning. The rise of mobile telephony, and especial y the rapid spread of smartphones, has given education new opportunities to choose when, where and how to learn. Increasingly powerful devices, tools and appli- cations are making m-learning more than just another form of first-generation e-learning, characterised by access to learning content and communication through mobile devices. Functionalities such as powerful video cameras, voice recorders, GPS systems, interfacing with other devices (e.g., Internet of Things (IOT) and Bring Your Own Device (BYOD)), open up new dimensions for tech- nology-enhanced education, whether it is delivered in the classroom, blended or online, or for non-formal education, such as on-the-job training. The break- through of m-learning is not only the result of the extremely rapid techno- logical development of mobile devices, but also of their high level of broad dissemination. According to Eurostat (Eurostat, 2019), in 2018, three-quarters of the adult population in the European Union (EU-28) had a smartphone, compared to only slightly less in Slovenia (71%). Smartphones have become an inseparable tool in our daily work and personal lives, and a natural step is to enter the field of education. The benefits of m-learning, especial y in relation to microlearning (Section 6.7), were first recognised in the business world, as learning shorter, focused content tailored to the learner’s circumstances (context) improves knowledge retention and increases employee activity and initiative (Simon, 1974; Cowan, 2001). The rise of m-learning is expected to continue. Docebo (2018) states that the mobile learning market will reach nearly $38 billion in 2020, with an impres- sive 36% annual growth rate. North America dominates the m-learning mar- ket, followed by Europe and Asia Pacific. Educause’s 2019 report on technology trends relevant to higher education (Alexander et al., 2019) highlights m-learning alongside LA tools as a tech- nology option that is suitable for immediate deployment in education (in one year or less). EI Design (Pandey, 2018a) states that the further development of m-learning will be characterised by: wider use of BYOD, transformation from adapting technical solutions to mobile devices to mobile-first design, increased use of m-learning in non-formal education, the personalisation of learning through the design of individual learning pathways and the develop- ment of content tailored to the needs of individuals, engaging learning, in- creased use of video and interactive video in the learning process, and the wider use of gamification. 260 Learning Approaches and Methods in a Digital Society When we talk about m-learning, we most often think of learning using mobile devices that allow location-independent learning. The literature is also domi- nated by definitions that consider m-learning as a learning process that takes place anytime or anywhere using handheld or mobile technology, such as per- sonal digital assistants (PDAs), smartphones or wireless laptops. This technocentric view of m-learning does not allow the identification of the es- sential features of m-learning that open up new specific advantages for education. Traxler (2007) also points this out. The definition of m-learning must be based on the main characteristics that distinguish it from other forms of learning and teaching. The specificity of m-learning as a particular form of e-learning is its high adaptability to the context of the learning process (Sharples et al., 2005). For example, learners learn about an artist through a discussion based on the shar- ing of pictures they have taken during a visit to a gallery, or by sharing resources about that artist that they have found online, depending on the context. Context is therefore a central category of m-learning. The context of learn- ing changes in interaction with other people, the environment and tools. Horton (2012) highlights the reality of m-learning as an essential element of m-learning. In m-learning, we learn from the world that real y surrounds us and is constantly changing (objects, the environment, peers, the people we interact with). Unlike m-learning, traditional learning is based on the il usion of stability of the learning context, which comes from a fixed location, prede- fined fixed sources of learning, a teacher and a fixed curriculum. M-learning assumes that the learner dynamical y designs the learning content according to their needs (Kukulska-Humle, 2013). M-learning is therefore not a static acquisition of knowledge; the learner’s knowledge is constantly changing and upgrading according to the changing circumstances in which the learning process takes place and according to dynamic interactions with teachers and peers, learning tools and learning resources (Sharples et al., 2005). So, what are the essential features that enable the contextualisation of the learn- ing process in m-learning? Stanton and Ophoff note that the characteristics of m-learning most frequently cited in the literature are: nomadicy, ubiquity, per- sonalisation, social interactivity and context sensitivity (Stanton and Ophoff, 2013, p. 504). Nomadicy is the main characteristic of m-learning. Mobile technological de- vices allow users to receive and send messages, search the web, create and share content, etc., regardless of their location, which means that the learning process can, in principle, be spatial y independent. Even if Internet connec- tions are interrupted or unavailable, mobile devices provide an alternative way to access information (for example, by downloading mobile data instead of using the Internet). Spatial independence in the concept of nomadicy is a nor- Mobile Learning 261 mal state of affairs (Patokarpi in Stanton and Ophoff, 2013), not an exceptional situation requiring special conditions, as is the case for e-learning 1.0 and 2.0. The ubiquity of mobile devices means not only the option to learn anywhere, but also at any time, under any circumstances, easily and without complica- tions (Patokorpi in Stanton and Ophoff, 2013). Ubiquity also brings spontane- ity in learning. You can learn in any situation (e.g., on the train to work, during breaks between work and while waiting for customers, in a waiting room, dur- ing traffic jams on motorways, using any mobile device). Mobile devices allow for different aspects of the personalisation of learning: depending on the learning content, the design aspects and the media used. The ability to adapt to the individual’s circumstances is very important. LA enable important advances in personalisation, especial y at the level of prescriptive analytics (Section 6.4). Mobile technology offers excellent opportunities for collaboration and social activity. Learners can col aborate with peers around the world, with tutors and teachers, without restrictions or costs, in different ways and with different types of interaction. Col aborative learning helps to make learners more active. Mobile device ownership is also an important feature of m-learning (Naismith et al., 2004). Using your own devices is more comfortable and easier, and gives you a sense of control. The learner can test the performance and usability of their mobile learning device at wil . Of course, having your own devices does not give the learning organisation control over the technology, tools and data available to learners in the learning process, which can be problematic for for- mal assessments, for example. 6.6.2 the Benefits of M-learning Modern m-learning goes beyond the use of learning material and mobile- ready applications. In addition to accessing educational material via mobile phone, mobile technology can be used for the context-dependent use of re- sources, field-based learning, the preparation of material with mobile devices (images, audio recordings, etc.), communication and col aboration with teach- ers, students and other groups of learners, the organisation of the learning process, etc. M-learning has a number of advantages for the learner. M-learning can reduce barriers and difficulties to learning, as it can be rela- tively easily adapted to the individual’s life circumstances. M-learning enables instant communication and data sharing. The contextualisation of learning enables the documentation of learning based on location identification. The 262 Learning Approaches and Methods in a Digital Society learning content is delivered in smaller chunks, which is particularly important for on-the-job training. Abstract concepts can be il ustrated with appropriate content. Peer networks and connections further emphasise learner-centred- ness. M-learning promotes active learning and makes education more ac- cessible for learners with disabilities. And after al , mobile devices, especial y smartphones, are a common tool for the digital generation. M-learning can also benefit the educational organisation. The personalisation of learning and more col aborative learning increase learners’ satisfaction and reduce drop-out rates, contribute to increasing digital literacy and thus their employability, which in turn increases the reputation and improves the stand- ing of the educational organisation. M-learning also has beneficial business implications, as the cost of the basic equipment and associated costs are now passed on to the learners. Its characteristics make m-learning in formal education useful as a comple- mentary form of education to traditional and online education. In particu- lar, its strengths should be harnessed when it is necessary to provide learning opportunities outside the usual locations and when authentic or situational learning is at the forefront of learning strategies. Center šolskih in obšolskih dejavnosti (The Cen- tre for School and Extracurricular Activities – CSOE), Slovenia, has developed a mobile ap- plication entitled CSOE Mission. It provides an innovative way to enable self-directed experien- tial learning on learning trails around the CSOE centres. The app is based on the concept of gami- fication, which adds game elements to the learn- ing process and further motivates users to achieve learning outcomes. It has achieved excellent re- sults for learners attending the CSOE. The train- ing of primary and secondary school teachers has extended the use to general population. The entry system is simple and even learners at school can create ”missions” (learning adventures), (Center šolskih in obšolskih dejavnosti, 2019). More than 300 learning adventures are already available, covering interesting locations across Slovenia (http://misija.csod.si/en). M-learning is much better suited to non-formal education, in particular for providing just-in-time and just-in-place training. These types of training are Mobile Learning 263 usual y limited to short, focused content pieces, which are delivered through microlearning (Section 6.7). Microlearning and m-learning complement each other perfectly, and by combining the two, their individual effects are multiplied. Microlearning, together with m-learning, brings additional benefits. It provides a better learning experience, learning can be continuous, without unwanted in- terruptions, it can be personalised and enhanced with modern learning approaches, it reduces drop-out and increases participation, and the rate of application of the acquired knowledge is increased (Pandey, 2018b). M-learning can also bring wider societal benefits. The GLOBE (Global Learning and Observation to Benefit the Environ- ment) project is dedicated to research on Zika virus infection and protec- tion. Data on the mosquitoes that carry the virus is collected by school children in the countries at risk using a special app on mobile phones. This gives young people a deeper understanding of the environment and the threat of this dangerous disease, thus contributing to raising aware- ness, as well as direct experience of data collection for research purposes, developing digital and col aborative competencies, etc. Zika Globe: Edu- cation and Prevention (https://www.globe.gov/web/globe-mosquito-project). JISC Infonet (2011) points out that, a strategy of modernising education with m-learning can be more effective than using desktop computers and LMS. Smartphones and other mobile devices are already known and used by teach- ing staff, so they are more willing to use them in the learning process than other forms of e-learning, where the first barrier is fear, resistance to unfamil- iar technology. Accordingly, they see m-learning as a backdoor to the wider institutional deployment of modern technology in education. 6.6.3 limitations of M-learning M-learning should not be seen as a magic formula to overcome the difficulties of modernising education, nor should its potential be idealised. The main limitations stem from the technical characteristics of mobile devices. These are the small screen, the dependence on or sensitivity to connection speed, and the battery capacity. The m-learning can be partly adapted to these constraints. For example, the problem of a small screen due to poor clarity can be solved either by using a tablet or by appropriate design approaches (font size, text layout, contrasting colours, etc.). 264 Learning Approaches and Methods in a Digital Society The most important limitation, that mobile devices are better suited for shorter texts and simple graphical displays, must of course be taken into account. Some even see it as having the advantage of forcing authors to be more thoughtful and rational in their presentation of information. This is of course welcome for the learner, given the limited capacity to remember information.39 However, m-learning does not provide an ideal learning environment, as it can in principle be interrupted at any time by a phone cal , an alert about a new email or Facebook post, or a short hop to a popular portal. M-learning blurs the boundaries between different activities (learning, work, leisure and family) and forces people to spend even more time in the digital environment. 6.6.4 Planning M-learning We can develop new courses for m-learning format or adapt existing ones. m- learning is particularly suited to just-in-time or just-in-place learning and train- ing, or to the preparation of small parts of larger training programmes. In this context, all the essential pedagogical issues need to be taken into account: what is the underlying purpose and learning objectives, at which part of the learning pathway will the learner encounter each unit of m-learning and under what cir- cumstances, which knowledge and competencies are particularly important for him/her? The design of m-learning can be based on different learning theories, depending on the pedagogical framework of the m-learning design. Naismith et al. (2004) mention, theory of behaviourism, theory of construc- tivism, situation theory or authentic learning theory, col aborative learning, non-formal and lifelong learning, and the provision of pedagogical support as theoretical background for developing M-Learning. M-learning is particularly suitable when the learning process is to be con- ducted as authentic learning (JISC Infonet, 2011; Herrington et al., 2009). A study ”New technologies, New pedagogies; Mobile Learning in Higher Education” prepared by Australian experts from the University of Wollon- gong, shows a number of interesting examples of the use of mobile learning in different fields (teacher training, kindergartens, environmental educa- tion, IT training, language and literacy training, critical thinking skil s de- velopment, sports education, art and design, etc.), (Herrington et al., 2009). As with other forms of e-learning, when planning the development of spe- cific educational programmes for m-learning, we need to take into account 39 For example, the Ebbingaus curve shows that 70% to 80% of the information learned is forgotten the next day if it is not actively reviewed or revisited (https://uwaterloo.ca/campus-wellness/curve-forgetting). Mobile Learning 265 not only the pedagogical aspects, but also other factors such as business and organisational contexts and financial, technical possibilities, etc. The essential characteristic of m-learning, i.e., contextuality, dictates that some of these factors are examined in particular detail and that this is taken into account when planning m-learning. Stanton and Ophoff (2013) highlight the status of the learner (in terms of personal characteristics, preferences and ex- pectations), the circumstances faced by the ’nomadic’ learner, and the learning environment, which encompasses both physical and digital components. Figure 45: characteristics and contextual aspects of m-learning Personalisation Context Ubiquilty Sensivity Social Nomadicy Interactivity texton Digital Sphere: Characteristics Of M-Learning text tional C ont C Situa As-Lived Experience onmenvir ning En Lear Learner Learner Personal Status Source: Stanton and Ophoff, 2013, p. 504. The context expressed in these three dimensions, is worth to consider, when the starting point for m-learning design is the way in which individuals engage with technology in a natural environment (as lived experience). This means that we are constantly looking at how learners actual y use technology, their attitudes towards it and how they manage it. Finnish researchers analysing mobile phone usage data in Finland and the UK for the 2006–2008 period found that users are most active when they are mobile, but that the use of voice messages and SMS is not af- fected by mobility. Micro-patterns of mobile device use vary between in- dividuals (Verkasalo, 2009). 266 Learning Approaches and Methods in a Digital Society Figure 46: average browsing session duration in different contexts WEB Session Duration 7.0 6.02 6.0 es] 5.15 5.17 4.96 inut 5.0 4.06 4.43 4.02 4.19 4.13 4.0 3.46 ation [Mur 3.0 2.0 ession D 1.0 WEB S 0.0 Mean WEB session duration Median WEB session duration Home Office On the move Home (weekend) On the move (weekend) Source: Verkasalo, 2009, p. 341. The high level of use of Internet services on mobile devices at a time when the population is moving is confirmed by Eurostat data. In 2018, 69% of the EU population and 67% of the Slovenian population on the move used mobile de- vices to access Internet services. The proportion is even higher for young people aged 16–24: 91% in the European Union and 97% in Slovenia (Eurostat, 2019). There is a wealth of practical advice and recommendations of providers of tools and systems for m-learning, professional associations, professional or- ganisations and individual experts on how to go about developing m-learning. The most common recommendations are (Shift. Disruptive Elearning, 2018; Horton, 2012): • M-learning takes place in specific contexts, so only what is necessary to achieve the learning objectives should be directly included in m-learning. Additional information can be provided in an attachment with a relevance flag (for example, as recommended reading). • In m-learning, participants expect an immediate response and are also rather impatient with the transfer of content. If it is not strictly necessary, we avoid graphical representations and try to present the content as con- cisely as possible. Content is presented in a structured way (for example, with simple diagrams), increasing clarity and reducing file sizes. • M-learning needs to be adapted to the nature of the mobile device and the way it is used. Users do not need to hold a mobile phone all the time or for long periods of time, as they can sit at a computer for hours at a time. They usual y use the phone during individual activities, often in adverse Mobile Learning 267 circumstances, distractions, noise, etc. The learning units in m-learning are relatively small chunks of content and are often based on the principles of microlearning. Ideal y, these units should last somewhere between 5 and 10 minutes, but in any case, no longer than 15 minutes. • Good navigation and a good website structure are important for m-learn- ing. Users expect the most important information as soon as they enter the course, followed by clear instructions and a logical y designed learning path. • Before putting a ready-made m-learning unit into use, it is essential to test and pilot it. • Most users use several electronic devices a day. The design of m-learning should automatical y adapt to the screen size so that the user sees the con- tent equal y well regardless of the device used. • Simplicity is key. Too many options and elements can confuse the user. Every button, image and paragraph make the screen less clear and more difficult to use. It should also be borne in mind that users perform most operations with one or at most two fingers. It is important to remember that complex images are not suitable for smaller screens, as details are lost. Such information is better sent as an attachment. • Mobile devices work by touching the screen, which is very different from clicking and using a mouse. It is therefore advisable to design simple menus and to maximise buttons and touch-sensitive areas. 6.6.5 tools for M-learning Today, there are many tools available to help us prepare m-learning. A search of the freely available Capterra digital tools database for the term ”mobile learning software” yields 273 hits, 60 of which are free trials (https:// www.capterra.com/mobile-learning-software/). The choice of tool should also take into account the learning objectives and the content of the m-learning (Section 4.5). For example, if we are develop- ing m-learning for ”just-in-time” courses that include some activities, we need to choose a tool that is compatible with video graphics and interaction. Vid- eos should be short, interactive elements compact and animations simple and short. In addition, it is important to bear in mind under what conditions users will use smartphones for m-learning (e.g., while commuting, with a poor In- ternet connection), (Habeeb Omer, 2018). Suitable tools for m-learning recommended by the eLearning Industry as Top Mobile Learning Platforms List for 2022 are Talent LMS, Adobe Learn- ing Manager, Docebo, 360Learning, Absorb LMS, iSpring Learn, Stream LXP From Learning Pool, Fuse, Rockstar Learning Platform, iTacit Front-Line Em- 268 Learning Approaches and Methods in a Digital Society ployee Platform + LMS (Pappas, 2020.). Most LMSs are also already mobile- 6.7 Microlearning friendly. LMSs provide access to specific free mobile apps, such as Blackboard Mobile Learn, Canvas Student or Moodle App. recommended links 6.7.1 characteristics of Microlearning UNESCO: Best Practice in Mobile Learning: Learning in smaller content units has been known in educational theory and https://en.unesco.org/themes/ict-education/mobile-learning/fazheng/case-studies practice for a long time, and the concept of microlearning started to take hold Capterra: around 2007 (Alqurashi, 2017). Its popularity seems to be growing (Hogle, 2018). https://www.capterra.com/ Experts do not have a single view of microlearning and therefore define it dif- Mobile Learning Portal.Org: ferently. Here are some definitions: https://www.mobilelearningportal.org/ • microlearning is a learning strategy that delivers information to learners in small chunks (Alqurashi, 2017; CommLab India, 2017); • microlearning refers to short learning activities using microcontent (Hug, 2010 in Buchem and Hamelmann, 2010); • microlearning offers the opportunity to learn using short focused chunks (Learning Seat, 2017). Micro-content is an integral part of microlearning. It refers to information, its length is determined by a narrowly defined subject matter, covering only one idea or concept, and is accessible on a website (Giurgiu, 2017, p. 19). Micro- learning can also include short activities such as short quizzes and various interaction-based activities. Duolingo is a popular tool for learning foreign languages and includes a foreign language placement test. Testing your knowledge of a foreign lan- guage (the language you want to learn, for example) involves translating short written sentences from English into your chosen foreign language, and then translating short spoken sentences from your chosen foreign lan- guage into English, etc. For more information, see https://www.duolingo.com/. Microlearning is characterised by its reliance on small meaningful entities. This approach can be used to learn new skil s, to refresh and update knowledge, or to quickly solve a problem encountered at workplace. It can be used in formal, non-formal and informal learning, standing alone or being a part of an educa- tional programme. The Khan Academy (https://www.khanacademy.org/) offers online courses for the free learning of content topics in various fields such as mathematics, science, economics, computer programming, etc. The main elements of the courses are videos, which are prepared according to the principles of mi- crolearning. Microlear Micr ning 269 6.7 Microlearning 6.7.1 characteristics of Microlearning Learning in smaller content units has been known in educational theory and practice for a long time, and the concept of microlearning started to take hold around 2007 (Alqurashi, 2017). Its popularity seems to be growing (Hogle, 2018). Experts do not have a single view of microlearning and therefore define it dif- ferently. Here are some definitions: • microlearning is a learning strategy that delivers information to learners in small chunks (Alqurashi, 2017; CommLab India, 2017); • microlearning refers to short learning activities using microcontent (Hug, 2010 in Buchem and Hamelmann, 2010); • microlearning offers the opportunity to learn using short focused chunks (Learning Seat, 2017). Micro-content is an integral part of microlearning. It refers to information, its length is determined by a narrowly defined subject matter, covering only one idea or concept, and is accessible on a website (Giurgiu, 2017, p. 19). Micro- learning can also include short activities such as short quizzes and various interaction-based activities. Duolingo is a popular tool for learning foreign languages and includes a foreign language placement test. Testing your knowledge of a foreign lan- guage (the language you want to learn, for example) involves translating short written sentences from English into your chosen foreign language, and then translating short spoken sentences from your chosen foreign lan- guage into English, etc. For more information, see https://www.duolingo.com/. Microlearning is characterised by its reliance on small meaningful entities. This approach can be used to learn new skil s, to refresh and update knowledge, or to quickly solve a problem encountered at workplace. It can be used in formal, non-formal and informal learning, standing alone or being a part of an educa- tional programme. The Khan Academy (https://www.khanacademy.org/) offers online courses for the free learning of content topics in various fields such as mathematics, science, economics, computer programming, etc. The main elements of the courses are videos, which are prepared according to the principles of mi- crolearning. 270 Learning Approaches and Methods in a Digital Society Micro-content is most often part of e-learning or blended learning using Web 2.0. This enables the active participation of learners who create, combine, use and reuse micro-content according to their needs. Web 1.0-based e-learning is also characterised by content that is divided into smaller learning units, but these are more extensive and require long- er learning times - for example, several hours. Therefore, web 1.0-based e-learning can be called macrolearning (Buchem and Hamelmann, 2010). Microlearning is prepared in such a way that it (CommLab India, 2017): • allows learners to achieve only one learning objective; • offers specific and targeted information on a very narrow topic; • is available on various mobile devices (smartphones, tablets, laptops); • it can be in the form of quizzes, videos, infographics, animations, interac- tive material in pdf format, games, etc.; • is available at different stages of the training, i.e., before, during and after the training. Microlearning is characterised by the shorter time required for learning activi- ties. Time in microlearning is defined differently by different authors. There is no standard length of time for microlearning activities – some au- thors limit it to one minute, others up to 15 minutes (Learning Seat, 2017). Microlear Micr ning 271 The microlearning survey, which was conducted with 61 global educa- tion professionals at the end of 2016, also asked which microlearning methods they had used or were still using in the 18 months prior to the survey. Their responses, shown in Figure 47, show that videos dominate (more than four-fifths), followed by a variety of multimedia interactives (three-quarters) and written material (two-thirds). Figure 47: Frequency of use of different microlearning modes Which of the following methodologies for delivering online/digital microlearning have you used? 90.0 % 83.3 % 80.0 % 75.0 % 70.0 % 66.7 % 60.0 % 54.2 % 50.0 % 50.0 % 45.8 % 40.0 % 30.0 % 20.0 % 10.0 % 0.0 % Quizzes Infographics Videos Games Multimedia Text/ or Polls interactives Reading Source: Adapted from Kasenberg (n.d.), p. 13. Research on the potential of technology-enabled microlearning is still in its infancy, focusing mainly on adult education and training in enterprises (Alqurashi, 2017). The effectiveness and applicability of the microlearning concept is supported by studies showing that learners are more likely to learn and retain learning content when it is divided into smaller chunks or learning units (Giurgiu, 2017). There have been some studies on microlearning in higher education. Micro- learning has been shown to motivate students to learn and also to increase the effectiveness and efficiency of their learning. The findings of two studies conducted by Zhamanov, Bruck, Motiwal a and Foerster (in Alqurashi, 2017) on the use of microlearning in univer- sity degree programmes showed that students were very satisfied with microlearning and that a lot or more of the learning materials was pre- pared for microlearning. 272 Learning Approaches and Methods in a Digital Society The Dresden University of Technology in Germany designed a study to examine the effectiveness of student learning in relation to the amount of e-learning content. The aim of the study was to investigate whether students respond better to questions that follow the learning of a piece of content if they process more smaller pieces of content followed by more questions, or if they process larger amounts of content with fewer ques- tions. The students were divided into three groups. The first group an- swered a question after reading each of the 16 chapters, the second group answered four questions after reading each group of four chapters, and the third group received eight questions after reading eight chapters. The best-performing group was the first group, which spent 28% less time answering questions than the third group and scored 8% better in the comprehension tests than the second group (Giurgiu, 2017). 6.7.2 Benefits and limitations of Microlearning As microlearning is a new concept in today’s context and technological pos- sibilities, it is much debated by experts, who point out its advantages and limi- tations. Microlearning has these advantages: • reduces cognitive load (the concept of microlearning is characterised by avoiding the inclusion of content that is not necessary in terms of the learn- ing objective); • encourages independent learning (by giving learners more control over their learning, as they can set their own pace); • makes information available when needed (just-in-time) (by facilitating learning when the individual needs it, the learner is more likely to be in- trinsical y motivated to learn and therefore to be more successful); • allows training to take place in intervals (different micro-content can be delivered to learners in several steps, gradual y over time, allowing for more long-term retention of knowledge), (Learning Seat, 2017). The real added value of microlearning is the ability to integrate short content with learner-generated content and social interactions, made possible by Web 2.0. (Giurgiu, 2017). One of the characteristics of microlearning is its permanent accessibility, i.e., that interested parties can find the information they need when they need it, for example in a library of microlearning units on a specific website. Microlear Micr ning 273 Experts stress the advantages of combining microlearning and mobile learn- ing, which provide personalisation, continuity, flexibility of learning (when the need for a particular skill arises) and lower drop-out rates (Pandey, 2018b). Microlearning has proved particularly suitable for professional development. Its advantages make it particularly useful for companies that need to provide efficient training or knowledge to their employees if they are to have an edge over their competitors and continue to survive in today’s global y competitive environment. Microlearning is used extensively in companies before training (training information), to highlight key aspects during training and to con- solidate or test the knowledge acquired after training. It is suitable for inform- ing employees about new developments in legislation, onboarding employees, updating employees on a particular area of expertise, etc. (CommLab India, 2017). Microlearning saves employees a lot of time, as they learn on the job and take less time off work than with traditional education or training. Micro- learning is also simpler mode of training for companies than more extensive e-learning courses, as it virtual y eliminates the need to fit it around work com- mitments. Some experts consider that microlearning is not suitable for certain broad and new areas of expertise (especial y if the learner has no experience in the subject area). A significant problem with microlearning is that some of the content is re- moved from the whole and there is a risk that the learner will not recognise the interconnectedness and interdependence of the learning content. The suitabil- ity of microlearning for learning more complex content is therefore question- able. Some therefore argue that the instructional designer for learning more extensive or in-depth content in the form of microlearning needs to bear in mind how the learner will achieve the broader objectives when designing it40. It is certainly advisable to have a recommended learning path for larger content consisting of a large number of microlearning units. 6.7.3 the Preparation of Microlearning The general principles for developing e-learning courses are also applicable for microlearning, but the specific characteristics of microlearning need to be respected. One of the most important challenges faced by microlearning de- velopers is how to present content in one item (unit) of microlearning that is concise, fol ows only one main objective, is understandable and user-friendly, and without unnecessary additional information. When designing a micro- learning unit based on an educational programme, the problem can be how 40 A good example of foreign language microlearning (with vast content) is the Duolingo app (Hogle, 2018). 274 Learning Approaches and Methods in a Digital Society to break down the learning content into microlearning units in a consistent way, so that each unit covers only one sub-objective that is an integral part of the main objective of the course. Simply dividing learning content into shorter units does not mean that the microlearning concept is applied. There are two main stages in the preparation of a microlearning project: • microlearning design, • development and implementation with appropriate technology. When preparing a microlearning session, we follow these steps: • identify the characteristics of the target group (what are the preferred learn- ing methods of the target group, what are the conditions for learning, etc.); • examine the content and processes that will be turned into smaller units – these will take the form of individual microlearning units; • focus on one main learning outcome/ objective in the design of a unit; • write down the most important messages that need to be included in the content to achieve the desired learning outcome; • decide on the appropriate media to deliver the main messages that will en- able the learning objectives to be achieved effectively; • write down and limit the time allowed for the main messages (content) and decide on the narrative, etc.; • identify the types of learning support to facilitate the achievement of learn- ing objectives that will be available to the learner (usual y after they have completed a review or unit of study – such as a quiz, email reminder, exam- ples of good practice, etc.); • we need to know the capabilities and limitations of the digital tools we are going to use (when making videos, check that the tool we choose allows interactivity, etc.); • because the time for delivering the main messages is very short, we check virtual y every word on the screen or in the narration and weed out unnec- essary or redundant ones (for example, when the video itself says enough); • test the product (i.e., the microlearning unit) from a content, technical and user experience point of view, preferably with the target group for which the product is intended. The development of microlearning usual y takes place in several iterations – de- velopment, validation, revision. This is because many content professionals are not familiar with microlearning and have yet to be trained in it (Kasenberg, n.d.). Microlear Micr ning 275 Learning objects are different from microlearning units. As with micro- learning, learning objects are also about learning content that follows a single learning objective. The most important thing for microlearning is that it is available online, but this is not necessarily the case for the learning object. Learning objects have been typical y designed to reduce the cost of education, to standardise learning content and to enable their reuse in the LMS (https://en.wikipedia.org/wiki/learning_object). For more on the learning elements, see Section 6.2. Microlearning is also offered by large MOOC providers such as Udemy and Coursera, and other online platforms such as Lynda.com (Learning Seat, 2017). In 2017–2018, the ACS developed video guides that follow the microlearn- ing concept. Self-Directed Learning Centres (SLC) portal at the web-site of ACS offers 161 video tutorials (70 on learning to use a computer, 48 on learning to use a smartphone, and 43 on learning to create and program digital y). Figure 48: Screenshot of the Slc portal Source: SLC Portal (https://portalssu.acs.si/gradiva/javna/vv/mob ). 276 Learning Approaches and Methods in a Digital Society recommended links 6.8 gamification Commonwealth of Learning. Introduction to Microlearning: https://www.colvee.org/course/technology-enabled-learning/introduction- microlearning 6.8.1 introduction ATD (Association for Talent Development). What is microlearning. Glossary: https://www.td.org/talent-development-glossary-terms/what-is-microlearning Over the last decade, there has been a growing interest among educators in the ELearning Learning: integration of digital games and education. Playing video, computer or online https://www.elearninglearning.com/micro-learning/ games is widespread around the world. According to the WePC website, there Duolingo: are 3.2 billion video game players worldwide and the market value of the gam- https://www.duolingo.com/. ing market is growing (WePC, 2022). Experts estimate that it will be worth SLC Portal: more than $268 billion in 2050. https://portalssu.acs.si/gradiva/javna/vv/ A Pew Research Centre survey (Perrin, 2018) found that 43% of adults in the US often or sometimes play video games on a computer, TV, games console or portable device. There are significant differences in this group by age and gender. Americans under 50 play more than twice as much as those over 50 (55% vs. 28%), and men play more than women. These differences are particularly large among young people: 72% of men aged 18–29 play video games, compared to 49% of women in the same age range (Perrin, 2018). Interest in the use of games in education has increased due to several factors. Recently, modern psychological and neuroscientific research in particular has increasingly pointed to the positive impact of games on children’s brain de- velopment, or the stimulation of certain brain centres. For example, Rosser and colleagues (2007) found that surgeons who played video games for three hours a week made 37% fewer errors in laparoscopic surgery simulations than surgeons who did not play video games. Jane McGonigal, a video game devel- oper who researches the impact of video games on individuals, has also found positive effects on social relationships, self-esteem and creativity (McGonigal, 2015). Due to the proven positive effects of gamification on the development of certain individual skil s and group dynamics, gamification has been used in a variety of contexts, marketing, mindfulness, health and medicine, and more. Gamification 277 6.8 gamification 6.8.1 introduction Over the last decade, there has been a growing interest among educators in the integration of digital games and education. Playing video, computer or online games is widespread around the world. According to the WePC website, there are 3.2 billion video game players worldwide and the market value of the gam- ing market is growing (WePC, 2022). Experts estimate that it will be worth more than $268 billion in 2050. A Pew Research Centre survey (Perrin, 2018) found that 43% of adults in the US often or sometimes play video games on a computer, TV, games console or portable device. There are significant differences in this group by age and gender. Americans under 50 play more than twice as much as those over 50 (55% vs. 28%), and men play more than women. These differences are particularly large among young people: 72% of men aged 18–29 play video games, compared to 49% of women in the same age range (Perrin, 2018). Interest in the use of games in education has increased due to several factors. Recently, modern psychological and neuroscientific research in particular has increasingly pointed to the positive impact of games on children’s brain de- velopment, or the stimulation of certain brain centres. For example, Rosser and colleagues (2007) found that surgeons who played video games for three hours a week made 37% fewer errors in laparoscopic surgery simulations than surgeons who did not play video games. Jane McGonigal, a video game devel- oper who researches the impact of video games on individuals, has also found positive effects on social relationships, self-esteem and creativity (McGonigal, 2015). Due to the proven positive effects of gamification on the development of certain individual skil s and group dynamics, gamification has been used in a variety of contexts, marketing, mindfulness, health and medicine, and more. 278 Learning Approaches and Methods in a Digital Society Mango Health has developed a smartphone app to motivate pa- tients to take their medicines on time. Users set the time to take their medication and the app re- minds them in time. The app also contains information about medi- cines and warns the user about drug interactions and side effects. By taking their medicines cor- rectly, users can earn points on gift cards or win prizes in prize draws. Source: https://www.mangohealth.com/ The term ’gamification’ started to be used in 2008, mainly in the digital me- dia industry, and only slowly spread until 2010 (Deterding et al., 2011; Groh, 2012). Gamification has been defined as ”the use of game design elements to stimulate user activity and engagement in non-game contexts” (Robson et al., 2015; Zichermann, 2010), as ”the use of gamification techniques to make ac- tivities more fun” (Kim, 2011), and as ”the use of game-based mechanisms, aesthetics and elements to engage individuals, motivate action, promote learn- ing and problem solving” (Kapp, 2012). 6.8.2 gamification in education The gamification of education aims to create a learning environment that mo- tivates and encourages the learner’s curiosity, activity and participation in learning (Kapp, 2012). Huang and Soman (2013) note that in the digital age, gamification is becoming a popular method to encourage certain behaviours and increase motivation and engagement. A meta-analysis by de Sousa et al. (2014) found that of the 357 studies analysed, 48 used gamification in relation to education, with the largest number – a quarter – in higher education. Huang and Soman (2013) presented a five-step process for the development of gamification in the learning environment. Gamification 279 Figure 49: the process of development of gamification Understanding the Defining Structuring Applying Target Audience Learning the Identifying Gamification and the Context Objectives Experience Resources Elements Source: Huang and Soman, 2013. As the figure above shows, the process of developing gamification in education follows the ADDIE model for educational programme design (Section 3.1). The process starts with getting to know the participants (age, background) and the learning context. This not only covers the content of the training, but also the delivery context (group size, learning environment, duration, etc.). Such a definition will help the teacher to define learning objectives and structure the placement of the game elements in the curriculum. Then it can start to identify resources – existing games or those to be developed, which can range from the complex to the very simple. The final step is to implement gamification. Horton (2012) considers gamification as the core of an e-learning course. The game can take different paths. The learner first learns the content of the course as usual – in a digital learning environment and with tutor support, and then works on the problem in the game, or learns through the game. In the second case, learning by playing goes like this: • the game starts with a short introduction, explaining what the content of the game will be and what the participant will learn in the game; • the learner can choose to learn by playing a game or by learning in a digital learning environment with the support of a tutor; • when the learner starts the game, he/she realises that he/she will have to study the content that is the subject of the game; • the learner then goes back to normal learning and returns to the game again, and can repeat this several times; • when the learner has achieved the objectives of the game, he/she quickly reviews the concepts learnt; • by successful y completing the game task, the participant is deemed to have the ability to successful y complete the task in real life; the game itself is a test, so there is no need to test the participant’s knowledge. The introduction of gamification is not only a feature of formal education, but also of entrepreneurship education. A good example of this is the Deloitte training for managers ”Deloitte’s Digital Centre for Immersive Learning”. The centre develops applications for learning and developing leadership com- petencies with the aim of high participant engagement. This includes game- based e-learning courses, training simulators, virtual and augmented reality (Monahan et al., 2016). 280 Learning Approaches and Methods in a Digital Society 6.8.3 types of gamification Kapp (2014) lists two types of gamification. The first type is structural gami- fication, the second is contextual gamification. It is important to note that the two species are not mutual y exclusive, as both can coexist. In fact, this is when their synergistic impact is at its greatest. Structural gamification Structural gamification means using game elements that stimulate the learner with content, without changing the content. The primary focus of this type of gamification is to motivate participants to explore the content and engage them in learning through rewards. The most common elements in this type of gami- fication are points, badges, achievements and levels. This type usual y includes a leaderboard, methods to track learning progress and a social component where participants can share and compare their achievements with others. The Khan Academy online platform is a non-profit organisation that has been using gamification elements (badges, levels of achievement and competencies, etc.) for its activities since the beginning. Khan Academy offers learning in different fields and levels of education. In some coun- tries, Khan Academy content is integrated into the curriculum of indi- vidual subjects. Figure 50: the World of Math Source: Khan Academy (https://www.khanacademy.org ). Gamification 281 Khan Academy gamifies many elements of the curriculum. The learner is mo- tivated by the goal they set at the start of their learning, and the web inter- face shows their progress and the challenges that they will face in the future throughout their learning. Achievement Awards Achievements are a very popular element when introducing gamification in education. This gives the participant a reward for their behaviour, for example, passing a test without making any mistakes. This is both a pleas- ant activity and a useful one, as it stimulates the user. The different levels of achievement are also shown in the Duolingo screenshot. It shows that the app encourages the learner to learn as much and as often as possible each day, to connect with others, to achieve a certain number of points, etc. (https://www.duolingo.com). It is important that the achievements are well-designed, otherwise they may mo- tivate users to do something that makes studying more fun but less effective. gamification of content The gamification of content, on the other hand, means applying elements of play to the content itself – for example, adding parts of a story to a training programme, or starting a training session by stating learning objectives rather than challenges. Adding these elements makes the content more game-like, but does not turn it into a game. 282 Learning Approaches and Methods in a Digital Society The TalentLMS online learning environment is built entirely around the concept of the gamification of learning. The entire online learning environ- ment (content, results pages, module progression, etc.) can be gamified. The figure below shows the Dashboard of the TalentLMS online learning environment. This shows the number of units of learning that a partici- pant has completed, the training programmes he/she is participating in, the time he/she has been in training and the progress made during his/ her training. Figure 51: talentlMS learning environment ”dashboard” Source: TalentLMS (https://www.talentlms.com ). Of course, an essential part of the gamification of content is that the entire learning environment is structured in the form of a game. 6.8.4 Benefits and limitations of gamification Research shows that gamification, when properly implemented, can have positive effects and benefits for education. In a review of 24 empirical studies on gamifica- tion, Hamari et al. (2014) found that e-learning research shows improved learning outcomes, increased motivation, greater engagement in learning activities and greater participant satisfaction when gamification is used (Hamari et al., 2014). Gamification 283 Kim et al. (2018) list the positive effects of using gamification in education: • increased participant engagement and motivation, • improving learning performance and attainment, • better retrieval and retention of information, • immediate feedback on learners‘ progress and activities, • influencing behaviour change, • monitoring your own progress, • promoting the development of cooperation skil s. It should be noted that gamification is not always an appropriate solution for all participants and for all educational contexts. Sometimes it can even dis- courage the participant from the substance in question and act as a demotiva- tor (Lister, 2015). 6.8.5 gamification tools There are many original tools online for integrating games into educational programmes, which can also be used for e-learning. The Digital Chalk website recommends five tools to help you ”gamify” your online content (Digital Chalk, n.d.): • Raptivity – a tool for integrating interactivity into content; • Articulate – this platform brings together different tools for the gamifica- tion of e-materials, such as Articulate Storyline, Presenter, Quizmaker and Engage; • iSpring – a PPT plugin that allows you to create online presentations, quiz- zes and interactions without leaving PPT; • Adobe Captivate – an app that lets you publish your e-material in a variety of ways, such as on mobile devices, the web, on your PC or in an LMS; • Elucidat – an e-learning platform that allows you to create HTML5 content – this ensures that it works on any device. recommended links Capterra – Gamification Software: https://www.capterra.com/gamification-software/ Quicksprout – How to Easily Add Gamification Techniques to Your Content: https://www.quicksprout.com/how-to-easily-add-gamification-techniques-to-your- content/ eLearning Industry – 6 Killer Examples of Gamification in eLearning: https://elearningindustry.com/6-killer-examples-gamification-in-elearning 284 Learning Approaches and Methods in a Digital Society 6.9 Simulations in education 6.9.1 concept and Meaning of Simulations Simulation is the process of replicating real-world processes or systems (Banks et al., 2009). Simulation as a tool in research or learning is based on the defi- nition of a model to describe the main characteristics, operation and func- tions of a chosen physical or abstract system or process. The model is used to represent the system and the simulation to show how it works over time. A well-designed and implemented simulation can bring the model closer to reality and show how the entity or phenomenon under study would behave or act in practice (Gundala and Mandeep, 2016). Simulation is thus a way to better understand the relationships between entities or phenomena, to solve problems more efficiently, as well as a method of research and teaching (Klein and Fleck, 1990). In theory and practice, we often encounter the simultaneous use of the terms simulation and game as if they were synonymous. However, their main pur- pose is different, and the design and implementation of games and simulations are subordinate to this. Horton (2012) considers games as a special group of simulations, and com- petition is an essential component of games. Games are also characterised by instant information and feedback. In the absence of this information, the simulation forces the participant to guess, try and test different strategies (on gamification see Section 6.8). Simulations can be used in a wide range of fields: testing technological inno- vations, scientific tests, safety verification, business, medicine and healthcare, law, etc. Education is also an important field of application. Simulations are not a new technological development. The history of simulations and games goes back five millennia, when the first begin- nings of war games and simulations can be found in ancient China. The immediate precursors of modern simulation games date back to around 1930, when Mary Birshstein at the Leningrad Institute of Engineering and Economics developed the first business simulations based on simu- lations for tactics in warfare. A decade after the end of World War II, the Monopologs business simulation was developed in the US to train administration to manage the Air Force supply system (Faria et al., 2009). Today, majority of simulations in general and simulations in education are computer-supported. Simulations in Education 285 6.9.2 Simulations in education Using simulations, participants learn by achieving learning outcomes or re- sults through specific activities, rather than passively – through reading, tests or discussions. This helps to improve the retention and application of knowl- edge. Simulation-based learning is also safe; any mistakes or wrong decisions do not cause harm. Participants can weigh up the reasons and consequences of mistakes, learn from them and prepare for real situations. Feedback is con- stant, consistent and immediate. A good simulation is a fun and exciting learn- ing experience for participants, which also contributes to increased motivation and learning effectiveness (Nielson, 2017). An essential feature of simulations in education is that they also contain peda- gogical elements that enable the learner to achieve the learning objectives. The Warplane Control Simulator cannot be classified as an educational simulation, as it is primarily designed to replicate procedures in the cock- pit and the environment in which the aircraft operates. While the tool can be used for educational purposes, but only with the help of an instructor or other pedagogical support that enables the learning objectives of the participants to be achieved (Instructional Simulations, 2019). Modern technology has enormously increased the usefulness of simulations as a pedagogical tool. Simulations, together with digital learning environments and gamification, are expected to play an even more important role in educa- tion in the future. Research in neuroscience shows that learning and compe- tencies acquisition are more effective when supported by concrete, hands-on experiences and activities (Aldridge and Powel , 2018). Simulations are one of the main forms of active and experiential learning, along with gamifica- tion and virtual reality methods. These tools are suitable for modern learning concepts of student-centred learning and the application of learning theory of constructivism. The use of simulations brings a range of benefits to education. Simulations encourage critical thinking and judgement skil s. Since the choice of solu- tions to a problem is left to the participants, they have to think and weigh the consequences of different choices. Simulations give the impression that the situation is real, so participants are more engaged and ready to participate. Simulations also contribute to a deeper understanding of concepts (UNSW Teaching, 2019). Learning and knowledge acquisition through simulations allow learners to gain experience in how to make decisions in an uncertain, changing and com- plex environment (Gundala and Mandeep, 2016). 286 Learning Approaches and Methods in a Digital Society Bates (2019, p. 510) states that simulations promote the development of com- petencies such as: • complex and real-time decision-making, • operating simulation facilities and complex equipment, • training and awareness-raising on safety procedures, • risk management and decision-making in a safe environment, • developing manual and cognitive skil s. 6.9.3 e-learning and Simulations The basis of simulations in e-learning is a digital learning environment. Simu- lations can be used at different educational levels and in different educational models, from traditional to online education. The experiential component makes them particularly suitable for adult education. The success of simula- tions often depends on the ingenuity and commitment of the participants. In e-learning, simulations are designed as different real-world scenarios that, with appropriate computer support, interact with the learner. The simulations can be run by the learner or by the teacher, depending on the learning strategy and objectives. Simulations work on the basis of different types of input and equipment (Instructional Simulations, n.d.): • body tracking, • physical controller, • voice/sound recognition. The development of input information retrieval is moving towards the integra- tion of the human brain and the computer (Brain Computer Interface – BCI). The results of the simulations are also presented in different ways (visual y, audibly, by touch and through the sensation of movement). 6.9.4 design and development of Simulations Similar to the design of other pedagogical approaches and e-learning pro- grammes, the ADDIE model (Section 3.1) is a suitable starting point for the design and development of simulations. In order to produce good-quality simulations, the characteristics of the system or process to be simulated must first be careful y considered. This determines the functional characteristics of the simulation. If it is a complex simulation, such as the simulation of the functioning of complex surgical equipment, im- mediate feedback must be provided with the chance to correct errors. When Simulations in Education 287 simulating surgical procedures, a wrong decision requires an immediate re- sponse and action. In simulations, it is not only the correct decision that mat- ters, but also the response time. Whatever the complexity and responsiveness of each simulation, we need to define the achievements in advance. For example, in business simulations, we need to determine what the amount of savings or profit achieved by the simulation means in terms of performance. Participants should also receive feedback on how close the simulation was to the optimal solution and what measures or sequencing could be used to achieve improvement. Participants also need to have insight into the conse- quences of wrong decisions. Horton (2012) recommends that the simulation navigation should include fea- tures for hints, demonstrations, navigation help, moving forward, backward, ending and restarting the application. 6.9.5 authoring tools for developing Simulations Today, a wide range of paid-for and open-source tools are available for devel- oping simulations. Berking (2016) classifies simulation authoring tools into the following groups: • System simulation development tools. These tools provide additional fea- tures for the operation of different systems (e.g., additional graphics, sound add-ons, interactions). Adobe Captivate (https://www.adobe.com/si/products/captivate/features.html) is a widespread tool for developing highly interactive displays and simulations that can be used on a variety of devices (laptops, tablets, smart- phones or desktops). • 2D simulation development tools. These tools are much cheaper and sim- pler than 3-D tools, but more complex than regular authoring tools for e- learning development. 288 Learning Approaches and Methods in a Digital Society SimWriter allows you to prepare a simulation storyboard in various ways: from scratch, using PPT documents, based on snapshots or using a ”wiz- ard”. We discuss e-learning storyboard in Section 4.1. Learning Material. Figure 52: SimWriter user interface Source: SimWriter (https://www.learningsolutionsmag.com/articles/1594/toolkit-nexlearn-simwriter ). • 3D simulation development tools. These tools are used to create three-di- mensional images (simulations) that look and act as if they were real. • Video role-play tools. They allow the creation of video scenarios to which learners respond, either with webcams or other tools. Participants can re- view, modify and share their achievements with peers and tutors. The United Nations offers a Carana role-play simulation to help countries manage situation after wars and other crises. This simulation is now also used by the World Bank to train staff working in third countries (https:// paxsims.wordpress.com/2009/01/27/carana/). The World Bank also offers countries simulations to train bank manage- ment in crisis situations. Crisis Simulation Exercises, World Bank (http:// documents.worldbank.org/curated/en/424141468245407598/crisis-simulation- exercises-cSes). • Transmedia story-based tools. These tools can be used to create so-called immersive simulations, where the virtual component is ful y merged with the real one. Various communication tools can be used, such as email, SMS, web, social media and the Internet of Things. Simulations in Education 289 recommended links Conducttr: https://www.conducttr.com/demo-selection/ SERC Portal for Educators. Teaching with Simulations: https://serc.carleton.edu/sp/library/simulations/index.html Educational Simulations: http://www.creativeteachingsite.com/edusims.html Lesson Planet: https://www.lessonplanet.com/search?keywords=simulations&type_ids[]=357917 290 Learning Approaches and Methods in a Digital Society 6.10 Virtual and augmented reality 6.10.1 the difference Between Virtual or augmented reality Virtual reality (VR) refers to a computer-generated simulation in which a per- son can move and interact in an artificial three-dimensional environment us- ing special electronic devices, such as special goggles with a screen or gloves equipped with sensors. In this simulated artificial environment, the user can explore the space around them, use objects and perform various activities. In addition to VR, which has a longer tradition, augmented reality (AR) has also been gaining ground in education (and elsewhere) in recent years. AR is defined as ”an enhanced version of the real world whose elements are aug- mented by computer-generated or acquired sensory plug-ins in the real world, such as sound, video, graphics or haptics”41 (Schueffel, 2017, p. 3). Mihelj et al (2014) define it as ”the enrichment of the real-world image (as seen by the user) with a computer-generated image that enhances the real image with ad- ditional information” (Mihelj et al., 2014, p. 195). The main difference between VR and AR is that the former allows us to enter a new, virtual world that is independent of the physical space we are in, while the latter ”merely” enriches the existing reality through the use of technology. Peddie (2017) lists mixed reality, sometimes called hybrid reality, as a sub-type of AR that also has elements of virtual reality. 6.10.2 Virtual reality VR allows individuals to learn by simulating the real world. According to Dede et al. (2017), VR simulations provide a unique learning experience, allowing the exploration of a wide range of objects, environments and phenomena in virtual space. This increases the level of engagement of the individual in the learning situation, improving their overall learning experience and motivation (Lau and Lee, 2015). In the image below, a European Space Agency researcher in Darmstadt, equipped with an HTC Vive VR headset and motion controllers, demonstrates how astronauts could use VR in their training in the future. 41 Haptics is the science of combining the senses of touch and control in computer applications. Haptic technology refers to any technology that can create an experience of touch for the user through the use of forces, vibrations or movements (Hannaford and Okamura, 2016). Virtual and Augment tual and A ed Realit ed R y 291 Figure 53: using Vr in astronaut training Photo: ESA 2017 (https://www.esa.int/eSa_Multimedia/images/2017/07/reality_check ). One of the important characteristics of the virtual reality learning experience is its immersion or fusion with the virtual space. This can be interpreted as an intense sense of situating oneself in a virtual environment (Ferdig et al., 2018; Lau and Lee, 2015). Participants psychological y perceive themselves in im- mersive virtual environments. Some approaches to learning, such as simula- tions and role-plays, can be very effective in such an environment (Blascovich and Bailenson, 2005). 6.10.3 augmented reality Many experts highlight the great potential of AR to improve learning and teaching (Billinghurst and Duenser, 2012; Dunleavy et al., 2009). AR makes teaching and learning more effective and engaging, and it affects how we in- teract with the learning content by making learning more authentic and no longer just part of a static experience. AR stimulates the individual’s percep- tion and interaction with the real world. As technology advances and becomes more accessible, the importance of AR in education is becoming more evident and more affordable (Garrett et al., 2015). 292 Learning Approaches and Methods in a Digital Society Figure 54: ar using a phone app Source: UNIBOA on Unsplash (https://unsplash.com/photos/nrMgl5Mr8uk ). For example, AR allows students to observe a virtual object together in the same room and communicate directly with each other. This gives them a three-dimensional view of the object under observation, thus improving their depth perception. According to Wither et al. (2009), the ability to present addi- tional information in the same place as the subject under observation is a ma- jor advantage of AR over printed textbooks or other printed resources (Wither, DiVerdi and Hollerer, 2009). Above al , it increases students’ understanding of the content by making it easier for them to put the information into context, as it has a direct link to the subject under discussion (ibid.). AR also facili- tates the pedagogical process, which can be supported by games or simulations (Koutromanos et al., 2015), while encouraging persistence and maintaining the students’ attention (Estapa and Nadolny, 2015). AR also enables learning for people with disabilities, such as people with autism (McMahon, Cihak and Wright, 2015) and adults with intellectual disabilities (Smith et al., 2017). The field of AR and VR is technical y very diverse and complex. At the highest level, Peddie (2017) distinguishes between wearable and static devices. Wear- able devices include headsets, VR glasses and, in the future, contact lenses. Static devices include mobile devices (smartphones, tablets, etc.), stationary devices (TVs, PCs) and ”heads-up displays”. Virtual and Augment tual and A ed Realit ed R y 293 6.10.4 using Virtual and augmented reality in e-learning VR and AR are used in many different fields and for many different purposes, for example in architecture, sports, medicine, art, military, media, rehabilita- tion and entertainment. However, the use of both technological approaches in education is also increasing, in a wide variety of fields and modes of educa- tion, as part of formal education, for short training courses in companies or in e-learning. The main limitation to the use of VR and AR in e-learning is that little or no additional hardware is available or it is expensive. The uptake of VR and AR equipment by individuals is still modest. Boland (2017) reports that the prevalence of VR equipment in US households was only 17% in 2017, with a forecast of 30% in 2020. In e-learning, as in traditional education, we can take advantage of AR and VR to make learning more authentic and active. AR and VR enable certain types of functionalities that are particularly relevant for e-learning. Laura Lynch (2018) lists three examples where VR can be used particularly effectively in e-learn- ing: virtual classrooms, virtual field trips and hands-on learning opportunities. Virtual Classrooms This may be the most obvious use, but it has advantages. Virtual classrooms can further alleviate the problem of no face-to-face communication between teacher and learners in e-learning, as enabled by Web 2.0 tools. Virtual Excursions Virtual tours allow you to visit locations that are inaccessible due to geographi- cal or temporal distance. Google ”Arts & Culture” (https://artsandculture.google. com/) is an online platform that allows three-dimensional exploration of the physical world, for example by visiting sites, museums or galleries. The app which is available on Android and iOS operating systems does not re- quire any special equipment for the participant to use, so it is easy to use even in a ”traditional” LMSs. The added value of these virtual environments is the additional descriptions of the individual objects that the participant observes. DART learning platform offers similar functionality (https://dartlearning.org.au/). Practical Training Opportunities The greatest benefit of immersive virtual environments is the accurate simula- tion of real-life situations. The related use of VR can be useful for public speak- ing training, training doctors, etc. 294 Learning Approaches and Methods in a Digital Society A good example of such use is the training of surgeons. Stanford Univer- sity uses VR to simulate endoscopic sinus surgery (https://web.stanford.edu/ group/sailsbury_robotx/cgi-bin/salisbury_lab/). The use of VR or AR technology is most common in blended learning, where educational technology is combined with traditional teaching approaches. For example, it is very common in chemistry, biology, physics or languages. 6.10.5 Virtual reality and augmented reality tools There are many examples of tools on the web and in the professional literature that can be used to use virtual or augmented reality yourself. There are more and more of these tools every year, and the user experience is getting better and better. This is good news, of course, because it means that teachers them- selves will be able to use these tools to prepare e-learning material. It is also welcome that, as technology advances, hardware for VR or AR is becoming more affordable. This is also pointed out by Brown and Green (2016), who argue that the low cost of VR technology opens many possibilities for the use of VR in teaching and media production. Brown and Green (2016) list the most affordable VR tools or technologies as: 1. Google Street View (https://www.google.com/streetview/) allows you to create photospheres and see other people’s photospheres. Photo spheres are pan-oramic images that can be taken on some Android devices. This feature is built into the camera and allows you to take 360-degree photos of the world around you. It’s very easy to make your photosphere ready for use with Google Cardboard. Virtual and Augment tual and A ed Realit ed R y 295 Figure 55: google cardboard Source: Google Cardboard (https://arvr.google.com/cardboard/ ). 2. Immersive VR Education https://engagevr.io/ develops the Engage VR platform based on the Unity Engine (a proprietary tool for creating 3D vide- ogames and other interactive content). The use of specific applications allows the creation of realistic environments depicting historical events, different virtual environments, or visualising scientific concepts that are difficult to represent with conventional media. recommended links Using Virtual Reality in Education: https://elearningindustry.com/using-virtual-reality-in-education Virtual Reality Society: https://www.vrs.org.uk/ The Multiple Uses of Augmented Reality in Education: https://www.emergingedtech.com/2018/08/multiple-uses-of-augmented-reality-in- education/ 296 Learning Approaches and Methods in a Digital Society 6.11 digital Storytelling 6.11.1 What is digital Storytelling Digital storytel ing is one of the innovative didactic approaches that can be used at all levels and formats of education (Sharples et al., 2014). Learning through storytelling has a long tradition in education. It is based on the fact that we hu- mans are storytellers – we tell stories about our experiences and the meanings these experiences have for our lives. Every culture or society is characterised by stories that tell of our past or present, and sometimes of our future. In the era of new technologies, storytelling has also moved into the digital domain and is being successful y used at all levels of formal education (Cooper, 2016; Robin, 2016), as well as in non-formal adult education (Andersen and Tisdel , 2016; Prins, 2017). As Sharples and colleagues (2014) note, with the increas- ing amount of information online, storytelling is becoming almost a necessary component of education to help individuals understand and evaluate different information sources. In this section, we will learn about the different aspects of digital storytelling and its usefulness for e-learning, and we will look at some of the tools that can be used for this purpose. With digital storytelling, teachers and learners can use technology to enrich or enliven the telling of a story or the presentation of an idea. Digital storytelling allows participants to become active and creative storytellers by choosing top- ics, researching, scripting and developing the flow of the story (Robin, 2008). This material is then combined with different types of multimedia content, including computer graphics, sound recordings, videos and music, and made available online or in other ways. This makes learning more interesting and engaging for the learner than traditional didactic approaches. As Karla Gutier- rez (2015) points out, storytelling is not only an effective teaching strategy, but also a motivational one. Participant engagement increases as the individual becomes more emotional y connected to the story, the content and reflection. From a teaching perspective, Gutierrez emphasises the context and relevance of the content and the integration of theory and practice (Gutierrez, 2015). Allen (2016) has a similar understanding of digital storytelling, arguing that this approach can be used effectively in learning and teaching a wide range of content. In his view, stories provide the framework in which the learning con- tent is embedded and make the learning material more relevant and engaging for the learners. Digital St al S orytelling 297 Among the main advantages of this approach, he highlights two in particular: • The content of the course is embedded in a real context that is perceived by the learners as authentic and credible – this increases the relevance and applicability of the learning content (e.g., in the workplace), as well as the transfer of learning. • The stories and the characters in the stories make it easier for learners to re- member different pieces of information, facts or theories. They do not have to recreate abstract constructs, but simply remember what a person did in a particular situation. This encourages the retention of new knowledge and newly acquired competencies (Allen, 2016, p. 292). 6.11.2 elements of digital Storytelling In the 1990s, Dana Atchley and Joe Lambert founded the Centre for Digital Storytelling (CDS, now known as the Story Centre, https://www.storycenter.org), a non-profit organisation that was among the first to offer training and support for people interested in creating and sharing personal narratives (Storycenter – Our story, 2018). CDS is also known for creating and promoting the ’seven elements of digital storytelling’, which are often cited as a fundamental starting point to begin creating digital stories (Lambert, 2010). Table 31: the seven elements of digital storytelling 1. Point of view What is the point of the story and what is the author trying to say? 2. Dramatic question A fundamental question that keeps the viewer’s attention and only unravels at the end of the story. 3. Emotional content Serious questions that come alive in the story in a personal and powerful way and with which the audience connects. 4. Gift of your voice A way to make the story more personal and help the audience understand the background. 5. The power of soundtrack Music or other sounds that support and reinforce the story. 6. Economy Moderate story length (2 to 3 minutes) that does not overwhelm the viewer. 7. Pacing The pace of the story, which determines how fast or slow the story develops or progresses. Source: Lambert, 2010 and https://digitalstorytelling.coe.uh.edu/page.cfm?id=27&cid=27&sublinkid=31. These seven elements of digital storytelling provide a general framework for storytellers (regardless of the medium they use). They are useful for the teacher and the learners. 298 Learning Approaches and Methods in a Digital Society 6.11.3 types of digital Stories Robin (2015) points out that there are many different types of digital stories, but they can be categorised into three groups: personal narratives – stories that relate experiences in an individual’s life; historical documentaries – sto- ries that examine dramatic events and help us understand the past; and stories intended to inform or educate the viewer about a concept or practice. 6.11.4 digital Storytelling in education There are many ways to use digital storytelling in education. One of the first decisions a teacher has to make at the beginning of the learning process is who will create the digital stories: the teacher or the learner? Some teachers create their own stories to introduce new content to the participants. Digital storytelling can also be a handy tool for learners to learn how to create stories and thereby engage with the learning content. After seeing examples of digital stories created by their teachers or other storytellers, learners can be given an assignment to first research a topic and then present their own take on it, tak- ing into account the seven elements of digital storytelling. An engaging digital story, rich in multimedia content, can be a good way to capture the attention of learners and increase their interest in exploring new ideas. This approach can have a number of positive effects, such as increasing motivation, improv- ing communication, cooperationcompetencies, etc. (Chun-Ming, Hwang and Huang, 2012; Yang and Wu, 2012). Hull (2018) highlights three good examples of the use of digital storytelling in e-learning: • We use stories to il ustrate the content being taught. Content that is not en- gaging for the learner can be made engaging through the use of stories. The content of the course can be enriched with stories that il ustrate the learning topic. The learner is first given some information, which is then explained with a case study that demonstrates this new information in a real-life setting. • Stories put the learning content in context. Instead of using the story as an il ustration in one part of the course, it is constantly interwoven with the learning content. The learner takes on the role of the main character of the story and observes the action from their point of view. • Stories are the topic of learning. You can learn about individual stories by reading a textbook or book, or you can watch a story live. As going to the theatre is not always possible, educators can use video to help. The follow- ing image is a screenshot of an interactive video of William Shakespeare’s Twelfth Night or Whatever You Want, which has been brought into the present day. Digital St al S orytelling 299 Figure 56: twelfth night or Whatever you Want (video screenshot) Source: Twelfth Night (https://vimeo.com/209553401/78a322b49e ). The video is not static, but interactive, as participants can click on icons that appear in the video to get more information if they do not understand the ter- minology or want to understand more about the characters in the frame, etc. The video brings the story closer to the viewer (learner) and the interactivity adds depth and clarity. 6.11.5 tools for creating digital Stories In principle, participants choose the appropriate tools according to the needs or requirements of the story they are creating. If the story contains multimedia material combining photos, video, music and voice, it often uses equipment that the individual already has and is not too expensive, such as smartphones, digital cameras, digital voice recorders and camcorders. The new technology allows individuals or groups to share their stories across the Internet on YouTube, Vimeo or other video platforms. Simple storyboard- ing software tools usual y do not allow interactivity that would allow learners to actively participate in watching or listening to the story. Although interac- tivity allows for the greater activation of the viewer/listener, this does not mean that all stories have to be interactive, as the choice of the story format depends mostly on the objectives and the content that the learner is engaged with. 300 Learning Approaches and Methods in a Digital Society Educational Uses of the Digital Storytelling website (http://digitalstorytelling. coe.uh.edu), founded in 2004 by Bernard Robin, is a useful resource for teachers looking for ideas on how digital storytelling can be integrated into various learning activities. Figure 57: educational uses of digital Storytelling web portal Source: Educational Uses of Digital Storytelling (http://digitalstorytelling.coe.uh.edu ). A good resource to find the right tool for you is the list created and main- tained on Pinterest by David Kapuler (2018). In October 2022, there were more than 100 tools pinned on the list (https://www.pinterest.com/dkapuler/ digital-storytelling-apps-sites/). The eLearning Industry Association recommends 18 free tools for creat- ing digital stories (Pappas, 2013). The recommended tools for digital sto- rytelling are divided into two groups: websites (11 examples) that allow the creation of stories, and free apps for digital storytelling (7 examples) https://elearningindustry.com/18-free-digital-storytelling-tools-for-teachers-and- students. Here are some examples of tools from each group. Digital St al S orytelling 301 Digital Storytel ing Websites PicLits (https://piclits.com/) PicLits is a tool that allows you to be creative by selecting the words and im- ages available in the app. The aim is to put the right words in the right place and in the right order to capture the essence and meaning of the image. Smilebox (https://www.smilebox.com/) Smilebox makes it quick and easy to create slideshows, invitations, cards, col ages and photo albums. Upload your photos to Smilebox, select the ones you want to use, then choose a template, add comments and music. Digital Storytel ing Apps ShowMe Interactive Whiteboard (https://www.showme.com/) ShowMe is an app available on Android, iOS or Chromebook that allows you to create and share instruc- tions, presentations or lessons. Toontastic (https://toontastic.withgoogle.com/) Toontastic is a storyboarding tool that makes it easy for users to draw, animate and create cartoons. Once produced, they can be shared with other e-learners. WeVideo (https://www.wevideo.com/) WeVideo is a web-based video editing and editing software available on Android, iPhone, iPad, Mac, PC and Chromebook. recommended links 18 Free Digital Storytelling Tools for Teachers and Students: https://elearningindustry.com/18-free-digital-storytelling-tools-for-teachers-and- students Educational Uses of Digital Storytelling: http://digitalstorytelling.coe.uh.edu Pinterest: Digital Storytelling Apps & Sites: https://www.pinterest.com/dkapuler/digital-storytelling-apps-sites/ ManageMent in the 7 deliVery oF e-learning 7.1 Management in e-learning – general aspects 7.1.1 the importance of Management Management is a process we face every day. It is about effective and efficient performance, leadership and decision-making. It is already there when we plan what to do, where to do it and how to do it. Management is present all organisations – it can be good or bad. When it is bad, too much money is spent, opportunities are missed, organisations become weaker, morale drops, and eventual y such organisations may even fail. Good management means governing organisations efficiently and effectively. Efficiency is demonstrated by achieving the best possible value for money and effectiveness by achieving the objectives set. Management is not only an issue of the private sector, with which it is most of- ten associated. Management is a common feature of all organisations, whether service or manufacturing sector, large or smal , private or public. Education is no exception. The efficient use of resources in relation to educational outcomes and the attain- ment of objectives is important for al stakeholders in education: for the state, which invests heavily in education, as education is an important factor in the competi- tiveness and success of modern societies in general and a necessity for democratic societies; for educational organisations (private or public), whose mission is, in one way or another related to efficiency and effectiveness, which is closely linked to the quality of human resources; and final y for individuals and their related persons, as their quality of life, future and careers depend on education. 304 Management in the Delivery of E-Learning The importance of management in education is confirmed by the development of a specific management discipline in recent decades, management in educa- tion, which is now a frequent topic of professional debate and literature, and is also often included in the offer of education organisations offering manage- ment and similar programmes. Most of the literature on management in education is concerned with tradi- tional education, which assumes that the learning process takes place through face-to-face communication between the teacher and the learners in the class- room or lecture hal . E-learning poses management new challenges and tasks. While the general knowledge and skil s needed for effective and efficient man- agement remain the same as for ”traditional” managers, the environment in which these skil s and knowledge are put into practice is quite different, re- quiring new knowledge and competencies. The management of e-learning is being challenged by the increased influence of external environmental factors, the different organisation of the educational process and the related technical and organisational-administrative activities, the need for different job profiles and, of course, the increased role of technology. 7.1.2 definition and levels of Management The term management is derived from the English word ”manage” , which means to administer, direct, manage or control. All four tasks are intertwined and have different emphases depending on the level of management and the circumstances. This basic description of the concept of management indicates the main func- tions of management. These are (Rozman, 2002): • planning, • organising, • leading, • controlling. Planning is envisaging goals and the paths to achieve them (Rozman, 2002). Planning, therefore, determines where an organisation wants to go in the future and how to get there. This is one of the main tasks of man- agement, as the fundamental objectives cannot be achieved by adapting to random events in the environment every day. Organising, leading and controlling are also important, but without planning, these processes are difficult. Planning can be broken down according to the criteria of time, type of organi- sational unit, content and subject matter (Možina et al., 2002). For manage- Management in E-Learning – General As ning – Gener pects 305 ment, the division between strategic planning and tactical planning is the most important. Strategic planning refers to defining the priorities and decisive di- rections for the development of the organisation, while tactical planning refers to defining in more detail how to use the elements of the production process to ensure that the strategic objectives are met. Organising is the assignment of tasks, responsibilities and authority to mem- bers of an organisation and the allocation of resources according to the tasks assigned. This process usual y involves defining the tasks, performers, tools, materials, time, the sequence of activities and the way in which the work will be carried out. The organising function usual y follows the planning function, which determines how we want to achieve our objectives. Leading is a manager’s ability to influence the behaviour and actions of employ- ees by directing their actions towards set goals. Leading is about creating a shared organisational culture and values, agreeing on goals and encouraging employees to perform at their best. In its broadest sense, it covers the management of activi- ties to initiate action, communication, motivation and recruitment (Dimovski et al., 2005). Leading is an important component of management, but it is also the part of the management function that is the most difficult to learn. A manager’s ability to create an organisational culture, set goals together and motivate em- ployees is essential for the success of any organisation. As managers, if we want to be sure that we are making adequate progress to- wards the goals we have set ourselves, we need to continuously monitor the work we are doing. Monitoring helps us to see what has been done and to compare it with what was planned. We identify the causes of any deviations and take appropriate action. This process is called controlling. Management is therefore the effective and efficient achievement of an organi- sation’ s objectives through planning, organising, leading and controlling (Di- movski et al., 2005, p. 3). Creative problem-solving, decision-making and task coordination are important for the achievement of organisational objectives. For an overview of the basic definitions, roles and responsibilities of man- agement, see the video presentation Principles of Management. https:// www.youtube.com/watch?v=0Bjl676Qb-k. The content and specific tasks or activities of a manager depend very much on the level of management . There are usual y three levels of management: top man- agement, middle management and lower management (Dimovski et al., 2005). Top managers are responsible for the entire organisation. They are responsible for setting objectives, formulating and implementing strategies, monitoring and analysing the external environment, identifying changes in it and making 306 Management in the Delivery of E-Learning strategic decisions. This issue is addressed in Section 2.1 Preparation the E- Learning Development Strategy. It is most important for the top management to be able to make use of the specific knowledge, skil s and abilities of each member of the organisation. Middle managers are responsible for business units and larger departments in the organisation, for example, as heads of departments, heads of quality con- trol and development departments, etc. They are the links in the organisation, being responsible for implementing the strategies and policies set by the top management, while at the same time being responsible for organising the work of and encouraging lower-level management and performers. Lower management is directly responsible for the ”production” of the output of the organisation and is responsible for implementing the rules and procedures to achieve efficient production. The time horizon of this level of management is short, with a focus on achieving daily objectives. 7.1.3 Specific Features of e-learning Management Management in e-learning retains some of the general characteristics and tasks of management, such as human resource development and management, financial management, quality control and marketing. Managers in e-learning are faced with some new and specific tasks that are not met in traditional edu- cation. These specificities of e-learning management stem from the fundamen- tal characteristics of e-learning, which concern the development and delivery of e-learning programmes and whose common denominator is complexity. The complexity of e-learning is reflected in: • the many interrelated factors that determine the development and delivery of e-learning programmes; • numerous e-learning actors, which can be considered from an institutional or individual perspective; • the diversity of e-learning processes and services. Badrul Khan, one of the world’s foremost experts in the field of e-learning management, points out that the development and delivery of e-learning pro- grammes is influenced by a number of factors (Khan, 2005, pp. 13–14): pedagogi- cal, technological, interface design, evaluation, managerial, ethical, institutional and resource support. The e-learning manager has to weigh up how to manage and balance the impact of these factors in order to make e-learning as effective and efficient as possible during the development and delivery phases. Management in E-Learning – General As ning – Gener pects 307 Figure 58: Factors of the development and delivery of e-learning (e-learning framework factors) Pedagogical Technolog ical Institutional I D nt esig er E-Learning fac n Ethical Framework e Resour Suppor tion ce t Evalua Management Source: Adapted from Khan, 2005, p. 14. These factors are underpinned by different stakeholders, acting either in the educational organisation environment (institutional aspect) or within the educational organisation (individual or human resources aspect). The insti- tutions that could have an influence on e-learning programmes are: funders, associations, government, competitors, external partners and, of course, users. Within an organisation, development requires different skil s and competen- cies, usual y concentrated in specific job profiles, such as didactics expert, con- tent expert, designer and IT and educational technology expert. The delivery of an e-learning programme also requires an adequate staffing team, usual y consisting of a teacher, tutor, administrator and IT specialist. However, the co- ordinating role of the manager is essential in the development and delivery of e-learning programmes. The complexity of e-learning is also reflected in the diversity of activities or processes: different combinations or implementations of these processes result in different e-learning products and services. The basic processes or activities of e-learning programme development and delivery are: analysis of educational needs, design of the delivery model, setting up the digital learning environment, preparation of different types of learning materials and other elements of the e-learning programme, testing and evalu- ation, the provision of pedagogical, technical and administrative support relat- ed to the delivery of the programme, and evaluation activities. Alongside these processes, there are business processes or activities (for example, preparing a business plan and monitoring its implementation), as well as other supporting activities (for example, advertising and research). An organisation may choose 308 Management in the Delivery of E-Learning to carry out all of the activities listed above itself, or it may choose to carry out only some of the activities and outsource some of them. An individual organisation may choose to carry out all of these activities on their own. Another organisation may decide to specialise, for exam- ple, only in developing the learning content of e-learning programmes. In this case, it will only carry out the design and development of the con- tent itself, while at the same time assuring the necessary business activi- ties. An organisation can also specialise in setting up and maintaining a digital learning environment. An organisation can also carry out all or some of the business activities itself. These decisions significantly shape the business model for e-learning in an organisation. E-learning is therefore a complex activity, given the multiplicity of factors and stakeholders, the diversity of processes and activities, and the variety of prod- ucts and services that can be offered; this requires well-designed and compe- tent management. Regardless of the combination of factors involved, which actors are present and what services or results are produced by the processes, it is essential for e-learn- ing management to ensure that it delivers functions in the development phase and delivery phase of e-learning programmes, in terms of stakeholders (People), pro- cesses or activities (Processes) and products or services (Products). Accordingly, Badrul Khan (2005, p. 4) talks about the 3P-model of e-learning management. The key areas of work of an e-learning manager are securing the necessary resources, staffing, providing the technological infrastructure, planning, or- ganising, leading and controlling all the phases in the e-learning process, and ensuring quality and relevant results. It is often believed that management in e-learning is only needed to ensure the smooth delivery of programmes, while the development of programmes is handled by the teaching staff. In the early years of e-learning, the development of e-learning programmes was in the hands of individual enthusiasts, usual y teachers, without the direct involvement and support of management. This practice is still very common in Slovenia. An analysis of the state of e-learning in the higher education sector in Slovenia in 2017 showed that almost half (44%) of the surveyed higher education organisations involved in e-learning leave the care of this form of education to the teachers (Bregar and Puhek, 2017). Studies on the characteristics of e-learning and the experiences of successful projects point to the need to consider the development of e-learning as a strate- Management in E-Learning – General As ning – Gener pects 309 gic decision, which is the responsibility of the top management. The manage- ment thus also has an important role to play in the development of e-learning. The introduction of e-learning brings many changes and innovations to an organisation, and top management in particular needs to be aware of this. Change management is therefore an indispensable part of management in e-learn- ing, especial y in the early stages of development. Lynch and Roecker (2007) rec- ommend that change management in e-learning should follow these principles: • we need to systematical y and continuously monitor the human side of e- learning implementation; • changes must first be accepted by the top management; • we need to actively involve all levels of management and staff in change processes; • changes must be clearly presented and recorded in the relevant documents; • change must be driven by appropriate incentive mechanisms; • information and communication about changes must be comprehensive and timely; • we need to integrate change into the core values and culture of the organi- sation; • all participants must be prepared for unexpected changes; • communication must also take place on an individual level. Change management is thus one of the essential features of e-learning man- agement. Closely linked to change management is risk management. The establishment of e-learning as a form of education on a par with traditional education also requires careful quality management and the continuous in- novation and modernisation of the educational process. For more on change management in e-learning and other specific aspects of e-learning management such as risk management, knowledge man- agement, quality management and innovation management, see Lynch and Roecker’s monograph ”Project Managing E-Learning. A Handbook for Successful Design, Delivery and Management” (2007) and in the pro- ceedings of ”Leading and Managing E-Learning. What the e-learning Leader Needs to Know” (Piña et al., 2016). 7.1.4 Project Management In recent decades, the work of the management has changed considerably, es- pecial y at the middle management level, which has started to shrink and turn into project management. 310 Management in the Delivery of E-Learning The essence of project management is to deliver results from a well-defined, 7.2 Services for learners one-off or non-recurring task within a set timeframe, taking into account the resources available. The project manager is responsible for independent work projects involving staff from different departments within and outside the or- One of the fundamental features of e-learning is the learner support systems. ganisation. The project’s organisational structure is temporary and linked to Their main purpose is to enable the learning process to take place with the the implementation of the project and coexists alongside the permanent or- spatial separation of teacher and learner. Support systems are made up of dif- ganisational structure of the organisational unit or the organisation as a whole. ferent forms of pedagogical and non-pedagogical support. Part 5, Pedagogical Effective project implementation and organisational performance require the Support, is dedicated to the presentation of pedagogical support systems in ongoing col aboration and coordination of project managers with the organi- e-learning. sation’s managers. Project management has developed some simple but very In non-pedagogical support in e-learning, e-learning management must offer effective tools for carrying out management activities. Among the most wide- all the services available to learners in a traditional educational organisation spread are network programming techniques. These provide a good overview and ensure that they are accessible to learners despite the spatial separation. If of project realisation of all phases, identifying critical points and showing tasks an e-learning programme is delivered using an LMS, most of the services, both in a logical time sequence and interdependencies. Among the best-known are pedagogical and non-pedagogical, can be delivered using the LMS. In this sec- the critical path method ( CPM) and programme evaluation and review tech- tion, we will limit ourselves to the content of the non-pedagogical services that niques (PERT). e-learning management must provide to learners, regardless of the technologi- cal infrastructure available. Some websites offer freely available tools for project management. An The core non-pedagogical services needed by e-learners are: overview of these can be found on the website: Project Management Tools Directory (http://www.startwright.com/project1.htm). • administrative and technical services (enrolment, access to records and certificates, helpdesk), The basic features of project management in e-learning are clearly pre- • library, sented in Claudia Dornbusch’s video E-Learning Project Management • counselling. http://www.youtube.com/watch?v=Wurexewxu2M&playnext=1&lis t=Pld21383899241B4aa. The specific range of non-pedagogical services that are available, usual y with technological support, depends on the characteristics and needs of the partici- recommended links pants, the type and content of the programme, the format of the e-learning, financial considerations, etc. Capterra. 5 Best Online Project Management Courses for Accidental Pro- ject Managers: https://blog.capterra.com/the-5-best-online-project-management-courses/ 7.2.1 administrative and technical Services Badrul Khan. E-Learning Management: http://www.youtube.com/watch?v=Xauo4hVgWze Administrative processes are usual y routine, so they can be carried out ac- cording to standardised procedures accessible in a digital learning environment. If these are not available, we can simplify some administrative processes by sending the relevant forms by email. Ordinary mail is still used only excep- tionally today. enrolment in the Programme One of these mainly routine administrative procedures is enrolment in a pro- gramme. Enrolment in an e-learning programme should normal y take place without the participant being physical y present. Participants are usual y en- Services for Lear vices f ners ner 311 7.2 Services for learners One of the fundamental features of e-learning is the learner support systems. Their main purpose is to enable the learning process to take place with the spatial separation of teacher and learner. Support systems are made up of dif- ferent forms of pedagogical and non-pedagogical support. Part 5, Pedagogical Support, is dedicated to the presentation of pedagogical support systems in e-learning. In non-pedagogical support in e-learning, e-learning management must offer all the services available to learners in a traditional educational organisation and ensure that they are accessible to learners despite the spatial separation. If an e-learning programme is delivered using an LMS, most of the services, both pedagogical and non-pedagogical, can be delivered using the LMS. In this sec- tion, we will limit ourselves to the content of the non-pedagogical services that e-learning management must provide to learners, regardless of the technologi- cal infrastructure available. The core non-pedagogical services needed by e-learners are: • administrative and technical services (enrolment, access to records and certificates, helpdesk), • library, • counselling. The specific range of non-pedagogical services that are available, usual y with technological support, depends on the characteristics and needs of the partici- pants, the type and content of the programme, the format of the e-learning, financial considerations, etc. 7.2.1 administrative and technical Services Administrative processes are usual y routine, so they can be carried out ac- cording to standardised procedures accessible in a digital learning environment. If these are not available, we can simplify some administrative processes by sending the relevant forms by email. Ordinary mail is still used only excep- tionally today. enrolment in the Programme One of these mainly routine administrative procedures is enrolment in a pro- gramme. Enrolment in an e-learning programme should normal y take place without the participant being physical y present. Participants are usual y en- 312 Management in the Delivery of E-Learning rolled by filling in an online enrolment form and the system automatical y notifies them when their enrolment is accepted or rejected. If this is not feasible, information on e-learning enrolment and choices should certainly be made available online. The simplest way to do this is to have the email address of the person responsible online to answer any questions you may have. Traditional written communication is hardly used today, especial y in e-learn- ing. For those who may have any further substantive questions or technical problems when registering, it is also useful to publish the relevant telephone numbers and other contact details. The Doba Faculty (https://www.dobabusiness-school.eu/) offers non-pedagogical support services to applicants, students and graduates. The main purpose is to help people make choices about their education, to support them during their education, in their choice of career and throughout their life. Support is continuous and systematic throughout: pre-enrol- ment activities and at the start of, during, at the end of or after studies. Support is provided by Student Affairs administrative staff, mentors, pro- gramme managers, and technical staff. The formats vary widely and are supported by different media and modes of communication (telephone, websites, video conferencing, printed materials, live and online informa- tion days, one-to-one counselling, email, discussion forums, webinars, social media, chat rooms, etc.). records of commitments and certificates For e-learners, easy access to a variety of information about their progress and achievements in the programme they are enrolled in is just as important as easy access to enrol in the programme. They need certificates of completion as evidence for various purposes (e.g., the reimbursement of expenses, job pro- motion, finding a new job, completing their education). One of the core administrative services is to provide learners with easy access to assessments and other data related to their participation in the e-learning programme. As with enrolment, this service can be offered as information on a website, usual y password-protected, or by email. Learners are also interested in more detailed information on their progress in the educational programme and their learning achievements (for example, through feedback from teachers). Services for Lear vices f ners ner 313 The following figure shows information about an e-learning student’s academic performance in a course, provided by Doba Faculty’s teacher the Blackboard learning environment. Figure 59: information about academic performance in the Blackboard learning environment (example) Source: Doba Faculty, 2019. technical assistance Technical (IT) support provides technical assistance to learners, tutors and ad- ministrative staff on the use of the LMS and other digital tools. Well-organised and user-friendly technical support is an indispensable component of the digi- tal learning environment. For this service to work effectively, an information database of potential problems and solutions needs to be created and main- tained. This database can then be used as an open advisory system (with free access). However, it is advisable for the educational organisation to make the IT technician contactable (by phone or email). When you are just starting a programme and have little experience in setting up and running a technical IT support service, you can outsource this service to an external contractual partner. 314 Management in the Delivery of E-Learning 7.2.2 library An essential service of e-learning that e-learning management must provide is access to information and learning materials. Digital materials are an im- portant element of pedagogical support. This material can be made available online or in the cloud as an OER (Section 6.2) or for a fee. Alternatively, it may be prepared specifical y for the participants of a particular educational programme and made available only under specific conditions. Participants will often be given assignments for which they will need additional resources beyond the recommended learning material, which they will have to find for themselves. These additional resources can also be found in libraries, which are divided into public and in-house libraries, according to accessibility, and into general, specialised and school libraries, as well as a national library with a special sta- tus and importance, according to professional focus. Modern, high-quality li- braries are characterised by providing their members with free access to a wide range of digital materials and databases. Information on more than 900 libraries in Slovenia and their offer, as well as information resources available in Slovenia, can be found on the COBISS web portal https://www.cobiss.si/knjiznice/. Digital libraries, also known as virtual libraries or e-libraries, are greatly improving access to information resources . Digital libraries store material in dig- ital form and make it accessible by computer. We usual y distinguish between material that was original y created in digital form and material that has been digitised subsequently. Most modern libraries have printed and digital materi- als (hybrid libraries). For more information on digital libraries, see Wikipedia (http://en.wikipedia.org/wiki/digital_library). Europeana Collections (https://www.europeana.eu/portal/en) is Europe’s digital library of digitised images, texts, audio and video. It currently pro- vides access to about 60 million items of digital content. In addition to materials from libraries, it offers materials from museums, archives and multimedia archives (e.g., radio and television archives, film archives). Services for Lear vices f ners ner 315 Figure 60: europeana collections: latest Stories, Screenshot Source. Europena Collections (https://www.europeana.eu/en ). In addition to libraries, public and private organisations, especial y universi- ties, are developing digital archives. They are characterised by their preser- vation of primary sources and digitisation allows them to access individual documents and reproduce them easily. Among the best-known is the Oxford Text Archive of the University of Oxford (http://www.ota.ox.ac.uk/). The development of digital libraries is driven by several major international projects, such as Google Book Search, Project Gutenberg, the Million Book Project, the Internet Archive and the World Digital Library. The National and University Library is developing the Digital Library of Slovenia with the help of local companies (https://www.dlib.si/). The IZUM digital library also offers access to a wide range of open resources (https://www.izum.si/en/library/). Digital libraries are also available at the University of Ljubljana (https://www.uni- lj.si/libraries/university_libraries/), University of Maribor (https://dk.um.si/info/index. php/eng/) and the University of Primorska (https://www.upr.si/en/university-library/e- resources/). 7.2.3 counselling and information Services Learners need occasional counselling support for each course . In traditional ed- ucation, learners are usual y referred to an appropriate counsellor or teacher to resolve the problem. In e-learning, we also need to provide learners with advi- sory services, of course according to the (technological and other) possibilities available. We can use a variety of synchronous communication methods for counselling, from telephone conversations and chat rooms to videoconferenc- ing. Alternatively, we can use various asynchronous communication methods (e.g., email, an online advice corner, blogs, discussion forums, etc.). When a programme runs for a long time, we usual y find that much of the counselling is routine and actual y means information provision. Thus, it can be offered in the form of frequently asked questions (FAQs), digital or printed manuals and guides, or in various publications (leaflets or brochures). 316 Management in the Delivery of E-Learning The most frequently asked questions are: 7.3 e-learning results • the curriculum/content of the programme, • the conditions for student’s promotion, The performance of an organisation is usual y monitored from two perspec- • certificates of completion, tives: effectiveness and business efficiency. • the price of the programme, • the ways in which knowledge is tested and assessed, In effectiveness monitoring, we are interested in the quality of the results • recognition of prior education or training achieved in the broad sense of the word. The definition of effectiveness is close- • the technical requirements for participation in e-learning programmes and ly linked to the achievement of the organisation’ s objectives: higher effectiveness the use of the digital learning environment. means a higher level of achievement of the objectives. In short, when business is good, we do the right things, certainly in terms of the objectives we set. The successful online education organisation Online Advance from the Business efficiency is measured by comparing business results with the re- USA (http://www.advanceonline.com/elearning-faq.asp) addresses the follow-sources (inputs) needed to achieve those results. An organisation will be more ing topics in its FAQs: the specificities of e-learning, the content of the efficient if it can achieve a more favourable outcome with the same inputs, or if programmes, the technological support, the learning requirements and it can achieve the same outcome with fewer inputs. Outcomes and inputs can assessment methods, and the digital tools used. be defined in very different ways. Depending on how these are defined, there are three basic indicators of business performance: cost-efficiency, profitab ility In addition to posting general information on the website, a range of other com- and labour productivity. These indicators are usual y calculated over a one-year munication tools can be used for counselling, such as chat rooms, forums, blogs, period. wikis, etc. For non-pedagogical counselling, we use mostly the same asynchro- Effectiveness and business efficiency reveal different aspects of an organisa- nous and synchronous communication tools as for pedagogical support. tion’s performance. It is entirely possible for an organisation to improve its Whatever the communication method chosen, we need to make sure that the business efficiency by reducing costs at the expense of a lower quality of ser- information is easily accessible, visible and understandable. vice, while delivering quality output is one of the organisation’s strategic ob- When giving advice, we need to be aware of the limitations that advisers have. jectives. So it could be that we are doing the wrong things efficiently, or that we For example, psychological, health or marital problems can be a major obstacle are not doing the right things efficiently. to learning, but the counsellor should limit himself/herself to issues directly When monitoring the results of e-learning, we need to take into account both relevant to the programme and should not turn into a ”temporary” marriage aspects, effectiveness and business efficiency, whether the e-learning is deliv- counsellor or therapist. When the problems go beyond the competence of the ered in private educational organisations or in the public sector. In practice, counsellor, the learner is directed to the appropriate address. the private sector often focuses on business efficiency and less on education In any case, it is recommended that the management of the organisation or- effectiveness. By contrast, in the public sector, aspects of business efficiency ganises appropriate training for the staff who will be involved in the consul- are often neglected. tancy . E-learning advice is often written, so we need to pay attention to the Managers in e-learning need to continuously monitor, through effectiveness linguistic appropriateness of the texts and to the clarity and unambiguity of the monitoring, whether the set e-learning objectives have been achieved and also expression, which does not allow misinterpretations of what is written. what business efficiencies have been achieved in doing so. The following criteria recommended links are mainly used in e-learning to assess performance: • the success rate of the education or training and the drop-out rate, COBISS: • quality of results, https://www.cobiss.si/en/libraries/ • business efficiency. Europeana Collections: https://www.europeana.eu/portal/en E-learning, and in particular online education as a specific form of e-learning, makes access to education and training much easier and simpler; this is par- ticularly important for certain population groups and for achieving certain E-Learning Results ning R 317 7.3 e-learning results The performance of an organisation is usual y monitored from two perspec- tives: effectiveness and business efficiency. In effectiveness monitoring, we are interested in the quality of the results achieved in the broad sense of the word. The definition of effectiveness is close- ly linked to the achievement of the organisation’ s objectives: higher effectiveness means a higher level of achievement of the objectives. In short, when business is good, we do the right things, certainly in terms of the objectives we set. Business efficiency is measured by comparing business results with the re- sources (inputs) needed to achieve those results. An organisation will be more efficient if it can achieve a more favourable outcome with the same inputs, or if it can achieve the same outcome with fewer inputs. Outcomes and inputs can be defined in very different ways. Depending on how these are defined, there are three basic indicators of business performance: cost-efficiency, profitab ility and labour productivity. These indicators are usual y calculated over a one-year period. Effectiveness and business efficiency reveal different aspects of an organisa- tion’s performance. It is entirely possible for an organisation to improve its business efficiency by reducing costs at the expense of a lower quality of ser- vice, while delivering quality output is one of the organisation’s strategic ob- jectives. So it could be that we are doing the wrong things efficiently, or that we are not doing the right things efficiently. When monitoring the results of e-learning, we need to take into account both aspects, effectiveness and business efficiency, whether the e-learning is deliv- ered in private educational organisations or in the public sector. In practice, the private sector often focuses on business efficiency and less on education effectiveness. By contrast, in the public sector, aspects of business efficiency are often neglected. Managers in e-learning need to continuously monitor, through effectiveness monitoring, whether the set e-learning objectives have been achieved and also what business efficiencies have been achieved in doing so. The following criteria are mainly used in e-learning to assess performance: • the success rate of the education or training and the drop-out rate, • quality of results, • business efficiency. E-learning, and in particular online education as a specific form of e-learning, makes access to education and training much easier and simpler; this is par- ticularly important for certain population groups and for achieving certain 318 Management in the Delivery of E-Learning educational objectives, especial y in the public sector. That’s why e-learning also considers access to education and training as an effectiveness criterion. 7.3.1 educational Success rate and drop-out rate We usual y consider any completion of a programme a success and dropping out a failure, although sometimes this is not the most appropriate measure. Many times, adults start, follow and progress through the modules, but then fail to take the final steps to formal y complete the training. For an educational organisation, a low completion rate is a bad sign, but it does not show how much the learners have real y learned and progressed in the meantime. The measurement of learning or training performance should also take into ac- count the additional knowledge and competencies acquired by learners, regard- less of formal y calculated drop-out or completion rates. A typical example of the inappropriate use of drop-out rates as perfor- mance indicators is the drop-out rate for MOOCs. This form of open education is characterised by extremely high drop-out rates of around 90%. In interpreting this data, we need to bear in mind that the goals of the individuals who enrol in a MOOC vary widely: some enrol with- out any intention of serious education or training, others find that the programme is not in line with their expectations, some find it too de- manding, boring, not interactive enough, and only a handful of those who enrol successful y complete the MOOC. By monitoring the behaviour of enrolees in a MOOC learning environ- ment, we can identify different groups of enrolees in great detail. It then makes sense to adapt the benchmarks to the characteristics and objec- tives of each group. Research shows that dropping out is a multifaceted phenomenon. Learners with a higher level of prior education tend to perform better than those with a lower level of education. Those who find it harder to combine education with work and family commitments will find it harder to cope with learning tasks. Those who are unable to learn independently and organise the space and time to learn also tend to have more problems. The most frequently cited reasons for learners leaving e-learning programmes are (Yorke, 1999; Davies and Elias, 2003): • choosing the wrong field of study or programme, • learning difficulties, • financial and other personal problems, E-Learning Results ning R 319 • lack of experience in self-paced learning, • dissatisfaction with the LMS, • wrong choice of educational organisation, • dissatisfaction with the services provided by the educational organisation. The drop-out rate of adult first-time learners in an e-learning programme is influenced by sociological, psychological, technical and cognitive factors, with the cognitive load of the learner. The learner’s ability to manage it is crucial. When first participating in an e-learning programme, the learner must not only master the content, but also the technology, the digital learning environ- ment, the new role and the new mode of communication (Smith, 2007, p. 7). The article High Attrition Rates in e-learning: Challenges, Predictors and Solutions points to some interesting aspects of the causes of drop-out that should not be ignored in e-learning management (Martinez, 2003). The author points out that successful learners in traditional forms of edu- cation are not necessarily successful in e-learning. This also shows that e- learners are expected to have different competencies and other attributes than those in traditional forms of education. The quality of the interaction between teacher and learner is important for the success of e-learning. Be- cause it is done differently, the teacher also needs the right competencies for successful e-learning. Genç and Tinmaz (2016) also point out the impor- tance of teacher-learner interaction in online programmes, emphasising the role of teacher feedback, complemented by discussion in forums. Drop-out rates are also significantly influenced by factors stemming from the characteristics of the learners. One such factor is the way in which the causes of success or failure are attributed. If an individual sees the reasons for success or failure primarily in external factors (such as ’luck’ or ’bad luck’) rather than in his or her own effort or ability, he or she is less intrinsical y motivated and thus more likely to leave education. Based on a systematic literature review, Kauffman (2015) identified the characteristics of successful and satisfied online students. They should be characterised by: higher emotional intelligence, operationalised as self-awareness of one’s own needs, and appropriate emotion manage- ment (and avoidance of frustration), self-regulation skil s, self-discipline, appropriate time management, good organisation, planning and self- evaluation, a predominantly visual learning style (as most information is presented visual y in online study), and an internal locus of control (a sense of control over one’s own actions, looking for the causes of one’s own (lack of) success within oneself, rather than in external factors). 320 Management in the Delivery of E-Learning Educational organisations are sometimes too quick to attribute drop-outs to a lack of motivation or ability on the part of learners. Highly motivated and ca- pable learners will learn even under adverse conditions. Despite having access to satisfactory and appropriate learning materials, most learners will need at least some support in their learning. We can reduce attrition: • by offering high-quality learning material; • by providing appropriate counselling before participation in the pro- gramme (identifying the needs and suitability of the programme for the learner) and by providing appropriate, prompt and effective counselling and support during the programme; • by creating an atmosphere that encourages learners to seek help and advice from tutors and advisors, and from their peers; this creates a supportive atmosphere, encourages ongoing participation in the learning group and thus maintains motivation and interest. Reducing drop-out is one of the core areas of application of learning analytics; this topic is discussed in more detail in Section 6.4. Learner attrition not only affects the business results of an education organi- sation and the efficiency of its use of resources. It also affects its reputation and that of the users of its educational services. The issue of drop-out must therefore find its place in the organisation’s strategic and business plan (for example, by outlining a strategy to reduce drop-out or re-engage early leavers in e-learning). Of course, a reduction in drop-out rates should not be the result of lowering the benchmarks for learning achievement. 7.3.2 the Quality of the results The quality of e-learning is confirmed in the eyes of e-learners in the same way as in traditional education, i.e., completing all the prescribed learning requirements as successful y as possible and within the prescribed deadlines. In formal education, this aspect of performance is expressed in terms of pass rates. However, assessing the success of e-learning from a quality perspective also has wider dimensions. In terms of cognitive achievement, e-learning can be as successful (of good quality) or even more successful (of exceeding qual- ity) than traditional forms of education. It is also effective in the emotional domain, encouraging the development of values, attitudes and emotional re- sponses. Digital learning material produced in high-quality e-learning pro- grammes are ful y comparable to textbooks in traditional education and can even surpass them thanks to certain features such as interactivity and the use E-Learning Results ning R 321 of different media. The issue of quality assurance in e-learning is dealt with in Section 7.5. In the early years of e-learning, the wider take-up of e-learning was hampered by prejudices and fears that e-learning was of lower quality than traditional education. The introduction of quality standards in e-learning over the last decade has certainly contributed to a higher quality of e-learning and also in- creased its reputation. The European Foundation for Quality in e-learning, which ran from 2005– 2015, played a prominent role in accelerating the take-up and promotion of e-learning in the European Union. Its aim was to promote and contribute to a better quality of e-learning in various aspects. Evidence of the relevance of e-learning programmes comes from accreditation and positive evaluations. External evidence of the quality of an e-learning programme is, of course, not enough. The quality of the knowledge and competencies acquired by learners is also important . the Kirkpatrick Model The Kirkpatrick model (Kirkpatrick, 1994) is often used as a starting point for measuring performance in terms of acquired knowledge and skil s. This model provides for the assessment of learning or training performance at four levels: reaction, learning, behaviour and results. The levels are hierarchical y linked, i.e. when evaluating a higher level, we need to take into account the information gathered at the lower level of evaluation. Any element of the learning process can be subject to evaluation at any level. Figure 61: Kirkpatrick’s evaluation model Results Behaviour Learning Reaction Source: Kirkpatrick, 1994. 322 Management in the Delivery of E-Learning Level 1 – Reaction. At this level, we identify the views and opinions of learn- ers about the training they are taking part in. We are particularly interested in their overall satisfaction with the curriculum and its delivery. This information is usual y gathered after the learning process, either informal y in conversation or through surveys. In e-learning programmes, this level of evaluation is most easily carried out through online surveys, for example after each learning unit or module. Level 2 – Learning. At the learning level, we measure what learners have ac- tual y acquired (learned) in the learning process (in terms of knowledge and competencies). This is a traditional area of evaluation for educational organisa- tions and is carried out through tests. In formal education, it is usual y the last and most important level of evaluation, showing whether the main objectives of the programme have been achieved through pass rates. However, collecting data on learning outcomes alone cannot satisfy the fundamental objectives of the evaluation. An analysis of achievements is needed to give feedback on how to improve the programme. In e-learning, this level of evaluation is relatively easy to implement with quizzes and other tools usual y available in digital learning environments. Of course, we need to bear in mind the limitations of this type of knowledge as- sessment. More on this in Section 4.3. Assessment Methods in E-Learning. In e-learning, evaluation at this level is often limited to what is known as a tracking strategy , which is supposed to demonstrate the active participation of learners in the e-learning programme. Of course, participation in a pro- gramme does not guarantee that a learner will progress in their knowledge. Rosenberg (2001) points out that only the third and fourth levels of evaluation provide decisive information on the performance and quality of a programme. These two levels are particularly relevant for the evaluation of (work-based) employee education, as the educational objectives in this segment are more complex compared to formal education. Level 3 – Behaviour. Evaluation at this level aims to find out what changes the learning has brought about in the learners, for example in their behaviour at work. We want to know whether learners have improved or acquired new com- petencies, whether they perform tasks more efficiently and to a higher quality; whether they make better use of their potential and talents. Formal assessments, expressed in terms of test scores, are not relevant at this level of evaluation. In modern e-learning, this level of evaluation can be carried out using a variety of new methods and approaches based on active learning principles, such as simulations and gamification, virtual and augmented reality; these methods are discussed in Part 6 (Learning Methods and Approaches in a Digital Society). E-Learning Results ning R 323 Level 4 – Results. The fourth level of evaluation assesses the wider implications of learning, i.e., the impact on the organisation. For business-oriented compa- nies, achieving better business results is actual y the ultimate goal of training and learning. The effects come in different forms: increased sales, increased productivity, fewer defects or complaints, ecological improvements, innova- tions. In terms of measurement, this level is the most challenging, as it is dif- ficult to disentangle the effects of training and learning from other factors that influence business performance. This is why, despite its importance, analyses of the fourth level of the Kirkpatrick model are still quite rare. They largely remain at the level of the business performance indicators shown below. new World Kirkpatrick Model Around the 50th anniversary of the original model, Kirkpatrick’s successors developed a new version of the model, called The New World Kirkpatrick Model. This name was intended to emphasise its essential characteristics, i.e., its adaptability to the changes in learning and teaching that have taken place over the last 50 years, as well as to the changed circumstances in which learn- ing and teaching processes take place today (Kirkpatrick and Kayser Kirkpat- rick, 2016). The New World Model has been designed to prevent or at least mitigate, through refinements and additions, the misuse or inconsistent ap- plication of the model in practice. As already noted, one such shortcoming is that the analysis of the fourth level (results) is rarely carried out in practice, as research efforts and resources are usual y exhausted at the first two levels. The authors of the New World Model thus define the levels of evaluation in reverse order, putting results first. In this way, they want to emphasise the importance of results as the guiding objective of learning and teaching. The implementation of the other three levels must follow from this level. The re- lationship and content of the different levels for the New World Kirkpatrick model is shown in Figure 62. 324 Management in the Delivery of E-Learning Figure 62: the new World Kirkpatrick Model MONITOR AND ADJUST LEVEL 1 REACTION Engagement Relevance Customer onitor Reinfo Satisfaction rce M LEVEL 4 RESULTS LEVEL 3 • Leading indicators LEVEL 2 BEHAVIOUR • Desired outcomes LEARNING Knowledge Skills O Attitude E n- n the c -job learning o Confidence urage Commitment Reward Source: Kirkpatrick and Kirkpatrick, 2019, p. 5. The new version of the Kirkpatrick model is also characterised by the fact that the individual levels are defined in more detail, giving users quite detailed guidance on how to derive the measurement of learning and teaching perfor- mance for each level in practice (Kirkpatrick and Kirkpatrick, 2019). For more details on the novelties of the New World model compared to the original Kirkpatrick model, please visit the Kp Kirkpatrick Partners website ”The One and Only Kirkpatrick” (https://www.kirkpatrickpartners.com/our-Philosophy). 7.3.3 Business efficiency Despite the high fixed costs, if resources are used economical y and enough learners are involved, e-learning can achieve better business efficiency than a comparable programme would achieve in traditional education. As we have already said, we determine business efficiency in principle by com- paring the results of a business process (outcome) with the resources spent on it. Labour productivity, cost-efficiency and profitability have become es- tablished in the business world as fundamental economic indicators of busi- ness performance. Measuring the business efficiency of education is therefore primarily a question of education and training for employees at the workplace. Labour productivity is defined as the ratio of output to labour time spent over a given period. We consider that monitoring the efficiency of e-learning through labour productivity is less appropriate, mainly due to the specific nature of the work in e-learning programmes. The productivity of teaching and non-teach- E-Learning Results ning R 325 ing staff hours in an e-learning context is highly dependent on how much and how well we have integrated technology into the educational process. The con- stant evolution of technological support in e-learning and the wide variation in the use of technology between e-learning programmes make comparisons of labour productivity for e-learning over time or space highly questionable. Cost-efficiency is the ratio of business output (in value terms) to costs (input). An organisation will be more cost-effective than others if it is able to achieve a more favourable output with the same inputs (costs) or if it achieves the same output with fewer inputs than comparable organisations. The rate of return or profitability is calculated as the ratio of the income or profit generated to the resources invested. Of the three indicators, this is the most commonly used in e-learning. In the English literature, the rate of return is abbreviated as ROI (return on investment). Technical y, calculating the rate of return or profitability is quite simple. Rate of return = [(income or profit): resources invested] ×100. For ex- ample, if €120.000 is invested in an e-learning programme and the profit (over one year) is €6.000, the rate of return is 5%. In other words, for every €100 invested, we ”earn” €5. Calculating the rate of return for a given e-learning programme is quite dif- ficult in practice. Difficulties arise in identifying which revenues or costs (as component of profit calculation) are related to e-learning and which are only partly or not at al . A similar problem arises in the allocation of resources for e-learning and other uses. The calculation of the rate of return is also com- plicated by the time factor, in particular regarding the length of the time lag between when the new programme is expected to start generating income and the period in which the investment has to be recovered. It should also be borne in mind that some e-learning services (especial y those that are highly special- ised and non-standardised) can quickly become outdated or unusable. It is necessary to complement e-learning business performance indicators with other indicators, firstly because of the difficulties in calculating and interpret- ing traditional indicators, but also because e-learning is usual y not only about narrow business-oriented objectives, but also about broader strategic objec- tives related to the mission and vision of the organisation and the role of the specific educational programme in delivering the organisation’s strategy (see Section 2.1). 326 Management in the Delivery of E-Learning Jacob and Sharma (2017) give a number of examples of how the concept of the rate of return can be linked to the non-financial objectives of cor- porate e-learning. Instead of monetary indicators, ROI calculations can take into account the reduction in the time to get a certain competency, the reduction in errors in work and customer complaints, the increase in customer satisfaction, the improvement in relationships, and the better satisfaction of customer needs. They suggest ROL (return on learning) as a more appropriate term. The Balanced Scorecard, introduced by Robert S. Kaplan and David P. Norton, provides a systematic framework to complement traditional business perfor- mance indicators, applicable also for monitoring e-learning. These indicators are used to monitor the broader performance of the organisation in the fol- lowing respects: • from a financial perspective (revenue growth, return on capital, net profit- ability); • in terms of customer service (customer satisfaction index, drop-out rate, number of complaints); • from an internal business process (value added per employee, total qual- ity costs, average delivery time, investment in development, investment in technology); • in terms of learning and growth (employee satisfaction index, number of training hours per employee, absenteeism, turnover), (Možina and Klemenčič, 2008). Benchmarking emerged in the business world at the end of the twentieth cen- tury as a specific method for monitoring performance. This method is based on comparing the processes and procedures of a given organisation with simi- lar processes, usual y in the best-performing organisations. The level of the selected characteristics achieved in the organisation chosen as a baseline for comparison is taken as a target to be achieved and possibly exceeded. The benchmarking method therefore enables development by meeting and exceed- ing the values of the selected benchmarks. The rapid and uneven development of e-learning makes the method of bench- marking particularly interesting in e-learning. The article E-Learning Benchmarking: Methodology and Tools Review provides an overview of the main studies on the use of benchmarking in e-learning up to 2010 (Šćepanović et al., 2011). E-Learning Results ning R 327 7.3.4 access to education and training It is clear that many potential e-learners are not able to attend either full-time or part-time training offered by traditional educational organisations. E-learning opens up new educational opportunities compared to traditional education. For the individual, these opportunities stem from the principle of flexible learning, which can take place at home or at work. The time spent studying can also be tailored to suit the individual’s abilities and commitments. E-learning also allows flexibility in teaching methods and content, so that the learning process and content can be tailored to the individual needs of the learners. One of the main features in the development of e-learning is the increasing personalisation made possible by the methods and approaches of the third and fourth generations of e-learning in particular. Experience shows that e-learning benefits many: • individuals, because they are given the opportunity to be educated in a way that would otherwise be impossible for them, and in a way that is more tailored to their needs; • employers, because they can flexibly plan the training of their employees, who often don’t even have to leave their jobs. At the same time, many em- ployees in different locations can acquire the skil s and competencies that the company needs most at any given time; • the country, because e-learning gives it the opportunity to provide educa- tion and training to different groups of the population relatively quickly, cheaply and using modern educational approaches and methods. Educause shows the key features and opportunities of personalised learn- ing in a video: https://www.youtube.com/watch?v=6olnlco0vfi. Of course, for some people, e-learning has drawbacks. The emphasis is on in- dependent learning, which is not everyone’s cup of tea – it can lead to learn- ing difficulties or problems with motivation. However, for those who cannot attend face-to-face lectures, e-learning is often the only option and solution. 328 Management in the Delivery of E-Learning In her PhD thesis, Penina Mungania from the University of Louisville, USA, used a sample of 847 individuals from companies with different activities to investigate what limits access to e-learning in the workplace at different stages of the learning process (at the start of the e-learning process, at the continuation and at the end). The survey highlighted the following as the most important factors hindering e-learning: personal characteristics of the learner, learning style, pedagogical approach, or- ganisation and context of learning, relevance of content and technologi- cal barriers. However, research has also shown that the impact of these factors varies between the organisations studied, mainly depending on the type of organisation, computer literacy and computer training, as well as the individual’s competencies in e-learning (Mungania, 2004). However, accessibility to e-learning can also be worse for people with disabili- ties. The spatial flexibility of e-learning also improves accessibility for them, but the use of computer technology in e-learning can bring new specific barriers, as well as opportunities for people with disabilities (these issues are discussed in Section 4.1 Learning Material and Section 7.4 Copyright and E-learning). In the US, an initiative to improve the accessibility of the web for people with disabilities was launched in the 1990s, developing web design guide- lines, tools, information resources, etc., tailored to their needs. Today, the Web Accessibility Initiative has grown into an international movement, the results of which can be viewed on the Web Accessibility Initiative website (http://www.w3.org/Wai/). Socio-economic status also has an impact on the accessibility of e-learning, as it typical y influences access to ICT and digital literacy. For example, the PI- AAC survey results show that Slovenia ranks above the international average of the frequency of ICT use at home and at work. However, differences in ICT use at home and at work are above international average between the more and the less educated in Slovenia (Dolničar et al., 2018). The older population also has specific requirements for access to e-learning. For older people, access to this education is usual y hampered by lower com- puter and digital literacy, as well as by some typical health problems (e.g., poor eyesight, reduced ability to concentrate and remember) and psychological in- hibitions (fear of the unknown, fear of damaging an electronic device or per- haps inadvertently allowing personal data to be misused). E-Learning Results ning R 329 Statistics Slovenia, in its annual survey on Internet usage https://pxweb.stat. si/SiStat/en/Podrocja/index/88/development-and-technology which covers the population aged 16–74, also collects data on how individuals subjectively assess their ability to use the Internet. In 2018, 60.5% of all people who had used the Internet in the last 12 months rated their ability to use the Internet as good or very good, while only 21.3% of older Internet users aged 65–74 rated their ability to use the Internet as good or very good. recommended links iSpring. How to Measure E-Learning ROI: From Smile Sheets to Hard Numbers: https://www.ispringsolutions.com/blog/how-to-measure-elearning-roi eLearning Industry. 5 Ways to Determine eLearning ROI: https://elearningindustry.com/ways-determine-elearning-roi Kp KIRKPATRICK PARTNERS. The One and Only Kirkpatrick: https://www.kirkpatrickpartners.com/ 330 Management in the Delivery of E-Learning 7.4 copyright and e-learning 7.4.1 the origins of copyright law One area that is taking on a whole new dimension with the use of modern technologies is copyright protection. The issue of copyright protection arose when the invention of the Gutenberg printing press made mass printing of written texts possible and the literacy of the population ensured sufficient readers. The first printers were also publishers. They took on all the costs of publishing and compensating authors, but they were not legal y protected, as others could reprint books in unlimited quantities and sell them cheaper. The new circumstances have raised the fundamental question of copyright pro- tection, namely how to strike a balance between encouraging and adequately rewarding the creativity of authors, adequate earnings for publishers and other contributors, and the accessibility of the works created for users. The regulation of copyright is even more important in modern societies be- cause copyright regulates access to information as one of the main resources for development. The Internet plays a central role in this, as it greatly increases the possibility of accessing information, but it also makes it possible to use this information in very different ways and for very different purposes. The wide accessibility and diversity of possible uses supported by modern technology gives authors new and better opportunities to promote their creations in a pro- fessional and commercial sense. At the same time, the risks of the illegal use of creations without adequate protection and of harm to authors are increasing. In the remainder of this section, we will first present the foundations of copy- right law, with an overview of its origins and international agreements, and the copyright regime in Slovenia. We will then briefly review the options for efficient copyright management in e-learning – taking into account the state of copyright protection for digital content and the current legislation, which does allow some copyright exceptions for education. The origins of copyright can be traced back to the so-called privileges that medieval rulers granted to individual printers as exclusive (monopoly) rights to print a particular book, to prevent cheap reprints by other printers. Such privileges only protected the interests of printers and cannot therefore be con- sidered a form of copyright. The first such privilege was issued in Venice as early as 1469 (Hofman, 2009, pp. 2–3). Copyright protection has only been an issue since 1709, when the Statute of Anne, also known as the Copyright Act, was published in England to address the need for the more comprehensive protection of authors and their works (Copyright Act). This gave authors the exclusive right to reproduce their books and maps for Copyright and E-Lear Cop ning 331 14 years. This period could be extended for a further 14 years if the author was still alive at the end of the first term. By the end of the eighteenth century, copy- right protection had spread to other European countries and the USA, and had a decisive influence on the growth of publishing, which today includes sound recordings, films, photographs, software and architectural works. 7.4.2 international conventions and agreements With economic development and increased international trade, the need for international copyright protection has also arisen, first in the form of bilat- eral agreements between signatories based on reciprocity. Citizens of the other country signing such an agreement are granted the same rights as nationals of the first country. Berne Convention However, the regulation of copyright at the level of bilateral agreements has not been effective. That’s why, in 1886, at the initiative of the French writer Victor Hugo and under the auspices of the International Association of Literati and Artists, representatives of ten countries met in Bern and adopted a special convention on copyright protection, called the Berne Convention.42 The basic text of this Convention has been amended several times, most re- cently in 1979 (Hofman, 2009, p. 14). Subsequently, the text was not amended, but the substantive amendments were included in separate agreements.43 The countries that are signatories to the Berne Convention form the Union for the Protection of Authors’ Rights in their Literary and Artistic Works.44 In 1967, at the Stockholm Conference, this Union merged with the Paris Union for the Protection of Industrial Property to form the new World Intellectual Property Organisation (WIPO). The fundamental principles of the Berne Convention are (Bahor, p. 4): • the principle of assimilation or national treatment: authors who are nation- als of other Member States enjoy the same protection as nationals and all the rights recognised by the Convention; 42 The first signatories of the Berne Convention were: France, Germany, the UK, Belgium, Italy, Spain, Switzerland, Haiti, Liberia and Tunisia. Yugoslavia signed the Berne Convention in 1928 and the US only in 1989. Slovenia ratified it in 1992. 43 The changes in copyright law brought about by the need to protect ICT-related creations and the new possibilities for copyright protection using ICT tools are regulated by a special WIPO Copyright Treaty. 44 According to the latest data from the WIPO homepage https://wipolex.wipo.int/en/treaties/ ShowResults?search_what=B&bo_id=7, the Union has 179 members. 332 Management in the Delivery of E-Learning • the principle of automatic protection: the enjoyment and exercise of rights are without special formalities; copyright is automatical y protected as soon as the work is created, i.e., when it is written or stored in a medium. No specific declaration or application for copyright protection by the author is required; • the principle of the independence of protection: obliges Members to protect au- thors’ rights in their laws at least to the extent provided for in the Berne Con- vention. Countries that have adopted the Convention can protect copyright to an even greater extent. For example, the Convention allows signatory coun- tries to extend the duration of the protection of rights. Protection is granted to authors who are nationals of or merely resident in the Member States, for both published and unpublished works. However, authors who are not nationals of a Member State only enjoy protection for works first published in a Member State. One of the fundamental principles of the Convention is that only the text of the Convention ratified by a signatory is binding on it. Universal Copyright Convention The Universal Copyright Convention is the second convention for the pro- tection of authors, initiated by UNESCO in Geneva in 1952 to complement the international copyright protection in those countries that had not signed the Berne Convention because of the large scope of copyright protection. The scope of protection is smaller than under the Berne Convention. Only the principles of independence of protection and national treatment are accepted. The duration of the protection is at least during the author’s lifetime and 50 years after the author’s death, or at least 25 years after the first publication of the work, or at least 25 years after the registration of the work, while protection under the Berne Convention lasts for 70 years after the author’s death. Although the Universal Copyright Convention is still in force, its importance has diminished. The main reason is that the Berne Convention, following the 1967 revision, has been approximated to the aforementioned Convention and is less binding, allowing Union Member States to restrict copyright protection under certain conditions45, while also allowing developing countries to restrict copyright protection to a certain extent.46 International TRIPS Agreement Copyright also affects international economic flows. The payment of copyright income to an author in another country (royalties and licence fees) is an inte- gral part of international trade in services and requires appropriate regulation. 45 These conditions are defined by the so-called three step test. The Berne Convention provides that European Union countries have the right to authorise reproductions of the works in question in their laws in certain specific cases, but only if such reproductions are not detrimental to the normal (ordinary) exploitation of the works and do not unduly prejudice the legitimate interests of the author (Xalabarder, 2004). 46 For an overview of the countries that are signatories to international copyright agreements, see Wikipedia. https://en.wikipedia.org/wiki/list_of_parties_to_international_copyright_agreements. Copyright and E-Lear Cop ning 333 To this end, the Marrakesh Declaration on the Agreement on Trade-Related Aspects of Intellectual Property Rights (TRIPS) was adopted in 1994. It obliges WTO members wishing to adopt TRIPS to adapt their national legislation to the Berne Convention beforehand. Marrakech Agreement to Facilitate Access to Published Works for Persons with Reading Disabilities Under the protection of the World Intellectual Property Organisation (WIPO), an agreement was adopted in Marrakech in June 2013 to facilitate access to published works for people who are blind, visual y impaired or have other reading disabilities. The agreement allows people with a reading disability to legal y use a copy of any published work to adapt it to their reading needs. Some use mainly printed books, others electronic books and others audio books. Adaptation of the medium to the needs of people with print-related disabilities is carried out in accordance with the Marrakesh Agreement, through an authorised national organisation defined by each country in its own national legislation. The agree- ment also supports the international exchange of copies of published work for people with print disabilities. However, any existing copyright of the owner of the work must be respected (Kačič and Zaviršek, 2013). 7.4.3 copyright in Slovenia Basic concepts and Provisions The Berne Convention and other international agreements are the cornerstone of copyright law in Slovenia. Slovenia also takes into account the relevant EU directives when legislating in this area. Copyright law in Slovenia is governed by the Act on the Protection of Copy- right and Related Rights (ZASP), which was adopted in 1995 (Uradni list Re- publike Slovenije, No 21/95) and subsequently amended several times, most recently in 2019. Below we present the basic concepts and provisions of copyright law in Slove- nia, with a particular focus on the provisions applicable to education. Copyright is a special form of intellectual property that ensures that the author can pursue the economic (material) and personal (moral) interests associated with the exploitation of the author’s work. Copyright arises by the creation of the work itself and no special registration procedure, such as for industrial property rights, is required. 334 Management in the Delivery of E-Learning The author is the natural person who created the copyright work. The author is presumed to be the person whose name, pseudonym or sign ap- pears in the work or in the publication of the work in the usual way. If the work does not contain such information, the person who publishes the work or the person who first published the work is deemed to be entitled to enforce copyright. If a work is created by several persons and is an indivisible whole, all joint authors have an indivisible copyright in the work. Each co-author must con- tribute at least to what is considered to be the creation that forms the basis of the copyright work. Technical assistance in the creation of a work does not constitute co-authorship. Copyright works are individual intellectual creations in the fields of literature, science and art, expressed in any form, unless otherwise provided for in the Copyright Act. Case law has developed additional criteria to help us decide whether a work is a work of authorship or not: • work must be a human creation, not the result of machines; • the work must come from a specific field of creativity (science, literature and art); • the work must be created in the ”spirit of the author” and must have a per- sonal touch (individuality); • the work must be perceptible to the human senses, and therefore it needs to be expressed either in a material medium or in other ways, such as at musi- cal events (Trampuž et al., 1997, p. 30). Typical works include: • spoken works (e.g., speeches, sermons, lectures), • written works (for example, belletrist works, articles, manuals, studies and computer programmes), • musical works with or without lyrics, • theatre, music theatre and puppetry, • choreographic and pantomime works, • photographic works and works made using a process similar to photogra- phy, • audio-visual works, • works of art, architecture, applied art and industrial design, • cartographic works, • presentations of a scientific, educational or technical nature (technical drawings, plans, sketches, tables, expert opinions, three-dimensional rep- resentations and other works of the same nature). Copyright and E-Lear Cop ning 335 Copyright does not include: • ideas, principles, discoveries, • official texts in the legislative, administrative and judicial spheres, • folk literary and artistic creations. types of copyright Copyright, which is a unitary right, gives rise to exclusive personality rights ( moral copyright), exclusive economic rights ( material copyright) and other rights of the author. Figure 63: types of copyright in Slovenia Copyright Related rights: • accessibility to the general public Moral rights: Material copyrights: Other copyright: • the right of first disclousure application of the work: • re-accessing for • the right of attribution • in a material form the purposes of • the right to respect for the work • in a non-material form reproduction • the right of withdrawal • in a modified form or processing • in the form of specimens (distribution and granting of use) Moral copyright is the spiritual link between the author and his/her work. Moral copyrights ensure that the author can exercise their moral interests in his work, even if he/she has transferred the right to exploit the work to another person. Moral copyright means that the author has an exclusive right: • to decide when and how his/her work will be published for the first time (right of first publication); • the right to the attribution of authorship of their work and the right to de- cide whether to attribute authorship when publishing their work (attribu- tion right); • to resist mutilation and any other interference with the work or any use of it (right to respect for the work); • to revoke the copyright in relation to the owner47, if it has serious moral rea- sons for doing so and if he/she first compensates the owner for the damage caused thereby (right of withdrawal or repentance). 47 The rightholder is the author, performer, phonogram producer, film producer, broadcasting organisation, publisher, database producer and any other person, other than a collecting society, to whom individual material copyright and other rights of the author or related rights have been transferred by law, contract or other legal transaction, or who is entitled to a royalty by virtue of a contract or other legal transaction or by law (https://www.zamp-zdruzenje.si/avtorji/). 336 Management in the Delivery of E-Learning Copyright belongs to the author by virtue of the creation of the work itself, so there is no need to go through any procedure for the work to be protected by cop- yright. Copyright lasts for the author’s lifetime and for 70 years after their death. Material copyright protects the authors’ economic interests and allows the au- thors to decide how to use their work. The authors will pursue their material interests by exploiting their work entirely themselves, or by leaving the exploi- tation of their work to someone else for a fee. Material copyright includes: • the use of the work in physical (object) form, in particular the reproduction right; • using the work in a non-corporeal form, such as: – the right of public performance, – the right of public transmission, – the right of the public performance of phonograms and videograms, – the right of public display, – the right to broadcast and retransmit broadcasting services, – the right of secondary broadcasting, – the right to make it available to the public; • the use of the work in an altered form, in particular the right of adaptation and the right of audio-visual adaptation; • the use of copies of the copyright work, in particular the right of distribu- tion and the right of rental. Other rights of the author allow the author to have access to his/ her work that has already been transferred to a third party, if this is necessary for the exercise of the reproduction right or the right to transform the work, and if this does not conflict with the legitimate interests of the copyright owner (the so-called right of access and delivery). In addition to copyright, the ZASP also deals with related rights (Article 118). Related rights are the rights of natural and legal persons who, through their contribution, make copyright works available to the public at large. Such rights are held by performers, producers of phonograms (such as CDs), film produc- ers, broadcasting organisations and publishers. A related right last, as a rule, for 50 years from its creation. The right to use (exploit) copyright works is usual y obtained through a copy- right contract concluded directly with the author or a collective society of au- thors representing authors. The law allows exceptions where the use of copyright works is free, i.e., without the author’s permission and without the payment of royalties to the author. Of particular relevance to the enforcement or respect of copyright in the field of Copyright and E-Lear Cop ning 337 education, including e-learning, is the provision in Article 49 of the Copyright Act that free use for educational purposes is permitted in the case of: • public performance of published works in the form of live classes; • public performances of published works at free school ceremonies, pro- vided that the performers do not receive payment; • secondary broadcasting of RTV (National Slovenian Radio and Television) school broadcasts. Such use must be accompanied by the source and the authorship of the work, if indicated on the work used. Reproduction of copyright works in up to three copies for private use by pri- vate individuals or for private use in certain public institutions is also free. To mitigate the damage suffered by authors, the law grants them a special flat-rate compensation, which is payable by the producers and importers of products that enable the mass copying and reproduction of copyright works. 7.4.4 copyright and e-learning In e-learning, we are faced with copyright issues even in the design of the pro- grammes themselves, when we decide whether to prepare the learning mate- rial ourselves or to use already prepared one, and also in the implementation of the e-learning programme itself, which usual y involves not only the use of our own learning material, but also the use of other sources of information and the sharing of material in the digital learning environment. Coursera offers several MOOCs on copyright, including Copyrights for Multimedia with 20,000 enrolments (https://www.coursera.org/learn/ copyright-for-multimedia) and Copyright for Educators & Librarians (https:// www.coursera.org/learn/copyright-for-education) with 16,000 enrolments. The courses are prepared by Duke University and Emory University. It is essential to address copyright issues as early and professional y as possi- ble in the development of an e-learning programme. Unprofessional licensing procedures, or even breaches of the law, cause harm to authors and users alike and can lead to the prohibition of use, especial y for expensive digital material. As we have shown in Parts 2 and 3 on the business, organisational and peda- gogical aspects of designing e-learning programmes, the selection of content and the structure of an e-learning programme, its delivery and business plan are interrelated elements that should not be considered in isolation. What strategy should be used by an organisation or author of an e-learning programme who wants to produce an e-learning programme of the highest 338 Management in the Delivery of E-Learning quality and cost-efficient while respecting international agreements or appli- cable copyright legislation? In fact, we have two routes: • obtain a licence from the copyright owner ; • use a work that is otherwise protected by copyright under exceptions al- lowed by international agreements and legislation that do not require copy- right to be obtained for certain purposes. Copyright exceptions (limitations) general y apply to: • use for teaching purposes, • citing works, • compilations (collections) of material, • use in public libraries and other public institutions. In any case, the second option is much more attractive for the organisation and the user, provided the conditions are met. Obtaining a licence often involves difficulties in identifying the author or copyright holder, lengthy copyright procedures and, of course, significant costs. When assessing the possibilities and limitations that we have to take into ac- count when using certain material with regard to copyright protection, we also have to take into account that international agreements mostly concern copy- right works on traditional (analogue) media. In May 2019, the Directive of the European Parliament and of the Council on Copyright in the Digital Single Market was adopted. It aims to bring copyright into line with the new forms and meth- ods of distribution of copyright works brought about by technological progress. The adaptation of the legislation to the new Directive is still pending in most EU Member States, including Slovenia. Below, we will briefly present the main developments in technological and legal measures for the protection of copyright in digital content48, and then we will look at what the new legislative developments have brought to e-learning and what the state of play is in this area in Slovenia. development orientations for copyright Protection for digital content Today, copyright legislation is developing in two main directions. The first is the development of technological measures and systems for Digital Rights Management (DRM). These are intended to give the author full control over the use of their works, in particular by preventing any infringement of the 48 For more on digital rights management issues, see Hofman (2009) and Bogataj Jančič (2008). Copyright and E-Lear Cop ning 339 use of a particular digital work. Because of the need for complete control over the use of digital works, this approach requires control over the communications (distribution) and computing facilities (copying) of the individual user. The problematic nature of this approach is particularly evident from a privacy per- spective. Due to its restrictive nature, it may have a negative impact on research and development in specific technologies and on the development of e-content. In practice, this approach has proved rather unviable (Doctorow, 2014). The second is the system of legal protection developed under Creative Commons (CC). The CC system offers pre-prepared licences that clearly define the permit- ted and unauthorised uses of authors’ works, so that works can circulate more freely among users. Marking copyright works with a CC licence does not mean that the author waives copyright. The system provides two fundamental rights: • using and modifying the work in a new creation, • sharing. The CC allows for certain limitations on these rights: use or exploitation for noncommercial purposes only, and the requirement that any new creation cre- ated by the author on the basis of an existing work must be shared under the same rights and conditions as the existing work (share alike). The latest version (4.0) allows CCs to use six different licences, depending on attribution, non- commercial, no derivative work and share alike options. Table 32: the creative commons license options Licence Features Symbol First Attribution + NonCommercial + No derivatives $ Second Attribution + NonCommercial + Sharing alike $ Third Attribution + NonCommercial $ Fourth Attribution +No derivatives Fifth Attribution + Share alike Sixth Attribution Source: Creative Commons (https://creativecommons.org/about/cclicenses/ ) 340 Management in the Delivery of E-Learning cc copyright The scope of the rights under each Creative Commons licence category is as follows: Attribution-NonCommercial-No derivatives CC BY-NC-ND This licence is the most restrictive, as it only allows the work to be used and shared with others, provided that attribution is given. The work may not be modified or used for commercial purposes. Attribution-NonCommercial-Sharing alike CC BY-NC-SA This licence permits the distribution, adaptation, modification and supple- mentation of the work for non-commercial purposes. The reworked work must credit the author of the original work; it must not be commercial and must be licensed under the same conditions. Attribution-NonCommercial CC BY-NC This licence permits the distribution, adaptation, modification and supple- mentation of the work for non-commercial purposes. The reworked work must credit the author of the original work; the work must not be commercial, but licensing under the same conditions is not compulsory. Attribution-No CC BY-ND This licence only allows you to use and share the work with others, with attri- bution. The work cannot be modified, but commercial use is allowed. Attribution-Sharing alike CC BY-SA This licence allows the distribution, modification and supplementation of the work, even for commercial purposes. In a reworked work, the author of the original work must be acknowledged; however, licensing is required under the same conditions. This licence is often compared to the ”copyleft” licences used in free and open- source programming (Free and Open-Source Software). All new works use the same licence as the original work, so even remakes are commercial y accept- able. This is the licence used by Wikipedia. Attribution CC BY This licence allows the distribution, modification and supplementation of the work, even for commercial purposes. The author of the original work must be acknowledged in the reworked work; licensing under the same conditions is Copyright and E-Lear Cop ning 341 not compulsory. Of all the licences available, this one give users the most free- dom. It is appropriate when the interest of the author is to use it as widely as possible and to share it as widely as possible. The CC concept is legal y compliant and based on copyright law, but limited by national law. The choice of combinations of rights and restrictions for your chosen licence is easy to understand, technology-enabled and available on the Internet. Authors can also create a licence on the Internet themselves, depending on what they want to make their work available or on the specific circumstanc- es in which they want to enable their work to be used and exploited (https:// creativecommons.org/choose/). On the Creative Commons website https://creativecommons.org/choose/ you will find clear guidance, in various languages, on how to choose the appropriate licence under this system. Today, over 2 billion CC-licensed works are available on millions of web- sites. The majority are hosted on content platforms that provide CC li- cense options for their users. In Figure 64, some of the best-known platforms for sharing CC content are presented. Content on these platforms is searchable and shareable across the web thanks to CC licenses. Figure 64: Some platforms for sharing cc content Source: Creative Commons Platforms (https://creativecommons.org/about/platform/ ). By setting rights and restrictions in advance, CCs allow users to exploit the work without the need for explicit permission. This is a major advantage for users, as obtaining licences to use digital works is time-consuming and com- plicated, sometimes impossible. 342 Management in the Delivery of E-Learning On the other hand, the possibility of multiple rights allows authors to make further use of the Internet as a medium to disseminate and promote their work, while still retaining the explicit right to decide on commercial use. So, the system gives authors more flexibility in how they choose to use their work, while at the same time making it easier for the community and society to use and exploit copyright works in accordance with the legislation copyright exceptions and e-learning Although the core of copyright law is the system of exclusive rights, which is designed to encourage creativity – to reward authors, copyright law does rec- ognise some limitations on copyright (exceptions to exclusive rights), (Jančič, 2008, p. 76). Copyright exceptions aim to create a balance between the inter- ests of the individual author and the wider societal interests in the free flow of information and the dissemination of knowledge. International conventions general y allow for copyright limitations, but countries have reserved the right to determine the areas to which these exceptions apply. One area where copyright exceptions are general y recognised is education. Article 10.2 of the Berne Convention states that State Parties to the Agree- ment may legislate to permit the use of literary and artistic works in the con- text of teaching for the purpose of il ustration by publication, broadcasting or sound and visual recording, provided that such use is in accordance with good practice. Since the adoption of the Berne Convention, it has been recognised that teaching not only encompasses teaching at the primary level, but also at higher levels and distance education (Xalabarder, 2004). However, the concept of use is not precisely defined and can be interpreted as mere reproduction or as transmission to the public. The possibility of transmission of the work to the public is essential for e-learning. A review of copyright legislation shows that the implementation of the edu- cational exceptions provision varies considerably across national laws (Tor- res and Xalabarder, 2019). While most countries allow certain exceptions for classroom teaching, they general y do not specify exceptions for e-learning, with a few exceptions such as Luxembourg, Portugal, France, Belgium, Italy and Germany (Kakoura, 2016, p. 22). Slovenia is among the countries that allow certain copyright exceptions for edu- cation but do not specifical y address e-learning. As we have already pointed out, Article 49 of the Copyright Act states that copyright works may be reproduced in reading materials and textbooks intended for teaching purposes without the transfer of the corresponding material copyright, but on payment of an ap- propriate remuneration. Parts of works of authorship and individual works in the fields of photography, fine arts, architecture, applied arts, industrial design and cartography may also be used for such purposes, provided that they are Copyright and E-Lear Cop ning 343 already published works by several authors or that the author has not expressly prohibited such use. The treatment of copyright exceptions for use in education in Slovenian leg- islation apparently applies only to direct instruction and to works of authorship published in traditional media, and is undefined for the purposes of e-learning, which primarily requires the right of transmission to the public. The new Directive on Copyright and Related Rights in the Digital Single Mar- ket (Official Journal of the EU, 2019) was expected to bring copyright into line with the new forms and methods of distribution of copyright works brought about by technological advances. At least for education, these expectations are not being met. Article 5 of the Directive provides for an exception to allow the free use of copy- right works in digital form for the purpose of il ustration in teaching. However, only on the premises of educational establishments and other premises for which they are responsible, and on secure electronic networks set up by the educational establishments (ibid.). This exemption is not mandatory for Member States, as it does not need to be introduced where easily accessible paid-for licences are available to educational institutions (MIZŠ, 2019). The exception defined in this way is, in the opinion of the MIZŠ in Slovenia, insufficient for education, as it could mean a new practice where schools and other educational institutions have to pay compensation to authors and other rights holders (e.g., publishers) for the use of copyright works in the class- room. What’s more, this Directive is also not in line with the concept and stage of development of e-learning. The position of the MIZŠ (2019) is that the educational exception should be as broad as possible and apply to all non-com- mercial educational activities of teachers and students as part of the learning process, which can take place in all environments (digital and non-digital) and in all formats (the use of works in all formats, in their entirety). From the point of view of e-learning, this is, on the one hand, an appropriate starting point for adapting copyright legislation in Slovenia to the new European Directive. At the same time, the copyright regime, as framed by the current European Directive, provides an additional incentive and opportunity for the wider use of the OER (Section 6.2). In any case, e-learning, especial y regarding the possibility of copyright excep- tions, deserves a more comprehensive and thorough treatment in copyright law and a corresponding amendment of the legislation. Such treatment should be based on a substantive analysis of the different forms and modes of use, processing and distribution of digital material, also considering the differences in the status of users. 344 Management in the Delivery of E-Learning In addition to the possibility of using copyright works in e-learning on the 7.5 Quality assurance in e-learning basis of a limitation of copyright for teaching purposes, the implications of other limitations of copyright, i.e., on the basis of citation, compilation (col- lection) of materials and use in public institutions, should be explored, taking 7.5.1 general aspects of the Quality in education into account the possibilities offered by the three-step test. The globalisation of e-learning and the use of the OER also raise specific issues, but due to their ubiquitous accessibility and openness, they cannot be considered as material Quality is one of the main strategic orientations in modern education. The intended for ”instruction” as copyright law provides for in recognising excep- quality of educational services is a generic concept; the understanding of qual- ity depends on the socio-political and cultural characteristics and institutional tions, since the use of the OER or MOOCs does not imply ”instruction” at al . contexts in which education takes place, from educational objectives to levels recommended links and forms of education. Notwithstanding the diversity of conceptions of the quality of educational ser- Creative Commons: vices, views on quality can be classified into two groups: http://creativecommons.org/ Creative Commons (in Slovene language): • quality as the set of service characteristics that meet the needs of the learners https://creativecommons.org/licenses/?lang=slhttp://www.ota.ox.ac.uk/ (i.e., expressed in terms of the level of satisfaction with the characteristics WIPO: of the educational service); • quality as the absence of imperfections and weaknesses or the conformity to http://www.wipo.int standards (this is the matching of the characteristics of educational services CCdigital aw.ch: to the requirements set out in standards or specifications). https://ccdigitallaw.ch/index.php/english/about The first aspect is at the heart of non-formal education. Quality is monitored through various methods of determining learner satisfaction, using a survey methodology to measure attitudes; and the quality of services is also moni- tored by various consumer protection agencies and associations. The second aspect relates to accreditation and quality assurance processes and is mainly established in formal education. Accreditation is a process in education that aims to ensure that higher education institutions and other organisations meet and maintain the minimum quality standards for academic, administra- tive and other education-related services (USNEI, 2001). The Glossary of Ba- sic Terms for Vocational and Professional Education defines accreditation as a quality assurance process that accepts an education or training programme and demonstrates that the programme has been formal y approved by the competent legislative or professional bodies on the basis of meeting predefined standards (Muršak, 2012, p. 98). Quality assurance is the planned and systematic review of an organisation or programme to determine whether the minimum standards for educational processes, academic content (programmes) and infrastructure are being achieved or improved (CHEA, 2001). According to the Law on Adult Education in Slovenia (Uradni list republike Slovenije, 2018, Articles 67, 68 and 69), ”public organisations providing adult education programmes and activities financed from public funds” are obliged to develop and assess quality. Quality as- sessment is about setting up an internal quality system (which includes ongoing monitoring and self-evaluation) and to participate in external evaluations. De- veloping quality is ”the planning and implementation of measures to maintain and develop quality” (ibid.). Quality Assurance in E-Lear y Assur ning 345 7.5 Quality assurance in e-learning 7.5.1 general aspects of the Quality in education Quality is one of the main strategic orientations in modern education. The quality of educational services is a generic concept; the understanding of qual- ity depends on the socio-political and cultural characteristics and institutional contexts in which education takes place, from educational objectives to levels and forms of education. Notwithstanding the diversity of conceptions of the quality of educational ser- vices, views on quality can be classified into two groups: • quality as the set of service characteristics that meet the needs of the learners (i.e., expressed in terms of the level of satisfaction with the characteristics of the educational service); • quality as the absence of imperfections and weaknesses or the conformity to standards (this is the matching of the characteristics of educational services to the requirements set out in standards or specifications). The first aspect is at the heart of non-formal education. Quality is monitored through various methods of determining learner satisfaction, using a survey methodology to measure attitudes; and the quality of services is also moni- tored by various consumer protection agencies and associations. The second aspect relates to accreditation and quality assurance processes and is mainly established in formal education. Accreditation is a process in education that aims to ensure that higher education institutions and other organisations meet and maintain the minimum quality standards for academic, administra- tive and other education-related services (USNEI, 2001). The Glossary of Ba- sic Terms for Vocational and Professional Education defines accreditation as a quality assurance process that accepts an education or training programme and demonstrates that the programme has been formal y approved by the competent legislative or professional bodies on the basis of meeting predefined standards (Muršak, 2012, p. 98). Quality assurance is the planned and systematic review of an organisation or programme to determine whether the minimum standards for educational processes, academic content (programmes) and infrastructure are being achieved or improved (CHEA, 2001). According to the Law on Adult Education in Slovenia (Uradni list republike Slovenije, 2018, Articles 67, 68 and 69), ”public organisations providing adult education programmes and activities financed from public funds” are obliged to develop and assess quality. Quality as- sessment is about setting up an internal quality system (which includes ongoing monitoring and self-evaluation) and to participate in external evaluations. De- veloping quality is ”the planning and implementation of measures to maintain and develop quality” (ibid.). 346 Management in the Delivery of E-Learning In Slovenia, the POKI (Offering Quality Education to Adults) pro- gramme, developed at ACS, has become a model for assessing and devel- oping quality in adult education (https://kakovost.acs.si/en). POKI is based on self-evaluation. It offers opportunities for educational organisations to decide on these issues: • to assess the quality of their own performance; • what parts of the education process or what effects will be monitored; • what quality standards they will set for themselves; • how they will treat the results of the evaluation; • what measures they will take on the basis of the assessments. The model is suitable for use in a wide range of adult education organisations (e.g., centres for adult education, secondary and higher education schools, pri- vate educational organisations). Self-evaluation using the POKI model can be carried out at the level of the entire educational organisation, a single pro- gramme or department, an educational group, or only by an individual teacher. The ACS has developed an online collection of recommendations, tools and good practices called Mozaik kakovosti (Quality Mosaic). The Qual- ity Mosaic is practical y oriented and summarises the achievements of the development work carried out at the ACS together with a number of educational organisations that have been involved in quality-related projects since 2001. The collection covers five topics: professional back- ground, embedding the quality system in the activities of the educational organisation, defining quality, assessing quality, and developing quality. It is designed primarily for use in adult education, but contains a number of recommendations, tools and good practices that can also be useful for educators in other areas of education (https://mozaik.acs.si/). 7.5.2 Quality assurance in e-learning Initial y, quality assurance in e-learning was primarily focused on demon- strating that e-learning is of the same quality as traditional education (Twigg, 2001). However, the concept that e-learning brings new dimensions of quality through innovative and creative use of technology (i., more personalisation and interaction in education) and can therefore be of even higher quality than traditional education is gaining ground. Globalisation processes are contributing to the growing importance of quality assurance in e-learning. Modern technology allows an e-learning programme Quality Assurance in E-Lear y Assur ning 347 developed in a local environment to be accessible without geographical re- strictions. An organisation offering a specific e-learning programme online may be thousands of kilometres away from the potential learner, operating in a completely different cultural and social environment, often on market prin- ciples alone, so quality assurance for the learner is essential. Quality assurance in e-learning requires different approaches and quality indi- cators compared to traditional education due to its specific features (flexibility of programme delivery in terms of time and space, openness of access and information resources, different forms of communication and support, etc.). Quality criteria such as the number of classroom hours, the size of the library stock and lecturing attendance are not appropriate for e-learning. Well-considered quality monitoring is also necessary because, unlike tradi- tional education, e-learning does not permit to directly monitor the behaviour of learners in the learning process due to spatial separation and teachers are general y not able to get immediate feedback directly from the learners. In the US, recommendations, guidelines and examples of quality e-learning practice began to be developed two decades ago. The Institute for Higher Edu- cation Policy, USA (http://www.ihep.org/) developed comprehensive guidelines for quality assurance in e-learning in 2000. E-learning has been called Internet-based distance education. Recommendations were made on how to ensure quality in the most important areas (institutional support, programme devel- opment, teaching/learning, programme design and structure, learner and staff support, evaluation and assessment). Bates has published a range of useful information on quality assurance activities in e-learning around the world, as well as a selection of research articles in this area, on his blog Online Learning and Distance Education Resources. https://www.tonybates.ca/?s=quality. The European Foundation of Quality in E-Learning (EFQUEL), a non-profit organisation with up to 120 members, ranging from universities to companies and international organisations, has played an important role in promoting quality in e-learning in Europe. EFQUEL has stimulated a number of success- ful initiatives and projects: • UNIQUe: accreditation for excellence in the use of ICT in higher educa- tion; • EFQUEL: The Forum – a community of all those interested in digital, open and innovative education; the Forum was one of the initiators of the Open Education Europa movement (for more on this movement, see Section 6.2 Open Education); 348 Management in the Delivery of E-Learning • SEEQUEL – a system of criteria for the quality of educational resources, processes and contexts; • SEVAQ+ – a self-assessment tool for e-learning and open education. Project SEQUENT is the result of col aboration between EFQUEL, the Euro- pean Association of Distance Teaching Universities (EADTU) and the Euro- pean Association for Quality Assurance in Higher Education (ENQA). The basic idea of this project is to promote excellence in higher education with a clear focus on preparing universities in Europe for change in line with the European Agenda for the Modernisation of Higher Education and for interna- tional cooperation in innovative and technology-driven partnerships. EADTU has also developed an online methodology for assessing the quality of e-learning, E-xcellence, which contains several tools (quick self-assessment and complete assessment questionnaires, a handbook, a glossary). The third edition of this manual was published in 2016 (EADTU, 2016). It covers the pedagogical, technical and organisational aspects of e-learning, paying par- ticular attention to accessibility, flexibility and interactivity. In 2014, the EADTU, in col aboration with partner institutions, published the OpenupEd Quality Label, which, in conjunction with the E-xcellence method- ology sets quality criteria for MOOCs (Rosewell and Jansen, 2014). The quality criteria for MOOCs are: • openness to learners, • digital openness, • learner-centred approach, • independent learning, • interacting through the media, • recognition options, • a focus on quality, • diversity of target groups. A recent and particularly high-profile development achievement is the com- puter-based authentication and authorship tool for online assessment ”An Adaptive Trust-based e-Assessment System for Learning” (TeSLA) https://tesla- project-eu.azurewebsites.net/. The project was led by ENQA under the Horizon 2020 programme, with seven European universities testing the tool’s usability. TeSLA also includes a module for the quality assurance of assessment in online and blended learning; taking into account the specificities of these forms of education, it is aligned with the standards for quality assurance in European higher education. Standards and Guidelines for Quality Assurance in the Eu- ropean Higher Education Area – ESG), (Foerster et al., 2019). Similar objectives, i.e., guidelines for quality assurance in e-learning, taking into account the existing ESG guidelines, are followed in a document pub- Quality Assurance in E-Lear y Assur ning 349 lished by the ENQA Quality Assurance Working Group in 2018 (Huertas et al., 2018). The document defines standards, describing the content of these stand- ards, and sets out quality indicators by quality assurance domains, specifical y for internal quality assurance in higher education organisations and specifi- cal y for external quality assurance, for national quality agencies. The domains of internal and external quality assurance are shown in Table 33. Table 33: enQa’s internal and external quality assurance domains Internal quality assurance External quality assurance Quality assurance policy Consideration of internal quality assurance Programme design and approval Designing methodologies fit for purpose Student-centred learning, teaching and Implementing processes assessment Admission (enrolment), progression, Peer-review experts recognition and certification Teaching staff Criteria for outcomes Learning resources and student support Reporting Information management Complaints and appeals Public information Ongoing monitoring and periodic review of programmes Cyclical external quality assurance Source: Huertas et al., 2018. A review of activities in the area of developing guidelines and recommenda- tions for quality assurance in e-learning shows that significant progress has also been made in Europe, particularly in the development of standards and guidelines for higher education. However, the consideration of the specificities of e-learning quality in the work of national quality agencies is still limited. Even though there is a growing number of educational institutions in Europe offering formal education programmes in some form of e-learning, according to the results of the 2014 EUA survey, only 23% of National Agencies have taken into account the specific quality requirements for e-learning in their work (Huertas et al., 2018). Slovenia was not among them. 350 Management in the Delivery of E-Learning 7.5.3 domains of Quality assurance in e-learning A review of the literature on quality assurance in e-learning shows some dif- ferences in the definition of the domains that should be subject to quality as- surance. However, a closer comparison of the domains and their classification shows that the core domains addressed by the quality recommendations are fairly uniform and follow the phases of the ADDIE model (Huertas et al., 2018; EADTU, 2016; Bates, 2016). These areas are: • e-learning planning, • development of e-learning, • e-learning management, • pedagogical support, • quality assurance and evaluation. e-learning Planning This involves examining the features of the organisation that provide appropri- ate conditions for the development of a quality e-learning programme. This includes: • a strategy with a clear definition of the organisation’s vision and mission, • the place of e-learning in the organisation’s strategy, • the financial and marketing aspects of developing and delivering e-learning, • analysis of learners’ needs, • staff development strategy, • a technology development plan. development of e-learning Many authors point to the importance of standards in the design and develop- ment of e-learning programmes. Standards usual y contain principles for: • setting the objectives and defining the content of the programme, • producing learning material and other learning tools and aids for the pro- gramme, • integrating educational media and tools into the programme, • assessment and evaluation of knowledge and comptencies, • providing pedagogical, administrative and technical support to learners. Lynch and Roecker (2007, p. 121) recommend other documents in addition to the standards for quality assurance, such as manuals, checklists, flowcharts and other templates, user guides, well-chosen examples and proven methodologies. Quality Assurance in E-Lear y Assur ning 351 The Mozaik kakovosti (Quality Mosaic) online database contains a wealth of useful information for a variety of activities related to the quality of edu- cation, from how to prepare a plan for discussing a self-evaluation report or conducting a SWOT analysis, to recommendations for designing survey questions and guidelines for creating a quality charter. Around 50 docu- ments are available, complemented by information on the scope of the tool and selected indicators (https://mozaik.acs.si/pripomocki/seznam-vseh). Figure 65: excerpt from the website Mozaik kakovosti (Quality Mosaic) Source: ACS, Mozaik kakovosti (Quality Mosaic) (https://mozaik.acs.si/ ). e-learning Management For management, the only successful e-learning programme is one that re- sponds quickly and immediately to educational needs and can therefore be delivered in a short timeframe, is cost-efficient and of high quality. Rosenberg (2001) discusses the quality of the management in e-learning more broadly, as cost-efficiency quality of programmes, service delivery and speed or flexibility of the organisation. Quality assurance from a cost-efficiency perspective means managing all the costs incurred in the development and delivery of programmes. E-learning brings a different cost structure in terms of the relationship between investment and run- ning costs, between fixed and variable costs, and the cost centres are also differ- ent. Unlike traditional education, effective management brings savings primarily to the learner and, to a lesser extent, to the educational organisation. 352 Management in the Delivery of E-Learning Development costs are usual y significantly higher than for traditional educa- tion, and staff costs cannot be particularly reduced. Rosenberg points out that e-learning benefits employers most by reducing opportunity costs (less time lost through absenteeism, travel, etc.). For more information on the costs of e-learning, see Section 2.2 Creating a Business Plan. Quality management must provide opportunities to achieve the key objectives of increasing the knowledge and competencies of learners, which also con- tributes to the efficiency and effectiveness of the organisation itself. Achieving these objectives depends most directly on the quality of the programme and the quality of the pedagogical support. These two aspects of quality are present- ed in some detail later in this section, as they are influenced by factors other than management (adequacy of staff, technical infrastructure, etc.). The quality management of e-learning must also provide appropriate services. This quality area focuses on the service provided to learners, which can be divided into three parts in terms of time: • pre-enrolment and early learning services for learners (information and presentation of the programme, financial assistance options, enrolment- related information, identification of the learner’s needs, etc.); • support during the programme (registration for exams, access to various materials, libraries, organising different types of support); • post-programme support (advice on how to continue education, or advice and support on finding a job). Pedagogical Support Pedagogical support includes learning materials and resources, as well as tu- toring support. Particular attention is paid to the quality of learning materials. Quality elements typical y include: • relevance to the learning objectives, • professionalism and relevance, • technical and design adequacy, • the compatibility of the digital learning environment with other IT systems and tools, • ease of use and reliability, • interactivity, • the possibility of personalising learning material. When developing learning material and e-learning tools, we also need to take into account some of the regularities of how people interact with the user in- terface of a website. Quality Assurance in E-Lear y Assur ning 353 Nielsen, an expert in website usability evaluation, recommends heuristic evalu- ation as an effective, cheap and simple method for evaluating the usability of websites. Heuristic evaluation is the most popular method of assessing the us- ability of websites, identifying the design weaknesses of such sites in terms of ease of use and effectiveness. A heuristic evaluation is characterised by a small group of evaluators examining the user interface of a website and assessing its compliance with accepted principles (Nielsen, n.d.). Managing the spatial separation of the learner is crucial for the quality of the e-learning programme. The most important principle in this area is to encour- age and facilitate interaction between learners and teachers or tutors, as well as between the learners themselves. Of course, some elements of the quality of the delivery of an e-learning programme are the same as for a traditional educational programme (i.e., the professionalism of the teachers, their organi- sational competencies to deliver the educational process, encouraging positive attitudes and giving feedback). The quality of teachers’ or tutors’ work therefore depends on their profession- alism and their organisational, motivational and communication competen- cies, as well as their mastery of the digital learning environment. 7.5.4 Quality assurance and evaluation The fundamental process by which we ensure that quality is achieved and maintained is called evaluation. Evaluating the quality of evaluation refers to the ways and processes by which we determine whether and how evaluation in an organisation is an integral part of the organisation’s educational process. Merisotis and Phipps (2000) distinguish three ways of conducting evaluation. We are evaluating: • the educational effectiveness of the programme/learning process through an evaluation process using a variety of methods and standards; • by relying on analysis of data on enrolment, costs, performance and inno- vative use of technology; • by reviewing the intended learning outcomes and objectives to determine the clarity, the relevance and appropriateness of the educational programme. Of course, simply identifying the areas in which we will achieve and maintain quality cannot guarantee that the education will in fact be of high quality. We need to choose evaluation mechanisms and methods to evaluate and assess the quality of our e-learning and ensure that we implement appropriate improve- ment actions based on the evaluation findings.49 49 The basic evaluation procedures are described in the handbook ”The Essentials of E-Learning”, pp. 276-300 (Bregar et al., 2010). 354 Management in the Delivery of E-Learning 7.5.5 Quality and development of e-learning Quality assurance in a context where learning materials are not fixed in ad- vance, where learning processes are very different, both within and outside formal education, and where they can be adapted to the individual needs of each learner, raises entirely new questions. The quality of e-learning 2.0 can no longer be assured by traditional meth- ods of expert evaluation and managerial approaches, but more participatory methods focused on assessing the quality of individual learning processes are needed (Ehlers, 2009, p. 137). The following table shows the differences between e-learning 1.0 and e-learn- ing 2.0 in terms of quality assurance areas. Table 34: Quality assurance areas for e-learning 1.0 and 2.0 Quality assurance area E-learning 1.0 E-learning 2.0 Experts Learners and peers Digital learning environment Digital personal learning environment Contents Content designed by the learner Curriculum Learner diaries and portfolios Duration of direct learning activities (in class) Communication Availability of tutors Interaction Multimedia (interactive) Social networks and interest groups Acquiring knowledge Cooperation and communication). Source: Ehlers, 2009, p. 137. Assuring the quality of e-learning 2.0 is even more important than assuring the quality of e-learning 1.0. In e-learning 1.0, quality assurance is limited to quality control, whereas quality assurance in e-learning 2.0 is becoming an essential factor in the effectiveness of learning processes. Teaching and learn- ing methods and quality development are closely linked. Quality monitoring methods such as self-evaluation, feedback and group reflection are becoming an integral part of the learning process, while external standards and tests are used less and less (Ehlers, 2009, p. 139). The development of Web 3.0, with the emergence of so-called big data and learning analytics, opens up a new dimen- sion of quality assurance, namely real-time quality assurance. Quality Assurance in E-Lear y Assur ning 355 recommended links ACS. POKI: https://www.acs.si/en/projects/national/offering-quality-education-to-adults/ ACS. Mozaik kakovosti (Quality Mosaic): https://mozaik.acs.si/pripomocki/seznam-vseh ENQA: https://enqa.eu/ Defining quality and Online Learning (by Tony Bates): https://www.tonybates.ca/2022/03/24/defining-quality-and-online-learning/ List of figures Figure 1. The extent and degree of integration of technological support at different forms of e-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 22 Figure 2. Classification of e-learning according to the integration of ICT in communication and learning content . . . . . . . . . . . . . . . . . 24 Figure 3. The evolutionary stages of the web and e-learning . . . . . . . . . . . . 41 Figure 4. Strategic objectives of the UK Open University . . . . . . . . . . . . . . . 51 Figure 5. Strategy implementation process . . . . . . . . . . . . . . . . . . . . . . . . . . . 54 Figure 6. Implementing the e-learning strategy . . . . . . . . . . . . . . . . . . . . . . . 58 Figure 7. Basic stages of the ADDIE model . . . . . . . . . . . . . . . . . . . . . . . . . . . 81 Figure 8. Instructional design with the Dick & Carey model . . . . . . . . . . . . 84 Figure 9. Elements of an Agile planning model . . . . . . . . . . . . . . . . . . . . . . . 86 Figure 10. Community of Inquiry Framework . . . . . . . . . . . . . . . . . . . . . . . . . 93 Figure 11. Framework of an instructional design for e-learning . . . . . . . . . 106 Figure 12. Example of a spreadsheet storyboard . . . . . . . . . . . . . . . . . . . . . . . 120 Figure 13. Example of a slideshow storyboard . . . . . . . . . . . . . . . . . . . . . . . . 120 Figure 14. Summary report of the analysis of the questions for the pilot test . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 139 Figure 15. eNefiks user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 142 Figure 16. Differences between traditional and technology-assisted assessment . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 146 Figure 17. Screen image for people with dyslexia and dysgraphia . . . . . . . . 148 Figure 18. Smart Sparrow Toolbox . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 149 Figure 19. Maths 4 homework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 150 Figure 20. The role of the media in the communication process . . . . . . . . . 153 358 Figure 21. 100 top-rated digital tools for learning in 2022 . . . . . . . . . . . . . . 161 Figure 22. Basic types of support in e-learning . . . . . . . . . . . . . . . . . . . . . . . . 174 Figure 23. Pedagogical support in e-learning . . . . . . . . . . . . . . . . . . . . . . . . . 175 Figure 24. E-Moderating Model . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 182 Figure 25. Components of open education . . . . . . . . . . . . . . . . . . . . . . . . . . . 193 Figure 26. Open Education Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 195 Figure 27. Number of MOOCs in the period between 2012 and 2018 . . . . 207 Figure 28. Coursera, Learning How to Learn, McMaster University, University of California San Diego . . . . . . . . . . . . . . . . . . . . . . . . . 212 Figure 29. Links in a cMOOC; provider’s view . . . . . . . . . . . . . . . . . . . . . . . . 213 Figure 30. cMOOC ”Creativity & Multicultural Communication”: a visualization of the participants’ activities . . . . . . . . . . . . . . . . . . 214 Figure 31. Determinants of xMOOCs and cMOOCs . . . . . . . . . . . . . . . . . . . 215 Figure 32. Plaque to the pioneers of artificial intelligence at Dartmouth . . 226 Figure 33. Overview of artificial intelligence methods . . . . . . . . . . . . . . . . . . 228 Figure 34. The guidelines as a framework for trustworthy AI . . . . . . . . . . . . 233 Figure 35. General framework for learning analytics . . . . . . . . . . . . . . . . . . . 238 Figure 36. The steps (stages) of the learning analytics process model . . . . . 240 Figure 37. Components of the integrated student database for the learning analytics . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 241 Figure 38. Gartner classification of analytics complexity . . . . . . . . . . . . . . . . 244 Figure 39. Screenshot of the Signals warning system . . . . . . . . . . . . . . . . . . . 245 Figure 40. Screenshot of the Check My Activity tool showing statistics . . . 247 Figure 41. Personalised and adaptive learning . . . . . . . . . . . . . . . . . . . . . . . . 249 Figure 42. Example of an AutoTutor user window . . . . . . . . . . . . . . . . . . . . . 255 Figure 43. MATHiaU . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 256 Figure 44. Personal assistant prototype for lifelong learning. . . . . . . . . . . . . 257 Figure 45. Characteristics and contextual aspects of m-learning . . . . . . . . . 265 Figure 46. Average browsing session duration in different contexts . . . . . . 266 Figure 47. Frequency of use of different microlearning modes . . . . . . . . . . 271 Figure 48. Screenshot of the SLC portal . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 275 359 Figure 49. The process of development of gamification . . . . . . . . . . . . . . . . . 279 Figure 50. The World of Math . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 280 Figure 51. TalentLMS Learning Environment ”Dashboard” . . . . . . . . . . . . . 282 Figure 52. SimWriter user interface . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 288 Figure 53. Using VR in astronaut training . . . . . . . . . . . . . . . . . . . . . . . . . . . . 291 Figure 54. AR using a phone app . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 292 Figure 55. Google Cardboard . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 295 Figure 56. Twelfth Night or Whatever You Want (video screenshot). . . . . . 299 Figure 57. Educational Uses of Digital Storytelling web portal . . . . . . . . . . . 300 Figure 58. Factors of the development and delivery of e-learning (e-learning framework factors) . . . . . . . . . . . . . . . . . . . . . . . . . . . . 307 Figure 59. Information about academic performance in the Blackboard learning environment (example) . . . . . . . . . . . . . . . . . . . . . . . . . . 313 Figure 60. Europeana Collections: Latest Stories, Screenshot . . . . . . . . . . . . 315 Figure 61. Kirkpatrick’s evaluation model . . . . . . . . . . . . . . . . . . . . . . . . . . . . 321 Figure 62. The New World Kirkpatrick Model . . . . . . . . . . . . . . . . . . . . . . . . 324 Figure 63. Types of copyright in Slovenia . . . . . . . . . . . . . . . . . . . . . . . . . . . . 335 Figure 64. Some platforms for sharing CC content . . . . . . . . . . . . . . . . . . . . 341 Figure 65. Excerpt from the website Mozaik kakovosti (Quality Mosaic) . . 351 List of tabLes Table 1. International comparison of Internet use for education, 2017 (Percentage of the total population aged 16–74) . . . . . . . . . . . . . . . 38 Table 2. Associated terms with Web 4.0 paradigm in the literature 2009–2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 44 Table 3. Concepts and technologies in the evolutionary stages of the web and e-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 47 Table 4. Types of costs by source of incurrence for the main phases of e-learning development and delivery . . . . . . . . . . . . . . . . . . . . . . . . . 63 Table 5. Average spending (in hours) per hour of training for different training modes, 2017 . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 64 Table 6. A sample example of the elements of a business plan in e-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75 Table 7. Comparison of the ADDIE model and the Agile planning model . 87 Table 8. Analysis of the characteristics of participants in the Food Security Assessment Programme . . . . . . . . . . . . . . . . . . . . . . . . . . . 103 Table 9. Bloom’s taxonomy of cognitive levels with a description of the learning objectives . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 109 Table 10. Theoretical background for designing learning objectives . . . . . . 110 Table 11. Absorb-type, do-type and connect-type learning activities . . . . . 125 Table 12. Reasons for testing knowledge and skil s . . . . . . . . . . . . . . . . . . . . . 132 Table 13. Bloom’s original taxonomy of cognitive learning objectives . . . . 133 Table 14. Bloom’s revised taxonomy of cognitive learning objectives, with a description of the characteristic categories . . . . . . . . . . . . . 135 Table 15. Bloom’s revised digital taxonomy . . . . . . . . . . . . . . . . . . . . . . . . . . . 136 Table 16. Verbal descriptions of learning objectives . . . . . . . . . . . . . . . . . . . . 140 362 Table 17. Example of a rubric . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 144 Table 18. Comparison of human and technology-based education . . . . . . . 158 Table 19. Example of comparing the characteristics of authoring tools . . . . 170 Table 20. Flexibility of education in time and place . . . . . . . . . . . . . . . . . . . . 176 Table 21. Comparison of the teacher’s role in traditional education and the tutor’s role in e-learning . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 177 Table 22. Roles of learners in traditional education and e-learning . . . . . . . 179 Table 23. Comparison of the competencies needed to teach in a traditional and virtual classroom . . . . . . . . . . . . . . . . . . . . . . . . . . . 181 Table 24. Learning and teaching concepts and innovative teaching and learning approaches and methods to realise the learner- centred paradigm . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 190 Table 25. General open education framework: definitions and components . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 196 Table 26. Components of the transversal dimensions of the Open Education Framework . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 199 Table 27. An overview of the main features of some typical MOOCs . . . . . 216 Table 28. The most important MOOC portals and their characteristics . . . 219 Table 29. The world’s largest providers of MOOCs . . . . . . . . . . . . . . . . . . . . . 220 Table 30. Example of the definition of the critical dimensions of LA framework for the SNAPP tool . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 239 Table 31. The seven elements of digital storytelling . . . . . . . . . . . . . . . . . . . . 297 Table 32. The Creative Commons license options . . . . . . . . . . . . . . . . . . . . . . 339 Table 33. ENQA’s internal and external quality assurance domains . . . . . . 349 Table 34. Quality assurance areas for e-learning 1.0 and 2.0 . . . . . . . . . . . . . 354 LitEraturE Alberta (2002). Instructional Strategies. Alberta Learning. Health and Life Skil s Guide to Implementation. Alberta . https://education.alberta.ca/media/482311/ is.pdf [accessed: 13 November 2018] Aldridge S. C. and Powel , M. (2018). The Online Classroom of the Future. Drex- el Online University. https://upcea.edu/wp-content/uploads/2018/08/Classroom-of- the-Future-Whitepaper.pdf [accessed: 8 August 2018] Alexander, B., Ashford-Rowe, K., Barajas-Murphy, N., Dobbin, G., Knott, J., Mc- Cormack, M., Pomerantz, J., Seilhamer, R. and Weber, N. (2018). Edu- cause Horizon Report: eLearning Trends and Predictions for 2018. https:// library.educause.edu/resources/2018/8/2018-nmc-horizon-report [accessed: 10 May 2019] Alexander B., Rowe K.A., Barajas-Murphy N., Dobbin G., Knott J., McCor- mack M., Pomerantz, J., Seilhamer R., Weber, N. (2019). Educause Ho- rizon Report: 2019 Higher Education Edition. https://library.educause.edu/ resources/2019/4/2019-horizon-report [accessed: 21 June 2019] Allen, I. E. and Seaman, J. (2014). Grade change: Tracking online education in the United States. http://www.onlinelearningsurvey.com/reports/gradechange.pdf [accessed: 19 December 2017] Allen, I. E. and Seaman, J. (2016). Online Report Card: Tracking Online Ed- ucation in the United States, Babson Survey Research Group. https://eric. ed.gov/?id=ED572777 [accessed: 13 September 2017] Allen, M. W. (2012). Leaving ADDIE for SAM. An Agile Model for Developing the Best Learning Experiences. ADTD Press . Allen, M. W. (2016). Michael Al en’s guide to e-learning: Building interactive, fun, and effective learning programs for any company (2nd ed.). Hoboken, New Jersey: Wiley. 364 Almeida, F. (2017). Concept and Dimensions of Web 4.0. International Journal of Computers and Technology. 16(7). https://doi:10.24297/ijct.v16i7.6446 Andrilović, V., Matijević, M., Pastuović, N., Pongrac, S. and Špan, M. (1985). Andragogija. Zagreb: Školska knjiga. Alqurashi, E. (2017) Microlearning: A Pedagogical Approach for Technology Inte- gration. https://www.researchgate.net/publication/319715909_Microlearning_a_a Pedagogical_Approach_For_Technology_Integration [accessed: 21 June 2019] Andersen, I. and Tisdel , E. (2016). Ghost and the machine: bringing un- told personal spiritual and cultural experiences to life through the me- dium of digital storytelling. Adult Education Research Conference. https:// newprairiepress.org/aerc/2016/papers/2 [accessed: 21 June 2019] Anderson, J. (2010). ICT Transforming Education. A Regional Guide. Bangkok: UNESCO. https://unesdoc.unesco.org/ark:/48223/pf0000189216 [accessed: 12 November 2019] Anderson, T. and Elloumi, F. (2004). Theory and Practice of Online Learning. Canada: Athabasca University. Anderson, T., Rourke, L., Garrison, D. R. and Archer, W. (2001). Assessing teaching presence in a computer conferencing context . Journal of Asynchro- nous Learning Networks, 5(2), 1–17. Andragoški center Slovenije (2014). Resolution on the National Programme for Adult Education in the Republic of Slovenia for the period 2013–2020 (ReN- PIO 13–20). https://arhiv.acs.si/dokumenti/reNPiO_2013–2020.pdf [accessed: 20 September 2018] Andragoški center Slovenije (2018). Offer of educational programmes for the elderly. Internal material. Arnold, K. E. and Pistilli, M. D. (2012). Course Signals at Purdue: Using Learn- ing Analytics to Increase Student Success. In Proceedings of the 2nd Inter- national Conference on Learning Analytics and Knowledge (LAK12). Vancouver, BC, Canada. New York: ACM. Arroway, P., Morgan, G., O’Keefe, M. and Yanosky, R. (2016). Learning Analyt- ics in Higher Education. Educause. Research report. Louisville https://library. educause.edu/-/media/files/library/2016/2/ers1504la.pdf [accessed: 12 March 2018] 365 Atkins, D. E., Brown J. S. and Hammond A. L. (2007). A Review of the Open Educational Resources (OER) Movement: Achievements, Challenges, and New Opportunities. The Wil iam and Flora Hewlett Foundation. https:// oerknowledgecloud.org/sites/oerknowledgecloud.org/files/reviewoftheOErMovement.pdf [accessed: 6 February 2018] Bahor, S. (2002). Knjižničarji in avtorske pravice. Knjižnica, 46(4), 87–123. http://revija-knjiznica.zbds-zveza.si/lzvodi/K0204/bahor.pdf [accessed: 1 February 2010] Baller, S., Dutta, S., Lanvin, B. (2016). The Global Information Technology Re- port 2016. Innovating in the Digital Economy. World Economic Forum, IN- SEAD. Johnson Cornell University . http://www3.weforum.org/docs/Gitr2016/ WEF_Gitr_Full_report.pdf. [accessed: 10 June 2019] Banks, J., Carson, J.S., Nelson, B. L. and Nicol, D. M. (2009). Discrete-Event System Simulation (5th ed.). Prentice Hal , Upper Saddle River. Bassi, L. (2000). How Much Does E-Learning Cost? Line Zine. http://www.linezine. com/2.1/features/lbhmec.htm [accessed: 20 July 2019] Bates, T. (2013). Examining the potential and reality of open educational resourc- es: the 2013 COHERE conference. Online learning and distance education resources. https://www.tonybates.ca/2013/10/26/examining-the-potential-and- reality-of-open-educational-resources-the-2013-cohere-conference/ [accessed: 20 July 2021] Bates, T. (2015). What do We Mean by ’Open’ in Education? Online Learning and Distance Education Resources. https://www.tonybates.ca/2015/02/16/what- do-we-mean-by-open-in-education/ [accessed: 12 April 2018] Bates, A. T. (2016). Teaching in a Digital Age. Guidelines for Designing Teaching and Learning. Tony Bates Associates Ltd Vancouver BC https://opentextbc.ca/ teachinginadigitalage/ [accessed: 13 June 2018] Bates, T. (2017). National Survey of Online and Distance Education in Cana- dian Post-Secondary Education. eCampus Ontario https://www.newswire.com/ files/e8/b0/f52d2613bf54ec6b35a454a344a0.pdf [accessed: 13 February 2018] Bates, T. (2019). Teaching in a Digital Age. Version 2. Guidelines for Design- ing Teaching and Learning. Tony Bates Associates Ltd. Vancouver https:// opentextbc.ca/teachinginadigitalage/ [accessed: 13 December 2019] Bates, T. and Poole, G. (2003). Effective Teaching with Technology in Higher Education. Foundations for Success. San Francisco: Jossey-Bass. Baule, S. (2019, October 9). From LMS to NGDLE: the Acronyms of the Future of Online Learning. ECampus News. https://www.ecampusnews.com/2019/04/09/ lms-ngdle-future-online-learning/ [accessed: 16 October 2022] 366 Becker A. S., Cummins, M., Davis, A., Freeman, A., Hall Giesinger, C. and Ananthanarayanan, V. (2017). NMC Horizon Report: 2017 Higher Educa- tion Edition. Austin, Texas, New Media Consortium. http://www.nmc.org/ publication/nmc-horizon-report-2017-higher-education-edition/ [accessed: 10 January 2018] Becker, S. A., Brown, M., Dahlstrom, E., Davis, A., DePaul, K., Diaz, V. and Po- merantz, J. (2018). NMC Horizon Report 2018. Higher Education Edition . Higher Education Edition. Louisvil e, CO: Educause. https://library.educause. edu/resources/2018/8/2018-nmc-horizon-report [accessed: 20 January 2019] Beetham, H. and Sharpe, R. (2007). Rethinking Pedagogy for a Digital Age. New York, NY: Routledge. Beguš, R. (2015). Raziskovanje pogojev za uvedbo e-izobraževanja v podjetju Pošta Slovenije, d.o.o. Master’ s thesis. Maribor: DOBA Faculty https://www. doba.si/ftp/diplome/1415360032.pdf [accessed: 10 October 2019] Berking, P. (2016). Choosing Authoring Tools. Advanced Distributed Learning (ADL) Initiative. https://adlnet.gov/assets/uploads/Choosingauthoringtools.docx [accessed: 5 October 2019] Berking, P. and Gal agher, S. (2016). Choosing a Learning Management Sys- tem. Advanced Distributed Learning (ADL) Initiative. https://adlnet.gov/ publications/2016/11/Choosing-a-Learning-Management-System-LMS/ [accessed: 17 November 2019] Billinghurst, M. and Duenser, A. (2012). Augmented Reality in the Classroom. Computer, 45(7), 56–63 https://doi:10.1109/MC.2012.111 BizMOOC (2018). Existing MOOC Business Models BizMOOC- Knowledge Al- liance to Enable a European-wide Exploitation of the Potential of MOOCs for the World of Business. Discussion paper 08, Programme: Erasmus. https:// bizmooc.eu/wp-content/uploads/2018/10/BizMOOC-r1.1-paper-08-MOOC-business- models-updated.pdf [accessed: 20 July 2019] Blackboard (2016). Blackboard Intel igence Helped Lewis Clark Community Col ege Develop a Data-Driven Retention Strategy. https://www.blackboard. com/sites/default/files/resource/pdf/case_study-student-lewis%26clark.pdf [accessed: 13 September 2022] Blaschke, L. M. (2012). Heutagogy and lifelong learning: A review of heuta- gogical practice and self-determined learning. The International Review of Research in Open and Distributed Learning, 13(1), 56–71 https://doi. org/10.19173/irrodl.v13i1.1076 367 Blascovich, J. and Bailenson, J. (2005). Immersive Virtual Environments and Education Simulations. In S. Cohen, K. E. Portney, D. Rehberger and C. Thorsen (eds.), Virtual Decisions: Digital Simulations for Teaching Rea- soning in the Social Sciences and Humanities (pp. 229–253). Mahwah, N.J: Routledge. Bloom, B. S. (1956). Taxonomy of educational objectives. The classification of educational goals: Cognitive domain. New York: Longman. Bogataj Jančič, M. (2008). Avtorsko pravo v digitalni dobi. Ljubljana: Pasadena http://www.pasadena.si/knjigarna/?id=52843 [accessed: 2 February 2010] Boland, M. (2017). Data Point of the Week: VR to Reach 30% of U.S. Households by 2020. https://arinsider.co/2017/12/19/data-point-of-the-week-vr-to-reach-30-of- u-s-households-by-2020/ [accessed: 2 May 2019] Branch, R., M. (2009). Instructional Design: The ADDIE Approach. Department of Educational Psychology and Instructional Technology (6th ed.). University of Georgia. Springer New York. Brandon-Hall (2001). E-Learning Guidebook. Six Steps to Implementing E- learning. http://www.cedma-europe.org/newsletter%20articles/Brandon%20Hall/e- Learning%20Guide%20Book.pdf [accessed: 14 October 2022] Branson, R. K., Rayner, G. T., Cox, J. L., Furman, J. P., King, F. J. and Hannum, W. H. (1981). Inter Service Procedures for Instructional Systems Develop- ment. (Vol. 1–5). Florida State Univ., Tal ahassee. Centre for Educational Technology. Bregar, L. (ed.) (2007). Spletni program o e-izobraževanju. Ljubljana: Andragoški center Slovenije http://www.naberi.si/znanje/eucenje/ [accessed: 22 October 2009] Bregar, L., Zagmajster, M. and Radovan, M. (2010). Osnove e-izobraževanja (The E-Learning Essentials). Ljubljana: Andragoški center Slovenije http:// arhiv.acs.si/publikacije/Osnove_e-izobrazevanja.pdf [accessed: 13 September 2019] Bregar, L. (2013). Desetletje izobraževanja: preskromno izkoriščene priložnosti ali dozorevanje pogojev za inoviranje izobraževanja? Mednarodno inova- tivno poslovanje, 5 (1) https://journal.doba.si/ojs/index.php/jimb/article/view/157 [accessed: 13 October 2022] Bregar L. and Puhek, M. (2017). Analiza stanja na področju digitalizacije in e-izobraževanja v visokem šolstvu v Sloveniji. Maribor: DOBA Faculty http://www.doba.si/ftp/dokumenti/fakulteta/gradiva/Bregar_Puhek_2017_iKtvVS.pdf [accessed: 10 December 2018] 368 Bregar, L. and Puhek, M. (2018). Razvoj modela uporabe učnih analitik na DOBA Fakulteti. Smart Research Project Report DOBA. Maribor: DOBA Faculty. Brown, M. (2018). Why Invest in MOOCs. Strategic Institutional Drivers. In Jansen, D. in Konings, L. (Eds.) (2018). OpenupEd Trend Report on MOOCs. EADTU. https://tinyurl.com/2018OpenupEdtrendreport [accessed: 18 June 2019] Brown, A., and Green, T. (2016). Virtual Reality: Low-Cost Tools and Resourc- es for the Classroom. TechTrends 60, 517–519 (2016). https://doi.org/10.1007/ s11528-016-0102-z Buchem, I. and Hamelmann, H. (2010). Microlearning: a Strategy for Ongoing Professional Development. eLearning Papers, No. 21. Caffarel a, R. S. and Ratcliff Daffron, S. (2013). Planning Programmes for Adult Learners. A practical Guide. (3rd ed.). San Francisco: Jossey-Bass. A Wiley Im- print. Carretero, S., Vuorikari, R. and Punie, Y. (2017). DigComp 2.1: The Digital Competence Framework for Citizens with Eight Proficiency Levels and Ex- amples of Use. Joint Research Centre. https://doi:10.2760/38842 Carril, P. C. M., Sanmamed, M. G. and Selles, N. H. (2013). Pedagogical roles and competencies of university teachers practicing in the e-learning en- vironment. The International Review of Research in Open and Distributed Learning, 14(3), 462–487 https://doi:10.19173/irrodl.v14i3.1477 Carver, C. S. and Scheier, M. (2012). Perspectives on personality (7th ed.). Essex: Pearson Education Limited. CEDEFOP (2014). Terminology of European Education and Training Policy (2nd ed.) http://www.cedefop.europa.eu/en/publications-and-resources/publications/4117 [accessed: 20 September 2017] Center šolskih in obšolskih dejavnosti (2019). CŠOD Misija. Outdoor experien- tial learning. Presentation, February, 2019. Chatti, M. A., Dyckhoff, A. L., Schroeder, U. and Thus, H. (2012). A Refer- ence Model for Learning Analytics . International Journal of Technology Enhanced Learning, 4(5/6), 318–331. https://doi.org/10.1504/iJtEL.2012.051815 CHEA – Council for Higher Education Accreditation (2001). Annual Confer- ence. New Orleans, LA, January, 23, 2001. [accessed: 13 October 2022] 369 Chun-Ming, H., Hwang, G. J. and Huang, I. (2012). A project-based digital storytelling approach for improving students’ learning motivation, prob- lem-solving competence and learning achievement. Journal of Educational Technology & Society, 15(4), 368–379. Churches, A. (2008). Bloom’ s Digital Taxonomy. http://burtonslifelearning.pbworks. com/f/BloomDigitaltaxonomy2001.pdf [accessed: 16 March 2019] Circuits e-Learning Glossary (n.d.) https://www.puw.pl/sites/default/files/ content_files/zasob_do_pobrania/355/elearn-gloss-learncircuits.pdf [accessed: 27 October 2022] Citizens Bank (2019) https://www.eiseverywhere.com/file_upload s/225df4b5402d474c14cb5bb5de2718ad_CaseStudy.pdf [accessed: 22 August 2019] Clark, C. R. and Mayer, E. (2016). E-learning and the Science of Instruction (3rd ed.). Pfeiffer. Clark, D. (2013). MOOCs: Taxonomy of 8 Types of MOOC. http://donaldclarkplanb. blogspot.com/2013/04/moocs-taxonomy-of-8-types-of-mooc.html [accessed: 13 September 2017] Clow, D. (2013). An Overview of Learning Analytics. Teaching in Higher Edu- cation, 18(6), 683–695. https://doi.org/10.1080/13562517.2013.827653 Collison, G., Elbaum, B., Haavind, S. and Tinker, R. (2000). Facilitating online learning: Effective strategies for moderators. Madison, WI: Atwood Publish- ing. Colman, H. (2019). eLearning Storyboard 101: How to Map Out an Online Court. https://www.ispringsolutions.com/blog/elearning-storyboard [accessed: 11 August 2019] Colvin, C., Rogers, T., Wade, A., Dawson, S., Gašević, D., Buckingham Shum, S., Nelson, K., Alexander, S., Lockyer, L., Kennedy, G., Corrin, L., Fisher, J. (2015). Student Retention and Learning Analytics: A Snapshot of Australian Practices and a Framework for Advancement. Sydney, Australia, Austral- ian Government: Office for Learning and Teaching http://research.usc.edu.au/ vital/access/manager/repository/usc:17648 [accessed: 11 April 2018] Commission of the European Communities (2000a). i2010 – European Infor- mation Society for European Council: Europe of Innovation and Knowledge. Lisbon. Commission of the European Communities (2000b). e-Europe Action Plan for the Period 2000–2003. Feira, Portugal. 370 CommLab India (2016). E-learning: Getting Your Organisation Ready. Glob- al Learning Solutions. https://www.commlabindia.com/elearning-components/ ebook/elearning-101-advanced-guide-for-elearning-implementation-commlab. pdf?utm_campaign=Ebook_nut [accessed: 13 September 2017] CommLab India (2017). Microlearning. A Beginner’ s Guide to Powerful Cor- porate Training. Global Learning Solutions. https://elearningindustry.com/free- ebook-microlearning-a-beginners-guide-to-powerful-corporate-training [accessed: 14 October 2022] CommLab India (2018). Instructional Design Strategies to Design Engaging eL- earning Courses. https://resources.commlabindia.com/ebook/instructional-design- strategies [accessed: 19 June 2019] Commonwealth of Learning (2017). Open Educational Resources: Global Re- port 2017. http://oasis.col.org/bitstream/handle/11599/2788/2017_COL_OEr- Global-report.pdf?sequence=1&isallowed=y [accessed: 22 October 2018] Conati, C. (2009). Intel igent Tutoring Systems: New Chal enges and Directions. Presented at the 21st International Joint Conference on Artificial Intelli- gence (IJCAI-09). Conrad, D. and Openo, J. (2018). Assessment Strategies for Online Learning: En- gagement and Authenticity. https://doi.org/10.15215/aupress/9781771992329.01 Cooper, M. (2016). Multimodal teaching and learning: researching digital sto- rytelling on iPads in the primary school classroom to develop children’s story writing. Journal of Literacy and Technology, 17(1/2), 53–79. Corea, F. (2018, August 29). AI Knowledge Map: How to Classify AI Technolo- gies. Blog. https://medium.com/@Francesco_ai/ai-knowledge-map-how-to-classify- ai-technologies-6c073b969020 [accessed: 1 November 2019] Cowan, N. (2001). The Magical Number 4 in Short-term memory: a Recon- sideration of Mental Storage Capacity. Behavioral and Brain Sciences, 24(1), 87–114. Craglia, M. (2018). Artificial Intel igence. A European Perspective. https:// publications.jrc.ec.europa.eu/repository/bitstream/JrC113826/ai-flagship-report- online.pdf [accessed: 23 June 2019] Cronje J. C. (2016). Twenty-first Century Learning, Rhizome Theory, and In- tegrating Opposing Paradigms in the Design of Personal Learning Systems. In: M. Spector, B. Locke, M. Childress (eds.), Learning, Design, and Technol- ogy (pp. 1–22). Cham: Springer International Publishing. Curran, C. (1990). The Economics of Distance Education. Heerlen: Open uni- versiteit. 371 D’Mello, S. and Graesser, A. (2012). Dynamics of affective states during com- plex learning . Learning and Instruction, 22(2), 145–157. https://doi:10.1016/j. lear- ninstruc.2011.10.001 Daniel, J. (2012). Making Sense of MOOCs: Musings in a Maze of Myth, Para- dox and Possibility. Journal of Interactive Media in Education, 2012(3), p. Art.18. http://doi.org/10.5334/2012-18 Davies, R. and Elias, P. (2003). Dropping out: A study of Early Leavers from Higher Education (Research Report RR 386). London: Department for Edu- cation and Skil s. Dawson, S., Gašević, D., Siemens, G. and Joksimović, S. (2014). Current State and Future Trends: A Citation Network Analysis of the Learning Analytics Field. Proceedings of the Fourth International Conference on Learning An- alytics and Knowledge (pp. 231–240). New York, USA, ACM. http://dx.doi. org/10.1145/2567574.2567585 [accessed: 12 October 2022] Dawson, S., Poquet, O., Colvin, C., Rogers, T., Pardo, A. and Gašević, D. (2018). Rethinking Learning Analytics Adoption through Complexity Lead- ership Theory. Proceedings of the 8th International Conference on Learning Analytics and Knowledge. University of South Australia. de Sousa Borges, S., Durelli, V. H. S., Reis, H. M. and Isotani, S. (2014). A Sys- tematic Mapping on Gamification Applied to Education. Proceedings of the 29th Annual ACM Symposium on Applied Computing (pp. 216–222). New York, NY, USA: ACM Dede, C. J., Jacobson, J. and Richards, J. (2017). Introduction: Virtual, Aug- mented, and Mixed Realities in Education. In D. Liu, C. Dede, R. Huang and J. Richards (eds.), Virtual, Augmented, and Mixed Realities in Education (pp. 1–16). Singapore: Springer Singapore. https://doi:10.1007/978-981-10-5490-7_1 Defelice, R. (2018). How Long to Develop One Hour of Training? Updated for 2017. ATD Association for Talent Development. https://www.td.org/insights/ how-long-does-it-take-to-develop-one-hour-of-training-2017 [accessed:14 October 2022] Deloitte (2017). AI and You. Perceptions of Artificial Intel igence from the EMEA Financial Services Industry. Efma. Grid by Deloitte , https://www2. deloitte.com/content/dam/Deloitte/cn/Documents/technology/deloitte-cn-tech-ai-and- you-en-170801.pdf [accessed: 10 October 2022] Dennis, M. A. (2019). Sir Tim Berners-Lee. Encyclopedia Britannica. https:// www.britannica.com/biography/tim-Berners-Lee [accessed: 13 January 2019] 372 Deterding, S., Dixon, D., Khaled, R. and Nacke, L. E. (2011). Gamification: Toward a Definition. CHI 2011. Gamification Workshop Proceedings, Van- couver, 2011, May, 12-15 (pp. 1–4). Vancouver, BC, Canada: ACM. Detrick, A. D. (2016). Learning Analytics. A Practical Pathway to Success. White Paper. Elearning Guild Research. Dick, W., Carey L. and Carey, J. (2005). The Systematic Design of Instruction (6th ed.). Pearson. Digital Chalk (n.d.). 5 Tools to Help You ”Gamify” Your eLearning Content https://www.digitalchalk.com/resources/blog/elearning-tools/5-tools-to-help-you- gamify-your-elearning-content [accessed: 5 August 2019] Digitalna Slovenija 2020 (2016). Strategija razvoja informacijske družbe do leta 2020. http://www.mju.gov.si/fileadmin/mju.gov.si/pageuploads/DiD/ informacijska_druzba/DSi_2020.pdf [accessed: 6 June 2018] Dimovski, V., Penger, S. and Žnidaršič, J. (2005). Sodobni management. Lju- bljana: Faculty of Economics. Doba Fakulteta (2019). Študijski dosežki v Blackboardu. Internal material. Maribor: Doba Faculty. Docebo (2014). E-Learning Market Trends and Forecast 2014–2016. https:// www.iconcept.nl/publicfiles/136/bestanden/elearning-market-trends-and-forecast- 2014-2016-docebo-report.pdf [accessed: 14 October 2022] Docebo (2016). E-Learning Market Trends and Forecast 2017–2021. https:// www.docebo.com/resource/elearning-market-trends-and-forecast-2017-2021/ [accessed: 14 October 2022] Docebo (2018). eLearning Trends 2019. https://www.docebo.com/resource/report- elearning-trends-2019 [accessed: 13 September 2018] Doctorow, C. (2014). What Happens with Digital Rights Management in the Real World? The Guardian: https://www.theguardian.com/technology/blog/2014/ feb/05/digital-rights-management [accessed 1 October 2018] Dolničar, V., Mrzel, M. and Šetinc, M. (2018). Digitalna pismenost in reševanje problemov v tehnološko bogatih okoljih: primer Slovenije. In P. Javrh (ed.), Spretnosti odraslih (pp. 165–180). Ljubljana: Andragoški center Slovenije. Donmez, M. and Cagiltay, K. (2016). A Review and Categorization of Instruc- tional Design Models. E-Learn 2016 – Washington, DC, November 14–16, 2016. 373 Downes, S. (2001). Learning Objects: Resources for Distance Education Worldwide . International Review of Research in Open and Distance, 2(1). https://doi.org/10.19173/irrodl.v2i1.32 Downes, S. (2005). E-learning 2.0. E-Learn magazine. http://elearnmag.org/ subpage.cfm?section=artides&artide=29-1 [accessed: 16 June 2006] Downes, S. (2007, 5 Feb). What connectivism is. https://www.downes.ca/cgi-bin/ page.cgi?post=38653 [accessed: 9 October 2018] Downes, S. (2012a). E-Learning Generations. Half an hour. http://halfanhour. blogspot.si/2012/02/e-learning-generations.html [accessed: 20 September 2017] Downes, S. (2012b). Massively Open Online Courses are ’Here to Stay’. https:// www.downes.ca/cgi-bin/page.cgi?post=58676 [accessed: 14 May 2018] Downes, S. (2013). The Quality of Massive Open Online Courses. https://www. downes.ca/cgi-bin/page.cgi?post=66145 [accessed: 20 June 2017] Downes, s. (2014a) The MOOC of One. INTED Valencia, Marc 10, 2014. https://www.downes.ca/presentation/336 [accessed: 3 July 2021] Downes (2014b). The Rise of MOOCs: Past Successes, Future Chal enges: Knowledge, Learning, Community. https://slidetodoc.com/the-rise-of-moocs- past-successes-future-challenges/ [accessed: 2 July 2021] Dron J. and Anderson, T. (2009). On the design of col ective applications. Pro- ceedings IEEE CSE’09, 12th IEEE International Conference on Computa- tional Science and Engineering, Vancouver, Canada. Dron, J. and Anderson, T. (2014). Teaching crowds: Learning and social media. Athabasca University Press. Dunleavy, M., Dede, C. and Mitchel , R. (2009). Affordances and Limitations of Immersive Participatory Augmented Reality Simulations for Teaching and Learning. Journal of Science Education and Technology, 18(1), 7–22, https://doi:10.1007/s10956-008-9119-1 Dutt, A., Maizatul, A. I. and Herawan, T. (2017) Systematic Review on Edu- cational Data Mining. (IEEE Access 5, 2017) https://ieeexplore.ieee.org/stamp/ stamp.jsp?tp=&arnumber=7820050 [accessed: 23 June 2019] EADTU (2014). Open Education Handbook. LinkedUpOpen Knowledge. https:// education.okfn.org/handbook/index.html [accessed: 14 October 2022] EADTU (2016). Quality Assessment for E-learning: a Benchmark- ing Approach Third Edition. https://oro.open.ac.uk/47597/1/E- xcellence_manual_2016_third_edition.pdf [accessed: 12 October 2022] 374 EADTU (2019). Empowering Higher Education. Glossary. https://empower.eadtu. eu/glossary [accessed: 20 September 2019] Educause (2015). 7 Things You Should Know About NGDLE. EDUCAUSE Learning Initiative. https://library.educause.edu/resources/2015/12/7-things-you- should-know-about-ngdle [accessed: 17 October 2022] Ehlers, U. D. (2009). New e-Learning Cultures for Adult Learning. Lline – Life- long Learning in Europe, 14(3), 134–141. eLearning Industry (2019). The Elearning Professional Guide to Becoming a Content Curation Hero. https://elearningindustry.com/free-ebooks/content- curation-expert-becoming-elearning-professionals-guide [accessed: 11 August 2019] E-learning Terms Glossary (n.d.). https://www.northpass.com/elearning-glossary [ac- cessed: 16 July 2019] Elias, T. (2011). Learning Analytics: Definitions, Processes and Potential. https:// landing.athabascau.ca/file/download/43713 [accessed: 2 September 2018] Elkins, D. and Pinder, D. (2015). E-learning Fundamentals. A Practical Guide. ATD Press. Emerich, P. (2018). Why I Left Silicon Val ey, EdTech, and ”Personalized” Learning. https://paulemerich.com/2018/01/15/why-i-left-silicon-valley-edtech-and- personalized-learning/ [accessed: 9 October 2018] Empirica (2014). e-Skil s in Europe. Slovenia Country Report. http:// eskills-monitor2013.eu/fileadmin/monitor2013/documents/country_reports/ country_report_slovenia.pdf [accessed: 13 September 2017] Estapa, A. and Nadolny, L. (2015). The Effect of an Augmented Reality En- hanced Mathematics Lesson on Student Achievement and Motivation. Jour- nal of STEM Education, 16(3), 40–48 https://www.learntechlib.Org/p/151963/ [accessed: 12 October 2017] ET 2020 (2009 ). Council conclusions of 12 May 2009 on a strategic framework for European cooperation in education and training (’ET 2020’), 2009/C 119/02. https://eur-lex.europa.eu/legal-content/EN/tXt/PDF/?uri=CELEX:52009XG0528(01 )&qid=1630251348045&from=EN [accessed: 7 May 2021] European Commission (2008). The Use of ICT to Support Innovation and Life- long Learning for All – a Report on Progress. https://www.cedefop.europa.eu/es/ news/use-ict-support-innovation-and-lifelong-learning-all-report-progress [accessed: 14 October 2022] 375 European Commission (2013a). Opening up education; Innovative teaching and learning for all through new technologies and open educational resourc- es. http://eur-lex.europa.eu/legal-content/EN/tXt/PDF/?uri=CELEX:52013DC0654&fro m=EN [accessed: 10 August 2017] European Commission, Directorate-General for the Information Society and Media (2013b). Survey of schools: ICT in education: benchmarking access, use and attitudes to technology in Europe’ s schools, Publications Office, 2013, https://data.europa.eu/doi/10.2759/94499 [accessed: 16 October 2022] European Commission (2015). The Changing Pedagogical Landscape. New Ways of Teaching and Learning and Their Implications. http://www. changingpedagogicallandscapes.eu/publications/ [accessed: 5 May 2017] European Commission (2018). Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions on the Digital Education Ac- tion Plan COM/2018/022 final. https://eur-lex.europa.eu/legal-content/EN/ tXt/?uri=COM:2018:22:FiN [accessed: 10 January 2019] European Commission (2019a). 2nd Survey of Schools: ICT in Education. https://ec.europa.eu/digital-single-market/en/news/2nd-survey-schools-ict-education [accessed: 14 October 2019] European Commission (2019b). Digital Economy and Society Index. DESI 2019 https://ec.europa.eu/digital-single-market/en/desi [accessed: 15 October 2019]. European Commission (2019c). Ethical guidelines for trustworthy artificial intel igence. Independent high-level expert group on artificial intel igence https://ec.europa.eu/digital-single-market/en/news/ethics-guidelines-trustworthy-ai [accessed: 23 June 2019] Eurostat (2017). Survey on Usage of ICT by Individuals, 2017. http://appsso. eurostat.ec.europa.eu/nui/show.do?dataset=isoc_ci_ac_i&lang=en [accessed: 19 August 2017] Eurostat (2019). Individuals Using Mobile Devices to Access the Internet on the Move. https://ec.europa.eu/eurostat/databrowser/view/tin00083/default/ table?lang=en [accessed: 19 November 2019] Falconer, I., McGil , L., Littlejohn, A. and Boursinou, E. (2013). Overview and Analysis of Practices with Open Educational Resources in Adult Education in Europe. Seville, Spain: European Commission Institute for Prospective Technological Studies http://oro.open.ac.uk/50933/1/JrC-adult Learning report 2013.pdf [accessed: 5 September 2018] FAO (2011). E-learning Methodologies. A Guide for Designing and Developing E-Learning Course. 376 FAO (2019). E-learning Methodologies. A Guide for Designing and Developing E-Learning Course. Second Edition. E-learning Centre . http://www.fao.org/3/ i2516e/i2516e.pdf [accessed: 10 May 2022] Faria, A. J., Hutchinson, D., Wellington, W. J. and Gold, S. (2009). Develop- ments in Business Gaming. A Review of the Past 40 Years. Simulation and Gaming, 40(4), 464–487. Farrel , T. and Rushby, N. (2016). Assessment and learning technologies: An overview. British Journal of Educational Technology, 47(1), 106–120. https:// doi.org/10.1111/bjet.12348 Fee, K. (2009). Delivering E-Learning. A Complete Strategy for Design, Applica- tion and Assessment. Kogan Page. London and Philadelphia. Ferdig, R. E., Gandolfi, E. and Immel, Z. (2018). Educational Opportunities for Immersive Virtual Reality. In: J. Voogt, G. Knezek, R. Christensen and K.- W. Lai (eds.). Second Handbook of Information Technology in Primary and Secondary Education (pp. 955-966). Cham: Springer International Publish- ing. https://doi:10.1007/978-3-319-71054-9_66 Ferguson, R. (2012). The State of Learning Analytics in 2012: A Review and Fu- ture Chal enges. Technical Report KMI-12-01. Milton Keynes, UK, Knowl- edge Media Institute, The Open University. Ferguson, R., Barzilai, S., Ben-Zvi, D., Chinn, C.A., Herodotou, C., Hod, Y., Kali, Y., Kukulska-Hulme, A., Kupermintz, H., McAndrew, P., Rienties, B., Sagy, O., Scanlon, E., Sharples, M., Weller, M. and Whitelock, D. (2017). Innovating Pedagogy 2017: Open University Innovation Report 6. Milton Keynes: The Open University https://iet.open.ac.uk/innovating-pedagogy [accessed: 22 October 2019] Ferguson, R., Brasher, A., Clow, D., Cooper, A., Hil aire, G., Mittelmeier, J., Rienties, B., Ullmann, T. and Vuorikari, R. (2016). Research Evidence on the Use of Learning Analytics - Implications for Education Policy. Joint Research Centre. JRC Science for Policy Report . https://ec.europa.eu/jrc/en/publication/ eur-scientific-and-technical-research-reports/research-evidence-using-learning- analytics-implications-education-policy [accessed: 9 May 2018] Ferguson, R., Coughlan, T., Egelandsdal, K., Gaved, M., Herodotou, C., Hil aire, G., Jones, D., Jowers, I., Kukulska-Hulme, A., McAndrew, P., Misiejuk, K., Ness, I. J., Rienties, B., Scanlon, E., Sharples, M., Wasson, B., Weller, M. and Whitelock, D. (2019). Innovating Pedagogy 2019: Open Uni- versity Innovation Report 7. Milton Keynes : The Open University. Institute of Educational Technology, The Open University, UK. https://iet.open.ac.uk/ innovating-pedagogy [accessed: 12 May 2020] 377 Foerster, M., Gourdin, A., Huertas, E., Mohren, J., Ranne, P. and Roca, R. (2019). TeSLA. Framework for the Quality Assurance of E-Assessment. ENQA. https://www.enqa.eu/wp-content/uploads/D4.7-Framework-screen-teSLa-2606.pdf [accessed: 12 October 2022] Fourtane, S. (2019). Augmented Reality: The Future of Education. https:// interestingengineering.com/augmented-reality-the-future-of-education [accessed: 6 August 2019] Frank, M., Miller, G. and Sener, J. (2015). Definitions of E-Learning Cours- es and Programs Version 2.0. Online Learning Consortium https:// onlinelearningconsortium.org/updated-e-learning-definitions-2 [accessed: 20 September 2017] Fritz, J. (2013). Using Analytics at UMBC: Encouraging Student Responsibility and Identifying Effective Course Designs (Research Bul etin). Louisville, CO: EDUCAUSE Centre for Applied Research. Gaebel, M., Kupriyanova, V., Morais, R. and Colucci, E. (2014). E-learn- ing in European Higher Education Institutions. Results of a Mapping Survey Conducted in October-December 2013. Brussels: European University Association https://www.eunis.org/blog/2015/02/09/e-learning-in- european-he-institutions-results-of-a-mapping-survey/ [accessed: 13 October 2022] Gaebel, N. and Zhang, T. (2018). Trends 2018. Learning and Teaching in the Eu- ropean Higher Education Area. European University Association https://eua. eu/resources/publications/757:trends-2018-learning-and-teaching-in-the-european- higher-education-area.html [accessed: 22 November 2018] Garrett, B. M., Jackson, C. and Wilson, B. (2015). Augmented reality m-learn- ing to enhance nursing skil s acquisition in the clinical skil s laboratory. In- tel igent Technology and Smart Education, 12(4), 298–314. https://doi:10.1108/ ITSE-05-2015-0013 Garrison, D. R., Anderson, T. and Archer, W. (2000). Critical Inquiry in a Text-Based Environment: Computer Conferencing in Higher Education. The Internet and Higher Education, 2(2-3), 87–105. https://doi:10.1016/ S1096-7516(00)00016-6 Garrison, D. R., Anderson, T. and Archer, W. (2001). Critical Thinking, Cognitive Presence, and Computer Conferencing in Distance Edu- cation. American Journal ofDistance Education, 15(1), 7–23. https:// doi:10.1080/08923640109527071 Gartner (2014). Gartner Says ’Advanced Analytics is a Top Business Priority’ https://www.gartner.com/newsroom/id/2881218 [accessed: 2 September 2018] 378 Gašević, D., Dawson, S. and Pardo, A. (2018). How do we start? State and Di- rections of Learning Analytics Adoption. Norway: ICDE. Genç, Z. and Tinmaz, H. (2016). The Perception on Fundamentals of Online Courses: A Case on Perspective Instructional Designers. European Journal of Contemporary Education, 15(1), 163–172. https://files.eric.ed.gov/fulltext/ EJ1095975.pdf Georgieva, V. (2019). How to Create Effective Storyboards for eLearning? eL- earning Industry. https://elearningindustry.com/create-storyboards-effective- elearning-tips [accessed: 11 August 2019] Gerlič, I., Debevc, M., Dobnik, N., Šmitek, B., Korže, D., Stjepanovic, Z. and Krašna, M. (2002). Načrtovanje in priprava študijskih gradiv za izobraževanje na daljavo. Maribor: Faculty of Electrical Engineering, Com- puter Science and Informatics. Gikandi, J. W., Morrow, D. and Davis, N. E. (2011). Online formative assess- ment in higher education: A review of the literature. Computers & Educa- tion, 57(4), 2333–2351 https://doi.Org/10.1016/j.compedu.2011.06.004 Giurgiu, L. (2017). Microlearning: an Evolving Elearning Trend. DE Gruyter Open. Scientific Bul etin. 22(1), 18–23. https://sciendo.com/pdf/10.1515/ bsaft-2017- [accessed: 16 September 2018] Glas, M. (1995). Poslovni načrt podjetnika. Študijsko gradivo. PIC-GEA Col- lege. Godsey, M. (2019). The 2019 Learning Management System Forecast. E-learning- Industry. https://elearningindustry.com/2019-learning-management-system-forecast [accessed: 12 September 2019] Goodyear, P. (2001). Effective networked learning in higher education: notes and guidelines. Networked Learning in Higher Education (JISC/CALT) http:// csalt.lancs.ac.uk/jisc/guidelines_final.doc [accessed: 4 October 2017] Gowda, S. M., Baker, R. S., Corbett, A. T. and Rossi, L. M. (2013). Towards Automatical y Detecting Whether Student Learning is Shallow . Interna- tional Journal of Artificial Intel igence in Education, 23(1), 50–70. https:// doi:10.1007/s40593-013-0006-4 [accessed: 4 October 2019] Greller, W. and Drachsler, H. (2012). Translating Learning into Numbers: A Generic Framework for Learning Analytics. Educational Technology and Society, 15(3), 42–57. Groh, F. (2012). Gamification: State of the Art Definition and Utilisation. In Proceedings of the 4th Seminar on Research Trends in Media Informatics pp. 39–46. Ulm: Institute of Media Informatics. Ulm University. 379 Gundala, R. R. and Mandeep, S. (2016). Role of Simulations in Student Learn- ing: A Case Study Using Marketing Simulation. Journal of Educational Re- search and Innovation, 5(2), 1–14. Gutierrez, K. (2015, 27 Aug). How Storytelling Can Improve Your eLearning Courses. https://www.shiftelearning.com/blog/how-storytelling-can-improve-your- elearning-courses [accessed: 4 June 2019] Habeeb Omer, A. (2018). Mobile Learning; Selecting the Right Authoring Tool for Mobile Learning. eLearning Industry https://elearningindustry.com/right- authoring-tool-for-mobile-learning-selecting [accessed: 11 May 2019] Haghshenas, M., Khademi, M. and Kabir, H. (2012). E-learning and Author- ing Tools: at a Glance. IJRRAS, 10(2), 259–263. https://www.arpapress.com/ Volumes/Vol10issue2/iJrraS_10_2_08.pdf [accessed: 7 May 2019] Hamari, J., Koivisto, J. and Sarsa, H. (2014). Does Gamfication Work? – A Literature Review of Empirical Studies on Gamification. 2014 47th Hawaii International Conference on System Sciences (pp. 3025–3034). https:// doi:10.1109/ HICSS.2014.377 Hampson, K. (2013). The next chapter for digital instructional media: content as a competitive difference. Vancouver BC: COHERE 2013 conference. Hannaford, B. and Okamura, A. M. (2016). Haptics. IN B. Siciliano and O. Khatib (Ed.), Springer Handbook of Robotics (pp. 1063–1084). Cham: Springer International Publishing. https://doi:10.1007/978-3-319-32552-1_42 Harasim, L. M. (2017). Learning theory and online technology (2nd ed.). New York, NY: Routledge. Hase, S. and Kenyon, C. (2010). From Andragogy to Heutagogy. Ulti-BASE In- Site. https://epubs.scu.edu.au/gcm_pubs/99/ [accessed: 5 May 2019] Henderikx, P. and Jansen, D. (2018). The Changing Pedagogical Landscape: In Search of Patterns in Policies and Practices of New Modes of Teaching and Learning. EADTU https://tinyurl.com/CPLreport2018 [accessed: 11 June 2019] Hendron, J. G. (2008). RSS for Educators. Blogs, Newsfeed, Podcasts and Wikis in the Classroom. Eugene, OR: International Society for Technology and Education. Herrington, J., Herrington, A., Mantei, J., Olney. I. and Ferry, B. (2009). New Technologies, New Pedagogies: Mobile Learning in Higher Education. Faculty of Education. University of Wollongong https://ro.uow.edu.au/edupapers/91/ [accessed: 4 June 2019] 380 Ho, A. D., Reich, J., Nesterko, S., Seaton, D. T., Mul aney, T., Waldo, J. and Chuang, I. (2014). HarvardX and MITx: The First Year of Open Online Courses, Fall 2012 – Summer 2013, HarvardX and MITx Working Paper No. 1, January 21, 2014. https://papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263 [accessed: 4 October 2022] Hofman, J. (2009). Introducing Copyright. Vancouver : Commonwealth of Learning http://oasis.col.org/handle/11599/41 [accessed: 15 January 2019] Hogle, P. S. (2018). The State of Microlearning. https://www.elearningguild.com/ content/5319/research-the-state-of-microlearning/ [accessed: 5 October 2019] Horton, W. K. (2012). E-learning by design (2nd ed.). San Francisco, CA: Pfeiffer. Hrastinski, S. (2008). Asynchronous and Synchronous E-Learning. EDUCAUSE Quarterly, 31(4). https://er.educause.edu/articles/2008/11/asynchronous-and- synchronous-elearning [accessed: 12 June 2019] Huang, W. H.-Y. and Soman, D. (2013). A practitioner’ s guide to gamification of education (Behavioural Economics in Action). Toronto: Rotman School of Management, University of Toronto. Huertas, E., Biscan, I., Ejsing, C., Kerber, L., Kozlowska, L., Marcos Ortega, S., LaurI, L., Risse, M., Schorg, S. and Seppmann, S. (2018). Considerations for Quality Assurance of e-Learning Provision. Report from the ENQA Work- ing Group VIII on Quality Assurance and e-Learning. Occasional Papers 25. https://www.enqa.eu/publications/considerations-for-qa-of-e-learning-provision/ [accessed: 31 January 2019] Hul , A. (2018). 3 Examples on How to Use Storytel ing in eLearning. https:// elearningindustry.com/storytelling-in-elearning-3-examples [accessed: 4 June 2019] Hussain, F. (2012). E-Learning 3.0 = E-Learning 2.0 + Web 3.0? In: D. G. Samp- son, J. M. Spector, D. Ifenthaler, and P. Isaias (eds.), Proceedings of IADIS International Conference on Cognition and Exploratory Learning in Digital Age, Madrid, Spain (pp. 11–18). IBM Watson (n.d.) https://www.ibm.com/watson/education [accessed: 23 June 2019] Ifenthaler, D. (2014). From Educational Data Mining to Automated Analysis of Semantics. The Learning Analytics Week @ epfl https://documents.epfl.ch/ groups/m/mo/mooc-admins/www/documents/ifenthaler-LaW.pdf [accessed: 2 September 2018] Inamorato dos Santos, A., Punie, Y. and Castaño Muñoz, J. (2016a). Opening up Education. A Support Framework for Higher Education Institutions: JRC Science for Policy Report. http://dx.doi.org/10.2791/553783 [accessed: 16 October 2022] 381 Inamorato dos Santos, A., Punie, Y. and Castaño Muñoz, J. (2016b). Opportu- nities and Chal enges for the Future of Moocs and Open Education in Europe. London: Portland Press. https://www.slideshare.net/inamor/pplwennerch07 [accessed: 15 October 2022] Inamorato Dos Santos, A., Punie Y. and Scheller, K. (2017). Going Open: Policy Recommendations on Open Education in Europe (OpenEdu Policies). Euro- pean Commission. https://publications.jrc.ec.europa.eu/repository/ [accessed: 16 October 2022] Instructional Simulations (2019). Wikipedia https://en.wikipedia.org/wiki/ instructional_simulation [accessed: 9 June 2019] I-slovar (2019). http://islovar.org [accessed: 15 January 2019] Jacobs, S. and Sharma, N. (2017). How to Quantify the ROI of Learning in Your Organisation. Elearning Guild. https://www.elearningguild.com/sponsored/1528/ how-to-quantify-the-roi-of-learning-in-your-organisation/ [accessed: 9 August 2019] Jacques, D. and Salmon, G. (2007). Learning in Groups: A Handbook for Face- to-Face and Online Environments (4th ed.). Milton Park: Routledge. Jansen, D. (2017). An Introduction to Open Online Education & MOOCs. SCORE 2020. Workshop. Ljubljana: University of Ljubljana. Faculty of Electrical Engineering. 20 February 2017. Jansen, D. and Konings, L. (2017). MOOC Strategies of European Institu- tions. Status report based on a mapping survey conducted in November 2016 – February 2017. EADTU. http://eadtu.eu/documents/Publications/OEenM/ MOOC_Strategies_of_European_institutions.pdf [accessed: 14 October 2022] Jarman, B. (2019). EdTech Trends You Should Be Excited About. Elearning In- dustry https://elearningindustry.com/2019-edtech-trends-excited [accessed: 23 June 2019] JISC Infonet (2011). Mobile Learning infoKit. https://2020research.files.wordpress. com/2011/07/mobile-learning-infokit.pdf [accessed: 2 June 2019] Jonsson, A. G. and Swingby, G. (2007). The use of scoring rubrics: Reliability, validity and educational consequences. Educational Research Review, 2(2), 130–144. Jung, I. (2008). Costing Virtual University Education. In W. J. Bramble and S. Panda. (ed.), Economics of Distance and Online Learning (pp. 148–161). New York and London: Routledge Taylor & Francis Group. Jung, I. (2019). Open and distance education theory revisited: Implications for the digital era. Singapore: Springer. Kakoura, A. (2016). Copyright Limitations in Distance Learning Education. A Study of the European Legal Context. International Hellenic University. Kanuka, H. (2008). Understanding E-Learning Technologies-in-Practice through Philosophies-in-Practice. In T. Anderson (ed.), The Theory and Practice of Online Learning (pp. 91–117). AU PRESS. Athabasca University. Kaplan-Leiserson, E. (n. d.). E-Learning Glossary. http://www.lupi.ch/Schools/ astd/astd2.htm [accessed: 16 June 2019] Kapp, K. M. (2012). The gamification of learning and instruction: game-based methods and strategies for training and education. San Francisco, CA: Pfeiffer. Kapp, K. M., Blair, L. and Mesch, R. (2014). The gamification of learning and instruction field-book: ideas into practice. San Francisco, CA: Wiley. Kapuler, D. (2018). Digital Storytel ing Apps & Sites. https://www.pinterest.com/ dkapuler/digital-storytelling-apps-sites/ [accessed: 10 November 2018] Kasenberg, T. (n.d.). Just One Thing - Microlearning. A Practitioners Guide. Raptivity https://www.raptivity.com/pdf/Just One Thing - Microlearning A Practitioners Guide.pdf [accessed: 17 December 2019] Kauffman, H. (2015). A Review of Predictive Factors of Student Success in and Satisfaction with Online Learning. Research in Learning Technology, 23 http://dx.doi.org/10.3402/rlt.v23.26507 Kačič, M. and Zaviršek, D. (2013). Marakeški sporazum odpira pot do zakonite objave literature. Rikos, 12(4), 3–5 http://www.zveza-slepih.si/rikoss/wp-content/ uploads/2015/11/marrakeski_sporazum_2013.pdf [accessed: 12 June 2019] Kel y, M. G. and Haber, J. (2006). Resources for Student Assessment. Eugene, OR: International Society for Technology in Education (ISTE). Khan, B. (2005). A Comprehensive E-Learning Model. Journal of e-Learning and Knowledge Society, 1(1), 33–44. https://www.learntechlib.org/p/43389/ [accessed: 22 October 2017] Kim, A. J. (2011). Smart Gamification: Designing the Player Journey. https:// youtu.be/B0H3aSbnZmc [accessed: 13 November 2019] Kim, S., Song, K., Locke, B. and Burton, J. (2018). Gamification in Learning and Education: Enjoy Learning Like Gaming. Springer International Publishing . Kinta, G. (2013). Theoretical Background for Learning Outcomes Based Approach to Vocational Education. International Journal for Cross-Dis- ciplinary Subjects in Education (IJCDSE), 3(3) http://dx.doi.org/10.20533/ ijcdse.2042.6364.2013.0215 383 Kirkpatrick, D. L. (1994). Evaluating Training Programmes . The Four Levels. San Francisco: Berettt-Koehler. Kirkpatrick, J. D. and Kayser Kirkpatrick, W. (2016). Four levels of training evaluation. ATD Press. Kirkpatrick, J. and Kirkpatrick, W. (2019). An Introduction to the New World Kirkpatrick Model. Kp Kirkpatrick Partners. The One and Only Kirkpatrick https://www.kirkpatrickpartners.com/wp-content/uploads/2021/11/introduction-to- the-Kirkpatrick-New-World-Model.pdf [accessed: 13 October 2022] Kirkwood, A. and Price, L. (2014). Technology-enhanced Learning and Teach- ing in Higher Education: What is ’Enhanced’ and how do We Know? A Critical Literature Review. Learning, Media and Technology, 39(1), 6–36. Klein, R. D. and Fleck, R. A. (1990). International Business Simulation/Gam- ing: An Assessment and Review. Simulation and gaming, 21(2), 147–165. https://doi.org/10.1177/1046878190212003 Klobučar, T. (2013). Sodobne e-izobraževalne tehnologije. Mednarodno inova- tivno poslovanje, 5 (1), https://journal.doba.si/ojs/index.php/jimb/article/view/154 [accessed: 5 August 2016] Knowles, M. S. (1986). Using Learning Contracts. San Francisco: Jossey-Bass. Knox, J. (2018). Beyond the ”c” and the ”x”: Learning with algorithms in mas- sive open online courses (MOOCs). International Review of Education, 64(2), 161–178 https://doi.org/10.1007/s11159-018-9707-0 Kop, R. (2011). The challenges to connectivist learning on open online net- works: Learning experiences during a massive open online course. The International Review of Research in Open and Distributed Learning, 12(3), 19–38 . https://doi.org/10.19173/irrodl.v12i3.882 Koutromanos, G., Sofos, A. and Avraamidou, L. (2015). The use of augmented reality games in education: a review of the literature. Educational Media International, 52(4), 253–271. https://doi:10.1080/09523987.2015.1125988 Kovanović, V., Gašević, D., Dawson, S., Joksimović, S. and Baker, R. S. (2016). Does Time-on-task Estimation Matter? Implications on Validity of Learn- ing Analytics Findings. Journal of Learning Analytics, 2(3), 81–110. Kovanović, V., Gašević, D., Dawson, S., Joksimović, S., Baker, R. S. and Hatala, M. (2015). Penetrating the Black Box of Time-on-task Estimation . Proceed- ings of the Fifth International Conference on Learning Analytics and Knowl- edge (pp. 184–193). New York: ACM. 384 Kulik, J. A. and Fletcher, J. D. (2016). Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review. Review of Educational Research, 86(1), 42–78. https://doi:10.3102/0034654315581420 Kukulska-Hulme, A. (2013). Limelight on Mobile Learning: Integrating Edu- cation and Innovation. Harvard International Review, 34(4), 12–16. Kuzilek, J., Hlosta, M. and Zdrahal, Z. (2017). Open University Learning Ana- lytics Dataset. Sci Data, 4 https://doi.org/10.1038/sdata.2017.171 Lambert, J. (2010). Digital storytel ing cookbook. Berkeley, CA: Digital Diner Press. Lambert, J. (2013). Digital storytelling: capturing lives, creating community. New York; London: Routledge. Lau, K. W. and Lee, P. Y. (2015). The Use of Virtual Reality for Creating Unu- sual Environmental Stimulation to Motivate Students to Explore Crea- tive Ideas. Interactive Learning Environments, 23(1), 3-18. https://doi:10.108 0/10494820.2012.745426 Lawless, C. (2018). The Ultimate Glossary of eLearning Terms. LearnUpon https://www.learnupon.com/blog/elearning-glossary/ [accessed: 26 November 2019] Learning Seat (2017) Microlearning. A New Way to Train Your Workforce. Whitepaper. http://www.learningseat.com.au/MicrolearningWhitepaper.pdf [accessed: 26 November 2019] Lee, M. J. and McLoughlin, C. (2007). Teaching and learning in the Web 2.0 era: Empowering students through learner-generated content. Internation- al journal of instructional technology and distance learning, 4(10), 21–34. Lexico Dictionary (2019). Artificial Intel igence. https://www.lexico.com/en/definiti- on/artificial_intelligence [accessed: 10 November 2019] List of artificial intel igence films (n.d.). Wikipedia https://en.wikipedia.org/wiki/ List_of_artificial_intelligence_films [accessed: 10 June 2019] Lister, M. C. (2015). Gamification: The effect on student motivation and per- formance at the post-secondary level. Issues and Trends in Educational Technology, 3(2), 1–22. https://doi:10.2458/azu_itet_v3i2_lister Littlejohn, A. and Hood, N. (2018). Reconceptualising Learning in the Digital Age: The [Un]democratising Potential of MOOCs. SpringerBriefs in Open and Distance Education. https://doi.org/10.1007/978-981-10-8893-3 385 Long, P. and Siemens, G. S. (2011). Penetrating the Fog. Analytics in Learn- ing and Education. Educause Review, September/October 2011. https:// er.educause.edu/articles/2011/9/penetrating-the-fog-analytics-in-learning-and- education [accessed: 10 October 2022] Lynch, L. (2018). What Does VR Have to Do with Online Education? https:// www.learndash.com/what-does-vr-have-to-do-with-online-education/ [accessed: 6 August 2019] Lynch, M. M. and Roecker, J. (2007). Project Managing E-Learning. A Hand- book for Successful Design, Delivery and Management. London, New York: Routledge. Madaio, M. A., Ogan, A. and Cassel , J. (2016). The Effect of Friendship and Tutoring Roles on Reciprocal Peer Tutoring Strategies. In A. Micarelli, J. Stamper and K. Panourgia (eds.), Intel igent Tutoring Systems. ITS 2016. Lecture Notes in Computer Science, 9684 (pp. 423–429). Cham: Springer. https://doi:10.1007/978-3-319-39583-8_51 Marentič Požarnik, B. (2003). Psihologija učenja in pouka. Ljubljana: DZS. Martinez, M. (2003). High Attrition Rates in e-Learning: Challenges, Predic- tors, and Solutions. The ELearning Developers’ Elearning Guild Journal. https://www.elearningguild.com/pdf/2/071403mgt-l.pdf [accessed: 6 September 2019] Mathes, J. (2019). Global Quality in Online, Open, Flexible and Technology En- hanced Education, An Analysis of Strengths, Weaknesses; Opportunities and Threats. ICDE https://www.icde.org/knowledge-hub/report-global-quality-in-online- education [accessed: 14 October 2022] Mayadas, F., Miller, G. and Sener, J. (2015). Definition of E-learning Courses and Programmes. Version 2.0. Online Consortium https://onlinelearningconsortium. org/updated-e-learning-definitions-2 [accessed: 13 September 2017] McCawley, P. M. (2004). Methods for Conducting Educational Needs Assessment. Guidelines for Cooperative Extension System Professionals University of Idaho. Extension. https://www.researchgate.net/publication/255627038_Methods_ for_Conducting_an_Educational_Needs_assessment [accessed: 27 October 2022] McGonigal, J. (2015). SuperBetter: A Revolutionary Approach to Getting Strong- er, Happier, Braver and More Resilient – Powered by the Science of Games. New York, NY: Penguin Press. McMahon, D., Cihak, D. F. and Wright, R. (2015). Augmented reality as a navigation tool to employment opportunities for postsecondary education students with intellectual disabilities and autism. Journal of Research on Technology in Education, 47(3), 157–172. 386 Medárová, V., Bureš, V. in Otčenášková, T. (2012). A Review of Obsta- cles to Successful E-Learning Deployment in SMEs. Journal of Innova- tion Management in Small & Medium Enterprises, 58(13), 1–9. https:// doi:10.5171/2012.715039 Merisotis, J.P. and Phipps. R.A. (2000). Quality on the line. Benchmarks for success in internet-based distance education. Policy. The Institute for Higher Education Policy. https://eric.ed.gov/?id=ED444407 [accessed: 14 October 2022] Mihelj, M., Novak, D. and Beguš, S. (2014). Virtual reality technology and ap- plications. Dordrecht: Springer. Miller, D. (2018). Top 5 Reasons Why eLearning Projects Fail. ELearning Indus- try. https://elearningindustry.com/elearning-projects-fail-top-5-reasons [accessed: 13 September 2019] Mil s, M. (2016). Artificial Intel igence in Law: The State of Play 2016. Thomson Reuters Legal Executive Institute https://www.thomsonreuters.com/en-us/posts/ legal/artificial-intelligence-in-law-the-state-of-play-2016/ [accessed: 14 October 2022] MIZŠ (Ministry of Education, Science and Sport) (n.d.). IKT v izobraževanju. Urad za razvoj izobraževanja. http://mizs.arhiv-spletisc.gov.si/si/delovna_podrocja/ urad_za_razvoj_in_kakovost_izobrazevanja/sektor_za_razvoj_izobrazevanja/ razvojna_podrocja/ikt_v_izobrazevanju/index.html [accessed: 27 October 2022] MIZŠ (Ministry of Education, Science and Sport), (2019). UNESCO podprl predlog MIZŠ za ustanovitev mednarodnega raziskovalnega centra za umet- no inteligenco pod okriljem Unesco v Ljubljani. http://mizs.arhiv-spletisc.gov.si/ si/medijsko_sredisce/novica/10715/ [accessed: 20 October 2022] MIZŠ (Ministry of Education, Science and Sport), (2006). Akcijski načrt nadaljnjega preskoka informatizacije šolstva . http://mizs.arhiv-spletisc. gov.si/si/delovna_podrocja/urad_za_razvoj_in_kakovost_izobrazevanja/ sektor_za_razvoj_izobrazevanja/razvojna_podrocja/ikt_v_izobrazevanju/index.html [accessed: 27 October 2022] MIZŠ (Ministry of Education, Science and Sport), (2007). Akcijski načrt SIO. http://mizs.arhiv-spletisc.gov.si/si/delovna_podrocja/ urad_za_razvoj_in_kakovost_izobrazevanja/sektor_za_razvoj_izobrazevanja/ razvojna_podrocja/ikt_v_izobrazevanju/index.html [accessed: 27 October 2022] 387 MIZŠ (Ministry of Education, Science and Sport), (2016). Strategic guidelines for further implementation of ICT in the Slovenian education until 2020 https://www.gov.si/assets/ministrstva/MiZS/Dokumenti/Zakonodaja/EN/ Slovenian_Strategic_Guidelines_iCt_in_education.pdf [accessed: 16 October 2022] Miranda, P., Isaias, P. and Costa, C. J. (2014). E-Learning and Web Generations: Towards Web 3.0 and E-Learning 3.0. International Proceedings of Eco- nomics Development and Research, 81 (pp. 92–103). Singapore: IACSIT Press. https://doi:10.7763/iPEDr.2014.V81.15 Monahan, K., Harr, C., Knight, M. and Crump, J. (2016). Gaming away the leadership gap. Linking gamification and behavioural science to transform leadership development. Princeton, NJ: Deloitte University Press. MOOC Glossary (2017). Extension Engine Blog. http://blog.extensionengine.com/ mooc-terms-glossary/ [accessed: 20 October 2017] Moor, J. (2006). The Dartmouth College Artificial Intelligence Conference: The Next Fifty Years. AI Magazine, 27(4), 87 https://doi.org/10.1609/aimag. v27i4.1911 Moore, J.L., et al. (2010). e-Learning, online learning, and distance learning environments: Are they the same?, Internet and Higher Education (2010). https://doi.org/10.1016/j.iheduc.2010.10.001 Morgan, A (2016). ADDIE vs. AGILE Model: An Instructional Designer’ s Per- spective. Training Need Analysis. https://www.iddblog.org/addie-vs-agile-model- an-instructional-designers-perspective [accessed: 11 August 2019] Možina, S., Rozman, R., Glas, M., Tavčar, M., Pučko, D., Kralj, J., Ivanko, Š., Lipičnik, B., Gričar, J., Tekavčič, M., Dimovski, D. and Kovač, B. (2002). Management: Nova znanja za uspeh. Radovljica: Didakta. Možina, T. and Klemenčič, S. (2008). Razvoj kakovosti izobraževanja odra- slih. Notranje in zunanje presojanje kakovosti. Študije in raziskave št . 13. Ljubljana: Andragoški center Slovenije https://www.acs.si/digitalna-bralnica/ razvoj-kakovosti-izobrazevanja-odraslih-notranje-in-zunanje-presojanje-kakovosti/ [accessed: 16 October 2022] Mulder, F. and Janssen, B. (2013). Opening up Education. In: Jacobi, R., Jelger- huis H., van der Woert N. (eds.). Trend report: Open Educational Resources 2013, SURF SIG OER, Utrecht, https://www.oerknowledgecloud.org/record542 [accessed: 15 October 2022] 388 Mungania, P. (2004). Employees’ Perceptions of Barriers In E-Learning: The Re- lationship among Barriers, Demographics, and Elearning Self-Efficacy. Elec- tronic Theses and Dissertations. Paper 1027. https://doi.org/10.18297/etd/1027 [accessed: 24 August 2019] Muñoz Castaño, J. Punie, Y., Inamorato dos Santos, A., Mitic, M. and Morais, R. (2016). How are Higher Education Institutions Dealing with Openness? A Survey of Practices, Beliefs and Strategies in Five European Countries. Institute for Prospective Technological Studies . JRC Science for Policy Report, EUR 27750 EN https://publications.jrc.ec.europa.eu/repository/handle/ JrC99959 [accessed: 16 October 2022] Murray, T. (2003). An Overview of Intelligent Tutoring System Authoring Tools: Updated Analysis of the State of the Art. In T. Murray, S. B. Blessing and S. Ainsworth (eds.). Authoring Tools for Advanced Technology Learn- ing Environments: Toward Cost-Effective Adaptive, Interactive and Intel i- gent Educational Software (pp. 491–544). Dordrecht: Springer Netherlands. https://doi:10.1007/978-94-017-0819-7_17 Muršak, J. (2012). Temeljni pojmi poklicnega in strokovnega izobraževanja. Lju- bljana: Center RS za poklicno izobraževanje https://cpi.si/aktualno/knjiznica/ gradiva/temeljni-pojmi-poklicnega-in-strokovnega-izobrazevanja/ [accessed: 16 October 2022] Naismith, L., Lonsdale, P., Vavoula, G. and Sharples, M. (2004). Literature Re- view in Mobile Technologies and Learning. University of Birmingham, Fu- tureLab Series. https://www.nfer.ac.uk/publications/futl15/futl15.pdf [accessed: 16 October 2022] Nielsen, J. (n.d.). How to Conduct a Heuristic Evaluation. Papers and Essays. http://www.ingenieriasimple.com/usabilidad/HeuristicEvaluation.pdf). [accessed: 10 November 2018] Nielson, B. (2017). Benefits of Using Simulations. https://web.archive.org/ web/20180426104428/https:/www.yourtrainingedge.com/benefits-of-using- simulations [accessed: 27 October 2022] Nkambou, R., Mizoguchi, R. and Bourdeau, J. (eds.). (2010). Advances in Intel- ligent Tutoring Systems. Berlin Heidelberg: Springer-Verlag. Nye, B. D. (2015). Intelligent Tutoring Systems by and for the Developing World: A Review of Trends and Approaches for Educational Technology in a Global Context. International Journal of Artificial Intel igence in Educa- tion, 25(2), 177–203. https://doi:10.1007/s40593-014-0028-6 OECD (2007). Giving Knowledge for Free. The Emergence of Open Educational Resources. Centre for Educational Research and Innovation. http://www.oecd. org/education/ceri/38654317.pdf [accessed: 5 November 2017] 389 OECD (2018a). Future of Education and Skil s 2030: Conceptual Learning Framework Education and AI: Preparing for the Future & AI, Attitudes and Values. 8th Informal Working Group (IWG) Meeting 29–31 October 2018, OECD Conference Centre, Paris, France. OECD (2018b). Understanding Innovative Pedagogies: Key Themes to Analyse New Approaches to Teaching and Learning. OECD Education Working Pa- per No. 172. http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cot e=EDu/WKP(2018)8&docLanguage=En [accessed: 6 September 2018] Official Journal of the EU (2015). 2015 Joint Report of the Council and the Commission on the implementation of the strategic framework for European cooperation in education and training (ET 2020). Official Journal of the Eu- ropean Union 2015/C 417/04 https://eur-lex.europa.eu/legal-content/EN/tXt/PDF /?uri=CELEX:52015XG1215(02)&from=EN [accessed: 13 September 2017] Official Journal of the European Union (2016 ). Regulation (EU) 2016/679 of the European Parliament and of the Council on the protection of natural per- sons with regard to the processing of personal data and on the free movement of such data, and repealing Directive 95/46/EC (General Data Protection Regulation). https://eur-lex.europa.eu/legal-content/EN/tXt/%20PDF/?uri=CELEX :32016r0679&from=EN [accessed: 2 September 2018] Official Journal of the EU (2019). Directive (EU) 2019/790 of the European Parliament and of the Council of 17 April 2019 on copyright and related rights in the Digital Single Market and amending Directives 96/9/EC and 2001/29/EC (2019). L 130/92 https://eur-lex.europa.eu/legal-content/EN/tXt/ PDF/?uri=CELEX:32019L0790&from= EN [accessed: 2 July 2019] OECD (2019). How’ s Life in the Digital Age? Opportunities and Risks of the Digital Transformation for People’ s Well-Being. Paris: OECD Publishing https://www.oecd.org/publications/how-s-life-in-the-digital-age-9789264311800-en. htm [accessed: 15 October 2022] Open University (2014). Policy on Ethical use of Student Data for Learning Analytics. https://help.open.ac.uk/documents/policies/ethical-use-of-student-data [accessed 2 September 2018] Open University (n.d.) . Open University Learning Analytics Dataset. https:// analyse.kmi.open.ac.uk/open_dataset [accessed: 6 May 2018] OpeningUp Slovenia (2017) http://www.ouslovenia.si [accessed: 13 September 2018] OpenupEd (2015). Definition Massive Open Online Courses (MOOCs). https:// www.openuped.eu/images/docs/Definition_Massive_Open_Online_Courses.pdf [accessed: 2 September 2018] 390 Origin Learning (2019). What Defines an Ideal Learning Management System?. https://blog.originlearning.com/what-defines-an-ideal-learning-management-system/ [accessed: 11 July 2019] Orr, D., Rimini, M., Van Damme, D. (2015). Open Educational Resources. A Catalyst for Innovation. Educational Research and Innovation. OECD, Publishing, Paris. https://read.oecd-ilibrary.org/education/open-educational- resources_9789264247543-en#page4link [accessed: 12 September 2022] Orr, D., Weller, M. and Farrow, R. (2018). Models for Online, Open, Flexible and Technology Enhanced Higher Education Across the Globe – A Compara- tive Analysis. Final Report, April 2018. ICDE, FiBS, Open University. https:// oofat.oerhub.net/OOFat/ [accessed: 16 October 2022] Palloff, R. M. and Pratt, K. (2009). Assessing the online learner: resources and strategies for faculty. San Francisco: Jossey-Bass. Pandey, A. (2018a). eLearning Trends and Prediction for 2018. http://www. elearningnews.it/_resources/images/ebook/eLearning-trends-and-Predictions- for-2018.pdf [accessed: 20 December 2018] Pandey, A. (2018b). 12 Examples that Prove Mobile Learning and Microlearn- ing is an Essential Combination to Meet Your Learning Mandate. https:// elearningindustry.com/free-ebooks/mobile-learning-and-microlearning-combination- training-examples [accessed: 8 October 2022] Papamitsiou, Z. and Economides, A. A. (2014). Learning Analytics and Educa- tional Data Mining in Practice: A Systematic Literature Review of Empiri- cal Evidence. Educational Technology & Society, 17(4), 49–64. Pappano, L. (2012, 2 Nov). The Year of the MOOC. The New York Times https:// nyti.ms/2kBM8OV [accessed: 20 October 2017] Pappas, C. (2013, February 28). 18 free digital storytel ing tools for teachers and students. https://elearningindustry.com/18-free-digital-storytelling-tools-for-teachers- and-students [accessed: 26 October 2018] Pappas, C. (2020, December 9). Top Mobile Learning Platforms in 2022. https:// elearningindustry.com/top-mobile-learning-platforms-lms-list [accessed: 12 June 2022] Parker, R. E. (2013). Redesigning courses for online delivery: Design, interaction, media & evaluation. Amsterdam; Boston: Emerald. Peddie, J. (2017). Augmented reality: where we will all live. New York, NY: Springer Berlin Heidelberg. 391 Pedro, F., Subosa, M., Rivas, A. and Valverde, P. (2019). Artificial Intelligence in Education: Challenges and Opportunities for Sustainable Development; Working Papers on Education Policy 7. UNESCO. https://unesdoc.unesco.org/ ark:/48223/pf0000366994 [accessed: 12 June 2019] Perrin, A. (2018, 17 September). 5 facts about Americans and video games https://www.pewresearch.org/fact-tank/2018/09/17/5-facts-about-americans-and- video-games/ [accessed: 10 August 2019] Pilli, O. and Admiraal, W. (2016). A Taxonomy of Massive Open Online Courses . Contemporary Educational Technology 7(3). 223–240. Piña, A. A., Lowel , V. C. and Harris, B. R. (2018). Leading and Managing e- Learning. What the e-Learning Leader Needs to Know. Educational Commu- nications and Technology. Issues and Innovations . Springer. Plotkin, H. (2010). Free to Learn. An Open Educational Resources Policy Devel- opment Guidebook for Community Col ege Governance Officials. https://wiki. creativecommons.org/images/6/67/FreetoLearnGuide.pdf [accessed: 20 May 2018] Poulin, R. and Straut, T. (2017). WCET Distance Education Price and Cost Re- port. https://wcet.wiche.edu/resources/price-and-cost-of-distance-education/ [accessed: 14 October 2022] Powel , B. (2009). Delivering the Promise. Making e-Learning Strategy Work. Leicester: NIACE. Powel , M. (2018). The Online Classroom of the Future. Drexel Online University Prensky, M. (2001). Digital game-based learning. New York: McGraw Hill. Prins, E. (2017). Digital storytelling in adult education and family literacy: A case study from rural Ireland. Learning. Media and Technology, 42(3), 308–323. Prusty, B. G. and Russel , C. (2011). Engaging students in learning threshold concepts in engineering mechanics: adaptive eLearning tutorials. Presented at the 17th International Conference on Engineering Education, Belfast, Northern Ireland. Pučko, D. (ed.) (2005). Planiranje v neprofitnem javnem sektorju. Nova Gorica: EDUCA. Puentedura, R. R. (2014). Building Transformation: An Introduction to the SAMR Model. http://www.hippasus.com/rrpweblog/archives/2014/08/22/ Buildingtransformation_anintroductiontoSaMr.pdf [accessed: 27 October 2022] 392 Purdue University (2013). Course signals. https://www.youtube.com/watch?v=- Bi9E7qP9ja [accessed: 2 September 2018] Qayumi, A. K. (2010). Centre of Excellence for Simulation Education and In- novation (CESEI). Journal of Surgical Education, 67(4), 265–269. https://doi. org/10.1016/j.jsurg.2010.05.015 [accessed: 1 June 2019] Raccoon Gang (2019). How much does it cost to create an online course? https:// raccoongang.com/blog/how-much-does-it-cost-create-online-course [accessed: 20 July 2019] Radovan, M., Kristl, N., Jedrinovič, S., Papić, M., Hrovat, L., Žurbi, R., Leskošek, B. (2018). Vključevanje informacijsko-komunikacijske tehnologije v visokošolski pedagoški process na članicah Univerze v Ljubljani. Ljubljana: University of Ljubljana. Rebolj, V. (2008). E-izobraževanje skozi očala pedagogike in didaktike. Radov- ljica: Didakta. Redecker, C. and Johannessen, Ø. (2013). Changing Assessment – Towards a New Assessment Paradigm Using ICT . European Journal of Education, 48(1), 79–96 https://doi.org/10.1111/ejed.12018 Reigeluth, C. M. and Carr-Chellman A. A. (2009). Instructional-Design The- ories and Models. Volume III. Building a Common Knowledge Base. New York: Routledge. RMIT University (n.d.). Library Subject Guides. Open Educational Resources. https://rmit.libguides.com/c.php?g=405195&p=5880171 [accessed: 10 September 2021] Robin, B. R. (2008). Digital storytelling: a powerful technology tool for the 21st century classroom. Theory Into Practice, 47(3), 220–228. doi: 10.1080/00405840802153916 Robin, B. R. (2015). The effective uses of digital storytelling as a teaching and learning tool. In J. Flood, S. B. Heath and D. Lapp (Eds.), Handbook of re- search on teaching literacy through the communicative and visual arts (pp. 429–440). New York; London: Routledge. Robin, B. R. (2016). The power of digital storytelling to support teaching and learning. Digital Education Review, 30, 17–29. https://files.eric.ed.gov/fulltext/ EJ1125504.pdf 393 Roblyer, M. D. and Wiencke, W. R. (2003). Design and Use of a Rubric to Assess and Encourage Interactive Qualities in Distance Courses. Ameri- can Journal of Distance Education, 17(2), 77–98 https://doi.org/10.1207/ S15389286aJDE1702_2 Robson, K., Plangger, K., Kietzmann, J. H., McCarthy, I. and Pitt, L. (2015). Is it all a game? Understanding the principles of gamification. Business Hori- zons, 58(4), 411–420. https://doi:10.1016/j.bushor.2015.03.006 Rosen, A. (2009). E-learning 2.0. Proven Practices and Emerging Technologies to Achieve Results. New York: American Management Association. Rosenberg, M. J. (2001). E-learning Strategies for Delivering Knowledge in the Digital Age. New York: McGraw-Hil . Rosewel , J. and Jansen, D. (2014). The OpenupEd Quality Label: Benchmarks for MOOCs . The International Journal for Innovation and Quality in Learn- ing, 2(3), pp. 88–100. Rosser, J. C., Lynch, P. J., Cuddihy, L., Gentile, D. A., Klonsky, J. and Merrel , R. (2007). The impact of video games on training surgeons in the 21st century. Archives of Surgery (Chicago, III.: 1960), 142(2), 181–186. https://doi:10.1001/ archsurg.142.2.181 Rossett, A. and Sheldon, K. (2001). Beyond the Podium: Delivering Training and Performance to a Digital World. San Francisco: Jossey-Bass/Pfeiffer. Rowntree, D. (1994a). Preparing Materials for Open, Distance and Flexible Learning. London: Kogan Page. Rowntree, D. (1994b). Teaching through Self-instruction. London: Kogan Page. Rozman, R. (2002). Pojmovanje in razvoj managementa. In S. Možina. (ed.), Management: nova znanja za uspeh. Radovljica: Didakta. Rutar Ilc, Z. (2004). Pristopi k poučevanju, preverjanju in ocenjevanju (2nd printing). Ljubljana: National Education Institute. Salmon, G. (2004). E-moderating: The key to teaching and learning online (2nd ed.). London and New York: Routledge. Sanagustín, M. P., Maldonado, J. and Morales, N. (2016). Building Management Capacity for MOOCs in Higher Education. Status Report on the Adoption of MOOCs in Higher Education in Latin America and Europe, MOOC- Maker. http://www.mooc-maker.org/?dl_id=30 [accessed: 21 April 2017] 394 Sangrà, A., Vlachopoulos, D. and Cabrera, N. (2012). Building an inclusive definition of e-learning: An approach to the conceptual framework. The International Review of Research in Open and Distributed Learning, 13(2), 145–159 https://doi.org/10.19173/irrodl.v13i2.1161 Schiaffino, S., Garcia, P. and Amandi, A. (2008). eTeacher: Providing person- alized assistance to e-learning students. Computers & Education, 51(4), 1744–1754. https://doi:10.1016/j.compedu.2008.05.008 Schueffel, P. (2017). The Concise FINTECH COMPENDIUM. Freiburg: School of Management. Sclater, N., Peasgood, A. and Mul an, J. (2016). Learning Analytics in Higher Education. A review of UK and International Practice. JISC, Bristol, UK, Janet Network https://www.jisc.ac.uk/reports/learning-analytics-in-higher-education [accessed: 2 September 2018] Seaman, J. E, Allen, E. I. and Seaman, J. (2018). Grade Increase. Tracking Dis- tance Education in the United States. Babson Survey research Group. [ac- cessed: 12 January 2019] https://eric.ed.gov/?id=ED580852 [accessed: 14 October 2022] Shacklock, X. (2016). From Bricks to Clicks. The Potential of Data and Analyt- ics in Higher Education. Higher Education Commission. United Kingdom, Higher Education Commission https://www.policyconnect.org.uk/research/ report-bricks-clicks-potential-data-and-analytics-higher-education [accessed: 2 September 2018] Shah, D. (2017a). By the Numbers: MOOCS in 2017. Class Central https://www. classcentral.com/report/mooc-stats-2017/ [accessed: 20 July 2019] Shah, D. (2017b). Massive List of MOOC Providers around the World. https:// www.class-central.com/report/mooc-providers-list/ [accessed: 21 November 2018] Shah, D. (2018). A Product at Every Price: A Review of MOOC Stats and Trends in 2017. https://www.classcentral.comreport/moocs-stats-and-trends-2017/ [accessed: 19 November 2018] Sharif, A. and Magril , B. (2015). Discussion Forums in MOOCs. International Journal of Learning, Teaching and Educational Research, 12(1). https://www. ijlter.org/index.php/ijlter/article/view/368 [accessed: 10 November 2018] Sharples, M., McAndrew, P., Weller, M., Ferguson, R., Fitzgerald, E., Hirst, T., Mor, Y., Gaved, M. and Whitelock, D. (2012 ). Innovating Pedagogy 2012: Open University Innovation Report 1. Milton Keynes: The Open University https://iet.open.ac.uk/file/innovating-pedagogy-2012.pdf [accessed:16 October 2017] 395 Sharples, M., McAndrew, P., Weller, M., Ferguson, R., FitzGerald, E., Hirst, T., and Gaved, M. (2013). Innovating Pedagogy 2013: Open University In- novation Report 2. Milton Keynes: The Open University https://iet.open.ac.uk/ innovating-pedagogy [accessed: 22 October 2019] Sharples, M., Adams, A., Ferguson, R., Gaved, M., McAndrew, P., Rienties, B., Weller, M. and Whitelock, D. (2014). Innovating Pedagogy 2014: Open Uni- versity Innovation Report 3. Milton Keynes: The Open University https://iet. open.ac.uk/innovating-pedagogy [accessed: 22 October 2019] Sharples, M., Adams, A., Alozie, N., Ferguson, R., Fitzgerald, E., Gaved, M., McAndrew, P., Means, B., Remold, J., Rienties, B., Roschelle, J., Vogt, K., Whitelock, D. and Yarnal , L. (2015). Innovating Pedagogy 2015: Open Uni- versity Innovation Report 4. Milton Keynes: The Open University https://iet. open.ac.uk/innovating-pedagogy [accessed: 22 October 2019] Sharples, M., de Roock , R., Ferguson, R., Gaved, M., Herodotou, C., Koh, E., Kukulska-Hulme, A., Looi, C-K, McAndrew, P., Rienties, B., Weller, M. and Wong, L. H. (2016). Innovating Pedagogy 2016: Open University Inno- vation Report 5. Milton Keynes: The Open University https://iet.open.ac.uk/ innovating-pedagogy [accessed: 22 October 2019] Sharples, M., Taylor J. and Vavoula, G. (2005) . Towards a Theory of Mo- bile Learning. Centre for Educational Technology and Distance Learning, University of Birmingham https://www.researchgate.net/ publication/228346088_towards_a_theory_of_mobile_learning [accessed: 14 October 2022] Shift. Disruptive elearning (2018). A Comprehensive Guide to Mobile Learn- ing Design. https://www.shiftelearning.com/blog/a-comprehensive-guide-to-mobile- learning-design [accessed: 20 April 2019] Shute, V. J., Leighton, J. P., Jang, E. E. and Chu, M.-W. (2016). Advances in the Science of Assessment. Educational Assessment, 21(1), 34–59 https://doi.or g/10.1080/10627197.2015.1127752 Shute, V. J., Ventura, M. and Kim, Y. J. (2013). Assessment and Learning of Qualitative Physics in Newton’s Playground. The Journal of Educational Re- search, 106(6), 423–430 https://doi.org/10.1080/00220671.2013.832970 Siemens, G. (2004). Categories of eLearning. Elearnspace. https://web.archive. org/web/20070209233644/http:/www.elearnspace.org/articles/elearningcategories. htm [accessed: 27 October 2022] Siemens, G. (2005). Connectivism: A Learning Theory for the Digital Age. In- ternational Journal of Instructional Technology and Distance Learning, 2(1) http://www.itdl.org/Journal/Jan_05/article01.htm [accessed: 8 May 2018] 396 Siemens, G. (2007). Connectivism: A Learning Theory for the Digital Age. el- earnspace. https://www.itdl.org/Journal/Jan_05/article01.htm [accessed: 14 October 2022] Siemens, G. and Tittenberger, P. (2009). Handbook of Emerging Technologies for Learning. University of Manitoba https://www.bucks.edu/media/bcccmedialibrary/ documents/academics/facultywebresources/Handbook_Emerging-technologies.pdf [accessed: 14 October 2022] Siemens, G. (2012). MOOCs are real y a platform. elearnspace http://www. elearnspace.org/blog/2012/07/25/moocs-are-really-a-platform/ [accessed: 10 November 2018] Siemens, G., Dawson, S. and Lynch, G. (2013). Improving the Quality of Produc- tivity of the Higher Education Sector: Policy and Strategy for Systems-Level Deployment of Learning Analytics. SoLAR. https://solaresearch.org/wp-content/ uploads/2017/06/SoLar_report_2014.pdf [accessed: 2 September 2018] Siemens, G. (2013). Massive Open Online Courses. Innovation in Education. In McGreal, R., Kinuthia, W., Stewart Marshal , S. Open Educational Resourc- es: Innovation, Research and Practice. Perspectives on Open and Distance Learning. Commonwealth of Learning and Athabasca University. http:// oasis.col.org/handle/11599/486 [accessed: 27 October 2018] Simon, H. A. (1974). How big is a chunk? Science, 183(4124), 482–488. https:// doi.org/10.1126/science.183.4124.482 Simpson, O. (2015). Student Support Service for Success in Open and Distance Learn- ing. CEMCA EdTech Notes. https://empower.eadtu.eu/images/fields-of-expertise/ StudentSupport/Student_Support_Services_for_Success_in_OED_SiMPSON.pdf [accessed: 10 October 2022] Sink, D. L. (2014). Design Models and Learning Theories for Adults. In E. Biech (ed.), ASTD Handbook: The Definitive Reference for Training & Develop- ment (pp. 181–199). Alexandria, VA: ASTD Press. SiolNET (2019). Uradno: Slovenija bo dobila globalni center za umetno inteli- genco https://siol.net/novice/slovenija/uradno-slovenija-bo-dobila-globalni-center- za-umetno-inteligenco-512727 [accessed: 12 March 2019] SIO: Slovenian Education Network. MOOC (2019) https://skupnost.sio.si/course/ index.php?categoryid=933 [accessed: 19 November 2019] Slovenian Artificial Intelligence Society (n.d.). History of AI in Slovenia. https:// slais.ijs.si/index.php/history [accessed: 8 October 2022] 397 Slovenija 2050. Prosto dostopni izobraževalni viri (OER) (2017) https:// slovenija2050.si/2017/09/20/prosto-dostopni-izobrazevalni-viri-oer [accessed: 5 March 2019] Smith, C. C., Cihak, D. F., Kim, B., McMahon, D. D. and Wright, R. (2017). Ex- amining Augmented Reality to Improve Navigation Skil s in Postsecondary Students with Intellectual Disability. Journal of Special Education Technol- ogy, 32(1), 3–11 . https://doi:10.1177/0162643416681159 Smith, K. T. (2007). Early Attrition among First Time eLearners: A Review of Factors that Contribute to Drop-out, Withdrawal and Non-com- pletion Rates of Adult Learners Undertaking eLearning Programmes. MERLOT. Journal of Online, 2 (2) 73–85. https://jolt.merlot.org/documents/ Vol2_No2_tylerSmith_000.pdf [accessed: 15 October 2022] Spivack, N. (2004). New Version of My ”Metaweb” Graph - The Future of the Net. http://www.novaspivack.com/science/new-version-of-my-metaweb-graph-the-future- of-the-net [accessed: 23 May 2017] Stanton, G. and Ophoff, J. (2013). Towards a Method for Mobile Learning De- sign. Issues in Informing Science and Information Technology, 10, 501–523 https://www.informingscience.org/Publications/1825 [accessed: 1 June 2018] Statista (2019). Artificial Intel igence. Revenues from the Artificial Intel igence (AI) Software Market Worldwide from 2018 to 2025. https://www.statista.com/ statistics/607716/worldwide-artificial-intelligence-market-revenues/ [accessed: 20 August 2019] Statistics Slovenia (2017). Metodološko pojasnilo. Uporaba informacijsko-ko- munikacijske tehnologije v gospodinjstvih in pri posameznikih. https://www. stat.si/StatWeb/File/DocSysFile/8277/29-105-MP.pdf [accessed: 12 November 2017] Storycenter – Our story. (2018) . https://www.storycenter.org/press/ [accessed: 16 October 2018] SURF (2014). Open education. Trend Report. A Publication by the Open Edu- cation Special Interest Group https://www.surf.nl/files/2019-04/trendrapport- open-education-2014-eng.pdf [accessed: 14 October 2022] Surfnet (2016). Trend Report 2016. How Technological Trends Enable Custom- ised Education https://www.surf.nl/files/2019-03/trend-report-2016.pdf [accessed: 14 October 2022] Sursock, A. (2015). Trends 2015: Learning and Teaching in European Universities. Brussels, European University Association https://eric.ed.gov/?id=ED588102 [accessed: 13 September 2017] 398 Swan, K., Garrison, D. R. and Richardson, J. C. (2009). Information Technolo- gy and Constructivism in Higher Education: Progressive Learning Frame- works. In C. R. Payne (ed.), Information Technology and Constructivism in Higher Education: Progressive Learning Frameworks (pp. 43–57). Hershey, PA: IGI Global. Šćepanović, S., Devedžić, V. and Kraljevski, I. (2011). E-learning Benchmark- ing: Methodology and Tools Review. The Second International Conference on E-Learning (e-learning-2011), Belgrade, Serbia. Š ooš, T. (2017). Strategija razvoja Slovenije 2030 Vlade Republike Slovenije. http://www.vlada.si/fileadmin/dokumenti/si/projekti/2017/srs2030/ Strategija_razvoja_Slovenije_2030.pdf [accessed: 6 February 2018] Toppo, G. (2015). The game believes in you: How digital play can make our kids smarter. New York: Palgrave Macmil an Trade. Torres, M. and Xalabarder, R. (2019). Interim Report on Practices and Chal- lenges in Relation to Online Distance Education and Research Activities. WIPO. Standing Committee on Copyright and Related Rights, 38th Ses- sion. https://www.wipo.int/meetings/en/doc_details.jsp?doc_id=432192 [accessed: 22 August 2019] Trampuž, M., Oman, B. and Zupančič, O. (1997). Zakon o avtorski in sorodnih pravicah s komentarjem. Ljubljana: Gospodarski vestnik. Traxler, J. (2007). Current State of Mobile Learning. International Review on Research in Open and Distance Learning, 8(2). https://doi.org/10.19173/irrodl. v8i2.346 Tsai, Y-S. and Gašević, D. (2017). The State of Learning Analytics in Europe. Executive Summary. SHEILA http://sheilaproject.eu/ [accessed: 2 September 2018] Tuomi, I. (2018). The Impact of Artificial Intel igence on Learning, Teaching, and Education. Policies for the future. Publications Office of the European Union, Luxembourg. https://doi:10.2760/12297JrC113226 Twigg, C. (2001). Innovations in Online Learning: Moving Beyond no Significant Difference. The Pew Learning and Technology Program. [accessed: 17 Oc- tober 2005] https://library.educause.edu/resources/2002/1/innovations-in-online- learning-moving-beyond-no-significant-difference [accessed: 14 October 2022] UNESCO (1952). Universal Copyright Convention. https://en.unesco.org/about- us/legal-affairs/universal-copyright-convention-appendix-declaration-relating-article- xvii-and [accessed: 14 October 2022] 399 UNESCO (2002). Forum on the Impact of Open Courseware for Higher Educa- tion in Developing Countries. Final report. Paris, 1–3 July 2002. (CI-2002/ CONF.803/CLD.1). http://unesdoc.unesco.org/images/0012/001285/128515e.pdf [accessed: 5 May 2017] UNESCO (2017). Second World OER Congress. Ljubljana OER Action Plan 2017 https://en.unesco.org/sites/default/files/ljubljana_oer_action_plan_2017.pdf [accessed: 2 February 2019] UNESCO (2019a). UNESCO Recommendation Concerning Open Educa- tional Resources OER . https://en.unesco.org/news/unesco-recommendation-open- educational-resources-oer [accessed: 20 November 2019] UNESCO (2019b). Artificial Intel igence in Education: Chal enges and Oppor- tunities for Sustainable Development. Working papers on educational Policy No. 7. https://unesdoc.unesco.org/ark:/48223/pf0000366994 [accessed: 12 December 2019] UNSW Teaching (2019). https://teaching.unsw.edu.au/simulations [Accessed: 2 May 2019] Uradni list Republike Slovenije (1995). Copyright and Related Rights Act. Of- ficial Journal of the Republic of Slovenia, No. 21/95. www.uradni-list.si/1/ index?edition=199521 [accessed: 15 January 2010] Uradni list Republike Slovenije (2004). Resolution on the National Programme of Adult Education in the Republic of Slovenia until 2010. Official Journal No. 70/2004, p. 8591 http://www.uradni-list.si/1/objava.jsp?urlid=200470&stevilka=3149 [accessed: 15 December 2009] Uradni list Republike Slovenije (2013). Resolution on the National Programme of Adult Education in the Republic of Slovenia 2013–2020 (ReNPIO13-20. https://www.uradni-list.si/glasilo-uradni-list-rs/vsebina/114925 [accessed: 6 May 2015] Uradni list Republike Slovenije (2016). Copyright and Related Rights Act. Official Journal of the Republic of Slovenia, No. 16 http://pisrs.si/Pis.web/ pregledPredpisa?id=ZaKO403 [accessed: 15 June 2019] Uradni list republike Slovenije (2017). Vključevanje uporabe informacijsko- komunikacijske tehnologije v visokošolskem pedagoškem procesu, No. 12, 10 March 2017. https://www.uradni-list.si/_pdf/2017/ra/r2017012.pdf [accessed: 12 April 2017] Uradni list republike Slovenije (2018). Adult Education Act. Official Journal of the Republic of Slovenia No. 6/XXVIII, https://www.uradni-list.si/_pdf/2018/ur/ u2018006.pdf [accessed: 2 March 2019] Urbančič, T., Polajnar. A. and Jermol, M. (2019). Open Education for a Bet- ter World: A Mentoring Programme Fostering Design and Reuse of Open Educational Resources for Sustainable Development Goals. Open Praxis, 11 (4), 409–426. Open Education Global Conference. Selected Papers. https:// doi.org/10.5944/openpraxis.11.4.1026 [accessed: 2 December 2019] US Department of Education. (2017). Reimagining the Role of Technology in Education: 2017 National Education Technology Plan Update. https://tech. ed.gov/files/2017/01/NEtP17.pdf [accessed: 10 October 2022] USNEI - United States Network for Education Information (2001). Accredita- tion Described. https://www2.ed.gov/about/offices/list/ous/international/usnei/us/ edlite-accreditation.html [accessed: 27 October 2022] VanLehn, K. (2011). The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems. Educational Psychologist, 46(4), 197–221. https://doi:10.1080/00461520.2011.611369 Verkasalo, H. (2009). Contextual patterns in mobile service usage. Journal of Personal and Ubiquitous Computing, 13(5), 331–342. Vipond, S. (2017). Is Instructional Design a Dying Art? Industry Thought Lead- ers Share Insights. The elearning Guild Research https://www.elearningguild. com/content/5081/research-is-instructional-design-a-dying-art [accessed: 22 February 2019] Weller, M. (2011). A Pedagogy of Abundance. Spanish Journal of Pedagogy, 249, pp. 223–236 http://oro.open.ac.uk/28774 [accessed: 13 September 2017] Weller, M. (2014). The Battle for Open: How Openness Won and Why it doesn’t Feel Like Victory. London, Ubiquity Press. https://www.ubiquitypress.com/site/ books/10.5334/bam/ [accessed: 5 May 2017] WePC. (2019). 2019 Video Game Industry Statistics, Trends & Data. https://www. wepc.com/news/video-game-statistics/ [accessed: 5 August 2019] Wheeler, S. (2009). eLearning 3.0. http://www.steve-wheeler.co.uk/2009/04/ learning-30.html [accessed: 13 September 2017] Wiley, D., Waters, S., Dawson, D., Lambert, B., Barclay, M., Wade, D. and Nel- son, L. (2004). Overcoming the Limitations of Learning Objects. Journal of Educational Multimedia and Hypermedia, 13(4), 507-521. Norfolk, VA: As- sociation for the Advancement of Computing in Education (AACE). https:// www.learntechlib.org/primary/p/6586/ [accessed: 22 August 2017] Wiley, D. and Hilton, J. (2018). Defining OER-enabled pedagogy. The Interna- tional Review of Research in Open and Distributed Learning, 19(4). https:// doi.org/10.19173/irrodl.v19i4.3601 401 Wilson, A. (2012). Categorising E-learning. Journal of Open, Flexible and Dis- tance Learning, 16(1), 156–165 https://www.learntechlib.org/p/147979/ [accessed: 2 February 2019] Wither, J., DiVerdi, S. and Hollerer, T. (2009). Annotation in outdoor aug- mented reality. Computers & Graphics, 33(6), 679–689. https://doi:10.1016/j. cag.2009.06.001 Woody, M (2016). Why so Many Companies Want to Replace the Online Train- ing Platform. Learning Technology Trends, Brandon Hall Group. https:// www.docebo.com/2016/05/24/online-training-platforms-trends/ [accessed: 10 July 2017] Woolf, B. P., Lane, H. C., Chaudhri, V. K. and Kolodner, J. L. (2013). AI Grand Challenges for Education. AI Magazine, 34(4), 66-84 https://doi.org/10.1609/ aimag.v34i4.2490 Xalabarder, R. (2004). Copyright Exceptions for Teaching Purposes in Europe. UOC. Interdisciplinary institute. Working paper series WP04–004. http:// www.uoc.edu/in3/dt/eng/20418.html [accessed: 3 February 2010] Yang, Y.-T. C. and Wu, W.-C. I. (2012). Digital storytelling for enhancing stu- dent academic achievement, critical thinking, and learning motivation: A year-long experimental study. Computers & Education, 59(2), 339–352. https://doi:10.1016/j.compedu.2011.12.012 Yeager, C., Hurley-Dasgupta, B. and Bliss, C.A. (2013). cMOOCs and Global Learning: An Authentic Alternative. Journal of Asynchronous Learning Net- works, 17(2), 133-147. https://www.learntechlib.org/p/154153/ [accessed: 15 October 2022] Yorke, M. (1999). Leaving Early: Undergraduate Non-completion in Higher Ed- ucation. London: Falmer. Zichermann, G. (2010). Fun is the Future: Mastering Gamification. https:// youtube/6O1gNVeaE4g [accessed: 2 August 2019] Zikmund, W. G. (1994). Business Research Methods. Mason: Thomson South- Western. Document Outline Preface Introduction About the Authors List of Abbreviations Theoretical and Developmental Aspects of E-Learning 1.1 What is E-Learning 1.1.1 Definitions of E-Learning 1.1.2 Characteristics of E-Learning in the Narrow Sense 1.1.3 E-Learning Classifications 1.1.4 Potential Benefits of E-Learning 1.1.5 Realising the Benefits of E-Learning 1.2 Overview of the State of E-Learning 1.2.1 The Global E-Learning Market 1.2.2 E-Learning in the European Union and Slovenia 1.3 Web Development and the Future of E-Learning 1.3.1 The Evolutionary Stages of the Web and E-Learning 1.3.2 Web 3.0 and Trends in the Development of E-Learning Business and Organisational Aspects of E-Learning Planning 2.1 Preparation of the E-Learning Development Strategy 2.1.1 General Aspects of Strategic Planning 2.1.2 Strategic Planning Aspects of E-Learning 2.2 Creating a Business Plan 2.2.1 Economic Aspects of E-Learning 2.2.2 Business Plan in E-Learning Pedagogical Aspects of E-Learning Planning 3.1 Instructional Design Models for E-Learning 3.1.1 Instructional Design 3.1.2 Instructional Models for E-Learning 3.2 Theories of Learning 3.2.1 Theory of Behaviourism 3.2.2 Cognitive Theory 3.2.3 Theory of Constructivism 3.2.4 Theory of Connectivism 3.2.5 The Theory of Community of Inquiry 3.2.6 Pedagogy of Abundance 3.2.7 Heutagogy 3.2.8 The Relevance of Learning Theories for the E-Learning Instructional Design 3.3 Educational Needs Analysis 3.3.1 Objectives of the Educational Programme 3.3.2 Programme Analysis 3.3.3 Analysis of Learners 3.3.4 Conducting an Educational Needs Analysis 3.4 Instructional Design of an E-Learning Programme 3.4.1 Framework of an Instructional Design for E-Learning 3.4.2 Learning Objectives 3.4.3 Learning Theories and Learning Objectives 3.4.4 Learning and Teaching Strategies and E-Learning 3.4.5 Linking the Pedagogical and Business-Organisational Components The Development of E-Learning Programmes 4.1 Learning Material 4.1.1 Starting Points for the Development of Learning Material 4.1.2 Content of the Learning Material 4.1.3 Storyboard for Learning Material 4.2 Learning Activities 4.2.1 The Role of Learning Activities in E-Learning 4.2.2 Types of Learning Activities 4.2.3 Learning Activities in E-Learning 4.2.4 Learning Activities as an Integral Part of the Learning Material 4.3 Assessment Methods in E-Learning 4.3.1 General Aspects of Assessment 4.3.2 Bloom’s Taxonomies of Learning Objectives 4.3.3 Traditional and Alternative Methods of Assessment 4.3.4 The Benefits of Technology-Enhanced Testing and Assessment 4.3.5 Specific Features of Assessment in E-Learning 4.4 Selecting and Integrating Media into the Programme 4.4.1 Media and Technologies 4.4.2 Criteria for Media Selection 4.4.3 Usability and Limitations of Media in Education 4.5 Digital Tools for E-Learning 4.5.1 Digital Tools for E-Learning 4.5.2 Learning Management Systems 4.5.3 Authoring Tools 4.5.4 The Choice of Digital Tools Pedagogical Support 5.1 General Features of Pedagogical Support in E-Learning 5.2 The Changing Role of Teachers and Learners 5.3 The Tutor’s Competencies in E-Learning 5.4 Gilly Salmon’s E-Moderating Model 5.5 Planning the Tutor’s Work Learning Approaches and Methods in a Digital Society 6.1 Introduction 6.2 Open Education 6.2.1 Opening up Education and Open Education 6.2.2 Categories of Open Education 6.2.3 Open Educational Resources (OER) 6.2.4 Massive Open Online Courses (MOOCs) 6.2.5 The State of Open Education in the World, the European Union and Slovenia. 6.3 Artificial Intelligence 6.3.1 What is Artificial Intelligence 6.3.2 The Impact of Artificial Intelligence on Education 6.3.3 Artificial Intelligence and E-Learning 6.3.4 The Limits and Challenges of Artificial Intelligence 6.3.5 Artificial Intelligence in Slovenia 6.4 Learning Analytics 6.4.1 The Concept and Importance of Learning Analytics 6.4.2 The Design and Development of Learning Analytics 6.4.3 Learning Analytics Implementation Circle 6.4.4 Learning Analytics Models by Level of Complexity 6.4.5 The State of Learning Analytics in Practice 6.5 Intelligent Tutoring Systems 6.5.1 What are ITSs 6.5.2 Virtual Assistants 6.5.3 Using ITS in Education 6.5.4 Benefits and Limitations of ITSs 6.6 Mobile Learning 6.6.1 What is M-Learning 6.6.2 The Benefits of M-Learning 6.6.3 Limitations of M-Learning 6.6.4 Planning M-Learning 6.6.5 Tools for M-Learning 6.7 Microlearning 6.7.1 Characteristics of Microlearning 6.7.2 Benefits and Limitations of Microlearning 6.7.3 The Preparation of Microlearning 6.8 Gamification 6.8.1 Introduction 6.8.2 Gamification in Education 6.8.3 Types of Gamification 6.8.4 Benefits and Limitations of Gamification 6.8.5 Gamification Tools 6.9 Simulations in Education 6.9.1 Concept and Meaning of Simulations 6.9.2 Simulations in Education 6.9.3 E-Learning and Simulations 6.9.4 Design and Development of Simulations 6.9.5 Authoring Tools for Developing Simulations 6.10 Virtual and Augmented Reality 6.10.1 The Difference Between Virtual or Augmented Reality 6.10.2 Virtual Reality 6.10.3 Augmented Reality 6.10.4 Using Virtual and Augmented Reality in E-Learning 6.10.5 Virtual Reality and Augmented Reality Tools 6.11 Digital Storytelling 6.11.1 What is Digital Storytelling 6.11.2 Elements of Digital Storytelling 6.11.3 Types of Digital Stories 6.11.4 Digital Storytelling in Education 6.11.5 Tools for Creating Digital Stories Management in the Delivery of E-Learning 7.1 Management in E-Learning – General Aspects 7.1.1 The Importance of Management 7.1.2 Definition and Levels of Management 7.1.3 Specific Features of E-Learning Management 7.1.4 Project Management 7.2 Services for Learners 7.2.1 Administrative and Technical Services 7.2.2 Library 7.2.3 Counselling and Information Services 7.3 E-Learning Results 7.3.1 Educational Success Rate and Drop-Out Rate 7.3.2 The Quality of the Results 7.3.3 Business Efficiency 7.3.4 Access to Education and Training 7.4 Copyright and E-Learning 7.4.1 The Origins of Copyright Law 7.4.2 International Conventions and Agreements 7.4.3 Copyright in Slovenia 7.4.4 Copyright and E-Learning 7.5 Quality Assurance in E-Learning 7.5.1 General Aspects of the Quality in Education 7.5.2 Quality Assurance in E-Learning 7.5.3 Domains of Quality Assurance in E-Learning 7.5.4 Quality Assurance and Evaluation 7.5.5 Quality and Development of E-Learning List of Figures List of Tables Literature International comparison of Internet use for education, 2017 (Percentage of the total population aged 16–74) Associated terms with Web 4.0 paradigm in the literature 2009–2017 Concepts and technologies in the evolutionary stages of the web and e-learning Types of costs by source of incurrence for the main phases of e-learning development and delivery Average spending (in hours) per hour of training for different training modes, 2017 A sample example of the elements of a business plan in e-learning Comparison of the ADDIE model and the Agile planning model Analysis of the characteristics of participants in the Food Security Assessment Programme Bloom’s taxonomy of cognitive levels with a description of the learning objectives Theoretical background for designing learning objectives Absorb-type, do-type and connect-type learning activities Reasons for testing knowledge and skills Bloom’s original taxonomy of cognitive learning objectives Bloom’s revised taxonomy of cognitive learning objectives, with a description of the characteristic categories Bloom’s revised digital taxonomy Verbal descriptions of learning objectives Example of a rubric Comparison of human and technology-based education Example of comparing the characteristics of authoring tools Flexibility of education in time and place Comparison of the teacher’s role in traditional education and the tutor’s role in e-learning Roles of learners in traditional education and e-learning Comparison of the competencies needed to teach in a traditional and virtual classroom Learning and teaching concepts and innovative teaching and learning approaches and methods to realise the learner-centred paradigm General open education framework: definitions and components Components of the transversal dimensions of the Open Education Framework An overview of the main features of some typical MOOCs The most important MOOC portals and their characteristics The world’s largest providers of MOOCs Example of the definition of the critical dimensions of LA framework for the SNAPP tool The seven elements of digital storytelling The Creative Commons license options ENQA’s internal and external quality assurance domains Quality assurance areas for e-learning 1.0 and 2.0