GUIDELINES FOR PREPARATION OF PUBLICLY VERIFIED EDUCATION PROGRAMMES FOR ADULTS 1 CONTENTS PREFACE .................................................................................................................................................... 4 1 TERMS AND ABBREVIATIONS ......................................................................................................... 6 GUIDELINES FOR PREPARATION OF PUBLICLY VERIFIED EDUCATION PROGRAMMES FOR ADULTS 2 INTRODUCTIONW ............................................................................................................................... 11 3 TYPES OF PUBLICLY VERIFIED EDUCATION PROGRAMMES FOR ADULTS ................ 15 Slovenian Institute for Adult Education - SIAE. (2020). Guidelines for Preparation of Publicly Verified Education Programmes for Adults. Ljubljana: SIAE 3.1 Programmes for acquisition and raising the level of literacy and key competences ................................................................................................................ 15 Original publication: Andragoški center Republike Slovenije - ACS. (2020). Izhodišča za pripravo javnoveljavnih izobraževalnih programov za odrasle. Ljubljana: ACS 3.2 Programmes for improvement of the level of general education ........................... 17 Publisher: Slovenian Institute for Adult Education 3.3 Public validity of programmes ............................................................................................ 18 Editors: Estera Možina, MSc., Tanja Možina, Ph.D., Natalija Žalec, MAEd 4 CONDITIONS FOR PREPARATION OF PROGRAMMES AND APPRAISAL AT THE Working group: Katja Dovžak, MSc, president. Members (in alphabetical order): Elido Bandelj, COUNCIL OF EXPERTS ......................................................................................................................... 19 Teja Dolgan, Alenka Grželj, Sonja Kump, Ph.D., Estera Možina, MSc., Tanja Možina, Ph.D., Janko Muršak, Ph.D., Ema Perme, Maja Radinovič Hajdič, MSc., Boštjan Rozman Zgonc, Natalija Žalec, 5 PRINCIPLES FOR PLANNING ADULT PROGRAMMES ............................................................21 MAEd 5.1 Andragogy cycle as the foundation for planning programmes .................................21 Other contributors: Andrej Sotošek, MSc., Olga Drofenik 5.2 Identifying the needs of individuals and society for education ................................21 English translation: Julija Translation Agency Ltd. 5.3 Selection of a development strategy .................................................................................. 22 Proof-readers: General Secretariat of the Government of the Republic of Slovenia, Translation Department, and Ana Peklenik 5.4 Programme structured in modules ..................................................................................... 23 Corrector: Laura Fink, Ph.D. 5.5 Programme openness .............................................................................................................. 23 Design and layout: Urška Stariha s.p. 5.6 Flexibility in programme planning .................................................................................... 24 Year: 2020 5.7 Programme monitoring and evaluation ............................................................................ 24 Digital edition: web page https://arhiv.acs.si/dokumenti/Guidelines_for_preparation_of_ 6 PROGRAMME COMPONENTS ........................................................................................................26 publicly_verified_education_programmes_for_adults.pdf 6.1 Programme name ...................................................................................................................... 26 The Council of Experts for Adult Education adopted the Guidelines for Preparation of Publicly Verified Education Programmes for Adults at its 103. correspondence meeting held on 14.5.2020 6.2 Purpose of the programme ....................................................................................................26 with the prior consent of the Minister of Education, Science and Sport, Jernej Pikalo, Ph.D. (No. 6.3 Target group ................................................................................................................................ 27 6036-256/2019/1 dated 29.7.2019). 6.4 Programme goals ...................................................................................................................... 28 6.5 Knowledge catalogue or programme contents ..............................................................29 6.5.1 Structure and content design of the programme .................................................29 Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID=37273859 6.5.2 Knowledge catalogue ...................................................................................................... 32 ISBN 978-961-7116-04-5 (pdf) 6.6 Duration of education ............................................................................................................. 39 6.7 Examination catalogue or other methods for assessment of knowledge ............40 6.8 Conditions for enrolment .....................................................................................................44 6.9 Conditions for progression and completion of the programme .............................. 45 6.10 Organisation of education ....................................................................................................46 6.11 Knowledge and training of education professionals ................................................... 47 6.12 Certificate of training ............................................................................................................ 50 7 REFERENCES .........................................................................................................................................51 2 3 Education, Science and Sports made the printed and online versions and the translation PREFACE into English possible, in order to keep the Guidelines at hand at all times, accessible to all, and to trigger the development of a number of new publicly verified programmes for adults. The text titled the Guidelines for Preparation of Publicly Verified Education Programmes for Adults (hereinafter Guidelines) lays theoretically sound bases and professional Estera Možina guidelines for programming and curricular planning in adult education for a segment Slovenian Institute for Adult Education of publicly verified education programmes for adults in Slovenia. Adult Education Law from 2018 (hereinafter ZIO-1) defined the types of publicly verified programmes for adults. Namely, programmes for acquisition and raising of the level of literacy and key competences, and programmes for improvement of the level of general education. The ZIO-1 also defined the mandatory components of publicly verified education programmes, which do not grant an officially recognised educational qualification. However, one of the novelties of this type of programmes is a certificate of training which is an official document that adults can use as evidence in the procedures of validation of acquired knowledge when entering further education or the labour market. The working group appointed by the Minister of Education, Science and Sports in May 2018 (names of the working group members are provided in the introduction) had a mandate to prepare the Guidelines based on the legal provisions. The Guidelines bring some novelties compared to the previous ones; for example, the principles and approaches to curricular planning in adult education, flexibility in the planning and implementation of educational programmes, emphasis on the basic skills, the possibility of modular design, and uniform or content area design of this type of programmes. The working group has discussed the purpose and expected outcomes of publicly verified programmes and acknowledged the potential of flexible programmes for adults. With their contents, forms of implementation and an open curriculum, flexible educational programmes are tailored to respond quickly to the needs of adults and society, tackling the environmental crisis as well as economic, migrant, value crisis, and last but not least, global health crisis. Publicly verified programmes equip adults with the knowledge and skills to be able to actively respond to these challenges and offer support to their loved ones. The text of the Guidelines contains professional guidelines and steps for the preparation of quality and tailor-made adult education programmes in various fields, such as education, work, agriculture, health, culture, environment, home affairs, etc. The Guidelines are also prepared in order to be useful as a tool for appraisal of the proposals of publicly valid programmes at the Council of Experts for Adult Education. Developers and auditors will be assisted in this by the conditions for the preparation of the programmes and the criteria for the appraisal of the programmes. The Ministry of 4 5 variety of skills), motivational and value orientations, views, emotions and other social TERMS AND ABBREVIATIONS 1 and behavioural components. Competences comprise internal mental structures – meaning the abilities, capacities and dispositions that are characteristic of an individual. They are divided into general or generic competences, which are developed mainly Adults (odrasli) in general education programmes, and vocation-specific competences, which are These are individuals who have fulfilled their legal compulsory education requirement characteristic of individual professions or areas of work. Publicly verified programmes and wish to acquire, update, broaden and deepen their knowledge, or individuals of are intended primarily for the development of general or generic competences. at least 15 years of age for whom it cannot be determined whether they fulfilled their legal compulsory education requirement or who failed to fulfil it due to exceptional Content areas (vsebinski sklop) circumstances where their life and health were at risk (e.g. war, natural disasters). Content areas, as these Guidelines determines it, are substantively coherent units in compulsory or elective parts of a programme structured into content areas and Adult education professionals (strokovne delavke oziroma strokovni delavci) not structured in modules. Each comprises a set of knowledge and skills that is part Adult education professionals (hereinafter “education professionals”) in the of a greater whole and is a coherent didactic unit allowing for ongoing monitoring programmes are teachers, adult education organisers and other professionals. All of and formative assessment of learning achievements and the assessment of learning them must meet the following legal requirements: knowledge of Standard Slovenian, outcomes. Content areas can be upgraded or just linked to each other. a relevant educational qualification from a second-cycle study programme or a level of educational qualification corresponding, in accordance with the act regulating Examination catalogue (izpitni katalog) higher education, to the educational qualification of the second-cycle, a pedagogical The examination catalogue is an integral part of publicly verified education programmes and andragogical educational qualification in accordance with the act regulating the that sets out the standards of knowledge and skills in accordance with the objectives organisation and financing of education and teaching certification examination of of publicly verified education programmes for adults and in knowledge catalogues. professional staff in education. It is the basis for measuring the learning outcomes of participants. The examination (Summative) Assessment of knowledge (ocenjevanje znanja) catalogue defines the knowledge and skills for all or part of the programme and enables uniform assessment criteria regardless of the provider, but may also include methods Assessment of knowledge is identification and validation to what extent the participant of formative assessment. meets the goals or standards laid down in the knowledge catalogue or the examination catalogue. It generally comes after or is linked to the final formative assessment. It is Formative assessment (preverjanje znanja) important, however, to differentiate between summative assessment of knowledge and formative assessment, which is unrelated to grades but enables the adult education Formative assessment is used to collect information about how or to what extent professional to adjust the implementation of the programme on an ongoing basis. the participant meets the goals or standards laid down in examination catalogues or knowledge catalogues. There are multiple types of formative assessments. Each Certificate of training (potrdilo o usposabljanju) is serving a different purpose: initial or diagnostic assessment to determine the previously acquired level of knowledge and skills set out in knowledge catalogues and This is the official document the participant receives upon completion of all obligations examination catalogues; ongoing formative assessment, which is used to determine the envisaged under the programme. It includes a supplement containing a list of the extent to which participants have achieved the partial goals of individual programme knowledge and skills acquired in the programme. It serves as proof that the individual components, what they have problems with and what their strengths are so as to has mastered all knowledge and skills determined by knowledge catalogues and improve and upgrade the learning process; final formative assessment performed upon examination catalogues. completion of a given module, content area or whole programme in order to determine whether participants have achieved the goals of the programme or its constituent Competence (kompetenca ali zmožnost) part. The basic purpose of formative assessment is to get feedback on the progress Competence indicates an individual’s ability to make appropriate use of their learning of participants. In these programmes the final formative assessment may also be achievements in a variety of circumstances (education, work, personal or professional performed as a part of the examination procedure. development). It cannot be limited to the cognitive dimension alone (use of theory, concepts and tacit or hidden knowledge); it also contains a practical aspect (including a 6 7 Knowledge (znanje) Open part of the programme (open curriculum) (odprti del programa, odprti kurikul) Knowledge is a cognitive aspect of competence and is directly measurable. It is a set This is a feature of programmes or parts of programmes where the goals, contents of facts, principles and theories related to a given field of education that is acquired and/or forms of education are fully or partially defined only tentatively or not at through learning. We distinguish between declarative (theoretical, propositional) and all. Providers determine programme contents, forms and methods of work together procedural (process, practical, dispositional) knowledge. Declarative knowledge with participants. In terms of openness, programmes range from completely open to includes statements about certain events, facts and empirically substantiated programmes where only one part or component is open (e.g. methods, forms of work). generalisations as well as in-depth principles about the nature of things. Procedural Programmes can include an open curriculum only up to the extent where it is still knowledge includes heuristic knowledge, methodology, planning of practice, guaranteed that participants will achieve the standard of knowledge determined by appropriate procedures and skills-related strategies. With respect to the knowledge knowledge catalogues and examination catalogues and master the skills. catalogue, knowledge is understood as a hypernym related to the entire content and expected learning achievements of the individual programme or its constituent part. Participants (udeleženke oziroma udeleženci) Participants are adults who enroll in or join a programme. Knowledge catalogue (katalog znanja) The knowledge catalogue is a document that forms the basis for the implementation of Provider (izvajalec) the individual parts, subjects or modules of programmes for adults. It contains the name The provider is a legal person that provides education programmes for adults and adult of the subject, module or other constituent part of the programme, time frame, main education activities. Pursuant to the act regulating the organisation and financing of goals, recommended contents and the knowledge and skills covered by the individual education, providers carrying out publicly verified education programmes for adults part, subject or module. It also sets out the scope of the elective and the open part of must be entered in the register of publicly verified education programme providers the programme (open curriculum), if these are envisioned by the programme unit. If a kept by the Ministry responsible for education. This gives them the right to issue a programme has no subjects or modules, a single knowledge catalogue is designed for certificate of training, which is considered an official document, including a supplement the whole programme. containing a list of the knowledge and skills acquired in the programme. Learning achievements (učni dosežki) Publicly verified education programmes for adults (javnoveljavni izobraževalni These have a broader meaning than learning outcomes and include the effects of programi za odrasle) education that are not measured directly in the process of formative assessment Publicly verified education programmes for adults enable adults to obtain and raise or summative assessment of knowledge which are important for monitoring the the level of literacy and key competences as well as to improve the level of general competences of participants. education. They are set out in the Adult Education Act (ZIO-1) and are adopted by the minister of education at the proposal of the Council of Experts. The certificate of Learning outcomes (učni izidi) training received after successfully completed programme is the participant’s proof of These are the sets of knowledge and skills the individual has acquired during the having mastered the knowledge and skills defined by the knowledge catalogue and the educational process. Learning outcomes are defined in the programmes and represent examination catalogue. the expected level of knowledge and skills the participant needs to demonstrate; they are examined in the assessment procedures. Skill (spretnost, veščina) In addition to knowledge, skills are a directly measurable component of competence. Module (modul) Together with the knowledge and experience acquired and the abilities developed, Programmes can be designed to include modules as their programme units. The module skills allow an individual to solve problems and perform certain tasks or jobs. in such cases corresponds to a part of the programme and is a coherent set of goals and contents. It has different aims and makes it possible to divide the programme into parts Target group (ciljna skupina) that the participant can complete gradually. Thanks to this, education can be organised The target group includes potential participants of publicly verified education economically, which is especially important for adult education. Modules can be either programmes for adults. As a rule, the programmes are designed to take into account compulsory or elective, whereby programme contents can be adjusted according to the the educational needs of adults and are intended for a group of adults with certain features of the different target groups. characteristics. 8 9 ABBREVIATIONS USED INTRODUCTION 2 EQF - European Qualifications Framework The second point of Article 17 of the Adult Education Act (Official Gazette of the Republic MIZŠ – Ministry of Education, Science and Sport of Slovenia, No. 6/18; ZIO-1) provides that the Guidelines for Preparation of Publicly Verified Education Programmes for Adults be produced by the Slovenian Institute for OECD – The Organisation for Economic Cooperation and Development Adult Education (hereinafter SIAE) and be adopted by the Council of Experts for Adult ReNPIO13-20 – Resolution on the Master Plan for Adult Education in the Republic of Education (hereinafter Council of Experts). By decision No. 024-24/2018 dated 8 May Slovenia for 2013–2020 2018, the Minister of Education, Science and Sport appointed a working group1 for the SIAE – Slovenian Institute for Adult Education drawing up of the Guidelines for Preparation of Publicly Verified Education Programmes for Adults (hereinafter Guidelines). The working group also invited other professionals SQF – Slovenian Qualifications Framework to participate in the preparation of the Guidelines.2 ZIO-1 – Adult Education Act, 2018 Publicly verified education programmes for adults (hereinafter programmes) are programmes that contribute to raising the level of literacy and key competences and to improving the level of general education of adults, but they do not grant an officially recognised educational qualification. In terms of pursuing the principles and goals of public interest, these programmes are important primarily because they: • pursue the principle of lifelong education and learning; • contribute to the connection between formal and non-formal education at the systemic level; • promote education and learning among low-educated and other vulnerable groups of adults; • enable the improvement of knowledge after acquiring an educational qualification or facilitate inclusion in formal education leading to an educational qualification or vocational qualifications; • enable the acquisition of the knowledge, skills and personality traits needed by adults for a successful and quality life and work and to be able to get involved and participate constructively in social processes; • strengthen competences for mutual tolerance, respect for human diversity and mutual cooperation and respect for human rights and fundamental freedoms; 1 Working group composed of: Katja Dovžak, MSc, president. Members: Elido Bandelj, Teja Dolgan, Alenka Grželj, Sonja Kump, Ph.D., Estera Možina, MSc., Tanja Možina, Ph.D., Janko Muršak, Ph.D., Ema Perme, Maja Radinovič Hajdič, MSc., Boštjan Rozman Zgonc, Natalija Žalec, MAEd (GB) The working group completed its work on drafting the proposal for the Guidelines in March 2019. 2 Other professionals: Andrej Sotošek, MSc., Olga Drofenik. 10 11 • ensure learning outcomes that match the SQF and EQF level descriptors. They The Survey of Adult Skills in Slovenia (“PIAAC 2016”) identified gaps in the basic skills include current topics and modern methodical education and develop critical of adults (literacy, numeracy and problem solving in technology-rich environments) in thinking, etc.; a significant portion of the adult population. These refer to the proficiency of adults in information-processing skills essential for full participation in the knowledge-based • include aspects of sustainable development, green economy, culture and health. economies and societies of the 21st century. One in four adults aged 16–65 scored lower in all measured skills than the average across developed OECD countries. According to With these programmes, we are fulfilling the commitments from the national and the survey results, these skills are necessary for fully integrating and participating in the the European strategic development documents related to adult education. These labour market and for lifelong learning, are transferable to a variety of social contexts programmes are driving the implementation of the Resolution on the Master Plan and work situations and can be developed and improved throughout a person’s life. for Adult Education in the Republic of Slovenia for 2013–2020 (Official Gazette of the Republic of Slovenia, No. 90/13; “ReNPIO13-20”), specifically its first goal, which refers Both in terms of the assessed needs of adults for the improvement of literacy, key to “raising the level of education of the population and the level of key competences”, competences and general education and in terms of national and European strategic and its fourth goal, which refers to “improvement of the level of general education”. By orientations, the development of the programmes is expected to bring about long-term drafting these Guidelines, we want to ensure the development of a quality and balanced positive effects in the following areas: offer of these programmes for various adult target groups. • The strengthening of key competences in vulnerable groups of adults drives economic efficiency and social justice, as they have a considerable impact on the In addition to the adult education goals from the ReNPIO13-20, these programmes distribution of economic and social achievements in society. If a large proportion also serve to meet the guidelines for the development of literacy, key competences of adults do not have sufficiently developed reading and mathematical skills, and general education of adults who are less involved in lifelong learning, which are introducing and disseminating productivity-enhancing technologies is more covered in several key European documents listed below. difficult, resulting in potential interruptions to the organisation of work. However, Among other things, the New Skills Agenda for Europe – Working together to the effects of developed skills affect not only earnings and employment but also strengthen human capital, employability and competitiveness (2016) emphasises many other areas, and they have been shown to have significant social effects: the priorities for action in improving the quality and importance of the acquisition better health, higher social inclusion, and they also support higher productivity of knowledge and skills. It underscores strengthening the foundations for acquiring and economic growth, thereby contributing to employment and improving the basic knowledge and skills for flexibility in adapting to change. Further development of quality of jobs (Javrh, 2018). the key competences is also important, especially from the viewpoint of higher, more • “Planned investments into general adult education can indirectly and through versatile knowledge and skills. adequately developed support activities contribute the most to bridging social, intergenerational and spatial differences as well as to social inclusion, active Council Recommendation on Upskilling Pathways: New Opportunities for Adults (2016) citizenship, cultural literacy and tolerance for otherness, and in the long-term, specifically highlights the principle under which everyone, including adults, has the they also contribute to raising the level of education of the population, sustainable right to quality and inclusive education, training and lifelong learning. This is one of consumption and production and greater equity in the education system” the 20 key principles of the European Pillar of Social Rights. The Recommendation (ReNPIO13-20, p. 42). notes that in today’s society everyone needs to have a wide set of skills, knowledge and competences, including a sufficient level of literacy, numeracy and digital competence, The programmes currently in force shall not be terminated with the adoption of the in order to achieve their full competences, play an active part in society and undertake Guidelines. their social and civic responsibilities. Such skills, knowledge and competences are also crucial for accessing, and progressing in, the labour market and for engaging in further These Guidelines shall be applied in drawing up new programmes and when making education and training. any potential changes to the currently valid programmes for adults. 12 13 Furthermore, developers of non-formal education programmes for adults can also TYPES OF PUBLICLY VERIFIED apply these Guidelines when appropriate. 3 EDUCATION PROGRAMMES With the adoption of the Guidelines for Preparation of Publicly Verified Education Programmes for Adults by the Council of Experts for Adult Education, the provisions FOR ADULTS relating to publicly verified education programmes for adults in Guidelines of the curricular reform of adult education and design of adult education programmes from Article 16 of the ZIO-1 sets out two types of programmes, i.e. programmes for 1998 (slo. “Izhodišča za kurikularno prenovo izobraževanja odraslih in oblikovanje acquisition and raising the level of literacy and key competences and programmes for programov za izobraževanje odraslih”) shall cease to apply. raising the level of general education. 3.1 Programmes for acquisition and raising the level of literacy and key competences Development of programmes for raising the level of literacy and key competences is vital for the preparation of comprehensive support for vulnerable groups of adults. These programmes are part of efforts to improve, systematically and in the long-term, these skills in adults who did not have the opportunity to develop or maintain these skills to a sufficient extent during full-time education or later in life. Notions regarding the level of literacy3 and key competences – as well as which key competences – adults need to lead a successful life in modern society change with the development of society. The national PIAAC 2016 survey in Slovenia was instrumental to understanding the issue and to assess the situation and needs. It revealed that one third or over 400,000 adults of working age in Slovenia achieve lower skills scores than adults in developed OECD countries.4 The results from the PIAAC 2016 survey on the progress so far in educational services for the development of literacy and key competences and educational practice demonstrate that achieving long-term effects in education requires developing a variety of programmes and adapting them to the target groups’ current needs. In deciding what competences should be included as key competences in these Guidelines, we relied on the Council Recommendation on key competences for lifelong 3 The concept of literacy keeps evolving as society develops. In the narrower sense, literacy is equated to communication competence, which includes reading, listening, speaking and writing. In its broader sense, literacy is the competence and social practice that keeps developing throughout one’s life in various circumstances and in different fields and which permeates all their activities. Today, literacy is increasingly understood as a tool for research, understanding, creating and communicating in a modern world filled with print- or screen-based texts and information that is becoming more and more digital and constantly evolving. 4 According to the results of the PIAAC 2016, these adults are only capable of reading short and simple texts and can successfully perform only simple mathematical operations (Javrh, 2016). 14 15 learning (2018). It includes eight key competences citizens need for personal fulfilment, 3.2 Programmes for improvement of the level of general social inclusion, active citizenship and employability in a knowledge-based society. education These key competences include: The development of programmes for improvement of the level of general education 1. literacy competence; ensures a balance between those based on the needs of the individual or the various adult target groups and those intended for solving issues in the community. The way 2. multilingual competence; the goals and the purpose of general education are understood is important when 3. mathematical competence and competence in science, technology and engineering; deciding about the content areas and promoting the preparation of programmes for 4. digital competence; improvement of the level of general education. The goal of general education is “to 5. personal and social competences and learning competence; understand oneself and the world we live in, as opposed to specialist and vocational 6. citizenship competence; education whose aim is in-depth knowledge about and mastery of a specific corpus 7. entrepreneurship competence and of knowledge and associated skills, and their application in a given theoretical and practical field” (Kodelja, 2004, p. 38).6 When developing programmes for raising the 8. cultural awareness and expression competence. level of general education, the balanced representation of these content areas is taken into account: The key competences are equivalent. However, the results of the PIAAC 2016 survey • personal development, health and a healthy lifestyle; show that the development of literacy and other competences requires strengthening • nature, the environment and sustainable development; of these basic skills: • national affiliation (national identity, cultural and historical heritage, etc.); 1. basic literacy skills, • European and world cultural heritage (knowledge and tolerance of other cultures, 2. basic numeracy skills and globalisation, etc.); 3. digital skills. • community participation (active citizenship, development and specific features of In accordance with these Guidelines, the indicated basic skills must be included as the community, knowledge of the legal system and state institutions, the role of appropriate in all programmes. civil society, etc.); • intergenerational cooperation, etc. Previous practice in the development of literacy and key competences in Slovenia has shown that the definition of key competences in the European document is too general for the purposes of planning programmes, formulating implementation plans and executing the education process. The development and execution of programmes for adults are founded on the educational needs and interests of target groups. This is why in Slovenia we have developed descriptors for all key competences at the lowest and at the optimum level.5 They describe the respective key competence and show what it encompasses. Descriptors can be used when planning a programme as well as when 6 The concept of general education can be summed up with the statement: “An educated person must formulating an implementation plan and monitoring progress. be able to think and write clearly and efficiently; an educated person must reach in-depth knowledge in certain fields of learning and develop the ability of logical reasoning and analysis; they must know how to critically evaluate the pathways used to obtain and use knowledge /.../ in particular, they must be instructed in aesthetic and intellectual experiences relating to literature and the arts, history, /.../ social sciences, /.../ 5 The Slovenian Institute for Adult Education creates descriptors of key competences within the framework physical and biological sciences; they must demonstrate a certain level of understanding of moral and ethical of the multi-annual project entitled Professional Support for the Area of Key Competences 2016–2022 (slo. issues; they must be familiar with other cultures, and what distinguishes them from an uneducated person is “Strokovna podpora razvoju področja temeljnih kompetenc 2016–2022”), link: https://pismenost.acs.si/ precisely the extent to which they are able to see their own life experiences in a broader context.” (Kodelja, projekti/ess-6/gradiva/knjizice-z-opisniki/ 2004, pp. 38–39) 16 17 3.3 Public validity of programmes CONDITIONS FOR PREPARATION 4 A certificate of training, which is an official document, can be obtained in both types of OF PROGRAMMES AND APPRAISAL programmes. The certificate of training is issued by the programme provider. AT THE COUNCIL OF EXPERTS Programmes can be designed so as to include external assessment of knowledge. In such cases, programmes also include examination catalogues to be used for Listed below are the conditions that the programme developer and the Council of external assessment of knowledge, which are submitted to the Council of Experts Experts observe when considering a programme. for confirmation along with the programme. In cases where a valid need for external assessment of knowledge arises in a programme that has already been adopted and is being implemented without it, a justification explaining why external assessment of Table 1: Conditions for planning and appraisal of programmes knowledge should be introduced can be submitted to the Council of Experts subsequently, along with the examination catalogue for external assessment of knowledge. Condition Pursuant to Article 15 of the ZIO-1, it is also possible to carry out procedures for a 1. condition The programme contributes to the achievement of national recognition of equivalence of educational standards for all programmes. Knowledge goals of adult education in various fields.7 from general education areas for which the Council of Experts has established an equivalent educational standard is taken into account in further education. In the 2. condition The programme contributes to the achievement of international certificate of successful completion of programme, which is an official document, the goals of adult education in various fields. provider shall state which educational standard the acquired knowledge is equivalent to. 3. condition The programme is a response to identified needs arising from a needs analysis, research, studies, etc. conducted at the national level. 4. condition The programme includes literacy, numeracy and digital skills. 5. condition The programme is at least 100 hours long. 6. condition The programme provides an assessment of the level of knowledge and skills according to the SQF descriptors. 1st condition: When submitting a programme for consideration, the programme developer justifies how the programme will contribute to the achievement of national goals of adult education in the field for which it is being prepared. These can also include other fields besides adult education: e.g. work and social affairs, health care, agriculture, culture, the environment, internal affairs, etc. The justification includes a description of the legal bases and references to the goals and orientations listed in 7 These include: education, work, agriculture, health care, culture, the environment, internal affairs, etc. 18 19 country strategy papers, such as strategies and national programmes in the field for PRINCIPLES FOR PLANNING which the programme is being prepared. The justification should be attached to the 5 programme. ADULT PROGRAMMES 2nd condition: When submitting a programme for consideration, the programme developer justifies how the programme will contribute to the achievement of This chapter provides further information about seven basic principles that should be international orientations and goals that Slovenia committed itself to achieve. The considered when planning education programmes for adults. justification also includes an indication of the international documents with reference to the goals or orientations that are important for the programme. The justification should be attached to the programme. 5.1 Andragogy cycle as the foundation for planning 3rd condition: When submitting a programme for consideration, the programme programmes developer explains which identified educational needs of adults the programme is a The developer designs the programme for adults in accordance with the phases of response to. The justification may include data and findings collected through own the andragogy cycle. This is a basic principle defined by the recurring cycle in which research relating to the identification of the needs of adults or by virtue of social individual learning changes take place. During planning, the programme is roughly8 partners, local communities and other stakeholders. Connections with national or defined by four phases: identification of educational needs, planning, execution and international research or studies should also be stated in the needs analysis. The needs evaluation of the learning process. It takes place at the level of the programme as a analysis should be attached to the programme. whole – from identified needs through its design and through implementation to its validation (during or after implementation). It also occurs in the course of the learning 4th condition: The developer includes the acquisition of basic literacy, numeracy and sequences taking place every time the programme is carried out. This means at different digital skills, as appropriate, in both types of programmes. With this, they specify levels: from the position of the programme developer and from that of the programme strengthening of the basic skills of adults that are defined by the national strategic provider or participant. In other words, the cycle set out by the programme developer objectives. in the programme’s design also occurs when the programme is being carried out, 5th condition: When preparing a programme, the developer takes into account its though it is slightly different every time – depending on the circumstances in which duration of at least 100 hours of organised education. The programme allows adults to the programme is taking place. For the developer, these phases serve as a meta-plan of expand and deepen their knowledge and acquire skills. Programme participants also the programme in which they also consider each of the components used to specify the develop their own views and mindset, e.g. attitude toward learning and the environment, programme. This enables the provider to choose and adapt goals, contents, methods of view on refugees, attitude toward cultural heritage and tradition, etc. The programme formative assessment and summative assessments as well as the forms and methods provides a sufficient number of hours of organised education to develop all three listed of learning for the learning group and the individual. The individual components of the programme are appropriately broken down and the programme is coherent. aspects of competence – knowledge, skill, mindset/attitude. Determination of the duration of the programme is well thought out, as it contributes to the realisation of the adult’s right to quality education – considering that the target group may also include 5.2 Identifying the needs of individuals and society for vulnerable groups of adults with low education or without it. In terms of achieving the desired effects, it also means the protection of public interest or the protection of state education investments in these types of programmes. During programme planning, the developer takes into account the identified interests 6th condition: When submitting a programme for consideration, the programme and needs of individuals and society for education resulting from a substantively and developer provides an assessment of the level of knowledge and skills according to the SQF descriptors. 8 Some define it in even greater detail: 1. identification of needs, 2. planning the programme, 3. programming the contents, 4. preparation and organisation of the educational process, 5. execution of education, 6. evaluation of outcomes. However, programming can be viewed as being part of planning, and the preparation and organisation of the educational process can be seen as being part of execution. Evaluation is not just about assessment learning outcomes in the narrow sense (of those listed in the knowledge catalogue) but also includes evaluation of the learning process as a whole from different perspectives. 20 21 methodologically adequately prepared and presented needs analysis. This also includes are formulated more at the level of principles and values that are a criterion for the an analysis of the characteristics of the target groups of the programme. This is the choice of learning contents and methods. This strategy focuses on the process of foundation for all subsequent phases of programme planning. Although discussions in acquiring knowledge and developing skills and virtues – in short, the comprehensive the field are mainly about the educational needs of vulnerable groups,9 it is important development of the personality of an adult – not only in the cognitive but also in the that the planning takes all adults into account. affective (emotional, sociomoral) and psychomotor areas. From the perspective of Slovenia’s development on a global scale, when it comes to Priority is given to the principles, learning contents and methods integrated into the needs of individuals and the wider society for education, it is important to note the process, which contribute to the development of the personality potentials of the key competences associated with the development of Industry 4.0 and Industry adults. In addition to the development of cognition, which is the first condition for 5.0. Both involve different levels of digital competence as well as general education critical thinking, it is also about the development of those cognitive and affective and key competences related to communication, teamwork and the development of characteristics that enable gradually building the value orientation of the adult and transversal competences that promote and enable the creativity of adults. They also autonomous morality. Furthermore, it is also about mastering the knowledge, skills encompass changes in life courses and altered social patterns (e.g. multiculturalism and habits that allow an adult to have quality interactions in various social positions or demographic change, global issues related to sustainable management of natural (private, professional, community activities). resources, the participation of individuals in solving community issues, etc.), which repeatedly require people to learn to live differently. The application of appropriate strategies and methods enables development of various competences, among them learning in particular, which is considered a meta-competence in process-development planning. It enables the development of other 5.3 Selection of a development strategy competences and the sustainable development of an individual. Such programmes are expected to have a greater choice of content and to use various forms and means The developer decides on the development strategy for the programme according to and methods of formative assessments and summative assessment of knowledge that the identified needs of the target group, potential participants and the purpose of the promote the comprehensive development of adults. programme. It is recommended to employ a combination of an objectives-driven and a process-development programme planning strategy. The first relates to learning objectives, while the process-development models focus on the learning process and 5.4 Programme structured in modules process-development goals. Where appropriate (e.g. for longer programmes, diverse content), the developer Process-development programming offers flexibility in the implementation of designs the programme in modules and thus increases its flexibility. Modules are the programme, because the programme can be adapted to the specific needs and rounded programme units that can differ in content, difficulty or optionality. The characteristics of adults in the learning group. This strategy also includes goals, but developer carries them out in a specific or optional order, in different time periods, and they are set less precisely and bindingly than in the objectives-driven strategy; they they can be compulsory or elective. This makes it easier to harmonise them with the educational needs of the participant or groups. They enable the participant to complete the programme as a whole or in parts. A programme that includes both compulsory and 9 Scientific literature and research cite a large number of vulnerable or marginalised groups, both those who are already in a distinctly marginalised position or even socially excluded as well as those at risk of elective modules allows the participant to choose from the offered elective modules marginalisation and social exclusion. Trbanc et al. (2003) use the term vulnerable groups to refer to groups those that are in line with their needs and interests. in which various forms of deprivation (e.g. material or financial, educational, employment, housing, etc.) are intertwined and which are often significantly disadvantaged in terms of access to important resources (such as employment or education). With the growth of socially excluded groups (e.g. the long-term unemployed, 5.5 Programme openness older pensioners, young adults – dropouts, especially those from less privileged backgrounds, people with disabilities, etc.), expert discussions increasingly drew attention to the need for society to act before certain The principle of programme openness means that a part of the programme is groups become socially excluded. This is how the term vulnerable groups came about, meaning groups that, supplemented in terms of execution. It means that 80% of the programme is prepared in terms of certain characteristics, are identified early enough as groups (or individuals) that need assistance and confirmed at the Council of Experts, while 20% remains open and is determined by to avoid becoming socially excluded (Vilič Klenovšek and Rupert, 2010). 22 23 the provider in collaboration with various stakeholders (local community, employers, the end of any given learning sequence, for example, when we want to find out what the Employment Service, social work centres, etc.). is happening in the programme, the provider can adjust the programme accordingly if the selected teaching methods are not working, etc. In addition to the above, programme openness is also related to the choice of educational resources, the choice of learning forms and methods of work as well as the methods According to modern expertise, programme monitoring and evaluation must be of formative assessments and summative assessment of knowledge. The principle of carefully planned, setting out their goals, criteria and instrumentation. The envisaged openness of the programme is also reflected in its flexibility. It can be executed in instrumentation is used to collect data on the quality and level of implementation of all different orders or by emphasising, in accordance with the educational needs of the components of the programme in order to change and improve the education programme. learning group and the individual, those contents and goals that are most important to Monitoring and evaluation are not specifically defined in the set of components, but it is them in a given period. essential that the developer considers these issues when planning the programme. This can be done when determining the organisation of the programme’s implementation or The principle of programme openness gives providers autonomy and responsibility for when formulating the examination catalogue or during the formative assessment and the quality preparation of the open curriculum. When planning the implementation, the summative assessment of knowledge. Consideration of issues related to monitoring and provider in this way thoroughly elaborates the various aspects of the programme – in evaluation allows the developer to evaluate the programme from different perspectives: relation to the compulsory components of the pre-determined part of the programme from the perspective of its components, participants and teachers, whereby it is also on the one hand and the specificities related to the concrete learning context(s) on the possible to evaluate the influence of factors that (also) affected its implementation other (e.g. characteristics of the adult target group, the needs of adult participants, the and their interaction in specific circumstances (e.g. teachers’ work, timing of modules, level of development and the needs of the environment), and then harmonises the two participants’ situational barriers, etc.). with each other. 5.6 Flexibility in programme planning Flexibility in programme planning means that the programme: • is based on the own activity and creative participation of programme participants; • takes into account the appropriate proportions between individual and group education; • leverages modern forms of education based on e-learning and the use of new and multimedia technologies; and • enables a variety of ways of assessment of the acquired knowledge and skills and progression through the programme. 5.7 Programme monitoring and evaluation Already at the planning stage, the programme developer takes into account issues relating to the implementation or fulfilment of goals, which means that they also plan monitoring and evaluation. This has a significant impact on programme quality. Even though it is the last phase of the andragogy cycle, evaluation does not take place only at the end, when the programme expires, but it is a process having determined intermediate steps and lasting throughout the implementation of the programme. At 24 25 the needs of the adult. The same is true in the case of programmes for acquisition and PROGRAMME COMPONENTS 6 raising the level of literacy and key competences, where the current needs of the adult arising from the requirements of their work and living environment are even more emphasised. Having a properly defined purpose of the programme ensures systematic This section sets out the compulsory programme components, which are the foundation further planning. for quality planning and a guiding principle for the appraisal of the programme at the Council of Experts. The developer outlines which educational needs the programme addresses or how this will impact the participant’s life. In doing so, they take equal account of the social needs, work needs and needs of the individual. 6.1 Programme name The name of the programme is short. It states that the programme is intended for adult education (example: Programme for adults, etc.). The shorter name of the programme Criteria for appraisal: should also be listed, or the abbreviation, if appropriate. • The purpose of the programme sets out the vision of general changes to which a particular educational programme aspires. Choosing a name requires consideration, as it should reflect the purpose of the • The purpose of the programme is achievable and consistent with other programme. The name has a motivating effect and appeals to the virtues (positive programme components. qualities) without highlighting the weaknesses/shortcomings of the participant. Careful choice of name helps the participant understand what to expect from the programme, how they will benefit by participating in the programme, what the programme will train them for, etc. For an adult who less frequently attends organised forms of education or 6.3 Target group is a member of a vulnerable group, an appropriately chosen name evokes positive and The target group comprises adults for whom the programme is intended. These adults encouraging associations related to education. It is important for the name to be derived are connected by some similar characteristics that also significantly shape the profile from the potential interests and circumstances of adults in terms of pragmatism. of their educational needs that can be met in the respective programme. Criteria for appraisal: Target group data helps the developer determine other programme components and is • The name of the programme enables the adult to identify the purpose of the vital for the programme developer, provider and adult education counsellor. With it, programme and encourages them to participate. the provider can adapt the implementation of the programme to the learning group • The name of the programme evokes positive and encouraging associations more easily, as they will know what needs to be changed and how by comparing the related to education. characteristics of the target group and the characteristics of the participants in the learning group. The record makes it easier for adult education counsellors and others who guide adults in education to determine whether the programme is appropriate for the adult to whom they are providing guidance. 6.2 Purpose of the programme The developer defines the target group on the basis of data derived from quantitative The purpose of the programme outlines the general changes that the developer wants and qualitative analyses, evaluations or research. The data expresses the scope and to achieve in the participant through the educational programme. social position of the target group relative to educational and socio-cultural needs. It also shows whether there are any particularities and deviations related to individual This section states which type of programme this is – a programme for acquisition and target groups. raising the level of literacy and key competences or a programme for improvement of the level of general education. The purpose of the programme for improvement of the level of general education derives from the general development needs of society and 26 27 The developer defines the target group according to all or only some of the indicated • skills (e.g. uses, makes, executes, reads, calculates, writes, etc.); and characteristics, e.g.: • mindset, attitude toward something, perception of oneself and others (e.g. develops an attitude toward ..., develops awareness, strengthens motivation, etc.). • gender; • age; It is important that the developer defines this component only at a general level, • educational qualification; without going into too much detail in specifying the goals of the programme. The first • status of participants (e.g. employed, unemployed, retired); and the second type of goal (knowledge and skills) will be detailed in the knowledge • psychophysical characteristics, abilities, skills, previously acquired knowledge, catalogues, while the third type (mindset, attitude) will be taken into account by the interests, etc.; developer when executing the education as process-oriented principles to be developed • other social characteristics (e.g. immigrants, rural population, etc.). as appropriate during the educational process. In addition to the educational needs of the target group, it also makes sense to point Under this component, the programme developer should provide an assessment of the out the factors that encourage or hinder adults in education. Internal barriers (bad level of knowledge and skills according to the SQF descriptors. experiences from past education) and virtues (e.g. commitment, diligence, a sense of belonging) are the factors that act either as a disincentive or an incentive to enroll and complete the programme. Criteria for appraisal: • The general goals of the programme are consistent with the purpose of the Criteria for appraisal: programme. • The description of the target group shows the educational needs the • The general goals of the programme show what its participant will achieve or programme is designed to fulfil. develop. • An assessment of the level of knowledge and skills according to the SQF • The description shows the social position of the target group. descriptors is provided. 6.4 Programme goals In this component, the developer defines the key competences and general goals that 6.5 Knowledge catalogue or programme contents the participant will achieve or develop in the programme. Key competences 6.5.1 Structure and content design of the programme Under this component, in the case of a programme specifically designed to develop Contrary to formal education programmes leading to an educational qualification, where key competences and, if applicable, a programme to improve the level of general the term “syllabus list” is most often still used to show the structure and contents of education, the developer should indicate the key competences that the programme the programme, divided into complete units, these programmes are characterised by participant will develop. This can be one or several key competences (e.g. literacy, the fact that the contents are not divided into subjects but into other content units, multilingual competence, digital competence, citizenship competence, etc.). In this such as a module or a content area. The term “syllabus list” was therefore replaced case, competence is understood holistically, from the perspective of knowledge, skills with the term “structure and content design of the programme”. and the developmental or relational aspects. These Guidelines envisages two versions of the programme structure: Programme goals • a programme structured in modules (Table 2); and In defining the general goals of the programme, we should specify the expected changes • a programme structured into content areas (Table 3). in: • knowledge (e.g. is acquainted with, knows, understands, etc.); 28 29 The Council of Experts confirms 80% of the programme, for the compulsory and the A: The compulsory part of the programme encompasses: elective parts. • at least 60% of the duration of the programme if the programme also has an elective part, or The provider sets out 20% of the programme, specifically for the open curriculum. • 80% of the programme duration if the programme has no elective part. The compulsory part is the basic core content of the programme. This part includes Table 2: Programme structured in modules contents intended for all programme participants according to the characteristics of Programme unit Number of hours the target group and the identified needs. The purpose and goals of the programme are A – Compulsory part of the programme also taken into account when designing such coherent content units. Module name 1 The minimum duration of the programme is 100 hours, which is why the compulsory Module name 2 part lasts at least 60 hours if the programme has an elective part, or 80 hours if the … Total A programme does not have an elective part. B – Elective part of the programme Module name 1 B: Elective part of the programme – comprises a maximum of 20% of the programme Module name 2 duration … Total B The elective part of the programme is not compulsory. The decision on whether the C – Open curriculum programme will also include an elective part is left to the developer. In this part, it Open curriculum is especially important that the developer follows one of the fundamental principles Total C for the preparation of programmes for adults, which is to enable participants to Total A + B + C find contents in the programme that match their interests and to realise their right to participate in the making of their own education. The developer designs multiple Table 3: Programme designed by content areas modules or content areas for the participant to choose from. Whether the participant Programme unit Number of hours will choose one module or content area or more depends on the number of modules available. A – Compulsory part of the programme Name of content area The developer designs this part of the programme to take up to 20% of the duration of Name of content area the entire programme. For programmes that are 100 hours long, the elective part can … last up to 20 hours. Total A B – Elective part of the programme Name of content area C: Open curriculum – comprises 20% of the programme duration Name of content area … In collaboration with the local community, the provider prepares and adopts a Total B knowledge catalogue covering 20% of the programme duration. The open curriculum C – Open curriculum lasts 20 hours in the case of a minimum programme duration of 100 hours. Open curriculum Total C The provider independently designs the open curriculum in line with regional and local Total A + B + C needs. If the provider has autonomy in this regard, this means that they are solely responsible for adopting this part of the programme, i.e. without appraisal at the Council In both cases, the developer takes into account the compulsory, elective and open parts of Experts. The principle of flexibility in programme preparation is in this part satisfied of the programme when planning the programme. 30 31 by the opportunity given to the provider to replace, adjust or re-develop it in accordance Orientations for preparing knowledge catalogues in programmes structured in with identified needs without having to launch any external procedures to reconfirm modules the modified programme. Knowledge catalogue in programmes structured in modules Criteria for appraisal: If the programme is structured in modules, the knowledge catalogue is prepared for each module separately with the following components: • Structure and content design of the programme include the compulsory and open parts, but they may also include the elective part. • module name; • The compulsory part comprises a minimum of 60% of the programme • module duration (number of hours); duration. • key competences that participants will develop in the module; • recommended content areas and contents or topics; • The elective part comprises a maximum of 20% of the programme duration. • knowledge that participants will acquire in the module; • The open curriculum comprises 20% of the programme duration. • skills that participants will acquire in the module, including the basic skills of • Duration of the programme is minimum 100 hours. literacy, numeracy and digital skills. Module name 6.5.2 Knowledge catalogue The name of the module should be informative and clearly express the content of the module. When creating a name, care should be taken to preserve the substantive Knowledge catalogues are drawn up according to the principles of objectives-driven connection with the defined purpose of the programme. planning. This means that the goals of the programme are operationalised in the This component is not defined in the case of an programme structured into content knowledge catalogue and defined as measurable learning outcomes. Procedures of areas. formative assessment and summative assessment can identify what a participant has learned in the programme. At the same time, on the basis of defined different types of Module duration (number of hours) goals that they set, the programme provider autonomously and responsibly selects, at their own professional discretion, the content, methods of work and educational Duration comprises the number of hours of organised education provided10 that are resources for achieving the goals of the programmes. planned for the module. This is prepared and executed by the programme provider.11 Considering the principle of autonomy of the provider and their responsibility, it is Knowledge catalogues are structured so that in their planning a connection is maintained not necessary for the programme developer to precisely specify e.g. the number of with the primary purpose and goals that we want to develop with the programme or hours for lectures, exercises, etc. out of the module’s total number of hours when that the participant will achieve in the programme. submitting the programme for appraisal. The provider will specify this when drafting If the programme is structured in modules, the developer prepares a knowledge the implementation plan for executing the programme. It is important, however, that catalogue for each module. the goals of the programme, and especially the knowledge and skills set out in the knowledge catalogue, are taken into account when determining the number of hours In the case of an programme structured into content areas, the developer prepares one for the individual substantively coherent whole (module or content area) in programme knowledge catalogue for the whole programme and adjusts the structure accordingly. 10 Certain parts of the education system use the term “contact hours” to refer to organised education. 11 This includes: lectures, practical work (exercises, etc.), excursions, field work, group and individual consultations, organised preparations for formative assessment and summative assessment of knowledge, learning support, defence of seminar papers, group meetings or individual work with a mentor in e-learning, formative assessment and summative assessment of knowledge and other forms determined by the educational programme. 32 33 preparation. If, for example, the goals in the knowledge catalogues envisage that the aspects: knowledge (cognitive dimension), skills (practical or action aspect) and the adult will be practically trained for something, the developer takes this into account developmental or relational aspect (motivation, views, values, social or relational when determining the number of hours of the programme by anticipating and providing dimensions, etc.).12 the appropriate number of hours for practical training or practical acquisition of skills The specification of knowledge and skills in programmes intended for the development in the programme. of key competences represents the operationalisation of the key competences in practice. On the other hand, if the programme developer finds that thanks to the nature of Moreover, in programmes for the improvement of the level of general education, it the programme it would make sense to precisely specify e.g. the number of hours for represents the operationalisation of the general goals set out by the developer in the lectures, exercises, etc. at the programme level, it is advisable that they do so, although component called programme goals. it is not required. In knowledge catalogues, we define how we will achieve the cognitive aspect of In the case of an programme structured into content areas, the developer specifies the competence, which is covered by the term “knowledge”, and the functional or action number of hours for each content area. aspect of competence, for which we use the term “skill”. Table 4: Guidance for determining knowledge and skills for individual modules Key competences that participants will develop in the module Module Knowledge Skills With regard to goals, the developer will determine the key competences that the participant will develop in the programme. However, it is not necessary for the participant to develop all the required key competences in all modules. For each module, the developer should therefore list only the competences that will be developed by the participant in that module. In the case of an programme structured into content areas, the key competences Table 5: Guidance for determining knowledge and skills for programmes structured developed in the programme should be indicated. into content areas Content area Knowledge Skills Recommended content areas and contents or topics Taking the principles of objectives-driven strategy into account, the knowledge catalogues may for each module also specify the content areas and contents or topics, but only in the form of recommended contents or topics that the programme provider (teacher, adult education organiser, etc.) can choose from; they can also decide to use Different taxonomic levels are taken into account when defining knowledge and skills, other contents or topics if they believe the participant will be able to achieve the goals and it makes sense to rely on taxonomies of goals, such as Bloom’s, Marzano’s, Gagne’s13 more easily in that way. This is why the choice of content or topics is left to the provider, or other taxonomies, which enable the classification of descriptors into different levels who drafts an implementation plan for each topic. In this respect, the importance of knowledge and skills. of the teacher’s professional judgment becomes especially evident, seeing that with 12 See also Terms and abbreviations. the appropriate choice of content or topics that are close to the real life or working circumstances of the participant, the teacher can enhance motivation to learn and 13 Bloom: knowledge, comprehension, application, analysis, synthesis, evaluation or creation (cognitive domain), receiving, responding, valuing, (self)-organising, (self)-characterising (affective domain – views, thereby increase the chances of the participant realising their goals and achievements. values); Marzano: substantive and process-based, includes e.g. complex reasoning (comparison, classification, deduction, investigation, problem solving), working with resources, collaboration, presenting ideas, habits Knowledge and skills of mind. Gagne: basic and conceptual knowledge (understanding), procedural knowledge (understanding and effective mastery of algorithms and procedures; this can be either routine or complex) and problem-When specifying the knowledge and skills, the developer should consider that in the based knowledge (using knowledge in new situations, applying combinations of rules and expressions when Guidelines the key competences or competences in general are defined as a set of three dealing with a new situation, ability to use conceptual and procedural knowledge). 34 35 Knowledge Examples of goals understood as “skills” that the participant will When defining “knowledge”, the developer should take into account that the cognitive acquire in the programme: aspect of competence is about declarative and procedural knowledge, mastery of principles, theories and concepts and the development of higher cognitive levels – • Makes a payment for goods or services using various online payment methods the systematic connection of knowledge, mastery of various mental operations, (UPN payment orders, PayPal, cash on delivery). development of critical and creative thinking, etc. The developer defines what the adult • Creates their own user profile on Facebook. should discover, know, understand, etc. • Uses the function to restrict access to shared documents. • Exercises the right to be forgotten electronically. • Writes a grammatically correct and contextually appropriate job application. Examples of goals understood as “knowledge” the participant that • Formulates an annual plan for spending the family budget. will acquire in the programme: • Develops a conversation about a read article or text. • Knows the difference between free and paid online apps. • Writes out unknown words from a text (in a foreign language) and tries to explain • Distinguishes between cultural and natural heritage. them meaningfully according to the context, then verifies the interpretation using an appropriate dictionary. • Tells the difference between organic and other types of waste. • Understands the operating principles of various mobile operating systems, e.g. Android, iOS, Windows mobile. • Explains the phenomenon of virtual reality. Basic skills • Explains the benefits of sharing data in the cloud. In the definition of skills in both types of programmes, the developer also includes the • Explains graphically displayed data. following basic skills, as appropriate: • Explains how to dispose of waste according to the structure of the substance • basic literacy skills, indicated on the label. • basic numeracy skills and • Breaks down the dangers of bionic technology. • digital skills. Basic skill means the use-oriented or applicable aspect of a key competence. Skills In defining the skills that the participant will develop and acquire in individual modules or content areas, the developer should take into account that skills are about realising the functional or action dimension of competence. It is about the willingness to commit, to get involved and to use knowledge meaningfully in life and at work. When defining skills, the developer takes into account that they allow the adult to solve problems in everyday situations, to perform their daily activities. 36 37 Examples of defining key competences and basic skills in the Digital Literacy programme Criteria for appraisal: • If the programme is structured in modules, a knowledge catalogue is prepared In the Digital Literacy Examples of definitions of basic skills14 for each module. programme, the participant develops • The knowledge catalogue for each module specifies these components: module the following key name, module duration, key competences, recommended contents or topics, competence: knowledge and skills that the participant will acquire in the module. • The knowledge catalogue in the case of an programme structured into Digital competence Reading various texts on artificial intelligence and robotics, etc. content areas specifies these components: scope of each content area, key Writing various texts and using advanced functions of the competences, recommended contents or topics, knowledge and skills that the selected program for references, the table of contents, grammar participant will acquire in the programme. checking, tracking changes, etc. • The basic skills of literacy, numeracy and digital skills are included in the knowledge catalogues as appropriate. Digital skills: e.g. preparing a laptop for use, creating folders, selecting apps on a mobile device, using browsers, taking photos with a mobile device and uploading and editing photos using the appropriate tools. Calculating using programs and apps, e.g. creating a spreadsheet 6.6 Duration of education in Excel to track income and expenses, preparing simple graphical representations, using online apps to convert currencies, calculate The developer sets out the duration of education as the number of hours15 of organised taxes, etc. education provided to participants. When defining the duration, they take into account the purpose and goals of the programme, the knowledge and skills defined in the knowledge catalogues and other programme components. Examples of definitions of key competences and basic skills in the programme Spanish for Adults Duration of the programme is minimum 100 hours. In the programme Examples of definitions of basic skills Spanish for Adults, the Criteria for appraisal: participant develops the following key competence: • The programme includes a minimum of 100 hours of organised education Communication in a Reading various texts dealing with everyday life, history, culture, provided to participants. foreign language language, natural heritage in Spanish, etc. • The duration of the programme is realistic to the other components of the programme. Writing various texts in Spanish. Digital skills: e.g. using browsers to look up information about Spain and Spanish-speaking countries, using online translation services and dictionaries. Calculating e.g. using Spanish, using Spanish when making payments. 14 The descriptors of individual key competences prepared by the SIAE can also be of assistance. 15 One hour of organised education = 45 minutes. 38 39 (knowledge, skills) the participant has achieved according to the programme. 6.7 Examination catalogue or other methods for assessment Considering the fact that knowledge and skills are defined already in the knowledge of knowledge catalogue, they are the starting point in defining what knowledge and skills are to be The primary purpose of the examination catalogue is to define the knowledge to be assessed in formative assessment or summative assessment. Often the examination assessed and to anticipate the methods, procedures and time-based milestones for catalogue does not include all knowledge and skills defined in the knowledge catalogue formative assessment or summative assessment of knowledge. and instead, for example, defines the minimum allowed scope of knowledge and skills that the participant needs to achieve in order to obtain an official document signifying If the programme envisages external assessment of knowledge, the developer also successful completion of the training. When knowledge and skills are defined at sets out an examination catalogue for this. If the examination catalogue envisaged for multiple levels of difficulty (e.g. basic and advanced level), the developer also takes this obtaining a certificate of training, which is issued by the programme provider upon into account in the examination catalogue. completion of the programme, will also be used for the external assessment, the developer should indicate this in the programme. In defining the knowledge and skills to be assessed in formative assessment and summative assessment under the programme, the developer can use a scheme similar The plan of formative assessment and summative assessments of knowledge should be to the one recommended for the preparation of knowledge catalogues. They can choose defined by the developer in the examination catalogue. The developer should consider other ways as well, but what is important is that it is clearly indicated which knowledge in how much detail the programme should be specified, taking into account the purpose or skills will be assessed. and nature of the programme. Table 6: Guidance for determining knowledge and skills to be assessed in formative The examination catalogue should have at least these components: assessment or summative assessments • defined knowledge and skills to be assessed in formative assessment or summative assessments; Module Knowledge Skills • defined forms and methods of formative assessment or summative assessments of knowledge; • methods of expressing learning outcomes; • defined time periods when the assessment of knowledge take place. When it makes sense in regard to the type of programme, the examination catalogue Table 7: Guidance for determining knowledge and skills to be assessed in formative also includes: assessment or summative assessments under a programme structured into content • criteria for assessing the acquired of knowledge and skills; areas • tools for measuring progress and acquired knowledge and skills. Content area Knowledge Skills Individual components of the examination catalogue are further clarified and detailed below. Defined knowledge and skills to be assessed in formative assessment or summative assessment Methods of formative assessment and summative assessments of knowledge The examination catalogue is closely related to the knowledge catalogue and the The means and methods of formative assessment and summative assessment of programme goals. The main purpose of formative assessment and summative knowledge are defined by the developer in the examination catalogue. In doing so, assessment of knowledge and skills is to determine which learning outcomes they take into account a number of programme goals that are assessed in formative 40 41 assessment or summative assessment with the chosen or other method. Even though and in situations where the participant can optimally demonstrate learning progress the examination catalogue mainly contains definitions of the knowledge and skills that (encouraging, stress-free circumstances and elimination of any factors that could will be assessed in formative assessment and summative assessment, in specifying the hinder the demonstration of knowledge). Furthermore, it is essential that the different means and methods of formative assessment and summative assessment, we should learning types and styles, etc. are taken into consideration when planning formative also include those that help monitor the strengthening of those aspects of competences assessment and summative assessment. that have been defined in the programme as “changes in mindset, attitude, etc.”.16 Methods of expressing learning outcomes In addition to the final formative assessment and summative assessment of knowledge, monitoring the participant’s progress during the course of education is also important The developer can express learning outcomes in various ways. The achievement in adult education. Therefore, the plan of formative assessment and summative of knowledge and skills can be assessed dichotomously (e.g. passed / failed to pass; assessment of knowledge set out in the examination catalogue can be of help in ongoing achieved / failed to achieve) or with descriptive assessments, e.g. with descriptors that monitoring and shows how the participant in the programme is progressing in terms define knowledge and skills at different levels of difficulty. In this, the developer needs of achieving the programme goals. This makes it possible to identify the incentivising to take care that the chosen method of expressing learning outcomes does not hinder or any disincentivising factors of learning that affect the recorded learning outcome. adults in the validation and recognition of knowledge for the purpose of inclusion in formal education. The achievement of knowledge and skills can also be expressed In formative assessment and summative assessment of knowledge progress, it numerically or in percentages or with different adjectives (e.g. satisfactory, good, very is important for the participant to have the possibility of self-assessment. This good, excellent). Other aids can also be used in self-assessment and in the ongoing strengthens their competences related to learning and develops good learning habits. assessment of learning progress, e.g. graphics (marking progress on a line, for example) It is good if there are several assessors participating in the formative assessment and or symbols that can be used as a means of motivation or to express satisfaction with summative assessment of knowledge and that summative knowledge assessment, and the progress, while allowing the participant to reflect on the factors that affected their formative assessment in particular, is not just a one-time event but rather a means to learning progress. gauge progress and a basis for further planning of learning. It is also important that Descriptive assessment is appropriate for those groups of adults that have a large different forms are used and that different circumstances are created in which formative deficit in knowledge and skills. It gives the participant a deeper and clearer insight assessment and/or summative assessment of knowledge takes place. This allows adults into their learning progress and highlights any weaknesses and the parts where they to apply their knowledge and skills multiple times and in different problem situations, should invest additional work, while also acknowledging their strengths. From this which triggers different metacognitive processes in the participant and deepens their perspective, it can have a favourable impact on further motivation to learn. Descriptive knowledge, thereby achieving higher levels of knowledge. Additionally, tasks with assessment focuses on quality, not just the amount of knowledge and skills acquired. complex design enable the development of skills and thus key competences. In terms of measuring learning progress, it needs to be noted that an examination Definition of time-based milestones is not the only way to determine knowledge and skills. Formative assessment of In the examination catalogue, the developer indicates or proposes the time, duration knowledge is performed contextually (e.g. within the framework of a learning project and frequency of formative assessment and summative assessment of knowledge. or real circumstances where the participant demonstrates their knowledge and skills) Criteria for measuring the achievement of knowledge and skills 16 In choosing forms, and teaching methods and methods of assessment of knowledge in particular, it is necessary to keep in mind that these play an important role not just in conveying and strengthening knowledge When appropriate and necessary given the nature of the publicly verified programme, but also in the transformation of knowledge that takes place in the participant as well as in the transfer of the the examination catalogue also sets out the criteria for measuring the achievement acquired knowledge into real life and working circumstances (also known as learning transfer). This is why of knowledge and skills. With them, assessors can assess the level of knowledge and it is important that they are interactive and encourage thinking, motivation for learning and creativity and skills the individual has achieved. If the knowledge catalogue contains the developer’s the development of various basic skills (e.g. literacy, numeracy, digital skills) and transversal competences definitions of minimum learning outcomes that participants need to achieve in order to (development of critical thinking, abstraction, concretisation, etc.). They also enable the disclosure and consideration of various internal and external factors that affect the participant’s learning. complete the programme successfully, the criteria will be designed in accordance with 42 43 this reference point. If the learning outcomes in the catalogue are defined at different as well, because this is important information for everyone who might want to join levels of difficulty, the criteria will be designed to allow such an assessment. the programme, for those who provide guidance and guide adults in education and for programme providers. Tools for measuring progress and the achievement of knowledge and skills The developer specifies the conditions for enrolment on the basis of an assessment The purpose of the tools for measuring progress and the achievement of knowledge and of what prior knowledge and skills adults needs to be successful in the programme. skills is to detect the knowledge and skills of the participant at the various levels laid A procedure may be envisaged and outlined to verify eligibility for enrolment in the down in the knowledge catalogue, or in accordance with the defined learning outcomes. programme. It can include, for example, proof by official document, an validation of The examination catalogue can, if appropriate, also contain sample tools for measuring the individual’s knowledge performed by an authorised organisation, the recognition progress in knowledge and skills. These can be instructions for preparing assessment of work and other experience that meets the conditions for enrolment, etc. tasks to be completed by the participant or various self-evaluation tools or sources thereof (e.g. websites) that may be used according to the programme developer, even When defining the conditions for entry and progress, the developer envisages different though they were not necessarily prepared by them. However, in terms of programme options for enrolment and progress tailored to the various needs, competences, learning flexibility and adaptation, it makes sense that these are ultimately chosen or prepared styles and other characteristics of individuals. Particularly in the case of adults who are by the programme provider themselves so that they can be adapted to the respective members of vulnerable groups and do not participate in organised forms of education, circumstances of implementation. have limited access to education or have weak key competences, the developer should avoid strict conditions for entry. They should consider opportunities and incentives so that as many adults as possible would choose education. Criteria for appraisal: • The individual’s learning progress can be demonstrated and recorded using Criteria for appraisal: methods of formative assessment and summative assessments of knowledge and skills. • The conditions for enrolment are designed so as to encourage and enable the inclusion of the target group for which the programme is intended. • The methods of formative assessment and summative assessment used are those that enable optimum demonstration of knowledge and skills according to the characteristics of the target group. • The examination catalogue is drawn up according to the type of programme and the conditions for obtaining the official document (e.g. the programme in 6.9 Conditions for progression and completion of the question features external assessment of knowledge). programme The conditions for progression of adults in the programme are defined by the developer where appropriate. Moreover, they also define, where appropriate, the conditions for completing the programme as a whole. It is also possible to set conditions for 6.8 Conditions for enrolment completing individual parts of the programme along with the conditions for completing the programme in such cases. The component sets out the conditions for adults in order to be enrolled in the programme. If the programme developer decides to set conditions for progression, e.g. mastery of specific knowledge, these conditions need to be elaborated precisely. The conditions for enrolment can vary between different types of programmes. When the programme has conditions for enrolment, the developer sets out all the required When there are no conditions for progression, this will be clearly stated and justified conditions, e.g. knowledge, skills, experience, psychophysical capacities and others. If by the developer. there are no conditions for enrolment in the programme, the developer records this 44 45 If the programme developer decides to set conditions for completing the programme, • teaching methods (lecture, conversation, discussion, role play, case studies, they will set out the conditions for completing the programme as a whole or for working with written sources, etc.); completing the individual parts of the programme. • internal relationships between learning forms and methods; • methods and forms of engaging with the environment and promoting the When there are no conditions for completing the programme, this will be clearly stated programme in the environment; and justified by the developer. • records and documentation on the programme kept by the provider; In cases where external assessment of knowledge is envisaged in the programme, they • methods and course of programme evaluation that allow the monitoring of will also state the provider. programme quality. The organisation of education also includes defining the relationships, roles and duties Criteria for appraisal: among all stakeholders of the programme (e.g. participants, teachers, counsellors, external associates). • Conditions for progression of participants in the programme are adequately specified according to the type of programme. The provider can tailor the organisation to the specificities of the learning group as • It can be concluded from the specified conditions that the conditions for needed. completing the programme are mainly related to the achievement of the specified learning outcomes or goals of the programme. Criteria for appraisal: • Organisation takes into account the characteristics of the target group and is in line with other components of the programme. 6.10 Organisation of education • Organisation indicates the proportion between group and individual learning. The organisation of education provides for the quality preparation and execution of • The roles and relationships between stakeholders in the programme are the programme, because it operationalises the proportions between its components. evident from the organisation. It determines the duration of the learning unit and the order of learning units. The organisation also determines the recommended number of participants in the group and how the group will be divided if necessary; in short, how the work of the programme participants will be organised. It is recommended that the proportion between group 6.11 Knowledge and training of education professionals and individual learning is indicated.17 In addition to the minimum allowed scope, it Education professionals in the programmes are teachers, adult education organisers makes sense to also indicate in the context of individual learning the purpose, content, and other professionals (“education professionals”). forms, timing and the professional staff involved. Furthermore, the organisation can also determine the basic: The programme developer defines the following in the programme: • andragogical forms of work, which can be either group or individual (e.g. course, correspondence, consultative, online, distance learning, instruction, study circles, 1. knowledge of education professionals in the programme. The educational etc.); qualifications of education professionals are determined by the minister, whereby the knowledge specified in the programme and the conditions from Article 19 of 17 Organised group work refers to the learning that takes place in various group-learning forms. Organised the ZIO-1 are taken into account; individual learning refers to the work in the programme that is envisaged for the individual, i.e. personal 2. training of education professionals. Education professionals acquire training consultations relating to the drafting and implementation of independent learning tasks or activities that qualifications through professional and occupational training programmes. help the participant to determine their position, concrete learning needs and personal adjustments (e.g. identifying and confirming acquired knowledge, formative knowledge assessment, preparing a portfolio, creating a personal education plan, determining learning support, etc.). 46 47 In regard to programmes for acquisition and raising the level of literacy and key Table 8: Example of defining professional knowledge and skills of education competences, the programme developer pays particular attention to how knowledge, professionals in the case of a programme structured in modules skills and competences of education professionals are defined. In doing so, they take into account some of the new features from these Guidelines, for example the Programme Education Knowledge Professional development definition of skills the adult should develop in the programme and the inclusion of structure professional literacy, numeracy and digital skills. Competences and training of teachers for working with vulnerable groups of adults are essential to ensure quality education and the Module I Teacher Higher Teaching illiterate achievement of envisaged effects, whereby it is key that a systematic approach to education ... immigrants professional development is employed on the basis of which teachers may develop an appropriate professional identity for working with these groups. Module II Teacher Higher - These Guidelines contain general recommendations regarding which areas the education ... education professional should develop professionally. It is recommended that the education professional working with vulnerable groups of adults in programmes for Module III Teacher Higher - acquisition and raising the level of literacy and key competences develop professional education ... competences in the following areas: • characteristics of adult target groups, with emphasis on vulnerable groups and the Condition Not a condition, but a impact of these characteristics on education; recommendation • identification of educational and other needs of vulnerable groups; • implementation planning at the group and individual level, which considers Table 9: Example of defining expert knowledge and skills of education professionals in the characteristics of vulnerable target groups and removes their obstacles in the case of a programme structured into content areas education; • selection and application of adult education methods and special didactic approaches for the development of literacy and key competences; Programme Education Knowledge Professional development structure professional • integrating the development of literacy and key competences with different areas of life to achieve success in personal fulfilment, participation in society, Content area Teacher Higher Teaching vulnerable groups competitiveness and flexibility in the world of work (ensuring transfer of learning education ... – knowledge is transferable and useful in everyday life). Two examples of defining professional knowledge and skills of education professionals Content area Teacher Higher Teaching vulnerable groups in the programme are provided in the tables below. education ... Condition Not a condition, but a recommendation Criteria for appraisal: • The professional knowledge and skills of education professionals are set out in the programme. 48 49 7 6.12 Certificate of training REFERENCES The certificate of training is an official document the participant receives upon completion of all obligations envisaged under the programme. It includes a supplement Bukovec, B., Preskar, S., and Zupanc Grom, R. (2013). Ugotavljanje potreb, organizacija in vodenje containing a list of the knowledge and skills acquired in the programme. It serves programov neformalnega izobraževanja odraslih. Učbenik programa za usposabljanje strokovnih as proof that the adult has mastered all knowledge and skills determined by the delavcev v izobraževanju odraslih. Ljubljana: Association of Slovenian Adult Education Centres. programme or the knowledge catalogues and examination catalogues. The certificate Council of the European Union. (2018). Council Recommendation on Key Competences of training is issued by the programme provider. for Lifelong Learning. Accessible at: https://eur-lex.europa.eu/legal-content/EN/TXT/ PDF/?uri=CELEX:32018H0604(01)&from=EN The certificate of training provides an assessment of the level of knowledge and skills according to the SQF descriptors. Council of the European Union. (2016). Council Recommendation on Upskilling Pathways: New In accordance with Article 15 of the ZIO-1, the Council of Experts establishes an Opportunities for Adults. 2016/C 484/01, 19th December 2016. Accessible at: https://eur-lex.europa. eu/legal-content/EN/TXT/PDF/?uri=CELEX:32016H1224(01)&from=EN equivalent educational standard and the provider indicates in the certificate of training which educational standards the acquired knowledge meets. European Commission. (2016). A New Skills Agenda for Europe – Working together to strengthen Adults can use this certificate as evidence in the procedures of validation of acquired human capital, employability and competitiveness. COM/2016/0381 final, 10th June 2016. Accessible at: https://ec.europa.eu/transparency/regdoc/rep/1/2016/EN/1-2016-381-EN-F1-1.PDF knowledge when entering further education or the labour market. The obtained official document is valid on the labour market and in the education system. Ferrari, A. (2013). DIGCOMP: A Framework for Developing and Understanding Digital Competence in Europe. European Commission. Luxembourg: Publications Office of the European Union. Accessible at: https://www.zrss.si/iekosistem/wp-content/uploads/digcomp2013.pdf. Javrh, P. (ed.) (2018). Spretnosti odraslih. Ljubljana: Slovenian Institute for Adult Education. Accessible at: https://pismenost.acs.si/wp-content/uploads/2018/06/Monografija-Spretnosti- odraslih_splet_april2018.pdf. Javrh, P. (2016). Raziskava spretnosti odraslih: metodologija in rezultati – na kratko. Ljubljana: Slovenian Institute for Adult Education. Accessible at: https://www.acs.si/digitalna-bralnica/ raziskava-spretnosti-odraslih-metodologija-in-rezultati-na-kratko/. Javrh, P. and Kuran, M. (eds.) (2012). Temeljne zmožnosti odraslih: priročnik za učitelje (2012). Ljubljana: Slovenian Institute for Adult Education. Accessible at: https://arhiv.acs.si/publikacije/ Temeljne_zmoznosti_odraslih.pdf. Javrh, P. (ed.) (2011). Obrazi pismenosti: Spoznanja o razvoju pismenosti odraslih. Ljubljana: Slovenian Institute for Adult Education. Accessible at: https://arhiv.acs.si/publikacije/Obrazi_pismenosti. pdf.. Kennedy, D. (2007). Writing and Using Learning Outcomes. Cork: University of Cork, Ireland. Kodelja, K. (2004). O pojmu splošne izobrazbe. In: Zbornik prispevkov mednarodnega posveta o splošni izobrazbi. Portorož, 16 and 17 April 2004. Kroflič, R. (1992). Teoretski pristopi k načrtovanju in prenovi kurikuluma. Ljubljana: University of Ljubljana, University Development Centre. 50 51 Kroflič, R. (1993). Procesno razvojna strategija načrtovanja kurikuluma. In: Sodobna pedagogika, volume 44, no. 9/10 (1993), pp. 473–487. Kroflič, R. (1997). Kurikulum: Raznovrstnost kurikularnega načrtovanja. In: Andragoška spoznanja, no. 1 (1997), pp. 3–12. Ministry of Education and Sport (2007). Strategija vseživljenjskega učenja v Sloveniji. Ljubljana: Ministry of Education and Sport. Možina, E., Javrh, P., Kuran, M., Vrbanjščak, K., Šmalcelj, P., Radovan, M., Jamšek, D. (2010). Evalvacija javno veljavnega programa Usposabljanje za življenjsko uspešnost (UŽU) in Temeljno usposabljanje za učitelje v programih UŽU. Ebook: executive summary of the final report. Ljubljana: Slovenian Institute for Adult Education, 2010. Accessible at: https://arhiv.acs.si/dokumenti/ Evalvacija_JVP-UZU_in_TU_UZU.pdf. National Examinations Centre. (2007). Pravila o izpitih tujih jezikov za kandidate. Ljubljana: National Examinations Centre. Accessible at: https://www.ric.si/mma/pravila%20za%20kandidate%20 %20-%20tjo/2006062614110983/. OECD. (2017). Skills Strategy. Diagnostic Report Executive Summary – Slovenia. Paris: OECD Publishing. Accessible at: http://www.oecd.org/skills/nationalskillsstrategies/Skills-Strategy-Diagnostic- Report-Executive-Summary-Slovenia-Slovenian.pdf. Skubic Ermanc, K. (ed.). (2012). Z evalvacijo do sprememb. Ljubljana: Institute of the Republic of Slovenia for Vocational Education and Training. Slovenian Institute for Adult Education. (2014). Resolution on the Master Plan for Adult Education in the Republic of Slovenia for 2013–2020. Ljubljana: Slovenian Institute for Adult Education. Trbanc, M., Boškić, R., Kobal, B., & Rihter, L. (2003). Socialna in ekonomska vključenost ranljivih skupin v Sloveniji - možni ukrepi za dvig zaposljivosti najbolj ranljivih kategorij težje zaposljivih in neaktivnih oseb: Raziskovalno poročilo. Ljubljana: Faculty of Social Sciences. Accessible at: https://www.worldcat.org/title/socialna-in-ekonomska-vkljucenost-ranljivih-skupin-v- sloveniji-mozni-ukrepi-za-dvig-zaposljivosti-najbolj-ranljivih-kategorij-tezje-zaposljivih-i- n-neaktivnih-oseb-raziskovalno-porocilo/oclc/441826607 . Vilič Klenovšek, T., Rupert, J. (2010). Značilnosti izbranih ranljivih skupin odraslih in svetovalno delo za izobraževanje odraslih. Nadaljnje usposabljanje delujočih svetovalcev v središčih ISIO, učno gradivo za 2. delavnico. Ljubljana: Slovenian Institute for Adult Education. 52