Tanja Možina
Quality definitions and structural quality indicators in adult education
Abstract: This article shows how quality can be defined and assessed only in its relative aspects and that the definitions of quality in education have a basis in the interests and values. This reasoning also applies to the various lists of quality indicators in education. The inclusion of the concept of interest groups and the relational and dynamic processes of defining quality becomes important for the contemporary epistemological quality theories. It enables various interest groups to get their voice heard in the discussion about what education system we wish to have and to realize their legitimate interests. But the normative thesis of the legitimacy of interests does not really guarantee that the interests of a particular group will be realized. Whether their voice is heard depends on if the group has enough power in society. These processes also include an ethical-political dimension. In this article, the conceptual framework of the collection of quality indicators for the field of adult education, the structure of quality fields and examples of developed quality indicators, quality standards, and quality measures are presented. A reflection is made on how to promote the development of educational institutions for adults as inclusive multicultural communities, in which the rights and interests of groups who have less social power are respected, and their active participation in quality issues become important indicators of the quality of their performance.
Keywords: quality, quality definitions, interest groups, quality indicators, accountability UDC: 374.7 Scientific paper
Tanja Možina, Ph.D., Slovenian Institute for Adult Education, Šmartinska 134 a,
Sl-1000 Ljubljana, Slovenia;
e-mail for correspondence: tanja.mozina@acs.si
JOURNAL OF CONTEMPORARY EDUCATIONAL STUDIES 2/2014, 60-81
Introduction
Adult education has a long tradition in Slovenia. It has continuously developed to address the learning and educational needs of the adult inhabitants of Slovenia and to develop andragogic theoretical thought, which to a large extent has been based on research on educational practice. At the same time, this field has been open to what was happening in other educational systems. Slovene andragogues who went abroad to gain new knowledge have always known how to connect domestic tradition with new approaches, thoughts, and methods of adult education.
In this rich tradition of Slovene adult education, we can trace the first beginnings of the development of the approaches to ensuring quality in education. The questions about what kind of adult education we wish for, such as what is good or not good and what kind of criteria we will use to assess it, are not new, but are inherently built into the development of andragogical theory and practice itself. However, it is true that in the past fifteen years, there have been many possibilities and opportunities for the specialization of this part of the andragogical profession which deals with questions about quality. In the beginning of the 1990s, we established, in the process of the young state's independence, legal frameworks of a new educational system, including the adult education field. Gradually, we developed organizational infrastructure and new methods and approaches. At that time we got the opportunity, as a newly established state, to become included in the International Adult Literacy survey. We were eager to get the results in 1998, which placed us for the first time on such a professional map in comparison with other European states and the world. Nevertheless, the results brought great professional discomfort to those in Slovene andragogy, and they caused a cultural shock to a certain extent, or at least they should have. The reason is that the survey data showed that in literacy tests, 65% to 77% of Slovenian adults between 16 to 65 years old scored at the two lowest levels (out of a total of five) in a group of 20 developed OECD member states. Regarding the literacy of its inhabitants, Slovenia was placed at the bottom of the ladder, which triggered vivid discussions among the professional public (Možina 2011).
This data is now outdated today; however, it still reminds us about the problems of education quality and about the influences of life and work environment
on maintaining and developing knowledge. A new international study about adult literacy achievements (PIAAC),1 in which Slovenia has participated since 2013, will give us information about new results. That professional and cultural shock which we experienced at the end of the 1990s opened new possibilities and opportunities for specialized professional work in researching how new definitions of quality emerge in the development of various approaches to the assessment and development of quality of adult education. Awareness that higher levels of education do not necessarily mean high quality of knowledge was also taken into consideration by educational policy makers so that they allocated professional space and financial means to this field, which has enabled continuous research, development, and applied work over a longer period of time. This has been a very fruitful period for that part of the development of adult education profession, including the work done by the Slovenian Institute for Adult Education (SIAE). In this article, some professional dilemmas and applied solutions that have been developed since the 1990s will be presented.
We provide an overview of the main topics which have been dealt with in detailing other works on quality in adult education in Slovenia. We present the main characteristics of the most recent collection of education quality indicators as described by Možina et al. (2013)2 as well as numerous activities in the field of quality of adult education3 that we developed in Slovenia after the 1990s. We have in mind a renewed collection of quality indicators which expands upon the first collection on the same topic by Klemenčič et al. (2003).4
In the introduction, we present some considerations about the nature of education quality and the complexity of its definitions before presenting the conceptual scheme of the collection of quality indicators. Namely, such considerations help so that when using these collections, or in the analysis of other collections of structural quality indicators, we remain not only on the instrumental level, but we can also do more in depth research on the hidden basics, interests, and values which oriented the formulation and selection of these quality indicators, and not others.
1	PIAAC - Programme for the International Assessment of Adult Competencies (Program za mednarodno ocenjevanje kompetenc odraslih.) More information is accessible at the following website: http://piaac.acs.si/
2	This work is accessible at the following website: http://kakovost.acs.si/doc/N-1028-1.pdf_
3	Among them we could mention at least the following development and research projects from which our experiences and considerations related to this article derive from: The Offering Quality Education to Adults Project (OQEA), in which we developed a self-evaluation model and which is used nowadays by various adult education organizations; The Quality Counsellors in Adult Education Project, with the help of which we developed and introduced into practice the role of quality counsellor in adult education; The Expert External Evaluation in Adult Education Project, in which we have developed a methodology of expert external evaluation since 2011, which has been tested in practice; the research project The Formation of National Standards and Quality Indicators and internal and external quality assessment, in which we theoretically and empirically researched how quality definitions are formed and what the opinions and viewpoints of adult educators are about which aspects of quality should be monitored externally and which should be assessed internally, etc. (For more information, see this website: http://kakovost.acs.si/home/)
4	This work is accessible at the following website: http://kakovost.acs.si/razvoj_podrocja/publici-ranje/ index.php?id=461
The nature of quality and the challenges of its definitions
It is important to understand the concept of quality in adult education as well as its characteristics because it is the foundation of the strategies of quality we will develop, what models and quality indicators we will use, and what instruments we will need for the assessment of quality. Following this research path, we can pose the following questions: What does quality mean to us? Can it be defined? Is quality a notion that has the same meaning for everyone in the field?
In identifying the various factors that influence the formulation of quality definitions, we cannot overlook one of the most original and detailed discussions of the characteristics of the term "quality", which is comprised in the book Zen and the Art of Motorcycle Maintenance (Prisig 2005^.5 This is a work that brings the discussions about the characteristics of quality back to their beginnings. Namely, it again unveils the relationship between the so called essentialist and instrumentalist approaches to quality. Kump (1994) writes that up to now, the essentialist approach to quality was marginalized. The nominalist approach prevails, and it includes the instrumental interpretation of quality and does not deal with the question of what the essence of quality is.
Pirsig (2005) started to research the question of what is good by describing the very practical example of motorcycle repair. In this way, he explains very complex epistemological questions to the reader with the help of the examples from everyday life. In order to explain how people perceive the world around us differently, he shows two extremes. He divides human perception into two kinds: a classical one (rational analysis) and a romantic one (creativity, imagination, personal engagement, and attitude). Pirsig (ibid.) uses the example of a motorcycle to show that it is possible to divide it into its constituent parts and in relation to its functions with a classical rational analysis. He illustrates this reasoning as follows: "The most basic division according to the constituent parts is the division into the power actuating system and according to its accompanying parts or the chassis. The power actuating system can be divided into the motor or the system for the transfer of power. But first we have to look at the constituent parts of the motor. The motor consists of the housing and the main part of the motor, of the system for the supply of the filling mixture, the ignition device, the feedback system and the lubricating system. The main part of the motor consists of the cylinder, the rammer, the connecting rod, the crankshaft and the engine flywheel etc." (Ibid., p. 87)
At first glance, there is nothing unusual in the description of a motorcycle, says Pirsig. The description seems like the beginning of a schoolbook or like the first hour of a training course for mechanics. Only later do we notice the extraordinariness of this description, when it stops being the style of our thinking and it becomes the object of our thinking. Then it warns us against some issues:
5 The work was first published in 1974. In 2005, it was also first published in Slovene. The detailed analysis of this work was presented in Možina (2010). It is accessible at the following website: http:// kakovost.acs.si/doc/N-463-1.pdf . In the article, we summarize some of the most important considerations from this long analysis, when we will analyze how structural indicators are formulated.
(1) The description is extremely boring, in the way that a "romantic person" perceives a classical style as boring, rigid, and ugly. Romantics that are willing to go further on from this description are rare. (2) One cannot understand such a description of a motorcycle, if one does not know in advance how it functions. Namely, the description does not include any direct superficial impressions, which are needed for initial understanding. It only consists of hidden bases. A further characteristic is that the observer does not exist in this text. (3) There are no descriptions of "good" and "bad" or their synonyms in the description. There are no value judgements. There are only facts. (4) The fourth characteristic is that the intellectual razor that moves up and down the text is so swift and sharp that sometimes one cannot notice its movement. It seems that all these constituent parts are simply there and that the text names them in the only possible way. However, it is in fact possible to name them differently and to arrange them differently. "It is important to see this knife," says Pirsig. "Otherwise we will be misled into silly thinking that motorcycles are as they are, only because the knife did this cut and not another one. It is important to focus on the knife itself. We all use this knife, we all divide the world into parts and construct various structures", continues Pirsig (ibid., p. 89).
The relativity of quality definitions
Pirsig's warnings also become important when we think about various models of quality. People who design such models use an "analytic knife" when they design the lists of quality indicators. We can quickly make an analogy that not even two designers of such quality models would define exactly the same fields and formulate the indicators in exactly the same way. In order to understand the models of quality themselves, it is therefore not enough if we only examine the structure, scheme, and lists of indicators. It is important to know who the person or group was who wielded the analytical knife and formed such and another structure. Furthermore, it is good to know whose interests and views influenced the decisions and cuts. We find that this analytical razor was led by a certain point of view and ideas about what is important or what is good. This means that we cannot define quality in the absolute sense or by its potential. Thus every definition we make will only be partial (Možina 2010, pp. 29-31). From this finding, we see that we will never be able to define the absolute criteria of quality and that we can only draw conclusions about the quality of educational inputs, processes, results, and effects indirectly - according to some measureable entrance, processual and exit headings of education (Kroflič 2002).
This means that with quality models we can "catch" only some dimensions of all the potential that a certain work or an activity can have. However, we will catch only the dimensions that we can see, recognize, and perceive. This depends on what is important to us, what our interests are in connection with education, and what we expect from it. Due to the fact that the expectations and interests of various groups who participate in education can be very different, this is also characteristic for quality models. Many different viewpoints will be interwoven
into them in relation to what quality is, as there will be interest groups who will have an opportunity to participate in these processes.
Stake (1983, 2000), like Pirsig (2005), paid attention to the individuals and groups who have their interests and expectations in education when he was formulating the model of responsive evaluation. In his opinion, it is necessary to assure the cooperation of the representatives of such groups already when the goals of a certain program or an educational activity are defined. If we take into consideration such a participatory approach to defining quality, the role of the evaluator who assesses the quality of education changes as well. Stake has already discussed the characteristic that later became the main driving force of the naturalist-constructivist evaluation, namely that when defining a relative characteristic of quality, the evaluator's role is not that he/she shows in the evaluation the only possible objective reality, but that he/she includes in a comprehensive way the various realities and viewpoints on the activities in a certain program or educational activity as they are perceived by the various actors who are a part of this activity and have their interests related to it (Stake 1983, 2000).
If we accept the relative nature of quality definitions and the fact that we cannot measure quality directly, we can find that we need tools for measurement such as quality indicators, which enable us to assess quality and to think about it indirectly. The already planned strategic goals of education reflect the interests and expectations of the various interest groups involved, which have an influence on the formation of different quality indicators (Lessnigg 2003).
The influence of experiences, environment, and interests on quality definitions
Let us come back to Pirsig for a while, who, when thinking about the influence of our experiences and interests on quality definitions, connects his reasoning to Kant's thoughts about the fact that we cannot see some aspects of reality directly from our perceptions. Kant calls them a priori aspects. In our mind, we have a very real a priori motorcycle, which has formed itself over the years through numerous sensory perceptions. It constantly changes when we accept new perceptions. When he tries to explain how quality definitions emerge and how they change over time, Pirsig reminds us of what Kant called "the Copernican revolution". "The principle - the finding of Copernicus that the Earth circles around the Sun did not change anything, nevertheless, everything changed. The objective world from which our sensory perceptions originate did not change, however, our a priori concept about it became completely upside down" (Pirsig 2005, p. 147).
From this observation, we can make a conclusion that helps us to understand how people define what is important to them. We could say that we ourselves have formed in our thoughts an a priori image about what quality education is. This image is formed over time, and it depends on the environment in which we live and work, and on the experiences that we have already gained in education; it also depends on the values which guide our thinking. This a priori image definitely has an important influence on how we will assess the quality of education. However, the Copernican revolution could in our case mean that in the moment
when we change our image of what is good, we assess the educational experience in a different way. When we speak about quality, this is important from another point of view. A teacher, who has his/her own perception of what good teaching is, will act according to this perception. Even if someone else would judge that these activities are not of a high quality, nothing will happen until the teacher develops new and improved concepts about his/her activities and changes his/her subjective theories (Korthagen 2004).
We can relate this observation to some characteristics of contemporary evaluation models. The typical representatives of such models are E. G. Guba and Y. S. Linclon, the founders of the so called constructivist-naturalist evaluation.6 Guba and Lincoln draw our attention to the fact that in order to understand this approach, we need to know the philosophy of the constructivist paradigm of research, following which the approach is formulated. They wrote, "If we assume that multiple realities exist and they depend on time and context in which their constructors exist, it is necessary that the study occurs in the same temporal-contextual framework, which is researched by a researcher who tries to understand it. Namely, there are the contexts which give life to constructions and on the contrary, the constructors make the contexts in which they act alive" (Guba and Lincoln 1994, p. 175). This is the first common point between Pirsig's thinking and the ideational framework set by Guba and Lincoln. Reality is not only one; there are many of them. They depend on time and the circumstances in which the people who form those realities live. Let us find an analogy to our research on quality. We can make similar conclusions: the quality concepts which are prevalent in a certain environment, society, or characteristic for an individual are not concepts per se because circumstances influence their formulation, as well as the social circumstances of the time in which those concepts emerge. It is very likely that in a certain period in similar social and cultural environments, similar views on quality would appear. However, the situation might change, if the concepts that were formulated under the influence of different cultural and social circumstances would encounter.
Having that in mind, Guba and Lincoln stop again at the constructions themselves, which exist, according to their opinion, merely in the minds of people and are not a part of some objective reality. The main task of the constructivist researcher is that he/she examines various constructions that were formed by different actors, and that he/she unites and merges them, if possible, and at the same time connects them with all other possible information which he/she can get about the matter that is the focus of the research (ibid, p. 14). Such considerations are important for quality definitions because they tell us that it is not enough merely to develop accurate systems of quality indicators and systematically gather the information and data. In order to change and improve something, it is necessary to unveil the meanings which are attributed to quality indicators by the various interest groups in education. Those meanings should be confronted in a dynamic dialogical process with those who contributed them (Možina 2010, p. 37).
6 The basic characteristics of this approach were described in detail in the book Quality in Education (Možina 2003, pp. 150-166). For this purpose, we will only focus on those characteristics of this approach that help us to understand how quality definitions are formulated.
From static to dynamic quality definitions
According to Guba and Lincoln, the right way to approach such processes is in the use of the so called hermeneutic-dialectical process. The process is hermeneutic because it is interpretative in its nature, and dialectical as well. That enables the confrontation of various points of view in order to reach their synthesis on a higher level (Guba and Lincoln 1994, p. 149). The goal of this process is to reach an agreement, and when this is not possible, at the end of the process various opinions and points of view become clear, so that it is possible to create a field for negotiations. If the process is successful, it ends so that all sides reconstruct their opinions and points of view on the topic at hand (constructions), with which they joined the process. According to Guba and Lincoln, this also happens when we do not reach an agreement. All sides are thus simultaneously exposed to education and enhancement (ibid., p. 150).
It is important to pay attention to the ethical aspects of such processes, which are based on the confrontation with and the acceptance of diversity and dialogue. Thus we can pose the question: In what way are we confronted with various views on what quality education is when we design the collection of quality indicators? Who has the possibility to discuss it and make decisions about it? What do we do when it is not possible to reach an agreement? We will lead these discussions about what kind of education we want and what its quality should be, taking into consideration and allowing that quality definitions have a relative, interest oriented and value-laden starting point. This approach enables various social groups' voices to be heard and to realize their legitimate interests.
At the same time, we are confronted with the danger of having too big of a dispersion of interests, and the danger that the interests of stronger groups prevail over those of others (Apple 2003; Laval 2005; Mitchell et al. 1997). This would lead to the various relative and diverse points of view on quality, which are difficult to manage, and to difficulty in finding their common denominators. In their analysis of the characteristics of interest groups, Mitchell et al. (1997) showed that the normative thesis of the legitimacy of interests in fact does not guarantee that the legitimate interests of a certain group will be realized. One of the key characteristics to which we should all pay attention when we analyze different groups and their interests in connection to quality is power. The analysis of the concepts of interest groups additionally unveils the topic to which Apple (2003) and Laval (2005) draw attention: we are encountered with it when some partial interests become realized with the help of a certain interest group's (or an interest coalition's) social power in a certain moment which is stronger than the power of other interest groups (Možina 2010, p. 216).
The topic of interest groups and the possibilities of the realization of their interests also always consists of an ethical-political dimension, especially due to these aspects. Therefore, we need to put accountability into the core of contemporary views on quality - a personal and common professional accountability for quality (Gardner 2001). The ethical imperative for the recognition and participation of diversity of life stories and individuality/diversity of personal meanings in the
inclusive environment concerns all often excluded social groups of individuals, such as handicapped people, migrants, women, the poor, and members of other races (Kroflič 2008b), and these are groups that we frequently encounter in adult education. We should contribute to a more just cooperation with those interest groups by defining, assessing, and developing quality with contemporary participative strategies of evaluation, which are based on the participation in decision making and on taking into consideration the theories of interest groups.
The dimension of accountability is at the core of the focus of contemporary epistemological theories about quality. Firstly, a personal professional accountability for quality is important (Moller 2005), and demands that we occasionally turn towards ourselves to reflect on our work and achievements, and based on those findings we come to new development solutions. It is a personal and professional commitment to our profession which demands from us such accountability. For this purpose, the methods of internal quality assessment have developed in their purest form; they are based on methods of self-evaluation. Such ways of assessment should be placed at the core of the quality systems, and at the same time they should be upgraded with external ways of quality assessment. Beside the fact that we are responsible to ourselves and to our commitment to a profession, we are also accountable to those people who entrusted us with this work and to those for whom we do this work. Therefore, in contemporary quality systems we should find room for ways of expressing accountability for quality externally as well as internally7. It is important that the aims of the first ones and the others do not mix and especially that the levels of decision making do not get mixed. If the internal methods of quality assessment are an especially important element of quality development in the educational system, we need to view them also from their external aspect, with the goal of protecting the participants and their rights to quality education, justice, equal opportunities, and equal treatment (Rawls 1999), as well as their participation in defining, assessing and developing quality.
In the development of epistemological theories, we can notice a move from thinking of quality and attributed quality models as static, according to their nature, to thinking of them as different relationally and dynamically conceived quality models. The dynamic nature of quality definitions contributes to the fact that they emerge all over again and become realized in relations. However, this also means that those people who form such or a different structure of quality indicators in adult education cannot define the position from which they do so as neutral because people who form quality indicators are not separate from this process, but are also participants in it. The inclusion of the so called dialogic approach into the epistemological theories about quality is, as suggested by the authors of the fourth generation of evaluations (Easton 1997; Guba and Lincoln 1994), decisive, because we need new communication competences (Bingham and Sidorkin 2004), so that we will be able to encounter all diverse interests and values
7 These processes are based on the viewpoint that the taxpayers have the right to have insight into the quality of the functioning of educational system (educational organizations) and into what the results and effects are.
which are characteristic for the societies in which we live. At the same time, we should take into consideration the need for trying to find those aspects of what can be agreed on and common with the help of dialogic instruments, bearing in mind and recognizing diversity. This will enable us to have a creative coexistence in a community when taking into consideration the principles of justice and respect for equal opportunities (Možina 2010, p. 221).
Quality as value
We find that when considering how quality definitions emerge, we cannot stop ourselves at the mere instrumental and methodological understanding of these processes, but we also have to return to an essentialist understanding. This is what Pirsig (2005) does when he expands the paradigm of traditional science so that he introduces the concept of value. ^^he value, which is a leading margin of reality, is not an unimportant side branch of the structure anymore. The value is a predecessor of the structure. Our structured reality is previously selected on the basis of values. And in order to truly understand structured reality, it is important to understand the origin of the values from which it originates" (ibid., p. 305). "Therefore", says Pirsig, "our rational understanding of motorcycle changes from one minute to another, when we deal with it and find out that there is more quality in our new and changed rational understanding. So we abandon our rigid old ideas on the basis of direct rational findings. He again calls our attention to the source of changing and growth which is hidden in a dynamic thinking about reality. Reality has its own forms, however, these forms are capable of change" (ibid., p. 306).
At this point it becomes evident that the question of what quality is to us also contains an even more important question: What is important for us? One or another quality definition always has its origins in a certain value framework. The research on the importance of quality always leads us to the following question: What kinds of values are embedded in such or another quality definition?
Apple (2003) criticizes the contemporary social system and shows that topics that are related to education have always included basic conflicts and compromises among groups, with competitive visions of what is considered legitimate knowledge, what can be counted as good teaching and learning, and what a just society looks like. In the past, we witnessed such encounters of different value systems in defining quality in education more often than today, when we live in plural social systems in which it is not possible to accept any discussion and interpretation as the only valuable one and the only one that is correct. Namely, modern pluralism leads to greater relativization of value and explanation systems (Berger and Luckmann 1999). Such relativism which we face in the contemporary world of Western societies to a great extent influences the search for answers to the question about what quality education is, what a high quality educational organization is, and what a high quality teacher is.
If we accept the value basis of quality definitions, we can more easily find the answer to the question: How is it possible that different people understand
quality differently? Our viewpoint is led by our value framework, which in a certain way defines our interests, and this basis that originates in values and interests means that, in the scope of the potential of various opportunities, we most clearly see those who are the most important for us. Someone else will see other shades of quality from his/her standing point.
A relative comprehension of the concept of quality leads us to the insight that we need to seek the answers to the questions about what quality education is and what quality we want in adult education in a democratic discussion in which various viewpoints and interests are included. This approach leads us to take into consideration various aspects of quality and enables us to take a comprehensive approach to caring for quality in adult education.
Quality indicators in adult education
All the above-mentioned professional considerations and findings of contemporary epistemological theories about quality have strongly influenced the work which has been done in recent years by the professional team at the Slovenian Institute for Adult Education when we were preparing the collection Quality Indicators of Adult Education (Možina et al. 2013), which we will present in the following section.
If we, at this point, come back to Pirsig's thinking (2005) about the analytical knife, which creates such and other structures, and we try to answer the question of who was the person or group who wielded it in the preparation of the Quality Indicators of Adult Education, we can say that it was the profession as a whole that predominantly used it, meaning the professionals who work in the theory and practice of adult education. Thus the prevailing interest which could be recognized when reading the fields, indicators, and standards of quality is therefore professional. Besides the interests and viewpoints of adult educators, we have also included in the development of quality indicators the viewpoints, opinions and interests of other interest groups, such as the participants, educational policy makers, sponsors, representatives of employers, etc. who have already been included in the above-mentioned research studies, whether we have in mind the participation in questionnaires or in focus groups.
The aims of using the collections of quality indicators
We can use the collection of quality indicators for two basic purposes: for the purpose of the internal as well as the external assessment and development of quality. From that perspective, we can recognize in the collection the application of those aspects of epistemological theories about quality dealt with in the first part of the article, regarding the internal and external accountability for quality (Anderson 2005; Moller 2005). On the one hand, such an approach is pursuant to the processes of the decentralization of education, for which the transfer of accountability is characteristic for the acceptance of decisions from the national
to the lower regional and local levels. In this way, the organization becomes more autonomous in adopting the decisions, and thus more accountable for the quality of education, which it plans and implements. Such accountability demands that the organization has also introduced the methods of internal quality assessment. On the other hand, we can notice in this approach the aspect of the so called enhanced accountability for quality. It is especially characteristic of the education financed from public funds that the actors of the education system are called to answer the questions about what has happened in their field of accountability. In educational systems, those assessments are usually performed by the ministries or various external agencies. Nevertheless, adopting accountability is an important aspect of professionalism. Namely, this dimension emphasizes that, for example, a teacher is ethically responsible to the participants and other important interest groups who participate in education or invest in this education. Those processes can also be seen as those which represent the commitment to ethical standards of all those who plan education and contribute to the quality of its implementation (Moller 2005).
The collection of quality indicators was designed with the intention of being used in this part of adult education, which is in the public interest and financed by the state; therefore, it is especially important that at least the minimum standards of quality of this education are assured. It can be used for the assessment of formal and non-formal adult education. In the first place, the collection is intended for the assessment and quality development on the level of educational organization. However, it is possible to use numerous quality indicators for the purpose of the assessment and quality development at the level of the educational system. This initial decision about the purposes for which we can use quality indicators later on also influences how an assessment and quality development will be implemented, and, what is even more important, who will be their operator. Accountability is an important part of the model. We have in mind accountability for this activity, for example, of (1) policy makers and sponsors of this activity, (2) professional institutions which contribute to the development of the andragogical profession, (3) faculties which develop andragogy and its related disciplines, and (4) educational and other organizations that implement this activity (their leadership and staff) and all other subjects connected to them or those who are interested in their activity (employers, developmental and other factors from the environment) (Možina et al. 2013, pp. 11-12). The collection of quality indicators can be used in various ways for the assessment and quality development.
Competent ministry oz. body responsible for accreditation
External expert institutions
External evaluators
EXTERNAL MONITORING
INTERNAL MONITORING
EXTERNAL EVALUATION
Figure 1: Levels, methods, and actors of quality assessment
These quality indicators are the following: (1) Self-evaluation - a periodic, in depth self-assessment and quality development of all the measures that are important in order to reach the defined national and own quality standards from the point of view of adult education. The actors of self-evaluation are the leadership and staff in an educational or other organization which educates adults. (2) Internal and external monitoring - continuous monitoring of the measures that provide basic information on the work of the educational organization in the implementation of adult education and enable quick reactions with the introduction of corrective measures and improvements. (3) External evaluation - the external quality assessment of some measures, which are important from the point of view of national policies and the system of adult education or quality indicators, which are especially important in a certain period. The external evaluation is performed by external professionals. (4) Accreditation 8 - an external quality assessment of the assurance of some basic measures which are, according to the opinion of the profession and its sponsors, necessary for quality planning and the implementation of adult education. The collection of quality indicators enables us that we use it in each of the shown approaches independently; however, we can also use it in a way that we combine the approaches among themselves (ibid., p. 12).
The structure of the collection of quality indicators
The way we have chosen to structure the collection of quality indicators in adult education originates from the methodological starting point that the fields
8 When the renewed collection of quality indicators in adult education was published, the accreditation had not been formally introduced into practice; however, the aim of a renewal and a new recommendation is to encourage its systemic introduction. The definitions about which indicators, standards and quality measures can be an object of the accreditation are only prepared in the form of recommendations and professional guidelines for competent bodies which have their competencies for the decisions about which quality indicators and standards related to them, as well as the quality measures, will be included in the accreditation procedures, and thus they will also get a normative value.
and subfields of quality, as well as the quality indicators related to them, should be connected among themselves. The external as well as the internal connectedness are both important. External connectedness emphasizes that it is necessary to connect fields, subfields, and quality indicators with the goals of the activities in which the collections of quality indicators will be used. Internal connectedness emphasizes that the quality indicators which we introduce into a certain quality field should be harmonized and connected, and they should not oppose themselves. When preparing the collection, we used the processual quality model9. We took into consideration the starting point that a theoretical model of a field that we measure should be an important conceptual basis for working with standards and quality indicators. In our case, these are predominately andragogical theoretical models. However, some quality fields, such as leadership, are also based on other disciplines, such as organizational theory, human resource theories, etc. (Bole Kosmač 2005; Lessnigg 2003).
The quality fields are, pursuant to this model, structured so that they represent the transverse, input or infrastructural, process, or output factors of quality. The illustration below shows how we have cut adult education into the undefined substance of quality and how we have "cut" the structure of quality fields with the help of the "analytical knife".
Cross factors
MANAGAMENT AND ADMINISTRATION
EDUCATIONAL PROGRAMS
PROMOTION OF
ADULT EDUCATION AND ENCOURAGING ADULTS TO
ENTER EDUCATION
STAFF
PREMISES AND EQUIPMENT
PLANNING EDUCATION
REALIZATION OF EDUCATION
DEVELOPING WORK IN SUPPORT OF THE EDUCATIONAL PROCESS
SUPPORT FOR INDIVIDUALS IN EDUCATION
RESULTS
EFFECTS
Input factors
Process factors
Output factors
Figure 2: Quality f ields of adult education
We defined each field in detail into subfields, and each subfield was "cut" into quality indicators. 98 quality indicators are included in the collection. This structure of the collection enables enough creativity in combining and connecting
9 The original economic model as a tool for narrow economic analyses was expanded into a whole development, which includes, besides social and political as well as other aspects. In education, this model is known as the CIPP model ("context", "input", "process", "output"). Stufflebeam (1983, 2000) is the author of its implementation in the field of education.
quality indicators for the purpose of various analyses. There are too many quality indicators to be presented here;10 therefore, in continuation we only illustratively enumerate the examples of the definitions of quality indicators for the field Supporting individuals in education and learning in the organization in order to achieve a better insight into the structure of the collection.
It is evident from the example below in what way we applied in the collection and developed the theoretical considerations that we presented in the introduction. They emphasize that we cannot measure quality directly, in its absolute form; nevertheless, we can assess it indirectly in a relative way (Pirsig 2005; Harvey and Green 1993). Or as written by Kroflič (1994) when he describes the relation between the absolute and the relative, "in order to successfully assess the curriculum, we need the adequate indicators for the assessment, that means the elements which tell us if the composition of the curriculum and its implementation are adequate." (Ibid., p. 239) He adds that over the years we have become aware that it will never be possible to formulate ideal absolute quality indicators that will be equally suitable for all fields and levels of education. Nevertheless, gradually the lists of some basic quality indicators are being formulated, which cannot be overlooked in the assessment of the curriculum and its implementation (ibid.). We had these considerations in mind when we defined which aspects that show the quality of the processes of the support for individuals in education and learning in the organization.
QUALITY AREA	QUALITY SUBAREA	QUALITY INDICATORS
SUPPORTING INDIVIDUALS IN EDUCATION AND LEARNING	Diversity and accessibility of support for an individual	Diversity of support for an individual in education and learning
		Accessibility of support for an individual
		Informing an individual about the forms of support within the organization
		Informing an individual about the forms of support outside the organization
	Study support for an individual	Study help
		Consultations and mentorship
	Guidance support for an individual	Guidance for an individual before and during matriculation into education
		Initial interview
		Personal education plan
		Guidance for an individual during education
		Guidance for an individual at the end of the education program
	Support for an individual in self-directed learning	Guidance and mentor support for self-directed learning
		Providing possibilities for self-directed learning
	Support in removing obstacles in education	Removing situational obstacles
		Removing institutional obstacles
Table 1: Subfields and quality indicators for the field supporting individuals in education and learning in the organization
10 In Quality indicators of adult education there is a table on pp. 403-408 that contains the comprehensive list of quality indicators, accessible at: http://kakovost.acs.si/razvoj_podrocja/publiciranje/ index.php? id=1028
When we choose quality indicators as the object of our analysis, we can help ourselves further on with the collection, which helps us to achieve a more in depth understanding of the contents to which a quality indicator directs us. In continuation, we will have a look at the elements which are included in each quality indicator and how can we help each other with them.
QUALITY INDICATOR
QUALITY STANDARD
QUALITY CRITERIA

What is the expected quality for the selected indicator?
How will we know if the quality standard is met?
What do we assess for each indicator?
Examples of possible measures for quality development.
Quality indicator can be used for:				
SELF-EVALUATION	INTERNAL MONITORING	EXTERNAL MONITORING	EXTERNAL EVALUATION	ACCREDITATION
ü			ü	
Figure 3: The content of the structure of an individual quality indicator
A quality standard is defined for each quality indicator. It describes what a defined or expected level of quality is for each indicator. Quality measures are defined as well, with the help of which we assess if the adult education organization entirely or only partially reaches the defined quality standard. Let us see the example for the selected field Educational programs and the subfield The development of own educational programs.
Quality indicator
The development and modernization of educational programs
Quality standard
Together with its partners, the adult education organization develops and reforms educational programs on the basis of the identified educational needs.
Quality criteria
The adult education organization has programs it has developed itself in its program scheme.
The methods are set which occasionally assess how the educational programs of the adult education organization still meet the educational needs in individual sectors or professions, the local environment, or the needs of individual target groups._
New educational programs - or reformation of the old ones - come from the identified educational needs.
Before creating a new educational program, the procedure to define its type is carried out.
Employees have adequate knowledge to develop educational programs.
Partners from the local environment participate in developing new programs or the reform.
Educational programs are created according to the methodologies that are prescribed for different types of programs by regulations and/or expert principles._
Table 2: The example of the defined quality indicator, quality standard and measures
It is an important element in defining standards and quality measures which shows that in developing the collection, we did not avoid the explicit definition of what level of quality we expect from a certain adult education organization. In such operationalization of the quality model which is presented by us, we had to move from essentialist epistemological quality theories to the thinking about the nature of quality itself, which means to more instrumentalist and functionalist theories. Namely, the defined quality standards represent certain partial, relative quality definitions. In this case, we assess quality so that we find out to what extent a certain product or service met the description that was written for it (Harvey and Green 1993; Sallis 1993). At this point, we again used Pirsig's (2005) analytical knife when we developed quality standards from a certain field of knowledge, in this case andragogy.
In the part which is entitled What do we assess for each indicator? Numerous examples are presented, such as what is it is that we have to pay attention to, which questions we need to pose in order to get answers, and to what extent the quality standard has been reached. Of course, here we have in mind only setting the examples; each evaluator will be able to form different questions when he/ she will consider them meaningful and necessary in relation to the quality aspect that he/she assesses and in relation to the aim and the kind of the evaluation he/ she performs.
We have to call attention to the fact that it is not possible to assess all quality factors using all methods of quality assessment. Therefore, for each quality indicator, recommendations are enumerated for the following question: when is it meaningful to use it in the process of self-evaluation, in internal and external monitoring, external evaluation or accreditation? It is possible to use the same indicator, for example, for the self-evaluation or external evaluation. The collection is useful again in the phase when we have already accomplished the collection of data and information, validated them, and found out to what extent the standard has been reached, and we plan the necessary improvements. For each quality indicator, the examples of possible actions for the development of quality are stated. The examples of possible actions encourage the investment in training of employees, the definition of procedures, the development of new forms and work methods, the development and the introduction of the tools for supporting the high quality implementation of the processes, etc. (ibid., pp. 23-27).
Conclusion
In the end, let us summarize the following development trends in the field of epistemological theories about quality:
-	the knowledge of relative and interest or value basis of quality definitions;
-	the knowledge of the characteristics of interest groups and their influences on quality definitions;
-	the introduction of a personal and common accountability for quality into the core of contemporary quality concepts;
-	a dynamic and relational nature of the processes of defining, assessing, and developing quality;
-	the need for the development of organizational structures and organizational culture which will develop a dialogical approach, mutuality, justice, and respect for differences;
-	the need for combined development of internal (a self-reflection for own development) and external (expressing accountability for quality) ways of quality assessment.
If we take into consideration such professional orientations, we can say that the development of approaches and models of quality in the field of adult education in Slovenia follows them to a large extent. In the past 15 years, the approaches of self-evaluation have become widespread in educational institutions. They are one of the instruments which help us to enhance the internal accountability of educational organizations for quality. However, it should be mentioned that the internal approaches to quality are mostly and predominately formulated by those educational institutions which are developmentally oriented and have enough self-initiative, while some other organizations are at the beginning of this process, and they need external encouragement. If we can, on the one hand, estimate that there were quite some such external encouragements in the scope of various projects which were led on a national level, and that they bring positive effects, we can, on the other hand, call attention to the fact that such activities are undefined in a normative sense. While some laws require that educational institutions are obliged to introduce the internal quality system and self-evaluation, such as the Vocational Education Act and the Post-Secondary Vocational Education Act, the Adult Education Act does not define such an obligation.
Taking into consideration the interest and relative nature of quality definitions (Guba and Lincoln 1994; Harvey and Green 1993; Pirsig 2005) and a related need for the inclusion of various interest groups into the definitions of quality and its assessment, we found out in a study we did at the SIAE in 2008 that the organizations to quite a large extent include the internal interest groups in the processes of self-evaluation (such as participants, teachers, the leadership, and professionals); however, they included external interest groups to a smaller extent (employers and other stakeholders in the local environment). Therefore, it would be important to also pay attention to this segment in the future in training the members of the committees for quality development and other professionals. At the same time, attention should be paid to the enhancement of the so called relational dimension, the encouragement of various actors which operate in adult education, and to the enhancement of trust and cooperation (Nahapiet and Ghoshal 2000), which is necessary for the disclosure of various views on quality and the search for consensus in defining quality.
If we can be satisfied with the development of the processes of self-evaluation in the field of adult education, and we can estimate that it is further necessary to invest in such development, to enhance it, and to develop its normative framework, we cannot give the same assessment of the so called external approaches, those
that are justified on the basis of expressing accountability for quality. In this article, we described the approach developed at the SIAE which conceptually presupposes a combination of various methods of assessment and quality development, including internal as well as external ones. Here, we would like to emphasize that the instrument of accreditation which we included in this approach has not been implemented in the field of adult education. We have the instrument of the subscription to the list of operators of the state-recognized educational programs, however, it is quite old-fashioned and modest with regard to the conditions which educational organizations must meet in order to get the right to implement a certain state-recognized program. At the same time, these conditions are not checked periodically. This causes considerable anomalies which we have already noticed in the above-mentioned research11. However, the implementation of the external evaluations in the field of adult education is still systemically undefined. Nowadays, the external evaluations are implemented by various national research institutions, faculties, and public institutions, but the choice of topics covered by such evaluation studies is mostly left to the partial interests which causes some topics that are important for the quality of adult education to remain uncovered. So we find again that there is a need for a systemic and normative framework in this field.
Slovenia received such a recommendation from the European Commission in the scope of the study on quality in adult education12. In the study which was carried out on the European level, it was found that Slovenia has quite well developed approaches to quality on the level of operators - as one of the 13 examples of best practice, which the European professionals identified in the European space of adult education, the OQEA model for self-evaluation was presented. It was also emphasized that in the field of adult education the coherent systemic approach, which would also find its place in the normative framework, has not been fully developed. Therefore, the consistent normative framework is the biggest inconsistency and at the same time a strategic and developmental challenge, which can be noticed in this period in the field of the development of approaches to assessment and quality development in adult education. This lack of definition from the systemic and normative point of view is very alarming because it influences the financing of these activities and affects the inclusion of such projects and approaches into the system. If the numerous activities which we have developed up to now will only remain at the project level, we can envisage that it is likely that they will gradually wither away.
Taking into consideration the warnings of many authors that it is necessary to pay more attention to essentialist approaches and to allocate enough space to the considerations about how we define quality and what kind of quality we wish to have in adult education, we think that it would be wise to think about the development of organizational structures that will respond to the alarming social problems of contemporary and future times. In the field of adult education, it would be important to encourage, with the assistance of quality models and
11	More data on the research results is available in The Development of Quality in Adult Education: http://kakovost.acs.si/doc/N-464-1.pdf
12	European Report on Quality in Adult Education. http://kakovost.acs.si/doc/N-1019-1.pdf
indicators, the further development of communities of care, which will, taking into consideration a relational dimension of quality, enhance mutual connectedness and affiliation, as well as the collegial power and social skills that are necessary for caring prosocial behavior (Eisenberg and Mussen 1989). In this way, inclusive and multicultural communities become connected. These are the communities which adapt to the diversity of their participants and which do not expect that all the participants will be able to adapt to the usually explicitly rigid performance of educational institutions (Gilligan 1993; Kroflič 2008b; Noddings 2002). Such communities are justified on the accountability as the form of respectful relation to the Other (Levinas 2006) and they function pursuant to the principle of justice in connection with the principle of fair equality of opportunity13 and by taking into consideration the difference principle14 (Kroflič 2008a; Rowls 1999). Taking into consideration the rights and interests of groups that have less social power and their active participation in assessing and developing quality thus become key quality indicators of the performance of the educational organization.
Therefore, we in the field of adult education do not advocate the unification of the models, approaches, and quality indicators for the whole field of education. We are more inclined towards the thinking about searching for that which is common to all kinds of education and what we could connect into a certain common wider framework, which should be such that it will enable a constructive co-existence of various approaches under the scope of it. If we have predominantly in mind in formulating quality models the benefits of, for example, a child in kindergarten, a student, or a participant in adult education, with all of his/her developmental and other characteristics, and especially if we will keep in mind the fact that the above-mentioned target groups are not internally homogenous, we will never be able to form equal quality indicators. However, it is possible in these approaches, to find a lot in common and to formulate a certain common professional framework which will be of assistance to all of us who deal with the development of quality in various educational fields.
References
Anderson, J. A. (2005). Accountability in education. Brussels: The International Institute for Educational Planning, Paris and the International Academy of Education.
13	It is not enough for them that social positions are available to all people formally; what is more, everyone must have fair opportunities that he/she can reach such positions - especially regardless of the social position of his/her family. This has implications for educational policies, namely that social inequalities need to be corrected, and the disadvantaged should benefit the most from the inequalities in expressed efforts and skills.
14	The difference principle legitimizes unequal treatment of those who are different, if this is for the benefit of the underpriveleged. It appears as a demand that the inequalities regarding the quantity of primary social goods to which they are exposed are in favor of the most underpriveleged people (Rawls 1999). It should be organized in accordance with the expectations of underpriveleged people, meaning that the inequalities are legitimate if they bring benefits to those who are in the worst situation in a society (Kroflič 2008a).
Apple, M. (2003). The State and the Politics of Education. New York: Routledge.
Berger, P. L. and Luckmann, T. (1999). Modernost, pluralizem in kriza smisla: orientacija modernega človeka. Ljubljana: Nova revija.
Bingham, C. and Sidorkin, A. M. (eds.). (2004). No Education Without Relation. New York: Peter Lang Counterpoints, Studies in the Postmodern Theory of Education.
Bole Kosmač, D. (2005). Metodološki in izvedbeni okviri indikatorjev za spremljanje razvoja Slovenije v družbo znanja. In: Indikatorji na znanju temelječe družbe - metodologija, pregled, nabori. Ljubljana: Inštitut za ekonomska raziskovanja, Fakulteta za družbene vede, Center za poučevanje organizacij in človeških virov, pp. 3-27.
Easton, P. A. (1997). Sharpening our tools: Improving Evaluation in Adult and Nonformal Education. Hamburg: Unesco Institute for Education: German Foundation for international Development.
Eisenberg, N. and Mussen, P. (1999). The Roots of Prosocial Behavior in Children. Cambridge: Cambridge University Press.
Gardner, H. (2001). The Ethical Responsibilities of Professionals. Available at: http://the-goodproject.org/wp-content/uploads/2012/09/GoodWork2.pdf (Accessed on 24.6.2014).
Gilligan, C. (1993). In a Different Voice. Cambridge: Harvard University Press.
Guba, E. G. and Lincoln, Y. S. (1994). Fourth generation evaluation. Newbury Park, London, New Delhi: Sage.
Harvey, L. and Green, D. (1993). Defining quality, Assesment and Evaluation in Higher Education, 18, issue 1, pp. 9-34.
Klemenčič, S., Vilič Klenovšek, T. and Možina, T. (2003). Kazalniki kakovosti. Ponudimo odraslim kakovostno izobraževanje. Ljubljana: Andragoški center Slovenije. Available at:http://kakovost.acs.si/doc/N-461-1.pdf (Accessed on 3. 5. 2014).
Korthagen, F. A. J. (2004). In search of the essence of a good teacher: Towards a more holistic approach in teacher education. Teaching and Teacher Education, 20, pp. 77-97.
Kroflič, R. (1994). Evalvacija visokošolskega kurikuluma kot sestavni del planiranja. Sodobna pedagogika, 45, issue 4-5, pp. 236-246.
Kroflič, R. (2002). Skupne vrednote in paradigmatske uganke evropske pedagogike. Ljubljana: Zavod Republike Slovenije za šolstvo.
Kroflič, R. (2008a). Modeli vzgoje v globalni družbi. Ljubljana: Drugi mednarodni kongres dijaških domov (17. and 18. april 2008).
Kroflič, R. (2008b). Novi pristopi k spodbujanju otrokovega prosocialnega in moralnega razvoja v predšolskem obdobju. Celje: Posvet vzgojiteljic celjske regije: Socialne interakcije v vrtcu (27. september 2008).
Kump, S. (1994). Modeli zagotavljanja kakovosti. In: Kakovost visokega šolstva. Ljubljana: Univerza v Ljubljani, Center za razvoj univerze.
Laval, C. (2005). Šola ni podjetje. Neoliberalni napad na javno šolstvo. Ljubljana: Krtina.
Lassnigg, L. (2003). Indicators for Quality in VET. European experience. Vienna: Institute for Advanced Studies. Available at: http://www.siswo.uva.nl/tlm/root_files/Loren-zLassnigg.pdf (Accessed on 24.6.2014).
Levinas, E. (2006). Entre Nous. New York: Continuum.
Mitchell, R. K., Agle, R. B. and Wood, D. J. (1997). Toward a Theory of Stakeholder Identification and Salience: Defining the Principle of Who and What Really Counts. Academy of Management Review, 22, issue 4, pp. 853-886.
Moller, J. (2005). School leadership in an age of accountability. Oslo: Univerza v Oslu.
Možina, E. (2011). Mejniki v razvoju področja pismenosti odraslih v Sloveniji. In: P. Javrh (ed.). Obrazi pismenosti. Ljubljana: Andragoški center Slovenije.
Možina, T. (2003). Kakovost v izobraževanju. Od tradicionalnih do sodobnih modelov ugotavljanja in razvijanja kakovosti v izobraževanju odraslih. Ljubljana: Andragoški center Slovenije.
Možina, T. (2010.) Kakovost kot (z)možnost. Ljubljana: Andragoški center Slovenije. Available at: http://kakovost.acs.si/doc/N-463-1.pdf (Accessed on 22. 4. 2014).
Možina, T., Klemenčič, S., Vilič Klenovšek, T., Zoric Frantar, M., Jurič Rajh, A. and Orešnik Cunja, J. (2013). Kazalniki kakovosti izobraževanja odraslih. Ljubljana: Andragoški center Slovenije. Available at: http://kakovost.acs.si/razvoj_podrocja/publiciranje/ index.php?id=1028 (Accessed on 22. 4. 2014).
Nahapiet, J. and Ghoshal, S. (2000). Social Capital, Intellectual Capital and the Organizational Advantage. In: E. Lesser (ed.). Knowledge and Social Capital, Foundations and Application. Oxford: Butterworth, Heinemann.
Noddings, N. (2002). Starting at Home. Berkeley: University of California Press.
Pirsig, R. M. (2005). Zen in umetnost vzdrževanja motornega kolesa. Raziskovanje vrednot. Ljubljana: Iskanja.
Rawls, J. (1999). A Theory of social justice. Cambridge, Massachusetts: The Belknap press of Harvard university press.
Sallis, E. (1993). Total quality management in education. Philadelphia, London: Kogan Page.
Stake, R. E. (2000). Program Evaluation, Particulary Responsive Evluation. In: G. F. Meadaus, M. Scriven and D. L. Stufflebeam (eds.). Evaluation Models: Viewpoints on Educational and Human Services Evaluation. Boston: Kluwer academic publishers, pp. 343-362.