DOI: 10.17573/ipar.2016.2-3.04
1.02 Review article
Valuational Overhaul of Regulation and Assessing and Maintaining Education Quality in Slovenia
Stanka Setnikar Cankar
University of Ljubljana, Faculty of Administration, Slovenia stanka.setnikar-cankar@fu.uni-lj.si
Franc Cankar
cankar.aco@gmail.com
Tomi Deutsch
National Education Institute of the Republic of Slovenia, Slovenia tomi.deutsch@zrss.si
ABSTRACT
The aim of this article is to present practices for regulating elementary and secondary education from an international perspective. It presents processes needed to introduce a system of external evaluation and takes account of the danger that if not carefully thought out, external evaluation can adhere to procedure to too great an extent, to the detriment of actual improvements in education. An external evaluation model that could be implemented in the Slovenian education system is then proposed. The proposed model stresses schools' accountability to the public, the dissemination of effective practices, and the delivery of relevant information to those in charge of national school policy. The model upgrades existing mechanisms for assessing and maintaining quality and links them with a legislation proposal to form a coherent whole.
Keywords: regulation, quality, external evaluation, elementary and secondary education
JEL: 128
1 Introduction
Regulation processes in the fields of public administration and public policy research are an important and current issue for professionals in these fields, both in Slovenia and throughout the world (Lodge, 2014). As an activity, these processes are seamlessly linked to the effective functioning of government, They involve classification, inspection, licensing, prohibitions and sanctions
Setnikar Cankar, S., Cankar. F., & Deutsch, T. (2016). Vaiuationai Overhaul of 75 Regulation and Assessing and Maintaining Education Quality in Slovenia.
International Public Administration Review, 74(2-3), 75-94.
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
(Carpenter & Moos, 2014). It comes as no surprise that the field of regulation has been the subject of numerous, often contradictory, discussions. While some actors call for an end to regulation, others are demanding even more regulatory processes. Regulatory processes have also been a topic of research, and as such have been subjected to a variety of methodological approaches and have led to findings in form of both empirical data and theoretical concepts. The financial crisis presents a sound point of departure for rethinking the practices and research of regulatory processes, as a number of authors (Carpenter & Moos 2014; Pitman, 2014) feel that both practice and research around regulatory processes are experiencing a crisis of their own,
All of the above is true of Slovenia, where current trends in state development highlight the importance of the quality governance of societal subsystems. This cannot be achieved without coordination between developmental policies linked to the introduction of reforms in specific sectors. The focal points of these reforms are sustainable development and strategic importance and accountability of policy. Points of departure and principles for regulation are the result of a need to (once again) reform the functioning of the state on the basis of two key common denominators:
•	The transfer of best practices facilitating the introduction of measures that would once again legitimate the necessity of power structures in society;
•	Close cooperation of authorities with all interested groups from the non-state environment, which would bring about a greater transparency and a greater degree of democracy and with it a new, important source of legitimacy for the functioning of authorities (Kustec Lipicer, 2013).
De Francesco (2012) recommends that regulatory reforms take place in three phases. The first phase represents 'deregulation'. It is followed by a phase called 'regulatory reform'. The process concludes with the third phase, known as the 'principle of regulatory governance'.
The field to which the above observations clearly apply is education. Substantial systemic reforms introduced in the 1990s placed schooling in Slovenia on a firm, modern footing. But with the development of society over the past decade, the need for certain systemic upgrades and enhancements in this field has appeared. Such upgrade is the augmentation and development of mechanisms through which the quality of the functioning of the school system is assessed and maintained.1 In 2014, the European Council acknowledged the importance of this. It encouraged the European Commission to support member states in their efforts to develop policies and systems that can help assess and maintain education quality (Eurydice Report, 2015). Quality education is of decisive importance for employment
1 Task of the Working Group for the Operationalization of Quality in the Field of Education, 2014.
76 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
prospects, for development of social cohesion and for general economic and social advancement in individual countries and in Europe as a whole,
This article presents the practice of regulation in the field of elementary and secondary education from an international perspective. It analyses processes needed to introduce a system of external evaluation and takes account of the danger that if not carefully thought out, external evaluation can adhere to procedure to too great an extent, to the detriment of actual improvements in education. The analysis presented here is based on the descriptive method, which was used to analyze similar solutions in school systems in other countries, The comparative method is then applied to compare the collected data and to search for commonalities and differences between countries, An external evaluation model that could be implemented in the Slovenian education system is then proposed. The proposed model is based on the experience of other countries and stresses schools' accountability to the public, the dissemination of effective practices, and the delivery of relevant information to those in charge of national school policy (Faubert, 2009; Nevo, 2001; Nutley, Morton, Jung, & Boaz, 2010), The model upgrades existing mechanisms for assessing and maintaining quality and links them with a legislation proposal to form a coherent whole,
2 Theoretical Points of Departure
The findings of the OECD (2013) and the European Commission (2015) show that mechanisms for assessing and maintaining quality in education are becoming an increasingly important catalyst for school development. The systems being introduced in European countries increasingly exceed narrow considerations of schools and focus on creating and expanding a culture that places quality teaching and learning at the forefront, as well as the inclusion of and dissemination of information to various stakeholders, Emphasis is on improving the school as a whole and not just improving particular aspects of school effectiveness (such as scholastic achievements),
The evaluation of a school is linked to a broad range of school activities that encompasses both teaching and learning as well as various aspects of school management. In practice, two basic methods of school evaluation have been implemented in most European countries (Eurydice Report, 2015): internal and external evaluation. Systems of self-evaluation and external evaluation underscore the development of educational institutions that can set areas for improvements in their own practices and assume responsibility for their own development. Responsibility for assessing and maintaining quality in education is also divided out among other important partners. One of these is the government. Governments are accountable for the condition of the school system, which is why they always seek out ways to harmonize internal and external quality control in the framework of their competencies. The search for the ideal equilibrium between both processes presents a considerable
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 77
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
challenge for professionals in the field. It therefore comes as no surprise that these questions have been addressed by a number of professionals,
2.1 Definition of Terms
Evaluation can be defined as the systematic collection of data pertaining to some phenomenon with the purpose of producing a valuation which can then serve to improve the said phenomenon. Scriven (1991) states that evaluation is a process through which the given and added value of that which is being evaluated is assessed. A given value is something which the evaluee has a priori, and an added value is that value which the evaluee develops in a given context. Patton (2002) stresses the importance of the use of evaluation results and defines evaluation as the systematic collection of information on activities, characteristics and results of a program, through which an assessment of the program is produced with the aim of enhancing its efficiency and making decisions regarding its future. Wiggins (1991) stresses the importance of the criteria used to judge the value of the evaluee. Preskill and Torres (1999) stress that an evaluation must serve as an opportunity for self-learning at the institution conducting the evaluation, and that it must be integrated into working processes at the institution. Although different authors define evaluation in different ways, their definitions do have four things in common:
•	Evaluation is a systematic process that is planned in advance and carried out with a specific purpose;
•	Evaluation demands a systematic collection of data about critical points at an institution or in a program;
•	Evaluation is conducted for the purpose of deepening understanding and making decisions about improving programs, processes and products on the basis of assessments of the given and added value of the evaluee.
Discussions on these questions have brought issues associated with assessing and maintaining quality at educational institutions to the fore. The term quality ties in with the expectations of a number of partners, including the government, school managements, school councils, pupils and teachers. Because the expectations of stakeholders differ and are diffuse, they must be delineated and specified based on where these stakeholders are coming from. Vanhoof and Petegem (2006, 2007) divide expectations into internal expectations at the school itself and external expectations. Some expectations are rooted in legislative documents, while others are not. The government supports and demands conformity with legislative provisions; external stakeholders, who usually have a broad range of expectations, also communicate demands for cooperation in decisions linked to the implementation of quality-enhancement measures in education. This is of course also true of internal expectations at a school. The expectations
78 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
of all stakeholders are legitimate and must be brought into harmony, but differences in views are also present, for instance around the question of who is to assume responsibility for assessing and maintaining quality at educational institutions (Scriven, 1991; Nevo, 2001).
2.2 The Potential Complementarity of Internal and External Evaluation Processes
Authors (Vanhoof & Petegem, 2006, 2007; Altrichter & Kemethofer, 2015) define the field of quality assessment and maintenance from two perspectives:
1.	Perspective linked to ensuring public accountability;
2.	Perspective oriented towards development and improving quality at schools,
The focus of the first perspective with its emphasis on public accountability are school reviews. This involves checking objectives and points of departure as outlined by the government (society) and defined in the applicable legislation. This aspect of quality assessment and maintenance is more direct and focuses on assessing the effective functioning of a school. Initiative for this approach comes from outside the institution,
The second perspective is more developmental in nature and is based on the concept of quality development. It focuses on processes outlined and developed by schools themselves. Planned improvements to internal quality demand dialogue between school management, teachers, pupils, parents and the local community. With this perspective, initiative comes from the school itself.
In the literature (Vanhoof & Petegem, 2006, 2007; Ehren, Altrichter, McNamara, & O'Hara, 2013) the aspect of 'providing for public accountability' has usually been linked to the process of external evaluation, while the aspect of 'improving school quality' is traditionally the providence of internal or self-evaluation. Over time, this traditional division of tasks and perspectives became a constraint, thus giving rise to demands for reconsiderations of the relationship between the two forms of evaluation. The question of linking up the two perspectives is a complex one. Nevo (2001) claims that from the perspective of public accountability, self-evaluation is subordinate to external evaluation. Viewed from the developmental perspective, it is the other way around. Nonetheless, authors (Van Hoyweghen, 2002; Christie, Ross, & Klein, 2004; Shewbridge, Hulshof, Nusche, & Stoll, 2011; Ehren et al., 2013, Altrichter & Kemethofer, 2015) have listed and broken down various features common to both processes. These authors conclude that the processes of internal and external evaluation are complementary and round each other out. When handled in a sensible manner, the introduction of internal and external evaluation augments the added value of both forms of evaluation; if one of the two is absent, the added value of the other is diminished.
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 79
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
2.3 External Evaluation and Its Role in Improving the Quality of Work at Schools
Although studies of the impacts of external evaluation on the quality of work at schools have not been particularly plentiful, this area has received a fair share of attention in recent years. In their in-depth research about the methods of work of the School Inspection Service in the UK, Matthews and Sammons (2004) conclude that without changes in the nature of the work of this body, it would not be realistic to assume external evaluation has a direct impact on the quality of work at schools. However, two recent studies from the Netherlands found that schools do use reports from external evaluations and that this does affect school development (Bekkers, Catalini, Martinelli, & Simcoe, 2012; Janssens, 2012). Another study from Australia (Nees, 2007) reported that all six of the schools included in the study took into account recommendations produced through external evaluation and did in fact improve working processes. This echoes the findings of a report from Sweden (Ekonomistyrningsverket, 2006) on an external evaluation process conducted between 2003 and 2006. The report states that most schools constructively applied the recommendations of external evaluations and improved the quality of their work. It is interesting to note that improvements in the quality of work at schools which paid heed to external recommendations were more rapid than is usually the case. Teachers' representatives from the French community in Belgium report similar experiences (Blondin & Giot, 2011). The application of recommendations and feedback from external evaluators is an important aspect of the culture of self-evaluation at schools.
One larger project that was financed by the European Community (Ehren et al., 2013) analyzed official documentation at institutions that perform external evaluations in six countries: the Czech Republic, Austria, Ireland, the Netherlands, Sweden and England. The authors collected data on a sample of 2,200 respondents, using in-depth interviews to identify commonalities that represent forces in quality improvement at schools. They reached the following conclusions:
•	External evaluation strengthens expectations regarding school quality (through criteria and indicators that tell what a good school is);
•	Stakeholders are also briefed on the results of external evaluation (school management, parents and pupils); their sensitivity to the results is enhanced, which leads to demands for improved quality at a school;
•	External evaluation promotes and encourages improvements in the self-evaluation process at schools.
The findings show that external evaluation methods at schools set in place clear expectations regarding areas for assessment and quality indicators, while the inclusion of stakeholders encourages their sensitivity with regard to the process of external monitoring and typically has an impact on schools. At the same time, all this typically ties in with improvements in the self-
80 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
evaluation process at schools. Expectations regarding external evaluation are directly linked to both a wish and willingness for improvement. This shows that the quality improvement process at schools includes systemic self-evaluation as a vital developmental strategy (Ehren & Hendriks, 2010, Ehren et al., 2013). At the same time, Jung, Namgoong, and Kim (2008) concluded, on the basis of reports from teachers, that the external evaluation process presents an opportunity to refresh the climate at a school and facilitates a range of discussions and consultations about a school's strong and weak points. In a recent study, Altrichter and Kemethofer (2015) researched the effect of 'accountability pressure' as an element in understanding how external evaluation works. The study included 2,300 principals from seven European countries. The findings show that those principals who feel greater accountability pressure when communicating with evaluators are more prone to quality control. They are more sensitive to initiatives from stakeholders linked to evaluation results and more consistent in their selection and application of activities for improving conditions at their schools. This pressure also produces a range of other effects. Principals in the countries included in the study view the external evaluation system as one of their responsibilities vis-a-vis demands for improving practices at their schools,
But there is the danger that if not carefully thought out, an external evaluation can adhere to the 'by the book' procedure to too great an extent, to the detriment of actual improvements at a school. The haphazard implementation of external evaluation can also encourage an excessive rise in administrative demands, which already represent a considerable burden on schools (Faubert, 2009). Teachers do not monitor processes that overemphasize a top-down approach as opposed to innovation; specifically, they view such processes as creating a 'culture of obedience' (Leithwood, Aitken, & Jantzi, 2001). The latter finding should sound the warning that external evaluation processes for the most part should not be an (additional) burden on teachers and other professionals at schools through their demands for reporting, recordkeeping and similar activities, and that the 'reporting burden' (filling out evaluation questionnaires and forms) should be shifted onto the external evaluators to the greatest possible extent. An excessive and unproductive burden can of course also result from reckless, overly nuanced self-evaluation,
2.4 Practices to Date
Models and experiences related to assessing and maintaining quality at educational institutions differ from country to country. In certain countries evaluation processes are more prevalent, and impact quality development at schools to a greater extent. Different countries naturally have different cultures and different school systems. A report published last year by the European Commission (Eurydice Report, 2015) shows that both internal and external evaluations take place in the school systems of 26 European countries. This mechanism is still in the test phase in Italy and Hungary. In most
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 81
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
European countries assessment of the quality of work at schools takes place at the central level. Evaluation work is conducted either by school inspectors or special evaluation groups. In Denmark, Latvia and Iceland, responsibility for carrying out external evaluations is shared among central and local school authorities. In most European countries, external evaluation is focused on a range of school activities with both evaluative aspects and management aspects. Some countries systematically evaluate only specific aspects of school work, but occasionally obtain additional data from studies. Despite differences in methods and types of approaches to assessing and maintaining quality at schools in different European countries, the implementation of external evaluation is everywhere based on a recognizable structure that includes analysis, school visits and reporting,
Despite the positive contribution of external evaluation to the holistic assessment and maintenance of quality, this form of evaluation has long been viewed with a fair degree of skepticism (Nevo, 2001). For this reason, and because of decentralization processes and the increasing autonomy of schools, evaluation processes in a number of countries have developed in the direction of greater participation and self-development (Robinson & Cousins, 2004; McNamara & O'Hara, 2005). Besides external evaluation, over the past decade processes and methodologies that encourage schools themselves to evaluate education quality and plan adequate measures for its enhancement have also taken shape in European countries. It is a fact that internal and external evaluation processes grounded in the concept of school system management and supported by evidence-based governance are becoming a strategy for ascertaining and improving the level of quality of education systems in most European countries (Boaz, Grayson, Levitt, & Solesbury, 2008; Nutley et al., 2010; Eurydice Report, 2015).
Self-evaluation is defined in legislation governing the school system in Slovenia (Articles 48 and 49 of the Financing and Organization of Education Act). Apart from examples of best practices at certain schools (KEKS, 2015), no information about how self-evaluation is to take place or about whether self-evaluation is fulfilling its intended purpose is provided. According to reports from a representative of school inspectors (records from a session of the Working group for operationalizing quality in the field of higher education, 20. 2. 2015) schools do prepare reports on self-evaluation. However, the quality of these reports is questionable. The main issue is whether the development and performance of self-evaluation at Slovenian schools is moving in the right direction. This is understandable to an extent, as self-regulation processes, that is, a culture of and protocol for self-evaluation, do not have a long tradition in Slovenia. Planned, systematic training for school managements is therefore needed, as is assistance for schools in developing self-evaluation processes. The introduction of a suitable external evaluation model would also be a significant step. Studies and reports (Conley-Tyler, 2005; Simola, Rinne, Varjo, Pitkanen, & Kauko, 2009; OECD, 2013) show
82 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
that even countries with a long tradition and strong culture of evaluation (for example, Scotland and Germany) have introduced and intensified self-evaluation processes in response to external evaluation processes. Some have even adopted the areas, criteria and indicators used in external evaluations, This should be taken to mean that external evaluation would also benefit schools without substantial experience with internal evaluation,
Experiences with external evaluation in Slovenia only date back 25 or 30 years, when the school authorities at the time began to monitor the work of schools in a fairly systematic manner. Of course, this monitoring was adapted to demands of the time concerning the development of schools, Unfortunately, these experiences and the solutions they produced were poorly documented in the professional literature. One must also ask whether these measures, which in many cases were bureaucratic and often too strictly top-down in their orientation, encouraged self-regulation processes and school development. Today, a school inspection agency has been written into law and is active. Its activities are a part of processes for assessing and maintaining quality at schools. In accordance with the School Inspection Act, the agency pursues specific objectives. This system sets Slovenia apart from other European countries. Besides Slovenia, the only other countries where external evaluation focuses exclusively on checking whether school operations conform to legislation are Turkey and Estonia (Eurydice Report, 2015, p. 8 and p. 51). In most countries external evaluation processes focus on multiple areas of school operations. Some of the most common areas are educational and managerial aspects of school governance (Eurydice Report, 2015, p. 8).
On the other hand, over the years the question of the public and its access to information has received greater emphasis. These questions basically express the public's concern with how good is the school their children attend. The parents of elementary- and secondary-school pupils are interested in where their child ranks in terms of quality standards; How do they fare compared to other children? If empirical evidence were available on all areas that can (still) be improved at a school or even about things a school has yet to do, parents could even demand these things through their representatives. The question of how good schools are is even more important for the state (Braithwaite & Coglianese, 2007). It is the state which must ensure that all children have access to quality education (Merljak, 2015). Without data and analyses, the management of education policy is a daunting task (Campbell & Levin, 2009). A recent report from the Court of Audit of the Republic of Slovenia (2016) that assessed the orderliness of the job duties of elementary-school teachers largely confirmed this. The findings show that the Ministry does not have adequate insight into how teachers fulfill their job duties.
The ministry has since outlined changes for the future (KEKS, 2015). A national framework for quality assessment and maintenance is supposed to uncover problems and also offer solutions. By using it, schools are expected to be able
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 83
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
to determine what they're good at and where they could stand to improve. The reports, which are to be prepared by schools every three years, are supposed to serve as a basis for any necessary systemic reforms. However, this national framework does not foresee an adequate system for the external review of schools; instead, it aims to encourage a culture of quality through internal evaluation conducted by school members themselves. But at a number of school, these same persons have already been overseeing quality for some time. And at a number of schools, the knowledge or desire to make use of the results of legally mandated self-evaluation to improve pedagogic processes is lacking. The fear that schools where reports from self-evaluations are treated as just another piece of paper will maintain this attitude is therefore justified, In environments where information tends to get swept under the rug, it is difficult to expect that steps will be taken towards implementing a culture of quality simply because the Ministry would like to nudge schools in this direction (Merljak, 2015).
Parents, teachers, the local community and the state would therefore need some other mechanism for setting minimum standards on the systemic level that would warn schools when they are sliding below a certain point and offer assistance in overcoming their problems, A sensible system of external evaluation is needed,
3 An External Evaluation Model
Below an external evaluation model is proposed. It can be put into place without making substantial changes to the Slovenian education system. The model places emphasis on quality and does not interfere with the organization of education or foresee additional allocations of funds. It does however demand the reorganization of existing support processes and produce satisfactory results both in terms of assessing quality (data collection) and maintaining quality (carrying out measures on the national and school levels).
The presented model, which is suitable for the implementation of external evaluation at elementary and secondary schools, can be introduced in practice without considerable difficulties, It can be conducted over a longer period of time and encompasses all schools in an adequately frequent cycle, that is, every five years, This is the most frequent time interval for external evaluations in European countries (European Commission, 2015, p. 23). The proposed external evaluation model presents minimum conditions for implementing external evaluation; over time, it can be supplemented with new areas of evaluation and new stakeholders. From a content standpoint, a key advantage of the proposed model is that it does not focus on just one aspect of quality, but rather stresses both the achievements of education participants and the pedagogic process.
The proposed model stresses schools' accountability to the public and provides for the transfer of effective practices and the provision of relevant information
84 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
to those in charge of national school policy. On the institutional level, this will require an adequate degree of organization, while the group conducting the evaluation will be expected to perform its activities in accordance with the fundamental values of integrity, objectivity and impartiality. Emphasis would be on considerations of school development, teaching quality, the attainment of high-quality knowledge, school management and continuous development. External evaluation under the proposed model would have the following goals:
1.	To evaluate the work of educational institutions and to encourage the improvement of quality;
2.	To ensure high-quality education at all educational institutions or, ipso facto, to all those enrolled in the education process;
3.	To provide information that enables overviews of conditions in the field of education and to monitor measures for improving these conditions,
The implementation of the model would be made possible with a clear definition of external evaluation in the Organization and Financing of Education Act, the adoption of a suitable protocol for conducting external evaluation and the establishment of an evaluation group or groups at an existing institution in the field of education. Another possible solution - though it does not seem to be an optimal one - would be to establish the evaluation group within the Inspectorate of the Republic of Slovenia for Education and Sport. This solution is interesting because over time it could have an impact on refurbishing the content of the inspections currently conducted at schools and kindergartens and bring about a new, different role for the inspection agency in assessing and maintaining quality in education processes.
Presentation of an External Evaluation Model
The external evaluation model would be carried out according to a predefined framework that outlines fundamental areas which serve as a basis for assessing the quality of work and preparing measures for improving quality in those areas where it is not satisfactory or fails to meet predefined standards, The framework for external evaluation would at the same time provide a framework for conducting self-evaluation at schools. The framework for quality assessment consists of the following areas:
1.	Working conditions
2.	Processes
2.1.	Pedagogic management and organization
2.2.	Conducting classes
2.3.	Inclusion of education participants and their parents
3.	Achievements of education participants
4.	Internal evaluation and development at a school
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 85
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
Working conditions are one of four external evaluation areas that are not completely dependent on how the evaluator or school does their job. Working conditions are primarily material resources, human resources, climate and culture, legislation, curricula and training possibilities. Any failure to achieve quality in those segments of this area which are not dependent on the school itself but on outside factors is not included in area 4 of the external evaluation.
The pedagogic process or processes form the area to which the greatest weight is applied in the external evaluation. The achievements of other education participants also depend on these processes, which in turn depend of working conditions. Due to its complexity, the pedagogic process is broken down into multiple sub-areas. The three main sub-areas are pedagogic management and the organization of work, teaching classes and including education participants and their parents in processes at the school (the democratization of these processes, user satisfaction).
In the framework of the external evaluation, the achievements of pupils would be monitored using external or comprehensive testing (at the end of elementary or secondary school). Testing would be conducted as an independent activity and would also serve other purposes besides inputting data in evaluation processes.
Internal evaluation or the self-evaluation and development of schools (area 4) has a special place in the framework. These are two activities that would continually take place at schools. Their aim is to ensure the quality performance of the processes in areas 1, 2 and 3. From the standpoint of external evaluation, area 4 involves checking how self-evaluation is being carried out and applying the findings towards the ongoing development of the school. Insomuch as a school fails to achieve satisfactory or target quality levels, activities for improving quality at the school would be conducted in the framework of this area. These activities would involve the evaluation group, the school and others. Data and findings from the external evaluation would also be used in self-evaluation processes, along with data obtained by the schools themselves in line with a prescribed methodology.
Although the proposed external evaluation model has been developed independently of the external evaluation models used in other countries, it does share certain commonalities with them. For example, the four main areas are identical to those in the external evaluation model used in Germany (Landesinstitut fur Schulentwicklung, 2011). Together with the tasks conducted by inspectors (assessing whether operations conform to regulations), the proposed external-evaluation model covers all the typical areas of evaluation: quality of teaching and learning, scholastic achievements, multiple aspects of school management and the conformity of school operations to regulations.
86 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
The proposed external evaluation model and its mandatory self-evaluation portion would be integrated in the main school legislation and defined in detail in a protocol for conducting external evaluation and self-evaluation at educational institutions,
4 Conclusions
In the process for assessing and maintaining quality in the field of education, both the self-evaluation process and the external evaluation process have important roles. The two forms of evaluation are complementary and round each other out. Whereas internal or self-evaluation processes at Slovenian schools have at least become a topic of discussion (KEKS, 2015), support for external evaluation processes is scarce. This course of action -or rather inaction - undermines the important impacts of a holistic learning environment capable of producing better results and creating greater added value than one made up of otherwise good components which, however, do not function in unison.
The mechanisms through which the quality of work in the school system is currently assessed and maintained must be developed further. The expectation that a school will regulate itself without outside, modern professional support that can deliver relevant information to the school, school authorities and other stakeholders, is unrealistic. The model proposed here upgrades existing mechanisms for assessing and maintaining quality and links them up with a legislative proposal to form a coherent whole,
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 87
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
Stanka Setnikar Cankar, PhD in Economic Sciences, is a professor at the Faculty of Administration of the University of Ljubljana. Her work encompasses a number of fields: public sector economics, assessments of efficiency and effectiveness in the public sector, entrepreneurship functions in the public sector, the centralization and decentralization of public sector services, public procurement, the transfer of best practices through cross-border cooperation, innovation and creativity in the public sector and education. She is the author of numerous articles and books on economics in the public sector and editor of the International Magazine for Public Administration.
Franc Cankar was Head of the Center for Quality and Research at the National Education Institute of the Republic of Slovenia in Ljubljana. His research and development work is connected with the theoretical basis of school curriculum development and implementation. He has published works on various topics in the field of education.
Tomi Deutsch is a senior adviser at the National Education Institute of the Republic of Slovenia. His research focuses primarily on the methodology and implementation of monitoring and evaluation of individual segments of education and on the performance of basic research tasks. His development and research work covers all areas of pre-university education.
88 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
References
Altrichter, H., & Kemethofer, D. (2015). Does accountability pressure trough school inspections promote school improvement? School Effectiveness and Improvement: An International Journal of Research, Policy and Practice, 26(1), 32-56. DOI: 10.1080/09243453.2014.927369
Bekkers, R., Catalini, C., Martinelli, A., & Simcoe, T. (2012). Intellectual Property Disclosure in Standards Development. NBER conference on Standards, Patents & Innovation, Tucson (AZ).
Blondin, C., & Giot, B. (2011). Cadres d'évaluation en vue d'améliorer les resultants scolaires - Etude thématique de l'OCDE: Rapport de base pour la Communauté française de Belgique. Brussels: Ministère de la Fédération Wallonie-Bruxelles.
Boaz, A., Grayson, L., Levitt, R., & Solesbury, W. (2008). Does evidence-based policy work? Learning from the UK experience. Evidence and Policy: A Journal of Research, Debate and Practice, 4(2), 233-253. DOI: 10.1 186/1478-4505-12-34
Braithwaite, J., & Coglianese, C. (2007). Can Regulation and Governance make a difference. Regulation & Governance, 7(1), 1-7. DOI: 10.1111/j.1748-5991.2007.00006.x
Campbell, C., & Levin, B. (2009). Using data to support educational improvement. Educational Assessment, Evaluation and Accountability, 27(1), 47-65. D0I:10.1007/s1 1092-008-9063-x
Carpenter, D. P., & Moos, D. A. (2014). Introduction. In D. P. Carpenter & D. A. Moos (Eds.), Preventing Regulatory Capacity. Cambridge University Press.
Christie, C. A., Ross, R. M., & Klein, B. M. (2004). Moving toward collaboration by creating a participatory internal-external evaluation team: A case study. Studies in Educational Evaluation, 30(2), 125-134. DOI: 10.1016/j.stueduc.2004.06.002
Conley-Tyler, M. (2005). A Fundamental Choice: Internal or external evaluation? Evaluation Journal of Australasia, 4(1/2), 3-11.
Court of Audit RS. (2016). Delovna uspešnost učiteljev v osnovnih šolah (revizijsko poročilo). Računsko sodišče Republike Slovenije.
De Francesco, F. (2012). Diffusion of Regulatory Impact Analysis Among OECD and EU Member States. Comparative Political Studies, 45(10), 26-38. DOI: 0.1177/001041401 1434297
Ehren, M., & Hendriks, M. (2010). Combinations of Internal Evaluations and External Accountability in Europe: Ideal Types and Existing Models.
Ehren, M. C. M., Altrichter, H., McNamara, G., & O'Hara, J. (2013). Impact of school inspections on improvement of schools - describing assumptions on causal mechanisms in six European countries. Educational Assessment, Evaluation and Accountability, 25(1), 3-43. DOI: 10.1007/s1 1092-012-9156-4
Ekonomistyrningsverket. (2006). Skolverkets utbildningsinspektion -ger den nâgra effekter? Effektutvardering genomford pâ uppdrag av Skolverket, Ekonomistyrningsverket. Swedish National Financial Management Authority, Stockholm.
Eurydice Report. (2015). Assuring Quality in Education: Policies and Approaches to School Evaluation in Europe. Luxembourg: European Commission/ EACEA/,Publications Office of the European Union.
Faubert, V. (2009). School Evaluation: Current Practices in OECD Countries and a Literature Review (OECD Education Working Papers 42). DOI: 10.1787/218816547156
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 89
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
Janssens, F. J. G. (2011). The evaluation of the impact of the publication of school performance indicators in the Netherlands. International Journal of Educational Policies, 5(2), 55-73.
Jung, T., Namgoong, J., & Kim, J. (2008). Synthetic Report of School Assessment. Seoul: Korean Educational Development Institute.
KEKS. (2015). Taštanoska, T. (Ed.), Krepitev in evalvacija kakovosti v sistemu vzgoje in izobraževanja. Zbornik nacionalne konference, Brdo.
Kustec Lipicer, S. (2013). Regulatorne politike [študijsko gradivo]. Ljubljana: Fakulteta za upravo.
Landesinstitut für Schulentwicklung. (2011). External evaluation for general education schools in Baden-Wuerttemberg. Stuttgart: Landesinstitut für Schulentwicklung.
Leithwood, K., Aitken, R., & Jantzi, D. (2001). Making Schools Smarter: A System for Monitoring School and District Progress. Corwin.
Lodge, M. (2014). Regulation in Crissis: reputation, capacity and limitations. In K. Soonhee, A. Shena, & W. H. Lambright (Eds.), Public Administration in the Context of Global Governance. Cheltenham, UK, Northampton, MA, USA: Edward Elgar.
Matthews, P., & Sammons, P. (2004). Improvement through Inspection: An Evaluation of the Impact of Ofsted's Work. Crowin.
McNamara, G., & O'Hara, J. (2005). Internal review and selfevaluation - The chosen route to school improvement in Ireland? Studies in Educational Evaluation, 31(4), 267-282. DOI: 10.1016/j.stueduc.2005.1 1.003
Meden, A., Petelinšek, K., Rupnik, D., & Tavčar, A. (2012). Pogled staršev na kakovost v šoli. Zveza aktivov svetov staršev Slovenije.
Merljak, S. (2015, December 12). Želja po kakovosti. Še bolj kot za starše je to, kako dobre so šole, pomembno za državo. Delo.
Nees, P. (2007). Schools and their ERO Recommendations: A Study of Six Wellington Area Schools (Sabbatical Report).
Nevo, D. (2001). School evaluation: internal or external? Studies in Educational Evaluation, 27(2), 95-106. DOI: 10.1016/S0191-491X(01)00016-5
Nutley, S., Morton, S., Jung, T., & Boaz, A. (2010). Evidence and policy in six European countries: diverse approaches and common challenges. Evidence and Policy: A Journal of Research, Debate and Practice, 6(2), 131-144. DOI: 10.1332/174426410X502275
OECD. (2013). School evaluation: From compliancy to quality. In Synergies for Better Learning: An International Perspective on Evaluation and Assessment. OECD Publishing.
Patton, M. Q. (2002). Qualitative Research and Evaluation Methods. Thousand Oakes, CA: Sage Publications.
Pitman, T. (2014). Reinterpreting higher education quality in response to policies of mass education: the Australian experience. Quality in Higher Education, 20(3), 348-363. DOI: 10.1080/13538322.2014.957944
Preskill, H., & Torres, R. T. (1999). Evaluative Inquiry for Learning in Organizations. Thousand Oakes, CA: Sage Publications.
Robinson, T. T., & Cousins, J. B. (2004). Internal participatory evaluation as organisational learning system: A longitudinal case study. Studies in Educational Evaluation, 30(1), 1-22. DOI: 10.1016/S0191-491X(04)90001-6
90 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
Scriven, M. (1991). Evaluation Thesaurus (4th ed.). Thousand Oakes, CA: Sage Publications.
Shewbridge, C., Hulshof, M., Nusche, D., & Stoll, L. (2011). Scholl Evaluation in the Flemish Community of Belgium. In OECD Reviews of Evaluation and Assessment in Education (pp. 15-24). DOI: 10.1787/97892641 16726-4-en
Simola, H., Rinne, R., Varjo, J., Pitkanen, H., & Kauko, J. (2009). Quality Assurance and Evaluation (QAE) in Finnish compulsory schooling: a national model or just unintended effects of radical decentralisation? Journal of Education Policy, 24(2), 23-32. DOI: 10.1080/02680930902733139
Vanhoof, J., & Petegem, P. V. (2006). How too much Internal and External Evaluation? Conceptual Reflections from a Flemish Perspective. ICSEI.
Vanhoof, J., & Petegem, P. V. (2007). Matching Internal and External Evaluation in an Era of Accountability and School Development: Lessons from a Flemish perspective. Studies in Educational Evaluation, 33(2), 101-119. DOI: 10.1016/i.stueduc.2007.04.001
Van Hoyweghen, D. (2002). Hoe complementair zijn interne en externe evaluatie? How complementary are internal and external evaluation. Kwaliteitszorg in het onderwijs, 2, 39-70.
Wiggins G. (1991). Standards, Not Standardization: Evoking Quality Student Work. Educational Leadership, 48(5), 18-25.
Zofvi - Zakon o organizaciji in financiranju vzgoje in izobraževanja (Uradni list RS, št. 16/07 - uradno prečiščeno besedilo, 36/08, 58/09, 64/09 - popr., 65/09 -popr., 20/11, 40/12- ZUJF in 57/12 - ZPCP-2D).
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 91
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch povzetek
1.02 Pregledni znanstveni članek
Vrednostna prenova regulacije in ugotavljanje ter zagotavljanje kakovosti izobraževanja v Sloveniji
Procesi reguliranja na področju javne administracije in raziskovanja javnih politik so eno od pomembnih in aktualnih strokovnih vprašanjtako v svetu kot pri nas (Lodge, 2014). To je aktivnost, ki je neločljivo povezana z učinkovitim delovanjem vlade. Ni presenetljivo, da je področje reguliranja predmet številnih, pogostokrat protislovnih razprav. Raziskovanje procesov reguliranja, podprto s številnimi metodološkimi prijemi, je prineslo ugotovitve v obliki empiričnih podatkov in teoretičnih konceptov. Finančna kriza je dobro izhodišče za ponovni premislek o praksi in raziskovanju procesov reguliranja, saj številni avtorji (Carpenter & Moos, 2014; Lodge, 2014; Pitman, 2014), menijo, da sta tako praksa kot raziskovanje procesov reguliranja v krizi. To velja tudi za Slovenijo, saj sodobni trendi razvoja države poudarjajo pomen kakovostnega upravljanja družbenih podsistemov. Tega ni mogoče opraviti brez koordinacije razvojnih politik, povezanih z uvajanjem sprememb na posameznih področjih,
Vse povedano velja tudi za področje šolstva. Večje sistemske spremembe, uveljavljene v devetdesetih letih, so slovensko šolstvo postavile na trdno sodobno osnovo. Z razvojem družbe v zadnjem desetletju pa se je tudi na šolskem področju pokazala potreba po določenih sistemskih posodobitvah in izboljšavah. Taka posodobitev bi bila tudi dopolnitev in razvoj mehanizmov, s pomočjo katerih je mogoče učinkovito ugotavljati in zagotavljati kakovost delovanja šolskega sistema. Ugotovitve OECD (2013) in Evropske komisije (2015) kažejo, da postajajo mehanizmi ugotavljanja in zagotavljanja kakovosti v izobraževanju vse pomembnejše gibalo razvoja šol.
Sistemi, ki jih uvajajo evropske države, vse bolj presegajo ozke šolske vidike in se usmerjajo na ustvarjanje in širjenje kulture, ki poudarja izboljševanje šole kot celote in ne le nekaterih ozkih vidikov šolske učinkovitosti (na primer učenčevih dosežkov). Praksa kaže (Eurydice Report, 2015), da sta bila v večini evropskih držav uvedena dva načina evalviranja šol: notranja in zunanja evalvacija. Odgovornost za ugotavljanje in zagotavljanje kakovosti v izobraževanju je porazdeljena. Eden od partnerjev je vlada. Je odgovorna za stanje v šolstvu, zato v okviru svojih pristojnosti vedno išče možnosti, kako najbolj optimalno uskladiti notranjo in zunanjo regulacijo kakovosti. Področje ugotavljanja in zagotavljanja kakovosti avtorji (Vanhoof & Petegem, 2006; Altrichter & Kemethofer, 2015) opredeljujejo z dveh perspektiv. Gre za perspektivo, povezana z zagotavljanjem javne odgovornosti (public accountability) in perspektivo, usmerjeno v razvoj in kakovost izboljševanja
92 International Public Administration Review, Vol. 14, No. 2-3/2016
Valuational Overhaul of Regulation and Assessing and Maintaining
Education Quality in Slovenia
šole. V literaturi (Vanhoof & Petegem, 2006; Ehren et al., 2013) je bil vidik »zagotavljanja javne odgovornosti« običajno povezan s procesom zunanje evalvacije, vidik »izboljševanja kakovosti šole« pa je bil v domeni notranje evalvacije ali samoevalvacije. Sčasoma je tako tradicionalno ločevanje postalo utesnjujoče, zato so se pojavile zahteve po ponovnem razmisleku o odnosu med obema oblikama evalvacije. Različni avtorji (Van Hoyweghen, 2002; Christie et al., 2004; Shewbridge et al., 2011; Ehren et al., 2013, Altrichter & Kemethofer, 2015) iz tega zaključujejo, da sta procesa notranje in zunanje evalvacije komplementarna in se dopolnjujeta. Pri smiselnem uvajanju notranje in zunanje evalvacije se povečuje dodana vrednost obeh oblik, v primeru odsotnosti ene od njih pa se zmanjša vrednost druge.
Čeprav raziskave o vplivih zunanje evalvacije na kakovost dela v šolah niso ravno številne, je v zadnjih letih tudi temu področju namenjeno kar precej pozornosti. Ugotovitve kažejo, da prijemi zunanje evalvacije v šolah vzpostavljajo jasna pričakovanja glede področijspremljanja in kazalnikov kakovosti, vključenost deležnikov pa spodbuja njihovo senzibilnost glede procesa zunanjega spremljanja in značilno vpliva na šole. Obstaja pa tudi nevarnost, da nepremišljena izpeljava zunanje evalvacije v preveliki meri sledi proceduralnim postopkom, namesto izboljšanju šole. Nepremišljena vpeljava zunanje evalvacije lahko spodbudi tudi pretirano porast administrativnih zahtev, ki jih je v šolah že tako ali tako preveč (Faubert, 2009).
Modeli in izkušnje, povezane z ugotavljanjem in zagotavljanjem kakovosti v izobraževalnih institucijah, so v različnih državah različne. V slovenskem šolskem sistemu je zakonsko opredeljena le samoevalvacija (48. in 49. člen Zofvi). Poleg primerov dobre prakse, ki jih predstavljajo nekatere šole (KEKS, 2015), natančnejših podatkov o tem, kako samoevalvacija poteka, ali služi svojemu namenu, ni. Po poročanju predstavnika šolskega inšpektorata (zapisnik seje Delovne skupine za operacionalizacijo kakovosti na področju VIZ, 20. 2. 2015) šole sicer pripravljajo poročila o samoevalvaciji, vendar je kakovost teh poročil vprašljiva. Vprašanje je, ali gresta razvoj in izvajanje samoevalvacije v slovenskih šolah v pravo smer. To je do neke mere razumljivo, saj procesi samoregulacije oziroma kultura in protokoli samoevalvacije pri nas nimajo tradicije. Hkrati je v Sloveniji uzakonjena in deluje šolska inšpekcija, ki je del ugotavljanja in zagotavljanja kakovosti. V skladu z Zakonom o šolski inšpekciji zasleduje specifične cilje. S tako ureditvijo Slovenija izstopa med evropskimi državami. Preverjanje skladnosti delovanja šole z zakonodajo poznajo le še v Turčiji in Estoniji (Eurydice Report, 2015, str. 8 in str. 51).
Na drugi strani pa se z leti vse boljpoudarjajo vprašanja javnosti glede dostopa do informacij. Gre za vprašanja, kako dobra je šola, ki jo obiskujejo njihovi otroci. Starše učencev in dijakov zanima, kakšne standarde kakovosti dosega njihov otrok, kje je v primerjavi z drugimi. Če bi imeli empirične dokaze, kaj vse bi na šoli še lahko izboljšali in kaj bi celo morali narediti, bi to lahko prek svojih predstavnikov tudi zahtevali. Še bolj kot za starše pa je pomembno za državo, kako dobre so šole, (Braithwaite & Coglianese, 2007). Prav država
Mednarodna revija za javno upravo, letnik 14, št. 2-3/2016 93
Stanka Setnikar Cankar, Franc Cankar, Tomi Deutsch
namreč zagotavlja, da imajo vsi otroci dostop do kakovostnega izobraževanja (Merljak, 2015). Brez podatkov in analiz pristojni težko vodijo izobraževalno politiko. Nedavno poročilo Računskega sodišča (Revizijsko poročilo, 2016), ki je preverjalo urejenost delovne obveznosti učiteljev osnovnih šol, to povsem potrjuje. Starši, učitelji, lokalna skupnost in država bi zato potrebovali še mehanizem, ki bi na sistemski ravni določal, kaj je minimalni standard, ki bi šole opozarjal, če zdrsnejo pod to raven, in bi jim zagotavljal podporo pri premagovanju težav. Potrebovali bi smiseln sistem zunanje evalvacije,
V nadaljevanju predlagamo model zunanje evalvacije, ki zahteva le reorganizacijo obstoječih podpornih procesov in daje zadovoljive rezultate tako glede ugotavljanja kakovosti (zbiranje podatkov) kot glede zagotavljanja kakovosti (izvajanje ukrepov na nacionalni in šolski ravni). Predlagani model poudarja odgovornost šol do javnosti, zagotavlja širjenje učinkovite prakse in informiranje nacionalne šolske politike. Sestavljajo ga naslednja področja: pogoji dela, pedagoško vodenje in organizacija, izvedba pouka, vključevanje udeležencev izobraževanja in njihovih staršev, dosežki udeležencev izobraževanja in notranja evalvacija ter razvoj šole.
Mehanizme, s katerimi že sedajugotavljamo in zagotavljamo kakovost delovanja šolskega sistema, je treba razvijati naprej. Pričakovanja, da se bo šola samoregulirala brez zunanje sodobne strokovne podpore, ki bi zagotavljala potrebne informacije tako šoli, šolskim oblastem in drugim deležnikom, so nerealna. Predlagani model zato nadgrajuje obstoječe mehanizme za ugotavljanje in zagotavljanje kakovosti in jih z zakonodajnim predlogom povezuje v smiselno celoto.
94 International Public Administration Review, Vol. 14, No. 2-3/2016