242  Sodobna pedagogika/Journal of Contemporary Educational Studies  Dragana Pavlović Breneselović and Živka Krnjaja Let./Vol. 69 (135) Št./No. 4/2018 Str. 88–104/pp. 242–259 ISSN 0038 0474 Potentials of the model of collaborative evaluation in developing the quality of the kindergarten practice in Serbia1 Abstract: The potentials of a collaborative model for evaluation of the quality building of the kindergarten practice are compared here with the external evaluation. Two dominant quality discourses are considered: the discourse of quality assurance and the discourse of quality building. Starting from the quality building discourse, we discuss the model of collaborative evaluation with its essential characteristics and possibilities for evaluation. We present an example of the implementation of the collaborative evaluation model within the pilot project, the new Preschool Curriculum Framework in Serbia. The results show how the space could be transformed through a collaborative evaluation. We point out comparative advantages of the model of collaborative evaluation vs. the dominating model of external quality evaluation in Serbia. Key words: kindergarten space, quality, pilot project, preschool curriculum framework, transformation of kindergarten practice UDC: 373.2 Scientific article Dragana Pavlović Breneselović, PhD, full professor, University of Belgrade, Faculty of Philosophy, Čika Ljubina 18–20, 11000 Belgrade, Serbia; e-mail: dbrenese@f.bg.ac.rs Živka Krnjaja, PhD, full professor, University of Belgrade, University in Belgrade, Faculty of Philosophy, Čika Ljubina 18–20, 11000 Belgrade, Serbia; e-mail: zivka.krnjaja@f.bg.ac.rs 1 This article is a product of a research project No. 179060 – Models of assessment and strategy for improving the quality of education in Serbia, funded by Ministry of Education, Science and Tehnological Development of Serbia.  Pavlović Breneselović, Krnjaja 243 Introduction Our dealing with the quality of preschool education started through the study Where does Quality Live – Policy of Building Early Childhood Education Quality (Krnjaja and Pavlović Breneselović 2013). We identified and analysed two dominant discourses about quality of education and training – the discourse of quality assurance and the discourse of quality building. The quality assurance discourse is based on the perception of educational practice as a rigid and / or deterministic system. Within such perspective, quality is perceived in positivistic manner, as something tangible and material, something that can be learned, tested and measured; something objective, regardless of our values. Therefore, our knowledge of quality is acquired by quantitative measurement, scales of assessment, correlation studies, experiments and quasi-experiments. Empirical research offers data and leads to theories and quality assumptions that are introduced into practice through standardization, and quality evaluation has the function of control through measuring the standards’ achievement (ibid.). In the discourse of quality building, the practice of the educational system is recognized as a complex and purposeful value-based system – a system of “search for meaning” (Banathy in ibid., p.114). Quality is a socially and culturally constructed concept, and as such it is contextual, subjective, pluralistic, multi-perspective and value-based. From this understanding of quality, we have developed the following guidelines for understanding and building the quality in education (ibid., pp. 107–112): Dynamic. Quality is not an issue of a singular prescription and once established state or level, but a constant, dynamic and continuous process. It is not achieved by defining a level or a threshold to be achieved by setting up some standards, but by realizing the assumptions for its permanent development. Multidimensional. Quality cannot be reduced to individual, isolated dimensions; for example - a structural dimension of the institution - but it includes a whole range of dimensions of a complex system such as the education system, the system shaped by the educational practice. Quality is determined by all dimensions of education and upbringing practice and not by the outcomes of practice (Dahlberg and Moss 2005). 244 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja Systemic. Quality is a construct of mutually interconnected, mutually interacting and mutually dependent components and must be grasped as a whole of relationships through which individual dimensions or components acquire meaning, rather than as a sole sum of individual components. This is followed by synchronization and co-ordination by which the system’s coherence is achieved. Contextual. Determination and access to quality arise from cultural, social, economic and political perceptions of childhood and from the approach to the function of preschool education. There is no universally valid and permanently invariable meaning of quality, but it is built up by reflexion in a given context and is always contextually shaped. Multiperspective. “Quality is in the eyes of the observer“ (Farquhar in Krnjaja and Pavlović Breneselovic 2013, p. 119), hence it has more dimensions and perspectives. Educational practice includes a number of different actors: policy makers, local authorities, professionals, children, parents, members of the local community... They differ in terms of interests, expectations, perceptions and roles, and thus in terms of understanding, demanding and assessing quality. Participative. Defining quality is a participatory process of building the meaning and deepening of our understanding of what is happening in the preschool program, of assessing the values of what is happening, and of the common pursuit of the arguments of these assessments. This is a process that is important in itself because it provides opportunities for exchange, dialogue and understanding. Quality is based on “the assessments as a participatory process of interpretation that involves dialogue and argumentation, where the assessment is the subject of reflection of the participants in a given context in relation to the key values underlying early childhood education and care” (Dahlberg et al. 2007, p. 10). The approach to the quality of preschool education in Serbia Evaluation of the quality of work in preschool institutions in Serbia is defined in the Rulebook on Evaluating the Quality of Institutions (2012) and the Rulebook on Standards of Quality of Work of Institutions (2012a), which are implemented from 2013 on. Evaluation of the quality of the work of preschool institutions in Serbia is based on external evaluation of quality and on self-evaluation. External evaluation of quality is completed on the basis of external evaluation standards and is carried out by external, independent evaluators who evaluate the work of an institution based on pedagogical documentation, observations in the institution and the interviews with the director, teachers, parents and other participants important for the life of the institution (Rulebook on Evaluating … 2012). Direct monitoring of educational work includes realization of activities in at least 40% of educational groups in at least 40% of the facilities of the preschool institutions. The external evaluators have to observe the activities for at least 15 minutes (ibid., p. 5). Based on the report of the external evaluation commission, the institution drafts a Plan for improving the quality of work in the areas defined by the standards of quality of work of the institutions, and submits it to the school administration Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... 245 (ibid., p. 7). Coordination between school administration and preschool institution after completion of external evaluation, with the aim of further improvement of quality, is not defined by the Rulebook on Evaluating the Quality of Institutions (2012). The function of the standards for external evaluation of the quality of work is to enable a “uniform and objective assessment of the quality of the work in the preschool institutions” (Quality standards … 2012, p. 1). The quality standards for preschool institutions are determined through 29 standards and 137 indicators for 7 areas of work: preschool program, annual plan, development plan; educational work; child development and advancement; support for children and family; ethos of kindergarten; organization and management; resources. The standards are defined in the document as “evidence of quality practice or conditions of practice” and indicators as “operationalized definitions by which the standards are met” (ibid.). According to the aforementioned Rulebook on Evaluating the Quality of Institutions (2012), self-evaluation is defined as the quality assessment performed by an institution and based on the standards of external evaluation. From the definition of self-evaluation it is evident that the function of self-evaluation is alignment with the set up norms rather than a practitioners’ reflexive review of quality and their role in achieving it. The research on the practitioners’ perspective (Krnjaja 2016) of quality of the kindergarten practice provided the data on the attitude that practitioners have towards external evaluation. Data of the questionnaires’ analysis in the survey, which involved 105 preschool teachers, showed that practitioners do not see the external evaluation as one among ten dimensions of the quality of educational practice that they consider important for the quality of their work. According to the opinion of the largest number of the practitioners who took part in this research (77%), the instant external evaluation “forced the practice in a form and focused practitioners on formal aspects of practice” (ibid., p. 50) (Nobody deals with the essence, only the form is important; It is important that you will get the grade and not what will you do with it). Only a bit more than one-fifth of the total number of practitioner from the sample (21%) said that the external evaluation standards provided the “key to improving practice” (I have it written down how the good quality practice should look like, and I am sure I will not make a mistake), and 2% think that external evaluation has no effects at all on the quality of their practice (Nothing has changed; We do as before) (ibid., p. 50). The fact that practitioners experience external evaluation of quality as a separate act that does not contribute to building quality in kindergarten because it reduces practice on technical procedures that end up with the commission’s report and numerical ratings, opens the question of the purpose of external evaluation and its effects on quality improvement. From the study of the practitioners’ perspectives of quality, the following two quality assessment guidelines have been identified, among others: 1) empowering practitioners for self-evaluation with a sense of reflection of practice, understanding of the quality of the practice and their own professional role in the development of quality; 2) reconstruction of the regulatory function based on the practitioners’ assessment and strengthening of the 246 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja supporting, developmental function of evaluating the quality achieved by providing support to the practitioners in developing the quality of the practice. Model of collaborative evaluation The model of collaborative evaluation stems from the “quality building” approach and can be broadly defined as a process through which groups and various stakeholders build knowledge-oriented action in their reality that contributes to their knowledge of their values and norms, which mutually gets harmonized (Cousins et al. 2013; Greene 2000; Patton 2008; Patton 2011). According to D. Pavlović Breneselović (2014) the model of collaborative evaluation implies: Participation - participation of all partakers in the process of evaluation of the practice of preschool education (practitioners, parents, children, researchers, education policy holders, local communities). Multiperspectivity - the perception of the quality of preschool education from different perspectives (perspectives of practitioners, children, family and community, perspectives of researchers / bearers of evaluation). Formativeness - monitoring and evaluation viewed as a process whose function is the development of quality of the system and practice of the preschool education and the practice of the kindergarten. Systemic - entirety and networking of different levels (level of direct work with children, level of practice of kindergarten, level of the system of preschool education, level of evaluation system) in the process of monitoring and data collection and synthesis of data from different levels. The model of collaborative evaluation in developing the quality of preschool education enables: –– –– –– linking and collaboration of practitioners, researchers, policy makers and NGOs to initiate and manage change in a coherent manner that ensures the building of the common meanings, synchronization and harmonization, as well as the synergy of activities of different stakeholders; building the quality of practice not through regulation based on research data as a prescribed (simplified) universal implementation of scientific knowledge, but rather through the contextual support of researchers to practitioners based on the identification of contextual specificities and problems and on reviewing and generating scientific knowledge from the context of practice; putting all stakeholders in the position of a partner, which deconstructs power relations, external control and partial interests (for example, focusing on “desirable” results that meet predefined norms and indicators or research interest in proving the hypothesis) and building a common interest that is in function of quality development in a real context. In such a relationship, real Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... –– 247 problems and failures are not concealed but are a starting point for re-examination and substantive changes that ensure a real step forward; creating potential for the sustainability of change through: synergy and harmonization of activities of different actors; justifying a unique value position; developing capacities at all levels (systems, institutions, individuals) for continuous change and quality development (Pavlović Breneselović and Krnjaja 2017). Introduction of the new Preschool Curriculum Framework in Serbia: An example from the application of the collaborative evaluation model The European Quality Framework for early childhood education and care (Key principles … 2014) identified five dimensions of quality development, one of them related to the curriculum based on the well-being of the child and on the play, which promotes quality interactions among children, joined participation of educators, children and families as well as the joint participation of educators with colleagues in the development of reflexive and democratic practice (ibid.). Starting from this quality framework as well as from the generally recognized importance of quality in the new Curriculum Framework of the Preschool Education Curriculum, the elaboration of the new Preschool Curriculum Framework has begun in Serbia in 2018. Enacting the new Curriculum Framework (2018) through the pilot project was a novelty; it interrupted the usual practice of writing a document as a separate and instant act accomplished by an expert group formed by Institute for the Improvement of Education, composed of individuals from the academic community and professional associations, and established by the administration body. For the first time, the new Curriculum Framework was developed through joint participation of relevant institutions such as the Institute for Pedagogy and Andragogy of the Faculty of Philosophy in Belgrade (further on: IPA), the Ministry of Education, Science and Technological Development, the Institute for the Improvement of Education, UNICEF and the practitioners from three kindergartens in the pilot project. The project “Piloting the Preschool Curriculum Framework – the Years of Ascent” is based on the following assumptions: a) The program of educational work with children is created and constructed in the real context of education practice. Through the intervention directed towards the introduction of a new program in the kindergarten, the basic assumptions of the program are built upon the core framework determined by the theoretical and value-based image of the child and understanding of how a child learns and what are the roles of adults in it; b) The change in the practice of pedagogical activity implies a change that occurs in the real program. Therefore, the essence of pedagogical intervention is not to introduce a new segment of the program as a “ready-made change” but to enable, encourage and support the development of changes in all dimensions of the program in the kindergarten; 248 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja c) On understanding quality as a contextual and dynamic process that is being examined through monitoring of the process’ dimensions of the context and its potentials for the well-being and learning of a child, rather than through measuring the achievement of a child (Final Report … 2018, p. 7). Piloting in kindergartens was aimed at providing support to practitioners in developing programs based on the concept of the new Curriculum Framework (2018). The system of support for practitioners included: training, consultations of IPA researchers in kindergarten, consulting meetings of IPA researchers with professional associates from kindergartens. Piloting was conducted in the period from March 2017 to June 2018 and involved 45 practitioners from 18 groups from 3 kindergartens – 12 teachers-nurses, 24 preschool teachers, 6 associates (2 psychologists and 4 pedagogues), 3 directors of preschool institutions. For the pilot project, a complex model of collaborative evaluation has been developed, envisaging the participation of all actors in the monitoring and using a set of different dual-purpose techniques and instruments – supporting practitioners in developing a program based on the new Curriculum Framework (ibid.) and pilot project evaluation. For the purposes of this paper, we will present one example of application of a collaborative evaluation model that relates to the transformation of the kindergarten space. The concept of the new Curriculum framework (ibid.) is founded on the assumption that space is one of the key dimensions of the quality of the program. The space in the quality assurance discourse is classified into the structural dimensions of quality. In this discourse space is a physical environment that is organized for children, and the quality of space is perceived through its functionality and organization (Pavlović Breneselović 2015, p. 266). The emphasis is on the health and hygienic-security dimension of the space (the issue of size, brightness, adequate aero-thermal conditions, equipment that is adapted and safe for children) and pedagogical-didactic dimension (age relevancy of the equipment and materials, diversity, availability, sufficiency...). Such reduction of space results in the fact that space is taken as independent of the program and not as an integral part of the program and educational practice. In the discourse of quality building, the accent is shifted from the physical features of the space to the symbolic plan - the meaning of the space (without neglecting the importance of health and hygiene and safety conditions, but perceiving them through the prism of pedagogical meaning). This means the following: –– Equipment and materials are elements of the physical space that reflect cultural and programmatic values, norms and assumptions about preschool education, the child, his / her learning and the role of the adult (Peterson and Deal in Pavlović Breneselović 2015, p. 265). The space is not only physical, but it reflects the expectations of the participants in that space, as well as their mutual relations. The social dimension of the space determines children’s experience and forms childhood through beliefs that are involved in the given environment. Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... –– –– 249 Learning environment is not only an instantaneous physical space; it also makes a whole with its social, cultural, discursive and physical characteristics that shape the interactions of participants in the practice of kindergarten children, educators, parents and the community. Thus, for example, in Reggio Emilia Pedagogy, the environment is the third educator, which emphasises the connection between the quality of the environment and the learning. Adults build a pedagogical space through the environment in which children can learn and become competent (Rinaldi 2006). Learning environment is not a physical space as a state by itself, but a process of developing interactions in real situations in a given context. The characteristics of the environment are not fixed attributes, but are the subject of constant negotiation and interaction with others in everyday experiences and relationships; the outcomes of these exchanges will affect the child’s future relationship with a space (Lester and Russell 2010). Because of this, the quality of the environment “is not only a question of how much the environment meets the challenges, expands and presents a challenge for the experiences and intentions of children, but also about how much a child can influence and shape both the environment and their own learning process” (Sheridan 2001, p. 92). This understanding of the meaning of space is implanted in the concept of the new Curriculum Framework (2018): “The physical environment (space-time organization) directly forms the child’s position in the program. Relationships that emerge from the physical environment and which at the same time create the physical environment make it an immediate learning environment. Therefore, the physical space must be in accordance with the conception presented in the new Curriculum Framework. Space is by no means something given by itself, regardless of the program, but opposite, it reflects in the most direct and concise manner the concept of the program and must be in accordance with the concept of the Curriculum Framework. That is why the educator devotes particular attention to space, to its constant restructuring, to development, enrichment and to designing” (ibid., p. 26). A collaborative evaluation in the transformation of space in kindergartens was realized through a formative process aimed at critical review of the existing state of the space, the construction of a common meaning of the quality of space and the transformation of the space. The presented example illustrates how the process of collaborative evaluation takes place through the collection of data, support measures and actions that can take place in different cycles (“data collection – support measures – action”, “action – data collection – support measures”, “support measures – action – collecting data”). Monitoring and collecting data is in the function of building the quality of practice as it serves for mutual (self) insights and exchange, for researchers to design the next support measures and a better understanding of the process of change, and for practitioners – for further action. In the process of collaborative evaluation, various methods of joint mon- 250 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja itoring and evaluation of practitioners and researchers were used by combining techniques of systematic observation, participatory observation, group interviews, content analysis, as well as various ethnographic techniques such as photo-novels, auto-journals, vignettes (micro-ethnographic notes), photo – tours through kindergarten. Critical review of the current situation A critical review of the existing state of the space by bringing it in connection with the premises on settings given in the new Curriculum Framework (2018) has been realized through: –– –– Evaluation of the existing space by the practitioners who used the protocol of space analysis in the dimensions of the quality of the space enabling: cooperation and positive interdependence; respect for commitment and initiative; research, experimentation, creativity; diversity; affiliation and personalization; aesthetic sense and astonishment (Pre–school Curriculum Framework … 2018, p. 27). In this step, the basic function of the evaluation was informative - for practitioners to get more familiar with the premises given in the new Curriculum Framework through refinement, and for the researchers to get feedback about the importance that practitioners attach to individual dimensions of space. Practitioners then, within the framework of their kindergarten team and based on the same scale of space quality assessment, evaluated their work rooms and on the basis of that they made a plan how to change the space. The data obtained by this self-evaluation showed that the majority of practitioners estimate that the work rooms generally meet the quality criteria for all the dimensions given in the protocol. The data obtained by this self-evaluation showed that most practitioners estimate that the work rooms generally meet the quality criteria for all the dimensions given in the protocol. Evaluation of the quality of the existing space in kindergarten by three researchers from IPA, was carried out on the basis of the same protocol for assessing the quality of space and collected photo-documentation during participatory observation in kindergartens. Table 1 presents the summary of data based on the 18 observation protocols (one for each of total of 18 education groups that took part in this project) and on the analysis of photo-documentation collected during the first two-day researchers’ visit to each kindergarten. Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... Criteria of assessment: 251 (# of groups) (# of groups) (# of groups) not present partly present Present Cooperation and positive interdependence 4 8 6 Respect for individual and commitment to the activity 4 11 3 Initiative and participation 7 10 1 Exploring, experimenting and creativity 9 8 1 Diversity 8 9 1 Afiliation and personalization 7 11 0 Aesthetic sense and astonishment 10 8 0 dimensions of space quality Table 1: Data on assessment of the quality space in kindergartens (18 education groups) Collected data show a big gap between the quality criteria given in the new Curriculum Framework (2018) and present situations in kindergartens. These data are also supported by direct researchers’ insights; they pointed out the essential characteristics of existing spaces: uniformity (all rooms resemble one another); inadequate space structuring (fixed group position of tables against large empty space, fixed boxes or centres); stereotypes (conventional decorations and aesthetic patterns); dysfunctional furniture (large cabinets and high ceilings); partial inaccessibility of toys and materials; industrial toys and equipment; insufficiently consumable and semi-structured and unstructured materials); insufficient utilization of common spaces. “The two rooms were equipped with high cabinets and high semi-circular or enclosed shelves for the toys and didactic materials along the wall, low tables and chairs spatially grouped in one place with a larger area of empty space. Only three or four typical boxes are there: a kitchen corner (equipped with plastic or wooden kitchen furniture, plastic containers and “fruit and vegetables”), a puppet corner (mainly scarcely equipped with several dolls of different size and a crib); a corner of a hairdresser (with a small shelf and / or a table with few packaging plastic bottles) and a constructor (equipped with plastic cubes and constructors)... Only in one object the working rooms were different and structured according the centres of interest. On the walls there are decorations mainly done by the preschool teacher (drawings of Disney’s characters, letters, big drawings of Santa /in May/). The common space in front of the room is used as a wardrobe although the corridors have the same function. The terraces are empty and are usually not in function, and the courtyards are very scarce”.  (From the researcher note, “The initial state of space”) The data obtained by researchers’ monitoring were in great disagreement with the data obtained by the practitioners’ self-evaluation. Also, the data on the introduced changes showed that the changes were mostly superficial (by introducing smaller changes and enriching the existing ones) rather than transformative. These data indicated at: 1. the absence of a common understanding of the quality of the space and the criteria given in the new Curriculum Framework (2018); 2. the connection between the appearance and the use of space, with more or less implicit beliefs of practitioners about child, learning and their own role. 252 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja Constructing a common meaning On the basis of the obtained monitoring data, the researchers have developed the necessary support measures and a further follow-up process has been realized through: –– –– –– –– Consultative dialogue of researchers and practitioners on quality of space based on the presentation of collected data of space characteristics. The presented data and photo – documentation were the basis for a dialogue on the meaning of the quality of space in the concept of the new Curriculum Framework (2018) and the ways of its reorganization. Joint practitioners’ and researchers qualitative analysis of the collected examples of good and bad practice. The researchers selected examples of good practice photographs of the same space before and after the changes from the collected photo-documentation. The practitioners analysed one and the other example of space in relation to the child welfare dimensions given in the new Curriculum Framework (2018). A comparative analysis of the monitoring examples was aimed at providing support to practitioners in understanding the function of space and linking program objectives and organization of space. Follow-up through horizontal learning. Practitioners, with the support of the researchers, organized mutual visits and discussion groups as an opportunity for practitioners to observe and assess the space in another kindergarten and to exchange their observations and experiences through discussions with colleagues from other kindergartens: “... We were very proud of our work rooms. For the past ten years, we have built and built the space to be divided into smallscale centres of interest. There were centres that were more or less the same: the centre of the role-play, construction, language, art, table manipulations, research or sensory-perceptive and the centre of a small school that would be formed in groups. All centres were equipped with typical equipment and materials. Toys were at children fingertips. At the beginning, it seemed to all of us that we had a space that looks very much like what one can see in the modern concepts we were trained for ... Attempts to apply in our space what we saw in other kindergartens in the project made us aware that we were topped with a lot of unpacked materials in wardrobes … “. (From the narrative note of the expert associates, “Changing the outer space comes from the inside”) Joint monitoring of the quality of the kindergarten space – the dialogue of the practitioners and researchers on the transformation of space in the context of practice. During the joint visit, researchers and practitioners analysed the characteristics of the specific space, jointly identified problems and necessary support measures, and fostered transformation patterns such as: i) The transformation of space in the kindergarten as a question of transformation of the kindergarten culture, especially the question of transformation of stereotypical relations among the structures of the employees – a quality space requires the inclusion of different structures in the kindergarten Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... –– 253 and their joint work on the transformation of the space: “We could not make ourselves the complex transformation of culture. When the space is in question, particularly important to us was the help of the technical service. They agreed to work with us, unaware of transformations they themselves will experience”. (From the vignettes of expert associates, “Transformation of ways of participation of participants from kindergarten”) ii) “... The kindergarten space finally began to change from its roots with almost daily visits of directors, technical services and with their active involvement. From the basic elements (wall–painting, replacement of worn-out doors, the removal of unnecessary things, ...), through the reconstruction of the existing cabinets, changing position of existing furniture, to the “inner”, essential changes - changing the relationship between people and adults toward children. Great attention and comprehensive organization of technical tasks, as well as the internal arrangement of kindergartens in accordance with the program ‘Years of Ascent’, finally led to the result.” (From the vignette “How it all disentangled”) iii) If common areas in the kindergarten (terraces, halls, atriums, courtyards) turn out to be inspirational and provocative, they can potentially become a kindergarten place for joint encountering and exploring for children of different ages and children and adults: “Children were first who reacted to changes in space: they explored the halls, the atrium, and the courtyard and pointed to their parents. Parents and children stayed around the mobiliary, played when they were arriving and leaving the kindergarten. This experience has contributed to the fact that the kindergarten team “discovered” the potential of common space not as “the place through which children pass”, but rather as the place of common play and exploration of children from different educational groups. The above mentioned spaces were seen as places for joint learning, socializing children of different ages, and as places that can yield a sense of wonder, aesthetic experience and research incentive”. (From the vignette of expert associates, “Changes in common areas”) iv) space is dynamic – it is constantly being constructed and (re)organized through the joint participation of children and adults in kindergarten: “The project of a group of children who built a house on a tree in the yard this time was a catalyst for change. We began to think about space in a different way - to look at this tree as an opportunity, to open up for change from the established patterns of thinking about the kindergarten space. To think of all the premises we have and do not use them“. (From the vignettes of professional associates “Unused space around us”) Reinforcement of practitioners for monitoring and evaluation through the development of sets of techniques and instruments for monitoring the change of kindergarten premises for professional associates. A set of techniques and instruments consisted of vignettes, tours through kindergarten, auto-ethnographic and narrative notes (The Years of Ascent … 2018). Vignettes have been used as “cuts” that document one sequence (micro-saturation) that reflects the 254 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja essence of space change in accordance with the concepts of the new Curriculum Framework (2018). Tours through the kindergarten include photographs of professional associates in kindergartens that were made during the entire process of piloting The New Basis, with the aim of documenting the changes in space in accordance with the characteristics of the space given in the new Curriculum Framework (ibid.). Narrative and auto-ethnographic notes were used in the development of observational studies by professional associates in identifying and documenting breakthrough development points. Transformation of space Transformation is not focused on a definitive outcome; it is a continuous process of developing programs and building a quality of practice. –– Monitoring transformation of space by the professional associates – identifying difficulties and impasses. During follow-up expert associates identified the difficulties and congestions from which they generated insights that were the guidelines for further action. The examples of their insights are as follows: i) The reorganization of space is not a question of enriching the space but of its transformation. ii) “This action has made us aware that we are overcrowded with a lot of unopened packages of toys around the wardrobes, although we all tend to complain that we do not have enough material. While cleaning the space, talking and negotiating, we finally saw something that initially we could not see, blinded with colours and surpluses. Only when we began to question each part of the existing whole and think about new purposes the change began to occur.” (A note from the narrative note of the expert associates “Changing the external space comes from the inside”) iii) Space transformation does not come by contemplation but along with action in a given space. iv) “The active role of all should have been encouraged because the previous exchange pattern was based on interpretations of theoretical assertions that were known to everyone at the declarative level, without being grounded in the practice from which the different values and beliefs of all of us came to the surface. The scale of the assessment of the kindergarten culture pointed to the problems that inhibited further development, and gave us guidance on the direction in which to go. At one point, long discussions and abstract analyses became too demanding and unproductive. We decided to make a step forward in the direction of concrete changes that are visible and based on the starting points of the “The Years of Ascent” program. Our capacity to change is based on the general consent of all kindergarten’s members that we are ready to reconsider and change the existing practice and not to give up or abandon good ideas.” (From the vignettes of professional associates “How the Curriculum Lives”) Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... 255 v) Transformation of space requires transformation of the professional role of an expert associate from an observer to the active participant who works together with the preschool teacher in a concrete situation in the kindergarten. vi) “Be there, bring in provocations every day and build a common vision of space. In our case, for the beginning, this means to live together with preschool teachers and remember that the decision “leverage is in your hands” means that you will often be the one who will encourage the preschool teachers to try, to explore and find ways to play with ideas and turn them into practice by finding means, acquiring funds and looking for associates. To translate modern concepts into practice means that you do it daily with preschool teachers even if that is just to move furniture or find a new purpose with old materials.” (From the narrative note, “Key Development Site”) –– –– –– Supportive measures for transformation. Based on the monitoring data in the transformation of the kindergarten premises, the researchers prepared the material for the expert associates “Steps of Action”, which assisted the professional associates to initiate the transformation of the space through taking initiatives in the team and through working together with preschool teachers in concrete situations in the study room and common spaces in kindergarten. Also, additional resources have been developed in the form of manuals and equipment guides and materials in the space in accordance with the new Curriculum Framework (2018). Follow-up data as reinforcement for reflexive practice. Based on the follow-up data, the practitioners prepared their presentations for other practitioners about the transformation of the space in their kindergarten. By taking such a meta-position relative to the data, they are strengthened to use the follow-up data in the self-evaluation of the transformation process and for a reflexive approach to practice. We cite few examples of the preschool teacher’s explanations why this was helpful to them: “Because I can see my mistakes and get new ideas; Because it is closely linked to what I do in practice and helps me expand the perspective; Because it helps me to evaluate where we are and give the idea so we can and desire to want ...; Because it helps me and it’s very important for me to see where I am and what can go on; Because I’ve seen what’s good for me and where I’m in all this; In this way I compare my work, analyse and learn”. (Data from third-level evaluation) “We were thinking about the kindergarten culture and the changes that we have made and what steps are to follow, so I think that the joint presentation of the project is ’celebration of change’ and working on togetherness that is a process that needs to be nurtured...”. (From a report of a professional associate) Repeated space evaluation was conducted by researchers at the end of the piloting. Graph 1 shows the evaluation data obtained by ISSA (International Step by Step Principles of Quality Pedagogy) procedure for program’s quality in the learning environment category. 256 Pavlović Bresenelović, Krnjaja Sodobna pedagogika/Journal of Contemporary Educational Studies Learning environment 10 7 8 7 0 4 Step forward Quality practice Good start frst follow - up second follow- up Graph 1: Program’s quality within the learning environment area – the results of the first and second follow-up The data show a significant change in the quality of the learning environment in 14 out of 18 groups according to the indicators given in the ISSA protocol. Data obtained by repeated assessment based on the quality of space protocol and qualitative analysis of photo-documentation show that all kindergartens experienced significant changes in the quality of space in accordance with the criteria of quality space given in the new Curriculum Framework (Table 2). Assesment Not present Partly present Mostly present Totally present Kindergartens V1 V1 V1 V1 V2 V3 Cooperation and positive interdependence X X X Respect for individual and commitment to activity X X X V2 V3 V2 V3 V2 V3 Initiative and participation X X X Exploration, experimenting and creativity X X X Diversity X Belonging and personalization Aesthetic sense and astonishment X X X X X X X X Table 2: Assessment of quality of space at three sites (kindergartens) based on the protocol on the quality of space and on the analysis of photo-documentation Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... 257 Conclusion Application of Collaborative Evaluation Model in the project “Piloting the Preschool Curriculum Framework – the Years of Ascent” demonstrates the potential of a collaborative evaluation to develop the quality of practice in comparison with the external quality evaluation. This potential is observed in: 1) overcoming the detachment of external evaluation and self-evaluation; 2) mutual listening and building up the common meaning of researchers and practitioners working together and negotiating future actions; 3) methodological approach to evaluation that is consistent with the context of practice and the specifics of each context; quality evaluation is a process inseparable from the context of the real program and kindergarten in which it is observed and based on the multi-perspective of their participants. The potentials of the model of collaborative evaluation indicate the necessity of redefining the concept of evaluation and self-evaluation in preschool institutions in Serbia through: –– –– –– –– transformation of the existing system of evaluating the quality of work of the preschool institutions, which - instead of the emphasis on bureaucratic norms that stimulate the “mimicry of practice” (the practice focuses on an instant presentation that meets the requirements of measurement) - supports the process of collaborative evaluation as mutual support of actors in developing the quality of practice; replacement of the present model of self-evaluation according to the external evaluation form with the affirmation of the concept of self-evaluation as a reflexive process of a review with function to develop the quality of practice; redefining the role of researchers, educational advisers and all other actors outside the kindergarten from value-neutral-observers and controllers to a “critical friend” who seeks to understand the real context and, through “a voice from outside”, opens up a new perspective, contributes to deepening and expanding insights and provides support to trying out something new and own in a different way; deconstructing the existing model of external evaluation based on power relations, on non – synchronized activities, on partial and opposed interests between individual actors (policy makers, local government, researchers, practitioners) through the operationalization of the model based on partnership relations, synchronization of activities and common interests of all actors, in the function of supporting the development of quality in a real context. At the same time, this would presuppose a transformation of the culture of silencing the problems towards a culture of dialogue and mutual support in achieving a fundamental shift in the quality of preschool education. 258 Sodobna pedagogika/Journal of Contemporary Educational Studies Pavlović Bresenelović, Krnjaja References Dahlberg, G., Moss, P., and Pence, A. (2007). Beyond Quality in Early Childhood Education and Care: Languages of Evaluation. London: Routledge. Cousins, J. B., Whitmore, E., and Shulha, L. M. (2013). Arguments for a common set of principles for collaborative inquiry in evaluation. American Journal of Evaluation, 34, issue 1, pp.7-22. Dahlberg, G. and Moss, P. (2005). Ethics and Politics in Early Childhood Education. London: Routledge Falmer. Fetterman, D. M. and Wandersman, A. (2005). Empowerment Evaluation Principles in Practice. New York: The Guilford Press. “Godine uzleta”– vodič stručnim saradnicima za dokumentovanje. [“The Years of Ascent” – Guide for Professional Associates]. (2018). IPA, internal material. Greene, J. C. (2000). Challenges in practicing deliberative democratic evaluation. In: K. E. Ryan and L. DeStefano (Eds.). Evaluation as a democratic process: Promoting inclusion, dialogue, and deliberation. San Francisco, CA: Jossey-Bass, pp. 27–38. International Step by Step Principles of Quality Pedagogy. ISSA Standards International. Step by Step web site. Retrieved from http://www.issa.nl/; http://www.issa.nl/docs_pdfs/ Pedagogical-Standards-final-WEB.pdf (Accessed on 3. 4. 2017). Key principles of a Quality Framework for preschool education. (2014). Retrieved from http://ec.europa.eu/assets/eac/education/experts-groups/2011-2013/ecec/ecec-qualityframework_en.pdf (Accessed on 2. 2. 2018). Krnjaja, Ž. and Pavlović Breneselović, D. (2013). Gde stanuje kvalitet. Politika građenja kvaliteta u predškolskom vaspitanju. Knjiga 1. [Where does Quality Live – Policy of Building Early Childhood Education Quality. Book 1.]. Beograd: IPA. Krnjaja, Ž. (2016). Gde stanuje kvalitet. Razvijanje prakse dečjeg vrtića. Knjiga 3. [Where does Quality Live – Developing Kindergarten Education Practice. Book 3.]. Beograd: IPA. Lester, S. and Russell, W. (2010). Children’s right to play: an examination of the importance of play in the lives of children worldwide. Working papers in Early Childhood Development. The Hague: Bernard Van Leer Foundation. Osnove programa predškolskog vaspitanja i obrazovanja “Godine uzleta”. [Pre–school Curriculum Framework – “The Years of Ascent“]. (2018). Beograd: Ministarstvo prosvete, nauke, tehnološkog razvoja Republike Srbije. Patton, M. Q. (2008). Utilization-focused evaluation. CA: Sage Publications. Patton, M. Q. (2011). Developmental evaluation: applying complexity concepts to enhance innovation and use. New York: The Guilford Press. Pavlović Breneselović, D. (2014). Dečji vrtić kao model centar. [Kindergarten as a Model Center]. Nastava i vaspitanje, 63 issue 3, pp. 369–385. Pavlović Breneselović, D. (2015). Gde stanuje kvalitet. Knjiga 2. Istraživanje sa decom prakse dečejeg vrtića. [= Where does Quality Live – Research with Children the Practice of Kindergarten. Book 2.]. Beograd: IPA. Pavlović Breneselović, D. and Krnjaja, Ž. (2017). Kaleidoskop – Osnove diversifikovanih programa predškolskog vaspitanja i obrazovanja.[Kaleidoscope – The Framework of Different Programs of Pre–school Education]. Beograd: IPA. Potenciali modela kolaborativne evalvacije pri razvijanju kakovosti prakse v otroškem vrtcu .../  Potentials of the model of collaborative evaluation in developing the quality ... 259 Pravilnik o vrednovanju kvaliteta rada ustanove. [Rulebook on the Evaluation of the Quality of Institutions]. (2012). Službeni glasnik RS, No. 9/2012. Pravilnik o standardima kvaliteta rada ustanove. [Rulebook on Standards of Quality of the Institution]. (2012a). Službeni glasnik RS, No. 68/2012. Rinaldi, C. (2006). In dialogoue with Reggio Emilia – Listening, researching & Learning. London: Routledge. Standardi kvaliteta rada predškolskih ustanova. [Standards of Pre–school Institutions Quality]. (2012). Retrieved from http://test.ceo.edu.rs/novost/114-standardi-kvalitetarada-predskolskih-ustanova (Accessed on 7. 5. 2018). Sheridan, S. (2001). Evaluation of Pedagogical Quality in Early Childhood Education: A Cross–National Perspective. Journal of Research in Childhood Education 16, issue 1, pp. 109 –124. Završni izveštaj – Pilotiranje nacrta Osnova programa „Godine uzleta“. [Final Report – Piloting the Pre-school Curriculum Framework “The Years of Ascent”]. (2018). Beograd: IPA.