Developing Leadership Capacity for Data-Informed School Improvement Training Course Developing Leadership Capacity for Data-Informed School Improvement School leaders are listed as a professional group for which a deeper evidence-based knowledge is needed. Our experiences from comparative studies of school leader training in range of European countries indicated that working with data is not highly prioritize in existing programmes. The urge to enhance school leaders’ competencies in the use of evidence to contribute to the improvement of educational practice is the basis for this project. The de l e c a project aims at improving intelligent accountability of schools by developing school leaders’ competencies in the use of evidence for the improvement of educational practice. Following the training module school leaders will be more competent in: • Building a culture of data use • Developing an inquiry habit of mind • Data literacy • Creating a culture of inquiry in their school • Using data to improve instructional practice, school effectiveness and professional accountability • Focusing professional conversations on instructional improvement • Ensuring that data are made available for use by school staff • Using data to stimulate discussion, rethink directions and monitor progress with external stakeholders Within this project a universally usable and locally/nationally adaptable knowledge-based competence training module for school leaders will be developed, piloted and disseminated. Furthermore, the project intends to develop a strategy of its implementation in the countries involved and wider. This way, both the groups of school leaders and groups of leadership trainers will be affected by the project. This will be reached by meeting specific objectives of the project: • Develop a curriculum of the training program for school leaders focused on data-driven school development. • Develop a strategy of implementing the above mentioned program into the training of school leaders. The training programme will help school leaders to: • get adequate overview/knowledge of the main sources of data useful for improvement of student learning and school development • develop school leaders’ competence to understand such data • develop school leaders’ competence to work with these data for the benefit of their schools’ development For more information please see www.deleca.org. D E L E C A Developing Leadership Capacity for Data-Informed School Improvement Training Course Developing Leadership Capacity for Data-Informed School Improvement: Training Course This training package is a product of de l e c a 526499-l l p-1-2012-1-si-c o m e n iu s-c m p project with the contribution of: Justina Erčulj Agnis Gleizups Torbjörn Hortlund Bohumíra Lazarová Paul Mahieu Kristina Malmberg Signe Neimane Milan Pol Andrej Savarin Kristin Vanlommel Eric Verbiest Published by National School for Leadership in Education Dunajska cesta 104, 1000 Ljubljana, Slovenia Ljubljana | 2015 Design and Layout Alen Ježovnik, Folio © 2015 Deleca This project has been funded with support from the European Commission. This publication reflects the views only of the author, and the Commission cannot be held responsible for any use which may be made of the information contained therein. Published under the Attribution-NonCommercial-ShareAlike 4.0 International (c c b y-n c-s a 4.0) licence (https://creativecommons.org/licenses/by-nc-sa/4.0/). c i p – Kataložni zapis o publikaciji Narodna in univerzitetna knjižnica, Ljubljana 37.091(0.034.2) dev e l opi n g leadership capacity for data-informed school improvement [Elektronski vir] : training course / [with contribution of Justina Erčulj . . . et al.]. – El. knjiga. – Ljubljana : National School for Leadership in Education, 2015 Način dostopa (u rl): http://www.solazaravnatelje.si/ISBN/978-961-6637-86-2.pdf i sb n 978-961-6637-86-2 (pdf) 1. Erčulj, Justina 281017856 Contents Introduction · 6 1 Meaning and Purpose of Working with Data · 10 2 Data Collection: Assessing and Searching for Data and Evidence · 15 3 Analysis and Interpretation: Discuss Alternative Perspectives · 19 4 From Results to School Improvement · 23 5 Creating the Culture of Inquiry · 27 Recommendations · 33 DELECA Glossary · 36 5 Introduction Rationale There has been a growing quest for school accountability across Europe. Increasingly, schools are expected to inform their stakeholders about many aspects of their operation. They are expected to be able to generate and use data for their own development. For this informed professionalism (Barber 2002; Fullan 2005), leaders need to become data-wise (Earl and Katz 2006; Earl and Timperley 2009). The need to train schools leaders in this field is evident. As indicated in the baseline study, some existing programmes tend to develop inquiry skills and a culture of inquiry. But in most cases the programmes do not focus on generating data and using them in the context of school accountability and school development. The deleca project was set up to develop universally usable and yet locally/nationally adaptable knowledge-based competence development programme for school leaders. In applying this programme, local providers can (and should) build on participants’ previous knowledge and skills (for example, knowledge about accountability, skills like giving feedback, general inquiry skills). Within the framework for the effective practice of school leaders (Leithwood et al. 2006), specific knowledge and skills about data use can be related to the more general competencies needed by school leaders. In setting a direction, leaders need to ground their data about school results in the expectations of society and government. In developing people, school leaders need to use data to determine the content of professional development, to engage teachers in professional development and to help them to become data-literate. In organizing the school, they need to create a culture of inquiry. Lastly, managing the instructional program, school leaders need to use data to provide evidence for alternative practices, to test the effectiveness of teaching practice and to identify issues or problems in the teacher – learner relationship (Timperley et al. 2007). The project also fits with (inter)national policy priorities. An oe c d study (Pont, Nusche and Moorman 2008) emphasizes the value of accountability systems, but warns that they only lead to improvement if it is based on ‘data-wise leadership.’ Our project is partly based on the results of the Evaluation Strategies for Improving School Leaders’ Training Programmes – A European Perspective (e s i s t) project1 in which the present partners also co-operated. One of the findings of the e si s t project was that there was little attention in training programs to principles of informed professionalism in the context of school development and accountability. In addition the results of the Local Evidence-Based Policy and Practice in Education project in which four of the present partners co-operated, have been used to inform the programme. In this project partners and representatives of education institutions in 6 medium sized cities in Europe analysed how these cities play a role as broker in the information transition between the stakeholders in evidence-informed policy and practice. The general aim of this project is to develop school leadership capacity for evidence-informed school improvement by meeting the following specific objectives of the project: 1 The overall aim of this Comenius project was to develop strategies, methods and materials for evaluation and improvement of quality of school leaders’ training courses/programmes. 6 Introduction Development of the Project Year 1 Year 2 Year 3 Developing the conceptual framework. Piloting the training course. Producing the training course package. Developing the baseline study. Producing the final version of the Developing the strategy for Devising the training course training course. implementing the training course into curriculum. national settings. Exploitation of results. Quality Assurance Dissemination 1. To develop content for a training program for school leaders focused on data-driven school development to help school leaders: • to get adequate overview/knowledge of the main sources of the data useful for improvement of student learning and school development; • to develop school leaders’ competence to understand such data; • to develop school leaders’ complex competence to work with these data for the benefit of their schools’ development. 2. To develop a strategy for implementing the program of school leader training in the participating countries and beyond. Following our training programme school leaders should have developed competences related to: 1. Building a culture of data use through: • Developing an inquiry habit of mind (value deep understanding, reserve judgement, taking a range of perspectives). • Becoming data literate (think about purposes, know which data can inform the school and external stakeholders; recognize sound and unsound data; make interpretation paramount; pay attention to reporting and to audiences). • Creating a culture of inquiry in their school (identifies priorities; understand student’s academic standing and monitor and assure progress). 2. Using data to improve instructional practice, school effectiveness and professional accountability by: • Focusing conversations on instructional improvement (use data to test the effectiveness of teaching practice; use data to identify issues or problems in the teacher – learner relationship; use data to provide evidence for alternative practices; en- gage in professional learning conversations, based on data). • Ensuring that data are made available for use by school staff (use data from already existing structures, build new structures; offer professional learning op- portunities for staff to learn from data; use data to determine the content of the professional development). • Using data to stimulate discussion rethink directions and monitor progress with external stakeholders. The target groups for our programme are trainers/developers of school. Leaders who can use the whole programme or its parts for the professional development of school leaders. In addition school leaders will also benefit indirectly by being offered this programme. The project was developed over a period of three years. Approach In order to achieve the aim of the project, which was to create the ‘Working with Data Training Modules’ for school leaders, it was necessary to take several steps. Specialist literature was examined to identify the main themes to be addressed in the project modules: 7 Introduction information, data, working with data, evaluation, accountability and so on. The starting point was research and theory relating to working with data in educational institutions. This formed the basis for a coherent ‘Theoretical Framework’ which linked working with data (data collection, processing, reading, interpretation, and so on) to school development. In the second stage of project development, the reality of working with data from current school leader training programs in the partner countries was recognized. This drew on a ‘Baseline Study’ which afforded an overview of current training programmes for school leaders provided in partner countries. We were interested in the selection of school leaders, their work in practice, autonomy in decision-making, obligation to self-evaluate and ways of external accountability. We also studied selected characteristics of the systems of school leader training: education commitment, training providers and the duration of courses. We analyzed selected training programmes in terms of their compo-sition and we were interested in whether they provided modules on working with data. A last step, prior to creating the training modules, was a questionnaire for school leaders to ascertain the reality of working with data in schools in the partner countries in order to identify respondent educational needs. The structure of the ‘Use of Data in Schools Questionnaire’ enabled us to gain an insight into how schools work with data, which data they work with, how the data is used to inform practice and how head teachers perceive their role, capabilities and educational needs. We were able to perform a partial compar-ison of situations in partner countries and identify possible differences between various types of school. We presented the research report which originated from these data at numerous workshops and conferences at both national and international level. The outcome of these three steps for the basis for the creation of the training modules. The Training Package The Training Package consists of three parts: • Introduction • Training Modules • Recommendations The introduction contains some information about the project and about the training modules which are the core of the Training Package. We used a common template for all modules. Beside aims and objectives we provide a detailed description of all activities that should be used to reach the stated objectives. Reflection is a part of every module because we wish participants to continuously reflect on their learning about various aspects of the use of data. Each module ends with a list of selected literature. It should be mentioned that this is not a complete list of titles we used to prepare the modules. It is rather a list of selected texts that may help the future trainers to complement reading already available at the website (see Theoretical Framework and Baseline Study). All training materials for participants are referred to as Annexes. They are numbered according to modules (1–5) and to the order they are supposed to be used within each module (i.e. 1.1, 1.2 ). The last Annex in every module provides power points which are numbered consecutively the order of the modules of modules. Although we used the same template, harmonisation of the modules was quite a chal- lenge as the partners are drawn from different educational contexts. We hope that after several reviews they are quite coherent. The Glossary at the end of the Training Package can help you further explain any possible vagueness. Before using the training materials, we would like to ask the trainers to attend to our recommendations which suggest range of ways materials might be delivered. The materials are intended to form a coherent five-day training program, however they may be used in different ways and for different purposes. The materials can (and in some cases should) be complemented by data from real contexts so that the programme becomes 8 Introduction more meaningful and useful to the school leaders who are our main target group of this training programme. We hope that you find the Training Package useful and applicable for your target groups and for your contexts. References and Further Reading Barber, M. 2002. ‘The Next Stage for Large Scale Reform in England: From Good to Great.’ Paper presented at Vision 2020 Second International Online Conference, 13–26 Oc- tober and 24 November–7 December. Earl, L., and S. Katz. 2006. Leading Schools in a Data-Rich World: Harnessing Data for School Improvement. Thousand Oaks, c a: Corwin. Earl, L., and H. Timperley. 2009. ‘Understanding How Evidence and Learning Conver- sations Work.’ In Professional Learning Conversations: Challenges in Using Evidence for Improvement, edited by L. Earl and H. Timperley, 1–12. New York: Springer. Fullan, M. 2005. Leadership and Sustainability: System Thinkers in Action. Thousand Oaks, c a: Corwin. Leithwood, K., C. Day, P. Sammons, A. Harris, and D. Hopkins. 2006. Successful School Leadership: What is It and How It Influences Pupil Learning. Nottingham: National College for School Leadership. Pont, B., D. Nusche, and H. Moorman. 2008. Improving School Leadership. Volume 1, Policy and Practice. Paris: oe c d. Timperley, H., A. Wilson, H. Barrar, and I. Fung. 2007. Teacher Professional Learning and Development: Best Evidence Synthesis Iteration. Wellington: Ministry of Education. 9 Module 1 Meaning and Purpose of Working with Data Objectives The aims of the module are: • To motivate participants to work with data in their own school. • To develop participants understanding of the meaning and purpose of work with data in schools. • To help participants recognise the need to work with data. • To encourage participants to view their school ‘through’ the lens of local data. • To clarify how self-evaluation, as a form of working with data, can contribute to the development of school. Outcomes of Learning After completion of training the school leader will be more: • willing and skilled to reframe problems from multiple view points; • willing and skilled to help staff to reframe problems from multiple view points; • knowledgeable about school self-evaluation. Competences • Being able to help staff to define information needs. • Being able to reflect on working with data in their own school. • Being able to understand a variety of data resources. Content • Block 1: Introduction of participants • Block 2: What is data and what kind of data is used in schools? • Block 3: How does data contribute to school development? Time Schedule • 9.00–10.20 Block 1 • 10.40–12.00 Block 2 • 13.00–14.20 Block 3 • 14.35–15.30 Reflection Preparation for the Participants • No preparation is needed. 10 Module 1 Participant Introduction The aim is to introduce participants to each other, to find out their course-related expec-Block 1 tations and motivate them to work with the data in schools. Activity 1: Welcome of Participants, Technical Issues, Introduction of Trainers Time 10 minutes. Activity 2: What Do I Know about my School? Time 45 minutes. Step 1 Participants will be divided into groups of three (or pairs) and will be assigned the task for the first exercise. Instructions Each of you will be asked to think about your school for three minutes: what its characteristics are, what you know about it in terms of the processes that take place in it, results or relationships among people in the school. Do not focus on the type of school or building, etc., but rather on some significant features of your school. Choose three of these characteristics and write them on paper. When all participants will have written ‘What I know about my school’ (3 significant features), then we proceed to the next Instruction. Step 2 Instructions Once you have written the characteristics of your school ‘what I know about my school,’ turn to your partners and: • introduce yourself, • tell others your expectations of the course, • briefly introduce your school (the type of school, size, etc.), • tell others what else you know about your school – just as you have written it, • tell others ‘how you have come to what you have written about your school’ or ‘how you know what you know about the school,’ • then exchange the role with your partners. You have a total of 10 minutes (each about three minutes). Step 3 After 10 minutes, finish the discussion in your groups/pairs. Instructions Now you have three minutes to decide how you will present yourselves and results of your group discussion to others. One of you will briefly introduce members of your group, the other one will introduce your shared expectations of the course (what you want to get from the course) and the last one how you know what you know about your school? Step 4 Presentations in groups. Lecturers will write expectations of the course and ‘how the participants know what they know about their schools’ (i.e. how they receive data) on the flip chart. An alternative option The questionnaire working with data in our school (Annex 1.1). Using the questionnaire the participants can identify the reality of working with data in their schools. The results of the questionnaire can be used also in following modules. Continued on the next page 11 Module 1 Continued from the previous page Activity 3: What Does the Course/Module Offer? Time 25 minutes. • It follows the expectations expressed by the participants. • The offer of the plan of the whole course and module one (Annex 1.3 – slide two). A short discussion with the participants over the plan. • Introduction of literature (see Annex 1.3 – slide three). • Framing into the context of requirements for today’s schools (Annex 1.3 – slides four and five). What do we consider to be data and what kind of data is worked with in schools? The aim is to enhance the knowledge of the data resources, to support the participants’ Block 2 awareness about a different kinds of data they can work with in their schools. Activity 1: What is Data? What Are the Sources of Data? What is the Significance of the Data? Time 20 minutes. A discussion lecture on what is considered to be data. What we consider to be data (Theoretical Framework; Annex 1.3, slides 6–8). Activity 2: What Kind of Data do Schools Collect? What is Your Experience in Your School Setting? Time 60 minutes. Step 1 Each participant will be given a worksheet and asked to complete the table with the data that is worked with in their school, from what sources and what their experience with it is. Instructions What data do you work in school with? Complete the table with what kind of data you have experience with, what sources you use in the mentioned areas. Worksheet Student Perceptions Student School Teacher Environ- demo- of learning learning processes character- ment data graphic environ- istics ment External statistics Internal survey Step 2 The participants in groups of four will discuss their experience and summarize all together answers to the following questions: 1. What data do you work with? What data do you want to work with? 2. What sources do you use? What sources do you plan to use? 3. What experience do you have? Step 3 The groups will present their results. Trainers will summarize it and write it down. Plenary discussion. 12 Module 1 And what does it mean for school development? The aim is to inform the participants on the data-wise leadership concept. The other aim Block 3 is to suggest the ways of facilitation the work with data in participantsśchools. Activity 1: Data-Wise Leadership Time 20 minutes. Step 1 Discussion lecture (Annex 1.3, slides 9–15). Step 2 Give participants the table Stages of evidence informed practice, sample competencies and samples of skills, knowledge and attitudes and provide them with time to study it and discuss it in pairs (see Annex 1.2 or Theoretical Framework). Plenary discussion What use of data is feasible in my school? How would I like to enrich the work with data in my school? Activity 2: Barriers and Limits of Data-Wise Leadership; From Identification of the Barriers to Implementation of Data-Wise Leadership Time 60 minutes. Step 1 The participants will divide into groups of four. Instructions Imagine the situation that the school management is requested to start working with data. You can imagine that this requirement awakes school managers fears, defence and resistance mechanisms. Putting yourself in their place will not be difficult. In your group identify all possible and impossible fears that headteachers/the school managers could encounter and write them down on paper. Working method – brainstorming. Step 2 Instructions Propose measures that could reduce the fears of headteachers / school managers. How you would advise to headteachers about what they could try to do to succeed in working well with data. Start with the identified fears. Write the measures down on the flipchart. Step 3 Team presentations. Plenary discussion. The summary of proposed measures. Activity 1: Summary of Outcomes of all Blocks Reflection Time 15 minutes. Activity 2: Feedback on Module 1 Time 25 minutes. Step 1 Trainers will encourage participants to provide feedback – the participants will stick points on areas of the target which is on the wall (see Annex 1.3). The point with high degree of fulfilment should be placed closer to the centre (to point 5); the point with low degree of fulfilment should be placed closer to the edge of the circle. Continued on the next page 13 Module 1 Continued from the previous page • The red area: To what extent the topic was new. • The blue area: To what extent the module has fulfilled expectations. • The green area: To what extent the module has motivated participants to work with data in school. • The yellow area: To what extent participants realize the educational needs essential for work with data. Step 2 Trainers will encourage participants to write down the educational needs essential for work with data. On their basis the final discussion will be carried out. Further Remarks Preparation for trainers: • Familiarization with the theoretical framework (Deleca) and further relevant literature. Preparation of exercises for participants. References and Further Reading Argyris, C., and D. Schön. 1996. Organizational Learning 2: Theory, Method and Practice. Reading, m a: Addison-Wesley. Earl, L., and M. Fullan. 2003. ‘Using Data in Leadership for Learning.’ Cambridge Journal of Education 33:383–394. Earl, L., and S. Katz. 2006. Leading Schools in a Data-Rich World: Harnessing Data for School Improvement. Thousand Oaks, c a: Corwin. Senge, P. 1990. The Fifth Discipline: The Art and Practice of the Learning Organization. New York: Doubleday Currency. Vanhoof, J., K. Vanlommel, S. Thijs, and H. Vanderlocht. 2013. ‘Data Use by Flemish School Principals: Impact of Attitude, Self-Efficacy and External Expectations.’ Educational Studies 40 (1): 48–62. Verbiest, E., and P. Mahieu. 2013. ‘Developing Leadership Capacity for Data-Informed School Improvement: Theoretical Framework.’ http://www.deleca.org Materials ☞ Annex . The Questionnaire ☞ Annex . Stages of Evidence Informed Practice ☞ Annex . Presentation 14 Module 2 Data Collection: Assessing and Searching for Data and Evidence Objectives The aims of the module are: • To actualize the knowledge and improve skills to identify important issues in school. • To identify and formulate an issue. • To enrich the knowledge and skills about the selection of different data collection methods. • To encourage use of different data collection methods. • To reflect on leadership practice and challenges in using data for school improvement. • To reflect about ethics, reliability and validity of data collection methods. Outcomes of Learning After completion of training the school leader will be more: • Conscious of possible use of data. • Knowledgeable about tools for data collecting. • More aware of obstacles and opportunities in the collection of data related to school improvement. • Willing and better able to support staff to identify problems and choose methods for inquiry. • To deepen the understanding of the importance of validity and reliability in research to guarantee the quality of research. • To raise the awareness of school heads about the ethics of research. • To encourage the staff to base their planning of improvement on data. • More flexible in everyday work knowing how to use data in real life. • More aware about the ethics of research. Competences Being able to: • to think critically about the choice of data collection methods, • to relate data and decisions leading to school improvement, • to collect and use a rich set of data to understand and assess the strengths and weaknesses of the school, • to know and understand a range of evidence to support, monitor, evaluate and improve school performance, • to develop a capacity to apply the principles of research ethics and trustworthiness. Content • Block 1: Identification and Formulation of an Important Issue in the School Where Further Data Would be Beneficial to Decision-Making • Block 2: Methods for Collecting Data • Block 3: Ethics, Reliability and Validity of Data and Methods 15 Module 2 Time Schedule • 9.00–10.30 Block 1 • 11.00–12.30 Block 2 • 13.30–15.00 Block 3 • Reflection Preparation for the Participants No preparation is needed. Identification and formulation of an important issue in the school The aim is to focus on the importance of identifying knowledge needs and framing prob-Block 1 lems for inquiry Activity 1: Ice-Breaking Activity ‘Value Line’ Time 15 minutes. Participants gather into one end of the room. The trainer explains rules: • If you agree with the statement, go to the other side of the room, if not, stay where you are, if you have doubts choose a place closer to one or another end. • Be ready to explain, give an argument why you have chosen the place. • You can change your place if you agree with an argument provided by others. The trainer can use statements prepared (Annex 2.1) or formulate different ones. The other trainer keeps records of the final solution of participants using the protocol (Annex 2.2). The trainer can discuss briefly with the group if this method can be used in data collecting process. Activity 2: Photo Language Time 75 minutes. For the final identification of an important issue in participants schools method called Photo language process is used. About 50 photos are needed (Annex 2.3). There are seven steps in the Photolanguage process (Annex 2.4). After activity and discussions in small groups participants write down their issue on a post-it and put them on a large sheet of paper or stick to the white board. Participants read all post-its and categorize them putting together similar issues trying to categorize them. Variant A The group chooses one issue they would like to work on to plan an inquiry where further data would be beneficial to decision-making Variant B Each participant defines his/her important issue and work with it the whole day. Methods for Collecting Data The aim is to help the participants see the relationship between different kind of data and Block 2 useful tools for quantitative and qualitative data collection. Activity 1: What Kind of Data? Time 10 minutes. Discussions in pairs and then in the groups – what kind of data do we need to learn more about the formulated issues? How to get data? What data collection methods would be the most useful? Why? Continued on the next page 16 Module 2 Continued from the previous page Activity 2: Variety of Data Time 45 minutes. • The trainer offers the ‘basket’ of data collection methods from theoretical and practical point of view (Annex 2.5). • Trainer can ask questions like: How often do you use method . . . ? What advantages of the method . . . have you noticed? What disadvantages of the method . . . have you noticed? Activity 3: Data Collecting Methods Time 35 minutes. The participants in the groups choose the data collection methods and create a mind map of the inquiry. The mind maps of the inquiry are put on the white board for common discussion and reflection. Variant A Task is done in the group. Variant B Task is done individually, consultations with critical friend takes place. Ethics, Reliability and Validity of Data and Methods The aim is to stress the awareness of ethics and understanding the significance of validity Block 3 and reliability of data and methods. Activity 1: Ethics, Reliability and Validity Time 60 minutes. Trainer gives a short introduction to ‘reliability,’ ‘validity’ and ‘ethics’ of data (Annex 2.6). Ethics, Reliability and validity of data and methods are being discussed in the group using: • Protocol of the activity Value line • Photo language process • Questionnaire Activity 2: Practical Work with Questionnaire Time 20 minutes. The trainer refers to the questionnaire used in the Module 1. The trainer runs discussion: how to communicate with staff to involve them into inquiry process? Activity 3: Protocol of the Learning Walk Time 30 minutes. Trainer presents a Protocol of the learning walk. It is discussed in small groups if the provided Protocol reaches participants needs, is it relevant. Participants can change and add their new ideas to the Protocol (Annex 2.7). Time 10 minutes. Reflection Reflection and evaluation of the day using rapid feedback card method (Annex 2.8). References and Further Reading Earl, L., and S. Katz. 2006. Leading Schools in a Data-Rich World: Harnessing Data for School Improvement. Thousand Oaks, c a: Corwin. Bell, J. 2014. Doing Your Research Project. Maidenhead: Open University Press. Creative Research Methods. N. d. ‘Mobile Nodes: Mobile and Locative Media, Everyday Life and Sense of Place.’ http://creativeresearchmethods.wordpress.com/category/ visual/ 17 Module 2 Cropley, A. 2002. Qualitative Research Methods. R¯ıga: Zin¯atne. Denscombe, M. 2001. The Good Research Guide. Maidenhead: Open University Press. Learning Space Toolkit. N. d. ‘Data Gathering Tools.’ http://learningspacetoolkit.org/ needs-assessment/data-gathering-tools/ Stoll, L., and D. Fink. 1996. Changing our Schools. Buckingham and Philadelphia: Open University Press. White, R., D. Sasser, R. Bogren, and J. Morgan. 2009. ‘Photos Can Inspire a Thousand Words: Photolanguage as a Qualitative Evaluation Method.’ Journal of Extension 47 (3). http://www.joe.org/joe/2009june/iw1.php Materials ☞ Annex . Activity Value line – Statements ☞ Annex . Protocol of the Value Line ☞ Annex . Photo Language ☞ Annex . Steps in the Photo Language Process ☞ Annex . Data Collecting Methods (ppt Presentation) ☞ Annex . Ethics, Reliability, Validity (ppt Presentation) ☞ Annex . Protocol for Learning Walk ☞ Annex . Rapid Feedback Card 18 Module 3 Analysis and Interpretation: Discuss Alternative Perspectives Objectives The aims of the module are: • To deepen the participants’ knowledge, skills and attitudes to reading, understanding and interpreting data. • To invite to discuss alternative (or contrary) perspectives. • To strengthen the school leaders’ abilities to encourage and help staff to use school data. • To initiate dialogue in the school in order to make sense of data. • To help staff to read, understand and interpret data. • To help staff to describe the central data concepts. • To help staff to integrate this information into other data sources. Outcomes of Learning After completion of training the school leader will be more: • To engage others (teachers, critical friends) in a professional dialogue around the data. • Skilled to read, understand and interpret data and to support the staff to get skilled in reading, understanding and interpreting data. • Able to recognize other kinds of data (not only numbers, but also opinions, anec-dotes, observations). • Willing and skilled to help staff to develop an attitude of critical thinking when analysing data (reserving judgment, tolerance for ambiguity, valuing deep understanding). • Willing and skilled to help staff to recognize other kinds of data. • Skilled to recognize sound and unsound data and to help staff to recognize sound and unsound data. • Willing and able to make collective sense of the data. Competences • Being able to initiate dialogue in the school in order to make sense of data together. • Being able to help staff to integrate this information into other data sources. • Being able to help staff to read, understand and interpret data. • Being able to help staff to describe the central data concepts. Content • Block 1: Introduction to analysis and interpretation of data • Block 2: Reading quantitative and qualitative data • Block 3: Assessing and valuing outcome/Working with school data 19 Module 3 Time Schedule • 9.00–9.45 Block 1 • 9.45–10.30 Block 2 • 11.00–12.00 Block 3 • 13.00–16.00 Block 3 • 16.00–17.00 Preparation for field visit and the learning walk in module 4 • Reflection Preparation for the Participants Read through all the data from the hosting school in order to be prepared for Activity 4. Introduction to Analysis and Interpretation of Data The aim is to get an introduction to the concept analysis. Block 1 Activity 1: In What Ways Does the Elephant Cartoon Illustrate Important Perspectives in Regard to Analysing Data? Time 5 minutes. The picture with the elephant (Annex 3.1, slide 2) leads the participants into discussions such as: • we see different things in our data due to our differences in understanding, experiences, assumptions and thinking, • what we see is subjective, • the analysis will not lead to ‘one truth,’ This exercise will help the participants to ask questions like how will (Annex 3.1, slides 3–4): • The analysis help us to new questions? • The interpretation of our data support our organizational learning? • Data and data analysis support our school improvement programme? Activity 2: Exercise with the Box: About Perceptions and Interpretation Time 25 minutes. Put a dismantled coffee machine in a box with holes. Let different participants stick in one hand in the box and describe what they feel. Through the different ‘pictures’ of what different participants feel and think they try to figure out the ‘whole picture’ – what’s in the box. Summarize insights and reflections from the group and relate it to the conceptions analysis and interpretation. Activity 3: Theoretical Input Time 20 minutes. Theoretical input (Annex 3.1, slides 5–14). Connection to the theoretical framework. Reading Quantitative and Qualitative Data The aim is to get more skilled in reading both quantitative and qualitative data Block 2 Activity 3: Reading Statistics Time 45 minutes. Hand-outs with tables and diagrams on collected quantitative data in the current context (Annex 3.1, slide 15): • Describe what you see? • Trends? Changes over time? • Patterns? • Big picture? 20 Module 3 Analysing and Valuing Outcome/Working with School Data The aim is to get more skilled in analysing and assessing data and be more aware of how Block 3 do lead a professional dialogue supported by data and how data can be a useful tool for learning by integrating different perspectives. Activity 4: Working with School Data – Workshop Time 135 minutes. Introduction to analysing data (Annex 3.1, slides 16–22). The distinctions between data collection, processing and interpretation is clearly emphasised. In the fourth module on Thursday you will do a field trip to a school. In the folder/box you will now find a lot of different kind of data collected in this school. There are input, process and output data on different levels in the school system. The principal in the school has picked a certain area/objective/question in the curriculum that he/she wants you to focus on in your analysis and interpretation of data. Before starting, read through the instruction. Make sure that you focus on and follow the instruction, keep to the time frame and also take notes at each step. Step 1: Formulating Criteria Time 30 minutes. Start formulating a few criteria in relation to the chosen objective. You will need these to assess and value how well the school is achieving the objective. Step 2: Collecting Data Time 30 minutes. With the criteria as the starting point, choose which data, in the folder are relevant to ex-plore? What data do ‘collect’? Step 3: Analysis – Processing and Interpretation of Data Time 30 minutes. • Describe what you see. This step is about sorting and categorizing data. What com-parisons are possible to make? Find suitable ‘labels’ for the categories you find. • What kind of relationships/connections is possible to make? This phase is about dis-cerning patterns, themes and relationships. For example, how instructional patterns are related to outcomes. • Interpretation – possible explanations and different ways of understanding. What reasonable explanations ‘outside the data’ need to be added to understand cause and consequence, what effects what? • Question and discuss limitations of the interpretation in the group. If the analysis doesn´t give you a reasonable explanation because relevant data are missing, what new questions need to be answered? Step 4: Assessing the Result of the Analysis Time 15 minutes. Use the criteria you formulated in step 1 to value your findings in the analysis. How do you value school achievement in relation to the objective? Step 5: Critical Friends Time 30 minutes. Two groups meet for presentation of their findings. Continued on the next page 21 Module 3 Continued from the previous page Activity 5: Preparations for Field Visit and the Learning Walk in Module 4 Time 60 minutes. Meeting with the head teacher of the school where the field visit and the learning walk will take place. The head teacher of the host school determines the focus of the learning walk. The learning walk will focus on certain instructional principles to improve teaching such as questioning strategies, classroom management, differentiated instruction and co-operative learning. • Theoretical input in the preparation (Annex 4.3) • Purposes of learning walks • What are learning walks? • Reflective questioning’ Time 45 minutes. Reflection Summarize what you learned in the workshop (activity 4), before activity 5. Further Remarks • Reading relevant documents about the host school. • Organizing groups for the learning walk. References and Further Reading Ackoff, R. 1989. ‘From Data to Wisdom.’ Journal of Applied Systems Analysis 16:3–9. Boudett, K., E. City, and J. Murnane, eds. 2005. Data Wise: A Step-by-Step Guide to Using Assessment Results to Improve Teaching and Learning. Cambridge, m a: Harvard Education Press. Earl, L., and S. Katz. 2006. Leading Schools in a Data-Rich World: Harnessing Data for School Improvement. Thousand Oaks, c a: Corwin. Fichtman, D. N. 2009. With Passion and Knowledge: The Principal as Action Researcher. Thousand Oaks, c a: Corwin. Lai Kuin, M., and K. Schildkamp. 2013. ‘Data-Based Decision Making: An Overview.’ In Data-Based Decision Making in Education: Challenges and Opportunities, edited by K. Schildkamp, M. Kuin Lai, and L. Earl, 49–67. Dordrecht: Springer. MacBeath, J., and A. McGlynn. 2002. Self-Evaluation: What’s in It for Schools? London: RoutledgFalmer. Timperley, H., and L. Earl. 2009. ‘Using Conversations to Make Sense of Evidence: Possibilities and Pitfalls.’ In Professional Learning Conversations: Challenges in Using Evidence for Improvement, edited by L. Earl and H. Timperley, 121–126. Dordrecht: Springer. Materials ☞ Annex . Power Point Presentation 22 Module 4 From Results to School Improvement Objectives The aims of the module are: • To learn about using data in practice for decision-making related to school improvement. • To practice learning walks as a strategy for collecting data and evidence. • To reflect about participants’ experience and challenges related to the learning walks. • To practice planning improvement strategies based on data and evidence. Outcomes of Learning After completion of training the school leader will be more: • aware of challenges and opportunities in the use of data and evidence for decision-making related to school improvement, • knowledgeable in learning walks as a strategy for collecting data and evidence, • skilled in analysing data and evidence collected during the learning walks, • aware of how can school leaders use data in practice. Competences • Being able to help staff to provide explanations of the strengths and weaknesses of their own school, as these appear from the analysis of data. • Being able to help staff to plan quality assurance initiatives or improvement actions on these explanations. • To practice and be able to plan improvement strategies based on data and evidence. Content • Block 1: Preparation for the Learning Walk • Block 2: Learning Walk in Practice • Block 3: Planning for Improvement Time Schedule • 9.00–10.00 Block 1: Preparation for the Learning Walk 10.30–13.30 Block 2: Learning Walk in Practice 15.00–17.00 Block 3: Planning for Improvement Preparation for the Participants • Reading relevant documents about the host school and meeting with the headteacher the day before, organising groups for the learning walk (Block 3, Activity 5 in Module 3). • Headteacher of the host school determines the focus of the learning walk and presents it to participants: 1. What are the target groups involved in the actions? 2. What are the improvement processes? 3. What is the evidence to demonstrate the planned actions? 23 Module 4 4. What are the various forms of evidence? • He/she should ensure that the evidence is available. Preparation for the learning walk The aim of this block is prepare participants for the learning walk and to check whether all Block 1 participants have common understanding of the learning walk protocol. Activity 1: Presentation of Common Features of Learning Walks and 5-Step Approach Time 15 minutes. The trainer refers to the preparatory activity in Module 3. He/she makes sure that all participants understand the focus of the learning walk. He/she presents and explains (Annex 4.3, slide 2, Common Features of Learning Walks, and slide 5). Activity 2: Before the Learning Walk Time 45 minutes. Participants work in groups of 4–6 for 20 minutes. They prepare for the learning walk by discussing questions on slide 6 (Annex 4.3) and items of Protocol 1 (to ensure they understand them), Annex 4.2. In the discussion each group focuses on 1 question, the others contribute to the discussion. Discussion is concluded by presentation of slides 7 and 8 (Annex 4.3) Learning Walk in Practice The aim of this block is to practice learning walk as a method of data collection. Block 2 Activity 3: Meeting with the Head Teacher Time 30 minutes. Participants meet with the headteacher to discuss: • practical issues of the learning walk (where and how they can collect evidence); • where groups will meet to discuss the first impressions and develop questions; • where they can meet teachers and head teacher for posing additional questions; • how feeding back to schools will be organised. Activity 4: Visiting the Classrooms, Collecting Data and Evidence Time 60 minutes. Groups of 4–6 participants visit classrooms, look at displays and other evidence that support the school improvement initiative(s) following the pre-determined schedule. Rounds of 20 minutes are suggested (i.e. visits to 3 classrooms or visit to 2 classrooms and one event, etc.); they look, listen and take notes. Activity 5: Individual Reflection Time 10 minutes. After the learning walk participants reflect on the walk and to collate their observations using Protocol 2 (Annex 4.2). Activity 6: Emerging Questions Time 40 minutes. Participants discuss for 20 minutes their impressions and in items of Protocol 2. The following questions may guide the discussion: • Have we collated the information needed? • What does this mean? • What questions and reflections emerge? Continued on the next page 24 Module 4 Continued from the previous page Questions are written on post-its (1 question on 1 post-it) and displayed. Participants look at questions, merge them where possible and eventually answer some of them among themselves. They decide about the turn of questions. Activity 7: Meeting with Observed Teachers and the Head Teacher Time 40 minutes. Representatives of the learning walk groups report about their first impressions (max 5 minutes each). Teachers and the head teacher listen and give additional explanations if they find it necessary. Participants ask the emerging questions to complete their evidence about school im- provement initiative(s) they focus on. The trainer concludes the activity by informing the school staff of the way they will get feedback and how it should be considered (based on very limited sources of data and little knowledge of the whole context so the suggested strategies can only be understood as one possible way to inform school improvement agenda). Planning for Improvement The aim of this block is to plan school improvement strategies on the basis of data col-Block 3 lected during the learning walk. Activity 8: Planning for Improvement Time 60 minutes. Participants work in the same groups for 30 minutes to inform school improvement agenda based on collated evidence they agree on one (the most relevant) necessary strategy and/or activity. Key ideas are written on the poster and displayed. After 30 minutes the groups are regrouped so that representatives from each group form a new group. They share for 30 minutes what they have discussed in their original (nut) groups, similarities and differences. Activity 9: Key Determinants of the School Improvement Process Time 45 minutes. Participants stay in the same mixed groups. They report about key issues raised in their groups. The trainers comment their reports by explaining the determinants on slide 12 (Annex 4.1) – Key Determinants of School Improvement. Time 15 minutes. Reflection Write on the post-it ticket, the lecturer identifies key positive and negative aspects of the learning walk: • What will you take back to your school from the learning walk? • What did you missed? Further Remarks • Preparation for Day 4 must be incorporated in Day 3. • Documents of the hosting school must be given to participants on Day 1. References and Further Reading Earl, L., and M. Fullan. 2003. ‘Using Data in Leadership for Learning.’ Cambridge Journal of Education 33:383–394. Earl, L., and S. Katz. 2006. Leading Schools in a Data-Rich World: Harnessing Data for School Improvement. Thousand Oaks, c a: Corwin. 25 Module 4 Institute of Educational Leadership. 2013. ‘From Research Policy to Effective Practice: Ideas into Action for School and System Leaders.’ www.education-leadership -ontario.ca Stoll, L., and D. Fink. 1996. Changing our Schools. Buckingham, Philadelphia: Open University Press. Vanhoof, J., K. Vanlommel, S. Thijs, and H. Vanderlocht. 2013. ‘Data Use by Flemish School Principals: Impact of Attitude, Self-Efficacy and External Expectations.’ Educational Studies 40 (1): 48–62. Verbiest, E., and P. Mahieu. 2013. ‘Developing Leadership Capacity for Data-Informed School Improvement: Theoretical Framework.’ http://www.deleca.org Materials ☞ Annex . Protocol  ☞ Annex . Protocol  ☞ Annex . Power Point Presentation 26 Module 5 Creating the Culture of Inquiry Objectives The aims of the module are: • to create an awareness concerning the importance of a culture of inquiry, • to motivate participants to work on a culture of inquiry, • to provide tools to the participants to create a culture of inquiry. Outcomes of Learning After completion of training the school leader will be more: • willing and skilled to help staff to develop an attitude of critical thinking, • willing and skilled to build a culture of academic optimism, • willing and skilled to enhance collective efficacy of staff concerning data use, • willing and skilled to enhance co-operation and shared vision among staff using data. Competences • Being able to support and create a culture of inquiry. • To practice and be able to help staff developing an attitude of critical thinking. • Being able use strategies as reserving judgment, having tolerance for ambiguity, valuing and promoting deep understanding in order to create a culture of inquiry. Content • Block 1: Analysis of Existing Culture of Inquiry, Introduction of the Course Objectives and Plans of Module 5 • Block 2: Importance of Culture of Inquiry and Definition • Block 3: Working on a Culture of Inquiry: From Reflection to Action • Block 4: Sustainability • Block 5: Concluding Principles: Enhancing and Hindering Factors Time Schedule • 9.00–10.00 Block 1 • 10.00–11.00 Block 2 • 11.00–13.00 Block 3 • 14.00–15.00 Block 4 • 15.00–16.00 Block 5 Preparation for the Participants • Familiarization with the theoretical framework and further relevant literature. • Preparation of surveys on paper for participants. 27 Module 5 Analysis of Existing Culture of Inquiry, Introduction of the Course Objectives and Plans of Module 5 Block 1 The aim is to analyse the existing culture of inquiry and to elaborate the content and objectives of the Module. Activity 1: Welcome of Participants, Technical Issues, Introduction of Trainers Time 15 minutes. Activity 2: Analysis of Existing Culture on Inquiry + Course Objectives Time 45 minutes. Step 1 Participants are reminded of the survey (Annex 1.1) that has been developed to measure the existing culture of inquiry. The participants will be asked to collect their forms and define the strength and weakness of their school. Step 2 Participants are invited to share their answers with the other participants. The strengths and weaknesses will be discussed briefly. This will lead to the identification of hindering and promoting factors concerning a culture of inquiry. This input will be used as a starting point for Block 2. Importance of Culture of Inquiry and Definition The aim is to define what we consider to be a culture of inquiry and to provide a theoretical Block 2 framework. Activity 3: A Lecture (+ Individual Filling in of a Template) on the Theoretical Framework Concerning a Culture of Inquiry, Definition and Influencing Factors Time 60 minutes. Content • Definition of culture: ‘The way we do things around here’ – shared beliefs, norms and values (Annex 5.1, slides 2 and 3). • The importance of culture of inquiry: it is more than analysing data! • The culture of inquiry will be related to concepts as ‘policy making capacity’ of schools, knowledge management, and integral management (Annex 5.1, slide 4). • Four approaches of organisational culture starting from 2 criteria (Annex 5.1) Instructions Before these approaches will be discussed more in detail, participants are asked: • To illustrate these approaches with their own experiences. • To translate these approaches into ‘culture of inquiry’ (e.g. from an environmental approach: data-use will be supported by governmental norms; from a metaphoric ap- proach: in primary schools data will be mainly ‘qualitative’ from a system approach: the role of the school leader is crucial; from ideological approach: some schools will use data to influence their stakeholders). School Culture As an Environmental Factor (Template) • What’s the overall culture about data driven school policy (accountability vs. development)? • What are the legal constraints and habits (privacy laws, inspectorate, league tables, . . .)? • What brokers are available (e.g. researchers, central and local governments, school counsellors, . . .)? Continued on the next page 28 Module 5 Continued from the previous page • What data are available (to whom)? • How (good) are teachers and school leaders trained in (aspects of) knowledge management? • What stakeholders are interested in (what) information about the school? School Culture as a Metaphor See block 1 activity 2. School Culture as a System Variable See block 3. School Culture as an Ideology Participants are asked to describe the way recent data-based policy decisions were communicated to the outside world (stakeholders, public, media, . . .) and the underlying use (instrumental, conceptual, symbolic, strategic, . . .) Was there feedback to the school (and consequences of that feedback)? Working on a Culture of Inquiry: From Reflection to Action The aim is: Block 3 • To summarize important principles for principals, to act on in stimulating collective efficacy, co-operation and critical reflection in using data by staff. • To practice those principles in a role-play about staff discussing of data. Activity 4: Role-Play Time 120 minutes. Some Principles • Give positive feed-back • Model behaviour (if possible) • Encourage teachers • Formulate attainable goals • Emphasize a shared vision • Inquire existing mental models • Ask questions that stimulate deep learning • Take different perspectives • Reserve judgement • Tolerate ambiguity Preparation Collect data of a school, i.e. cohort-analysis about a subject, suitable for a vivid discussion. (These data must be country-specific, i.e. data from schools and school types the students are familiar with.) Form a group of 7 people (inner circle, role-players = Group A). The other participants are the outer-group (observers = Group B). Role Descriptions Group A: • Person 1 = school leader. He/she has organised a meeting with the teachers to discuss the results of pupils (cohort analysis, results of tests . . .) He/she will chair the meeting. He/she is asked to demonstrate behaviour that stimulate collective efficacy, co-operation and critical reflection in using data by staff, i.e. demonstrate the principles as mentioned before. Continued on the next page 29 Module 5 Continued from the previous page • Person 2–6 of the inner group the role of teacher. For example, ‘teachers of grades 1–8,’ or ‘teachers belonging to one department’ (depending on the type of data). Group B: • Observers. They will observe if and in which degree the ‘principal’ that stimulate collective efficacy, co-operation and critical reflection in using data by staff, i.e. demonstrate the principles as mentioned before. They can use the observation form in ap- pendix. • Person 1, 5 and 7 observe the occurrence of principles 1–3. • Person 2, 6 and 8 observe the occurrence of principles 4–6. • Person 3, 4, and 9 observe the occurrence of principles 7–10. Time Table • 20’ – Explaining the principles/criteria for a professional learning conversation that stimulate collective efficacy, co-operation and critical reflection in using data by staff. • 10’ – Introduction: explaining the goals and the approach. • 20’ – Preparation by the ‘principal’ and the teachers: study the data. In the mean time: 1. dividing the group of observers according to the different principles, 2. handing over the observation form, 3. study of the data by the observers. • 50’ – Role playing. • 20’ – Feed-back for the ‘principal.’ Sustainability The aim is: Block 4 • to describe main principles concerning sustainability, • to provide a framework for the participants to analyse the existing situation. Time 45 minutes. Participants are asked to judge the relevance of a recent data-based policy decision from a personal and institutional perspective. Concluding Principles: Enhancing and Hindering Factors The aim is to make a summary of the outcomes that will lead to 10 major principles en-Block 5 hancing a culture of inquiry. Activity 5 Time 10 minutes. Participants will be divided in groups of 3 (or pairs) and will be assigned to discuss the outcomes of this training module and write down 10 major principles to create a culture of inquiry. They will be asked to choose a rapporteur in each group. Activity 6 Time 15 minutes. Rapporteurs will briefly present the ‘10 principles to create a culture of inquiry’ of their groups. Activity 7 Time 25 minutes. A discussion of the presented principles will lead to a socially constructed concluding framework with 10 principles to create a culture of inquiry. Continued on the next page 30 Module 5 Continued from the previous page Trainers will complete these answers with insights from research: 1. Factors that stimulate/hinder information use 2. Contextual factors: • Expectations arising from an accountability vs. a developmental perspective • Support offered to schools (No one-size-fits-all model!) 3. School and user-related factors: • Attitude of users regarding information • Academic optimism • Efficacy • Features of a Professional Learning Community (i.e. trust, vision) Activity 8 Time. 10 minutes. Participants will be asked to fill out a feedback form concerning Module 5. Reflection See Block 5. References and Further Reading Argyris, C., and D. Schön. 1996. Organizational Learning 2: Theory, Method and Practice. Reading, m a: Addison-Wesley. Bandura, A. 1997. Self-Efficacy: The Exercise of Control. New York: Freeman. Biesta, G. 2012. Goedonderwijs en de cultuur van het meten: Ethiek, politiek en democratie. The Hague: Boom Lemma. Copland, M. 2003. ‘Leadership of Inquiry: Building and Sustaining Capacity for School Improvement.’ Educational Evaluation and Policy Analysis 25 (4): 375–395. Deci, E. L., and R. M. Ryan. 2000. ‘The ‘What’ and ‘Why’ of Goal Pursuits: Human Needs and the Self-Determination of Behaviour.’ Psychological Inquiry 11:227–268. Earl, L., and M. Fullan. 2003. ‘Using Data in Leadership for Learning.’ Cambridge Journal of Education 33 (3): 383–394. Earl, L., and S. Katz. 2002. Leading Schools in a Data-Rich World. In Second International Handbook of Educational Leadership and Administration, edited by K. Leithwood and P. Hallinger, 1003–1022. Dordrecht: Kluwer. Hoy, W., and C. Miskel. 2001. Educational Administration: Theory, Research and Practice. Boston: McGraw-Hill. Levin, J. A., and A. Datnow. 2012. ‘The Principal Role in Data-Driven Decision Making: Using Case-Study Data to Develop Multi-Mediator Models of Educational Reform.’ School Effectiveness and School Improvement 23 (2): 179–201. Nonaka, I., and H. Takeuchi. 1995. De kenniscreërende onderneming: Hoe Japanse bedri-jven innovatieprocessen in gang zetten. Schiedam: Scriptum. Schildkamp, K., M. Ehren, and M. K. Lai. 2012. ‘Editorial Article for the Special Issue on Data-Based Decision Making Around The World: From Policy to Practice to Results.’ School Effectiveness and School Improvement 23 (2): 123–131. Schildkamp, K., and W. Kuyper. 2010. ‘Data-Informed Curriculum Reform: Which Data, What Purposes, and Promoting and Hindering Factors.’ Teaching and Teacher Edu- cation 26 (3): 482–496. Schildkamp, K., L. T. M. Rekers-Mombarg, and T. J. Harms. 2012. ‘Student Group Differences in Examination Results and Utilization for Policy and School Development.’ School Effectiveness and School Improvement 23 (2): 229–255. 31 Module 5 Senge, P. 1990. The Fifth Discipline: The Art and Practice of the Learning Organisation. New York: Doubleday. Standaert, R. 2014. De becijferde school: Meetcultus en meetcultuur. Leuven and The Hague: Acco. Van Gasse, R., J. Vanhoof, P. Mahieu, and P. Van Petegem. 2015. Informatiegebruik door schoolleiders en leerkrachten. Antwerpen and Apeldoorn: Garant. Vanhoof, J., P. Mahieu, and P. Van Petegem. 2009. ‘Geinformeerde schoolontwikkeling: van een nieuw gegeven naar een beleidsinstrument.’ Kwaliteitszorg in het onderwijs, 17–51. Vanhoof, J., and P. Van Petegem. 2007. ‘Matching Internal and External Evaluation in an Era of Accountability and School Development: Lessons from a Flemish Perspective.’ Studies in Educational Evaluation 33 (2): 101–119. Vanhoof, J., K. Vanlommel, S. Thijs, and H. Vanderlocht. 2013. ‘Data Use by Flemish School Principals: Impact of Attitude, Self-Efficacy and External Expectations.’ Educational Studies 40 (1): 48–62. Visscher, A., and M. Ehren. 2011. De eenvoud en complexiteit van opbrengstgericht werken: Vakgroep onderwijsorganisatie en-management. Twente: Universiteit Twente. Wayman, J. C. 2005. ‘Involving Teachers in Data-Driven Decision Making: Using Com- puter Data Systems to Support Teacher Inquiry and Reflection.’ Journal of Education for Students Placed at Risk 10 (3): 295–308. Weggeman, M. 2000. Kennismanagement: De praktijk. Schiedam: Scriptum. Materials ☞ Annex . Presentation 32 Recommendations First, we would like to recall the general aim of the project: to develop school leadership capacity for evidence-informed school improvement. The first specific objective of the project was to develop a curriculum for a training program for school leaders focused on data-informed school development. This curriculum you are now holding in your hand. This last chapter is more closely related to our second specific objective aiming at finding ways and strategies to implement this curriculum into the training of school leaders. In this last chapter we will present to you some recommendations of how to make best use of the d e l e c a course curriculum. These recommendations build on experiences won within this project, by collecting opinions from all partners throughout the working process and specially the valuable experiences from school leaders and school leader trainers taking part in national and international workshops carried out within the project. The Importance of Putting the Topic of Data-Informed School Leadership into a Broader Management/Leadership Context The curriculum that we have developed is aimed at being part of a more comprehensive training program for school leaders. As a consequence, not all competencies necessary for school leaders will be addressed in this curriculum. We have restricted ourselves to those competencies that are more directly connected to the collecting, analysing, making sense of data and use it to plan actions. For example, a competency like thinking strategically, building and communicating a coherent vision in a range of compelling ways will not be addressed in a direct way in the curriculum. The same goes for a competency such as building a professional learning community. Even when we emphasize the process of organizational learning in using data, the realization of such a competency goes beyond the possibilities we have, as far as we are restricted to developing a part of the curriculum of a five days training course. That does not exclude, however, the notion that (a) in the formulation of competencies we emphasize the process of organizational learning about data use and (b) that the training school leaders to become data literate goes hand in hand with the application of methods of organizational learning. For example, a competency such as the ability to read, understand and interpret data can be reformulated as being able to help staff to read, understand and interpret data. And to develop this competency, applying a process of organizational learning between the school leaders in the course is both possible and desirable (Verbiest and Mahieu 2013). Stress of the Ethical Aspects of Using Data Leading a self-improving school through the routine of gathering, interpretation and use of data is in many ways a challenge. The building of a research capacity of school leaders should involve developing a capacity to critique the trustworthiness of the data and the data collection process and most essentially the awareness of the need to recognise and consider ethical issues at all levels of the process. To foster a data wise school leader includes never losing sight of the student and staff embedded in and related to the data being used. An awareness of the way this is being addressed is crucial. 33 Recommendations Sometime Small is Beautiful One important purpose when preparing the training modules was to develop and pi- lot a large variety of student-active exercises. The project group found that the activities worked well when trialled. This has resulted in an extended range of activities within the modules. Some of these exercises (workshops etc.) can be time consuming, but are important activities which facilitate insights and conclusions. In addition time space was needed for post activity individual and shared reflection. When using the manual be ware not to overload the curriculum. Don’t hesitate to take away activities and make sure to allow time for reflection. A Quick Fix Course or a Process Oriented Training Programme? This training programme can be used for multiple purposes in a variety of contexts. It was primarily developed and piloted to serve as a five-day intensive course for school leaders interested in developing the skills and knowledge to lead data-wise schools. The modules consists of five separate, but connected areas of interest created in a way intended to address appropriate areas of learning for individuals acting leaders in a data-rich school environments. But in addition the modules have been created for use with little adaption in other training contexts. Your own understanding and imagination is the limit! Our recommendation is to consider the advantages of arranging the modules in for example three blocks, 2 days +2 days +1 day and make time space in between the blocks. This would increase the learning opportunities and open for opportunities to relate the issues addressed more directly to the school leaders own school context, involve school staff and make use of authentic data. Also consider the possibilities of developing a university credit course. The curriculum also includes an impressive list of relevant literature (found in the appendices) that can serve as source for reading. Several of the activities can also be transformed into exercise to prepare in between the modules. Training for School Leaders or for Leadership Teams? Even if training needs of school leaders are the focus we are aware of that in many national contexts systematic quality work is also the responsibility of the pedagogical school staff. Also from an organisational learning aspect it is beneficial to involve staff especially in the analysing and improvement processes. Based on experiences from the Belgium context we know that this curriculum can easily be adjusted to a course organized for teams from schools including the school leader. Using Common Prepared Data or Participants’ Own Data? There are several occasions in the de l e c a training modules when participants are supposed to work with data in different ways. There are different possibilities here. Either you as trainer provide external prepared data in a relevant mix of data both quantitative and qualitative data, from different sources preferably input, process as well as output data. In this case data will be decontextualized but also less authentic. Another option is to ask school leaders to bring their own data. The advantage of using data brought by the participants’ themselves is that the data will be authentic and relevant to the school leader and the work that is done within the training can be of direct value for the school leader. If using authentic data make sure to take into account the level of trust and the climate of the group you are working with. Are the participants willing to share their experiences or can this be threatening and unpleasant? Our recommendations from piloting the training modules is that the amount of data need to be restricted. 34 Recommendations And Finally: Don’t Lose the Critical and Developmental Approach Accountability, now at the centre of policy in many countries, forces school leaders to respond to stakeholders and to prove they are effective and responsible working with data. They are expected to become data-driven (Earl and Katz 2006). This training course curriculum addresses the needs of a professional data-wise school leader. That means a school leader needs the capacity to critically judge policies on the use of data in schools (Verbiest and Mahieu 2013). This requires critical reflection on information regarding the policy context of accountability and school development and a focus on learner results and critical reactions to this policy. It also includes recognising the distinction between an external accountability perspective and an internal development and internal accountability perspective, the distinction between and interdependency of data for improving student results or for developing the school; the necessity of intelligent accountability systems, including broad self-evaluation; the importance and relevance of data in developing student learning as well as the development of the school, and the importance of engaging staff in using data for purposes of improvement, with an emphasis on the developmental approach and with the overall objective to create the better education for students. References and Further Reading Earl, L., and S. Katz. 2006. Leading Schools in a Data-Rich World: Harnessing Data for School Improvement. Thousand Oaks, c a: Corwin. Verbiest, E., and P. Mahieu. 2013. ‘Developing Leadership Capacity for Data-Informed School Improvement: Theoretical Framework.’ http://www.deleca.org 35 DELECA Glossary c i p o Acronym standing for ‘context,’ ‘input,’ ‘process’ and ‘output,’ to be used as a framework to categorize information. Culture of inquiry A learning environment within a school that includes attitudes, values, goals, norms of behaviour, and practices, accompanied by an explicit vision for data use by leadership that characterizes a group’s appreciation for the impor- tance and power that data can bring to the decision-making process. Data Facts and figures which relay something specific, but which are not organized in any way and which provide no further information regarding patterns, context, etc. Data-literacy The ability to ask and answer questions about collecting, analysing and making sense of data. Data wise leadership: leadership encompassing data literacy. Data wise leadership Leadership encompassing data literacy. Evidence Information that verifies effective practice. Information: data that is processed to be useful, providing answers to the questions of ‘who,’ ‘what,’ ‘where,’ and ‘when.’ Feedback The return of information about the results of a process or activity. Feed forward An anticipatory response to expect changes of a process or activity. Information Data that is processed to be useful, providing answers to the questions of ‘who,’ ‘what,’ ‘where,’ and ‘when.’ Knowledge The application of data and information and provides answers to ‘how’ questions. Knowledge management The process of capturing, developing, sharing, and effectively using organizational information. Learning walk A brief, structured, non-evaluative classroom observation by the principal or peers that is followed y by a conversation between the principal or peers and the teacher about what was observed. Reliability The consistency or repeatability of research measures. Stages of evidence-informed practice Consecutive steps in the process of arriving at a correct interpretation of data. Validity The extent to which a concept, conclusion or measurement is well-founded and corresponds accurately to the real world. 36 Document Outline Developing Leadership Capacity for Data-Informed School Improvement Title Page Contents Introduction 1 Meaning and Purpose of Working with Data 2 Data Collection: Assessing and Searching for Data and Evidence 3 Analysis and Interpretation: Discuss Alternative Perspectives 4 From Results to School Improvement 5 Creating the Culture of Inquiry Recommendations DELECA Glossary