c e p s Journal | Vol.8 | No1 | Year 2018 9 Professionalising Physics Teachers in Doing Experimental Work Claudia Haagen-Schützenhöfer*1 and Birgit Joham2 • It is commonly agreed that experiments play a central role in teaching and learning physics. Recently, Inquiry-Based Learning (IBL) has been intro- duced into science teaching in many countries, thus giving another boost for experiments. From a didactical point of view, experiments can serve a number of different goals in teaching and learning physics. First of all, experiments can support learners in understanding some of the central concepts of physics. Besides this function of “learning physics”, empirical evidence shows that experimental work in general has a high potential for promoting “learning about science” and finally “doing science”. Promoting aspects of how science works has become important, as the ideas of scien- tific literacy and competence orientation have been established as central educational goals in many national education systems. However, empiri- cal studies show that the reality in schools does not match these expecta- tions. Conventional physics classes still aim only at the mastery of content, and experiments that cognitively activate students and address issues re- lated to the Nature of Science (NOS) have not been implemented exten- sively. The reasons for this can be found in teachers’ attitudes and beliefs, as well as in their PCK concerning experiments and scientific knowledge production. In past decades in Austria, teacher education did not focus a great deal on the didactical aspects of experiments or their integration into physics classes in order to promote aspects of scientific literacy and competence orientation. Furthermore, there is a lack of high quality con- tinuing professional development courses that promote the concepts of Inquiry-Based Learning (IBL) in combination with relevant ideas of NOS. The present study examines inservice teachers’ beliefs about the function of experiments in science teaching and their meaningful integration into science classes. In the form of case studies, we follow the professional de- velopment of teachers in this field during continuing teacher training. Keywords: experiments in science teaching, continuous professional development course, Inquiry-Based Learning 1 *Corresponding Author. University of Graz, Institute of Physics, Austria; claudia.haagen@uni-graz.at. 2 KLEX – Klusemann Extern, Austria. focus doi: 10.26529/cepsj.333 10 professionalising physics teachers in doing experimental work Profesionalizacija učiteljev fizike v izvajanju eksperimentalnega dela Claudia Haagen-Schützenhöfer in Birgit Joham • Pogosto se strinjamo, da eksperimenti igrajo osrednjo vlogo v poučevanju in učenju fizike. Pred kratkim so učenje z raziskovanjem v veliko državah vpeljali v poučevanje naravoslovja, kar je dalo eksperi- mentiranju nov zagon. Z didaktičnega vidika lahko poskusi služijo vrsti različnih ciljev v poučevanju in učenju fizike. Prvič, poskusi lahko pod- pirajo učence pri razumevanju osrednjih fizikalnih konceptov. Poleg te funkcije »učenja fizike« empirični podatki kažejo, da ima eksperimental- no delo na splošno visok potencial za promocijo »učenja o naravoslovju« in ne nazadnje za »izvajanje naravoslovja«. Promocija tega, kako deluje znanost, je postala pomembna, saj je ideja naravoslovne pismenosti in razvoja naravoslovnih kompetenc postala osrednji izobraževalni cilj v veliko nacionalnih izobraževalnih sistemih. Empirične raziskave pa kažejo, da se realnost v šolah ne sklada s tem. Konvencionalni pouk fizike še vedno temelji na obvladovanju učne vsebine, medtem ko poskusi, ki kognitivno aktivirajo učence in naslavljajo zadeve, povezane z naravo naravoslovja, še niso pogosto implementirani. Razloge za to lahko najdemo v stališčih in prepričanjih učiteljev pa tudi v njihovem pedagoško vsebinskem znanju, ki vključuje poskuse in naravoslovno znanje. V zadnjih desetletjih se izobraževanje učiteljev v Avstriji ni os- redinjalo na didaktične vidike poskusov ali njihovo vključevanje v pouk fizike z namenom promocije naravoslovne pismenosti in naravoslovnih kompetenc. Še več, gre za pomanjkanje visokokakovostnih programov za stalno strokovno spopolnjevanje učiteljev, ki promovirajo učenje z raziskovanjem v kombinaciji z relevantnimi idejami narave nara- voslovja. Ta raziskava preučuje prepričanja učiteljev o vlogi poskusov v poučevanju naravoslovja in njihovo smiselno integracijo v pouk fizike. V obliki študije primera sledimo profesionalnemu razvoju učiteljev na področju stalnega strokovnega spopolnjevanja. Ključne besede: poskusi v poučevanju naravoslovja, programi za stalno strokovno spopolnjevanje, učenje z raziskovanjem c e p s Journal | Vol.8 | No1 | Year 2018 11 Introduction: Motivation and Starting Point Experiments and practical work play a central role in science educa- tion. In general, both teachers and students have a very positive attitude to- wards practical work: they “like doing experiments”. The reasons and aims of the two groups are, however, different, as are the perspectives about what can be achieved in terms of affective and cognitive student variables. While, from a science education perspective, experiments can contribute to a variety of facets of science learning, they are frequently implemented only for a limited number of aims in everyday science classes. A new focus on experiments has been introduced by Inquiry-Based Learning (IBL), which has become very popular in the last decade. In Austria, many engaged teachers are implementing IBL environments in their classes. In addition, at schools, the number of newly established science labs that are informed by the idea of IBL is growing at all age levels. At the same time, it is known that preservice training in Austria generally does not put a lot of effort into achieving a differentiated view of the use of experiments in science classes. The belief that experiments, irrespective of how they are implemented in learn- ing environments, facilitate understanding of subject matter and raise levels of interest is very common among science teachers (Haagen & Mayer, 2015). As far as IBL is concerned, this method is only now being implemented in science teacher education in Austria. In contrast to the situation on the level of teacher education, national standards have been designed on the level of students’ learning outcomes, and the idea of experimental work and inquiry has been introduced into our na- tional competence models for secondary science education. There is therefore a clear gap between what teachers learn during their preservice training and the requirements of the national competence models for secondary education in science. The continuous profession development programme “Competences in Mathematics and Science Education” (CMSE) is one of several actions taken by the Ministry of Education to support teachers in adapting their teaching to the requirements of the national standards. The present paper provides an insight into the development of science teachers’ beliefs about the implementation of Inquiry-Based Learning and practical work during their participation in CSME. 12 professionalising physics teachers in doing experimental work Standardisation in Science – A New Impetus for Experimental Work Like in many other European countries, Austria implemented national standards and competence models after achieving poor results in PISA and TIMSS. The medium-term aim is to improve the quality of teaching and learn- ing by shifting instructional practices from an input to an output orientation, and from a transmissive view of teaching to a constructive one. The Austrian education system is organised into a primary level (four years), a lower secondary level (four years) and an upper secondary level, which ends with A-levels (four or five years, depending on the school type). In addi- tion, there are other types of upper secondary education that do not end with A-levels and thus do not qualify students for direct access to university. As far as science instruction is concerned, the subjects Biology, Chemistry and Phys- ics are taught separately in Austrian secondary schools. In primary schools, we have the subject “Sachunterricht” which combines science and humanities such as local history and geography, and social learning. In general, competence models for all subjects were developed for year 8 and later year 12 (Haagen & Hopf, 2012; Weiglhofer, 2008). For science subjects, a common model was developed based on the construct of scientific literacy used in the PISA 2006 framework (Bybee, McCrae, & Laurie, 2009), as well as on existing competence models of other countries. The models for the subjects Biology, Chemistry and Physics (see Fig. 1) differ only on the subject matter di- mension, whereas competence domains and complexity levels are identical. On the level of primary education, so far, standards have only been implemented for the core subjects Mathematics and German. The Austrian competence model for Science year 8 (see Fig. 1), consists of three dimensions (axes): content, complexity and competence domains. The competence domains are subdivided into three facets, reflecting the core ideas of scientific literacy: • Knowledge meaning “Scientific knowledge and use of that knowledge […] to acquire new knowledge [and] to explain scientific phenomenon”. • Science Methods as “understanding of the characteristic features of sci- ence as a form of human knowledge and enquiry” as well as the ability “to identify [scientific] questions” and answer them with the help of inquiry. • Judgement describing “[the] willingness to engage in science-related issues, and with the ideas of science, as a constructive, concerned, and reflective citizen to draw evidence-based conclusions about science-re- lated issues” (OECD, 2006, in Bybee et al., 2009) c e p s Journal | Vol.8 | No1 | Year 2018 13 More details concerning the Austrian competence model can be found in Haagen et al. (2012) and Weiglhofer (2008). Figure 1. The Austrian competence model for Science, year 8. Inservice Teacher Professionalisation in Austria In Austria, inservice teacher training is not compulsory, so only one third of teachers attend trainings on a regular basis, while one third attend oc- casionally and one third do not attend training courses at all. What makes the situation even worse is that typical inservice trainings are very short – lasting for half a day or a day – and mostly focus on subject matter only. According to numerous research findings, effective professionalisation means to change teaching practices. Such a change is more likely to be achieved in programmes that enable activities of longer duration, that integrate subject matter, pedagogy and teaching strategies, and that include practice experiences that can be re- flected on (cf. Garet et al., 2001). Competences in Mathematics and Science Education (CMSE) is a na- tional Continuous Professional Development Programme (CPD) that supports science and mathematics teachers from different school types in implementing teaching innovations linked to the introduction of subject-specific competenc- es and standards in the Austrian education system. CMSE is one of five thematic teacher professionalisation programmes within the IMST framework (Innovations in Mathematics, Science and Tech- nology Teaching), which is another initiative launched by the Ministry of Edu- cation in 2010, after the PISA shock. The core idea of CMSE is to simultane- ously intervene on the level of the teacher and the student. For one school year, 14 professionalising physics teachers in doing experimental work teachers work together with teacher trainers and science education researchers, who help them to address the concept of subject-specific competences in their teaching (Langer, Mathelitsch, & Rechberger, 2014). The framework of CMSE was designed to initiate professional learning communities among the participating teachers. The aim is to support them to shift their teaching practice from input orientation to output orientation by integrating the concept of subject-specific competencies in their instructional practice (Haagen-Schützenhöfer et al., 2015). A main focus is the integration of experimental work. Theoretical Framework Research on Experiments and Practical Work in Science Teaching and Learning It is undisputed that experiments are an essential part of science teach- ing and learning. Their contribution can be seen on at least three levels, as sum- marised by Hodson (2014): learning science, learning about science, and doing science. Existing research results regarding the effectiveness of practical work and experiments in science teaching are heterogeneous: they do not confirm that experiments generally enhance the quality and effectiveness of science teaching (Lunetta, Hofstein, & Clough, 2007; Singer, Hilton, & Schweingru- ber, 2006). Research shows that, in many cases, there is a significant conflict between the aims teachers attribute to the implementation of experiments and the way in which experiments are implemented in science classes. According to the results of America’s Lab Report (2005), the most prominent motives for integrating practical work and experiments into science instruction are: • enhancing mastery of subject matter; • developing scientific reasoning; • understanding the complexity and ambiguity of empirical work; • developing practical skills; • understanding the nature of science; • cultivating interest in science and interest in learning science; and • developing teamwork abilities (Singer et al., 2006, p. 3). This list is long and undoubtedly incomplete; nevertheless, the motives followed in everyday school reality are usually very limited and centred on the mastery of subject matter. Even though teachers intend to attain all of the c e p s Journal | Vol.8 | No1 | Year 2018 15 desirable goals summarised by Singer et al. (2006), data indicate (Lunetta et al., 2007; Singer et al., 2006) that they are not successful in providing appropriate practical experiences with the kind of learning environments currently in use. Typical learning environments involve students following rigid proce- dures, but fail to integrate reflection or discussion (Lunetta et al., 2007; Maltese, Tai, & Sadler, 2010; Millar & Abrahams, 2009). Frequently, practical activities are used to verify or apply rules that are already part of instruction. In addition, they tend to be quite “tightly constrained” (“cookbook” or “recipe following” practical tasks) (Millar & Abrahams, 2009, p. 62), mainly focusing on procedures. The focus of practical work is “manipulating equipment [rather than] ma- nipulating ideas” (Hofstein & Lunetta, 2004, p. 39). This supports the develop- ment of manipulation abilities instead of establishing solid scientific concepts. Students are trained to aim at task completion as a major goal, while reflective processes are neglected. One reason seems to be that reflective phases are fre- quently regarded as too time consuming. In addition, some papers (Hart, Mul- hall, Berry, Loughran, & Gunstone, 2000) report that tasks are rather complex and may result in a cognitive overflow, as students have to perform numerous tasks simultaneously. Another problem area identified by several authors (Lu- netta et al., 2007; Singer et al., 2006) is the lack of integration of practical activities into general instruction. In many cases, hardly any relationship is established be- tween the experiment carried out and its theoretical background. Consequently, students lack the appropriate conceptual frameworks that help them to adequate- ly integrate the experiences acquired during practical work (Driver, 1983). Research data show this clash between the intended goals and the general reality of practical work. A survey of the existing research (Lunetta et al., 2007) on practical work yields a variety of outcomes. However, widespread beliefs that practical activities automatically improve student achievement – especially the mastery of subject matter – cannot be supported empirically. Americas’ Lab Re- port concludes that “Laboratory experiences have the potential to help students […], [but] [t]he potential is not being realized today” (Singer et al., 2006, p. 9). The idea of IBL is seen as a way out of this unsatisfactory situation. It may help to shift the focus from a hands-on attitude aimed at task completion and manipulating equipment, to a minds-on attitude. The method of IBL and relevant research results will be discussed in the following section. Inquiry-Based Learning (IBL): Models and Research Results As in many other countries, Inquiry-Based Learning has become a ma- jor trend in Austria in recent years. However, IBL is defined in different ways, 16 professionalising physics teachers in doing experimental work and it can barely be separated from other open methods of instruction (Min- ner et al., 2010). In addition, there is a second dimension that is independent of normative definitions but influences instruction: how teachers interpret the idea of IBL on a personal level and, consequently, how IBL is implemented in the classroom depending on this individual perspective of the teacher. For our work, we concentrated on the essential features extracted from the definitions of IBL used in NRC 2000 and NRC 1996 (National Research Council, 1996; National Research Council (NRC), 2000). Characteristic of IBL scenarios is that students: • are “engaged by scientifically oriented questions; • […] give priority to evidence, which allows them to develop and evalua- te explanations that address scientifically oriented questions; • […] formulate explanations from evidence to address scientifically ori- ented questions; • […] evaluate their explanations in light of alternative explanations, par- ticularly those reflecting scientific understanding; • […] communicate and justify their proposed explanations” (cited from: Pathway UK 2013). These features of IBL are well matched by the competence facets defined in the Austrian competence model for secondary science, as discussed above. Within the method of Inquiry-Based Learning, a number of subvarieties can be identified. For the professionalising processes, we focus mainly on the di- mension of openness. This aspect is well differentiated in the model of Blanchard et al. (2010), who define levels of IBL based on the distribution of responsibilities between teachers and students during the three phases of the IBL process: Source of the question Data collection methods Interpretation of results Level 0: verification teacher teacher teacher Level 1: structured teacher teacher student Level 2: guided teacher student student Level 3: open student student student Figure 2. Levels of Inquiry-Based Learning (Blanchard et al., 2010). What distinguishes IBL as defined by Blanchard et al. (2010) from mere c e p s Journal | Vol.8 | No1 | Year 2018 17 exploration is that a concrete and researchable question is always the starting point of the practical student activity. Data is systematically collected and the methods of data collection are aligned to the initial questions. Consequently, this model of IBL covers all goal-oriented and result-targeted experimental stu- dent activities. Mere exploration without a defined knowledge interest (ques- tion), as well as experimental demonstrations of phenomena (which usually lack data collection methods), are excluded from this definition. Blanchard’s model of IBL therefore fits our needs well, as it represents a large variety of stu- dent activities that support the development of experimental competences as defined in the Austrian competence model for secondary science. The levels of Inquiry-Based Learning represent a good basis for differentiating experimental student activities. In addition, they can support teachers in structuring experi- mental activities according to students’ pre-knowledge and skills. When we shift our focus to the output side of IBL in terms of learning processes, empirical evidence is, however, heterogeneous. One reason may be that a wide range of activities are labelled as IBL, which, of course, has a nega- tive effect on comparability. A common point of many studies seems to be that the level of guidance, especially when first commencing IBL, is crucial for the effectiveness of student learning and retention (Hattie, 2013; Kirschner, Sweller, & Clark, 2006; Minner et al., 2010). A slow progression from close guidance to more open scenarios seems to be advisable. In addition, a basic but solid knowledge base of the subject matter and experimental skills are necessary for students to be able to engage cognitively in more open forms of inquiry without being overtaxed. As reasons for this, Kirschner et al. (2006) mention “expert– novice differences, and cognitive load”. As it is undisputed that students need to develop certain abilities before they are able to carry out experimental work in the form of open inquiry, in the present paper we focus, inter alia, on this issue in our evaluation of the CPD course. Design and Methods Participants in CMSE – The Sample CMSE participants represent a selected sample. It can therefore be as- sumed that they belong to the more active, innovative and informed group of teachers, as they had to apply for the CMSE programme by submitting a pro- posal in which they outlined a school project aimed at implementing a teach- ing innovation related to subject-specific competences. Their submissions were 18 professionalising physics teachers in doing experimental work reviewed by external education experts and by CMSE staff. Only 20 projects are accepted for the programme each year, while the number of applications is typically around 45. The sample of our study consists of a total of 39 teachers who were se- lected for the CMSE programme in the 2015/16 and 2016/17 school years. The participants of CMSE teach science subjects and mathematics at different types of schools and at different age levels, from primary to upper secondary. Our sample of 39 teachers consists of two cohorts: one participated in CMSE in the 2015/16 school year, and the other in the 2016/17 school year. The 2015 cohort consists of 20 teachers and the 2016 cohort of 19 teachers. Out of the full sample (N = 39), 77% of the teachers are female and 23% male. Some 36% of the participants are primary teachers, who teach children aged between 6 and 10 years, while 64% of the sample are secondary teachers who teach students aged between 10 and 19 years. On entering CMSE, the majority of the teachers (39%) had more than 10 years of teaching experience, 37% had between 5 and 10 years of teaching experience, and 24% had less than 5 years of teaching experience. Research Questions Our guiding research questions can be summarised as: • RQ 1: What beliefs do the inservice teachers of the CMSE programme have regarding experimental work and IBL in general when they enter CMSE? • RQ 2: Do the inservice teachers of the CMSE programme categorise their school projects as Inquiry-Based Learning when they enter CSME? • RQ 3: Does their view about their previous categorisation change during their participation in CSME? • RQ 4: What beliefs do the inservice teachers of the CMSE programme have about the Nature of Science aspects of Inquiry at the end of the programme? • RQ 5: What beliefs do the inservice teachers of the CMSE programme have about the characteristics of effective experimental work and Inqui- ry-Based Learning at the end of the programme? Interventions – The CPD Programme CMSE The CMSE programme lasts for one year and supports 20 school pro- jects. After succeeding in the application phase for the programme, all CMSE c e p s Journal | Vol.8 | No1 | Year 2018 19 participants meet for the first time at the start-up workshop (cf. Fig. 3) at the beginning of the school year. The aim of this start-up is to make participants familiar with the ideas of competence orientation and to provide them with new impulses for their projects. In the more general part, we treat organisa- tional issues concerning project management, followed by inputs on subject- specific competences and practical work with a focus on Inquiry-Based Learn- ing. CMSE team members then work with the participants on their individual project aims, fine-tuning them and deducing a first rough set of interventions and evaluation strategies. The second day of the start-up is dedicated to the formation of focus groups (FG), which are proposed by the CMSE team. Each focus group consists of four to five participants and is coached by two coaches. In the focus group, each participant presents his/her current working version of the project, fo- cusing on project aims, corresponding instructional measures and initial ideas about evaluation. Within this session, the participants get feedback and advice from the focus group coaches and the other participants. Figure 3. Intervention – the format of the professional development programme CMSE (cf. Haagen et al., 2017). As a tool for experimental tasks, we introduce the spider-web model by Schecker et al. (2013), as shown in Figure 4. The axes of the spider-web represent 20 professionalising physics teachers in doing experimental work different experimental competence facets that students should develop: inquiry competences (e.g., develop questions, hypothesise) and experimental skills. The experimental skills mirror the sub-facets of experimental processes, which are typically divided into three phases of experimenting: preparation (e.g., plan- ning experimental procedures), performance (e.g., setting up the apparatus) and evaluation (e.g., interpreting results) (Schreiber et al., 2016). Figure 4. The Spider-Web tool for experimental student activities, adapted from Schecker et al. (2013). The scale can be used to assess an experimental task (0: not part of the task, 1: part of the task but not emphasised, 2: focus of the task). It can also be used to assess student competences (0: competence facet not shown, 1: competence partly shown, 2: competence shown to a high degree). The spider-web can be used for different purposes. It can help to analyse and/or plan experimental tasks, but it can also be used for assessing students’ experimental competences. Finally, as a self-assessment tool, it can support stu- dents in judging their own experimental competences. In our CPD programme, the spider-web is intended to help participants to analyse their projects and identify possible shortcomings related to certain experimental competence facets. In addition, they are encouraged to use the spider-web as a basis for the design of their own learning environments. An- other important step in the start-up workshop is the didactical analysis of the projects. In their focus groups, the participants work on the alignment of goals, intended learning processes and the design of appropriate interventions. After the start-up workshop, the teachers are supported by their fo- cus groups in the implementation of their project. The focus group func- tions as a “critical friend”, with members supporting each other with project c e p s Journal | Vol.8 | No1 | Year 2018 21 implementation and reflection work. The focus groups operate in different modes: there are interim face-to-face meetings, materials and interim reports are exchanged or participants visit each other in their schools. The final phase of the CMSE year is dedicated to the project report (see. Fig. 2), which is more or less a portfolio portraying the evolution of the innova- tive project. It describes the project starting from the teacher’s motivation, the aims pursued by the innovative school project, the learning objectives on the level of students, and the interventions carried out, as well as the design of the evaluation and its results. The process of writing the report is supported by the implementation of various scaffolding strategies during the project year. The start-up workshop is, for example, devoted to generating the first part of the re- port, which specifies the intended learning outcomes. From these learning out- comes, interventions and evaluation strategies are deduced. Each of these steps is documented during the individual phases of the project. CMSE participants are also supported in finalising the project report, meeting with their coaches for three days in April. There, the participants get specific input on data analysis and academic writing. However, most of the time of this workshop is dedicated to individual counselling and support. At this stage, professional communities play a crucial role. Participants support each other by reading their drafts and giving feedback as critical friends. Within this process, the intended goals are contrasted with the evaluation results. Thus, the participants get an opportunity to reflect on the output of the project and their individual teaching practice, again with the input and help of their colleagues and coaches. Evaluation The evaluation of the professionalisation processes with a focus on ex- perimental work and Inquiry-Based Learning was conducted with a pre-post design that uses two different data sources: written documents produced in the course of the project by the teachers, and answers gathered by questionnaires (see Fig. 5). Data of each type is collected before the actual implementation of the school projects and after the completion of the projects: the project pro- posals produced by the teachers for their application for CMSE, and the final project report. The application form for the CMSE programme contains sections con- cerning the participants’ innovative projects and, among other issues, focuses on their intended goals related to students’ subject-specific competences, stu- dents’ learning outcomes, instructional interventions, and evaluation strate- gies. At the end of the project year, a final project report is required in order to 22 professionalising physics teachers in doing experimental work complete the CMSE programme. This report develops during the project year and serves also as tool for structuring and reflecting on the participants’ pro- fessionalisation process. The structure given in the template of the final report corresponds partly to the structure of the application form. Thus, it is possible to extract data from one document that is produced before the start of the pro- ject – the project proposal – and from one that portrays the final development stage at the end of the project – the final project report (see Fig. 5). The follow- ing points are included in the application and/or the final report, and serve as a data corpus for our analysis: • analysis of the status quo of the individual teaching and possible areas for improvement/innovation (application & final report); • goals on the level of students and teachers, in order to improve the issues identified in the first point (application & final report); • development of innovative interventions to achieve the set goals (appli- cation & final report); • implementation of the teaching innovation (final report); • evaluation of the teaching innovation (final report). In order to place special emphasis on practical work and Inquiry-Based Learning, questionnaires were used as a second data source on the level of teacher professionalisation. They were administered at the start-up meeting and in a similar form at the end of the project year (see Fig. 5). The question- naires contained open questions as well as multiple choice questions on the following topics: • students’ competences in the context of science education; • the Austrian competence model for science subjects; • competence domains in science according to the Austrian competence model; • characteristics of Inquiry-Based Learning in science subjects; • levels/types of Inquiry-Based Learning and their implementation. c e p s Journal | Vol.8 | No1 | Year 2018 23 Figure 5. Research design and data collection (Haagen et al., 2017). Qualitative and quantitative methods were used for data analysis: statis- tical frequency analysis with SPSS was used for the demographic data as well as for the multiple choice questions in the questionnaires. Qualitative content analysis (Mayring, 2014) was used for the open questions of the questionnaires as well as for the written documents (project proposals and final reports). We worked with the free online software QCAmap (https://www.qcamap.org/). Categories were built deductively based on research on practical work and on models of IBL taken from research literature, as discussed in section II of the present paper (Theoretical Framework). These deductively generated categories did not portray the full data material, so it was necessary to extend the categories inductively. Results The general beliefs of CMSE participants about experimental work and IBL were collected with help of a pre-questionnaire. These findings were trian- gulated with the project proposals. Figure 6 shows the most important motives of the participants to use IBL when entering CMSE. 24 professionalising physics teachers in doing experimental work Figure 6. Motives identified by participants for using the IBL method. It can be clearly seen that subject-matter knowledge is the most impor- tant function of IBL, followed by motivation, a learning by doing approach and individualisation in learning. The acquisition of experimental skills, the devel- opment of scientific reasoning skills, or gaining knowledge about how science works are not mentioned. When participants were asked about the added value of IBL compared to other teaching methods, experimental skills (23.5%) were mentioned in the first place and inquiry competences (8.8%) were listed by at least a small mi- nority. In addition, critical thinking (5.9%) and the development of problem- solving abilities (8.8%) play a role for some participants. Again, categories such as NOS aspects or how science works are neglected. Figure 7. Possible negative effects of IBL for the learning process. Absolute num- bers are given per category. c e p s Journal | Vol.8 | No1 | Year 2018 25 As far as negative effects for students’ learning processes are concerned, the largest group of teachers (6) did not answer this question or denied nega- tive effects (6). Another six teachers stated that students need well-structured learning environments for effective learning. Obviously, they related IBL only to unstructured open scenarios, or even to exploration. Other arguments ad- dressed organisational issues, e.g., that IBL is very time consuming (4) or causes additional workload for teachers (3). Four participants mentioned that different students might profit differently in terms of knowledge gain, while another four thought that students might get on the wrong track without guidance. Another three teachers stated that not all topics are suitable for IBL. Learning about the Nature of Science was again not explicitly mentioned by any participants. Teachers’ knowledge about different phases or elements of Inquiry- Based Learning were investigated. Only two-thirds of the participants were able to name the different phases or elements of Inquiry-Based Learning. Figure 8 shows that the focus is clearly on observing (or collecting data) and docu- mentation, as well as on setting up equipment, that is, on hands-on elements. Discussion was also frequently mentioned, although analysis or interpretation of data was, in most cases, not mentioned explicitly. From the descriptions, it could be deduced that, in most cases, experiences and observations should be discussed rather than data and interpretations deduced from data. The phase of developing or posing questions often seems to be part of experimental work. The descriptions of these phases suggest that the questions given are typically not research questions in the narrow sense, but rather focus either on organisational issues or observables (“Does the piece sink or float?”). Figure 8. Focus on different phases or elements of Inquiry-Based Learning. 26 professionalising physics teachers in doing experimental work Another clue as to how diverse teachers interpret IBL can be seen from the categorisation of their projects concerning the use of IBL. When, during the entrance phase, participants were asked whether they used IBL in their projects, 84.6% categorised their project as inquiry-based. The same question was part of the post-questionnaire, at which point only 65.5% categorised their project as inquiry based. This effect is also in line with the analysis of project proposals and final-reports. Participants had obviously had a very vague idea of IBL: when entering the programme, they subsumed almost any experiment planned for the project as Inquiry-Based Learning. At the end of the course, however, the participants seemed to be much more aware of the concept and phases of IBL, and they consequently used the term in more reflected way. Analysis of the data collected at the end of CMSE shows that the view on Inquiry-Based Learning had become more differentiated. Figures 9, 10 and 11 show the results of an item complex using a four-point Likert scale, where 1 means that the feature described is not prototypical of IBL, while 4 denotes that it is very prototypical for IBL. Figure 9 shows that teachers still emphasise the hands-on character of IBL. On the one hand, they categorise the inquiry process itself as open to multiple solutions but, at the same time, they restrict inquiry to right and wrong in terms of experimental procedure. Together with the emphasis on targeted research questions and hypothesising, this implies that, at this stage, the participants differentiate more clearly between inquiry and exploration than at the entrance phase. Figure 9. Characteristics of IBL as a method (perspective of participants, 1 = not prototypical; 4 = very prototypical). As far as learning processes connected with IBL are concerned, Figure 10 shows that the initial belief that IBL works through a “learning by doing” ap- proach (cf. Fig. 6) is still strong. In addition, content knowledge is still not seen c e p s Journal | Vol.8 | No1 | Year 2018 27 as a prerequisite for successful learning through IBL. On the other hand, the idea that learning processes in IBL are not pre-structured is disappearing, while the role of the teacher – who is not longer only in the background – is empha- sised. Furthermore, the idea that IBL is highly typical of knowledge acquisition is no longer very strong. Figure 10. Prototypical aspects of learning processes in IBL (perspective of par- ticipants, 1 = not prototypical; 4 = very prototypical). Finally, the aspect of how science works was analysed (see Fig. 11). The idea that students can act as scientists has been relativised. However, we obtain more ambiguous results regarding the characteristics of research. The partici- pants do not view research as a typically targeted and systematic endeavour. On the other hand, they do not judge research as trial and error, either. In contrast to the entrance phase, the idea that IBL can support students to learn about the Nature of Science is present at the end of the CMSE programme, although there is still room for improvement. Figure 11. Aspects of IBL concerning NOS (perspective of participants, 1 = not prototypical; 4 = very prototypical). 28 professionalising physics teachers in doing experimental work Disscusion The evaluation of our continuous professionalisation programme CMSE showed that the teachers in our sample are motivated by the national science standards to shift their teaching practice from input-oriented, teacher-centred teaching practices to more student-focused ones. A new impetus for experi- mental work can be observed, and Inquiry-Based Learning is a major trend among our sample. However, analysis of the data shows that ideas about the added value of experimental work and inquiry, and about their effective imple- mentation, are very vague. When entering the CPD programme, the motives for implementing IBL focus mostly on subject-matter acquisition and motivational issues. It is inter- esting that several motives for practical work and IBL known from literature (cf. section II of the present paper) were not mentioned at this stage: for exam- ple, the acquisition of experimental skills, the development of scientific reason- ing skills, or gaining knowledge about how science works. Before the CPD programme, the majority of the teachers viewed the added value of experimental work as being in student activity, learning by do- ing and the acquisition of subject-matter knowledge. Distinctions between tar- geted, science-oriented student activities such as IBL and the mere exploration or demonstration of phenomena are frequently not made. When we contrast these results with the definition of IBL used by the NRC (see section II.2 of the present paper), for example, we see that there is much more focus on experi- mental skills than on collecting evidence to “develop and evaluate explanations that address scientifically oriented questions” (NRC, 2000, 2006). Cognitive abilities such as “critical thinking” and “problem solving” are only mentioned as added value of IBL by a small minority of participants. The idea of “evidence- based reasoning” is not explicitly mentioned in either the questionnaires or the project reports. This can be seen as an indication that many teachers may have a very unclear picture about the potential of practical work and IBL, a result that is in line with other research results, such as those discussed above by Singer et al. (2006). When entering CMSE, the sample’s view of the restrictions of inquiry and potential learning difficulties is very limited. It is astonishing that in both the pre-questionnaire and the project proposals there was no mention of IBL as a learning opportunity for aspects of NOS or for investigating how science works. In addition, it is very interesting that teachers, when asked about the possible negative effects of IBL on learning processes, use arguments on the c e p s Journal | Vol.8 | No1 | Year 2018 29 organisational level (e.g., “time consuming”, “additional workload for teach- ers”). We can infer from this that some teachers tend to plan their lesson activi- ties based on organisational aspects rather than focusing on learning processes. This impression was frequently confirmed in draft versions of the project re- ports discussed in our workshops. The participants’ lesson planning was guided by organisational aspects or teaching methods rather than by student learning. Hardly any of their first drafts on interventions were inspired by a causal se- quence of steps such as: intended learning goals / learning outcomes – students’ prerequisites / needs / interests – interventions operationalising learning goals and considering students’ prerequisites. Furthermore, there was a great deal of resistance among some of the participants to use any form of planning tool that follows the idea of didactical reconstruction, or to use planning processes starting from specific learning goals and student prerequisites. Such planning process were frequently seen as too time consuming and not helpful at all. Such attitudes and beliefs reflect a still very input-oriented view of education, which is not in line with competence orientation, but which seems to be deep rooted in many teachers. As far as practical work is concerned, it was found that such an atti- tude is often accompanied by the use of cook-book style experiments, focusing mainly on content knowledge and taking into account only a few experimental competences facets, such as setting up apparatus, and collecting and docu- menting data. In the majority of cases, all other experimental competence fac- ets, as described by Schecker et al. (2013) (see section III.3 of the present paper), were neglected at the beginning of the CPD. To sum up, when entering CMSE, the sample’s view of the restrictions of inquiry and the potential learning difficulties is very limited. The same is true of the differentiated perspective on the added value of practical work and IBL. After the professional development course, the teachers’ beliefs had only changed in certain aspects. It seems that some beliefs are very deep rooted, and that our participants tend to return to old, familiar patterns. On the oth- er hand, the positive effects of the CPD course are that the participants get a much broader view of experimental competences and how they can be imple- mented in their teaching. They are much more likely to plan interventions that cover more experimental competence facets than observing, collecting data and discussing. In addition, their initial low reflective level concerning the way learn- ing processes need to be structured was partly improved. It was observed that it is very difficult to initiate the conceptual development of teachers in this perspective. A minority found it reasonably easy to adopt planning tools 30 professionalising physics teachers in doing experimental work that consider students’ prerequisites and to set clear learning outcomes, which are operationalised in teaching interventions. However, the majority struggled with implementing such tools in their teaching practice, and some participants even rejected them openly from the beginning. In general, IBL seems to be a very trendy teaching method. However, many practical activities are labelled as IBL despite failing to fulfil many of the criteria or characteristics of inquiry (cf. Hodson, 2014; Singer et al. 2006). This holds true for our sample, as well. When entering CMSE, the majority of the participants (nearly 85%) categorised their project as inquiry based in the ap- plication, although, as the results of our pre-questionnaire show, most of them were not familiar with characteristics of IBL. Conceptual changes on the level of our participants were, however, triggered concerning some characteristics of IBL, as the analysis of the project reports and the post-questionnaire show. In particular, the importance of guidance (cf. Kirschner et al., 2006) is recognised better and viewed as independent of students’ expertise in IBL. As far as aspects of how science works are concerned, the initial naïve ideas could only be partly changed. On the one hand, it seems to be much clearer now that students can- not really act as real scientists and that research is not only trial and error (cf. Singer, 2006; Hodson, 2014); on the other hand, the idea that research is sys- tematic and targeted, even on the level of students (NRC, 2000, 2006), has still not been fully internalised. To sum up, we can conclude that our sample’s beliefs concerning IBL were, in the majority of cases, not aligned with normative concepts. Our sam- ple’s level of professionalisation concerning these issues was low when they en- tered CMSE. Since the participants were selected by a review process, they rep- resent a positively selected sample. Many of their colleagues may have an even less developed professional knowledge concerning experimental competences and IBL, despite the fact that these issues have dominated our education system for several years. Our programme helps the participants to achieve better use of IBL as a method, with more aspects and facets of IBL being implemented in their teaching practice. In addition, the participants’ view of learning processes within IBL was partly modified. However, there are still several aspects, such as the idea of “learning by doing”, in which pre-knowledge does not play any role in student learning. All in all, CMSE can be seen as a first step in individual professionalisation, although there is still room for improvement and further development. Some general points for school practice can be deduced from these find- ings. First of all, teachers need to be aware that practical exercises involve more than just “doing” an experiment; experiments need to be well implemented in c e p s Journal | Vol.8 | No1 | Year 2018 31 physics or science lessons. This means that the implementation needs to be planned in a more student-centred way. Intended learning processes and learn- ing goals, in combination with students’ prerequisites, can determine whether experiments or IBL are an adequate means to support learning processes; not the other way round, whereby planning considerations begin from the experi- ment, ignoring systematic learning progressions and students’ prerequisites as determining factors. Secondly, instruments such as the spider-web (cf. Schecker et al., 2013) can help to ensure that all experimental competence facets are supported in physics or science instruction. Thirdly, it is important that teachers are aware that IBL is not just “doing an experiment”; they should recognise that the acquisition of inquiry skills is a long-lasting and slow process. It takes students quite a long time to be able to develop adequate and meaningful research questions and an ability to hypoth- esise. Consequently, it is more frustrating than motivating for students when they are confronted with higher-level IBL tasks without thorough preparation. Finally, it is important that teachers themselves develop a solid view of how science works and how IBL can help to make students familiar with an adequate view of the Nature of Science. References Altrichter, H., & Kanape-Willingshofer, A. (2012). Bildungsstandards und externe Überprüfung von Schülerkompetenzen: Mögliche Beiträge externer Messungen zur Erreichung der Qualitätsziele der Schule [Standardisation and first test results on students’ competencies: Possible external contribu- tions to increase the quality of the school system]. Nationaler Bildungsbericht Österreich, 2, 355–394. Blanchard, M. R., Southerland, S. A., Osborne, J. W., Sampson, V. D., Annetta, L. A., & Granger, E. M. (2010). Is inquiry possible in light of accountability? A quantitative comparison of the relative eff ec- tiveness of guided inquiry and verification laboratory instruction. Science Education, 94(4), 577–616. Bybee, R., McCrae, B., & Laurie, R. (2009). PISA 2006: An assessment of scientific literacy. Journal of Research in Science Teaching, 46(8), 865–883. Bybee, R. W. (2002). Scientific Literacy–Mythos oder Realität? In W. Gräber, P. Nentwig, T. Koballa & R. Evans (Eds.), Scientic Literacy (pp. 21–43). Opladen: Leske. CMEC (1997). Common framework of science learning outcomes. Toronto: Council of Ministers of Edu- cation. Garet, M. S., Porter, A. C., Desimone, L., Birman, B. F., & Yoon, K. S. (2001). What makes professional development effective? Results from a national sample of teachers. American Educational Research Journal, 38(4), 915–945. Driver, R. (1983). The pupil as scientist? Milton Keynes: Open University Press. 32 professionalising physics teachers in doing experimental work Haagen-Schützenhöfer, C., Rath, G., & Rechberger, V. (2017). Teachers’ Beliefs About Subject Specific Competences and Inquiry Based Learning. In T. Greczyło & E. Dębowska (Eds.), Competences in Phy- sics Teaching and Learning (pp. 177–190). Bern: Springer International Publishing. Haagen-Schützenhöfer, C., & Mayr, S. (2016). Vorstellungen von PädagogInnen zum “Forschenden Lernen”. In C. Maurer (Ed.), Authentizität und Lernen - das Fach in der Fachdidaktik [Authenticity and Learning – the content in didactics]. Gesellschaft für Didaktik der Chemie und Physik Jahrestagung in Berlin (pp. 584-586). Regensburg: Universität Regensburg. Haagen-Schützenhöfer, C., Mathelitsch, L., Knechtl, W., Rechberger, V., & Rath, G. (2015). Compe- tencies in mathematics and science education (CSME): A programme promoting in-service teachers’ professional development. In K. Maaß, B. Barzel, G. Törner, D. Wernisch, E. Schäfer, & K. Reitz-Kon- cebovski (Eds.), Educating the educators: international approaches to scaling-up professional develop- ment in mathematics and science education (pp. 273–278). Conference Proceedings, 15-16 Dec 2014, Essen: WTM. Haagen, C., & Hopf, M. (2012). Standardization in Physics - first steps in the Austrian educational sys- tem. In C. Bruguiere & A. Tiberghien (Eds.), E-Book proceedings of the ESERA 2011 conference: Science learning and citizenship. Lyon: European Science Education Research Association (ESERA). Hattie, J. (2013). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. London, UK: Routledge. Hart, C., Mulhall, P., Berry, A., Loughran, J., & Gunstone, R. (2000). What is the purpose of this experiment? Or can students learn something from doing experiments? Journal of Research in Science Teaching, 37(7), 655–675. Hofstein, A., & Lunetta, V. N. (2004). The laboratory in science education: Foundations for the twenty- first century. Science education, 88(1), 28–54. Hodson, D. (2014). Learning science, learning about science, doing science: Different goals demand different learning methods. International Journal of Science Education, 36(15), 2534–2553. Kirschner, P. A., Sweller, J., & Clark, R. E. (2006). Why minimal guidance during instruction does not work: An analysis of the failure of constructivist, discovery, problem-based, experiential, and inquiry- based teaching. Educational Psychologist, 41(2), 75–86. KMK (2004). Bildungsstandards im Fach Physik für den Mittleren Schulabschluss [Intermediate-level education standards for the subject physics]. Sekretariat der Ständigen Konferenzn der Kultusminister der Länder in der Bundesrepublik Deutschland. Reterevied from www. kmk.org Langer, E., Mathelitsch, L., & Rechberger, V. (2014). Synergistic cooperation of school-based action re- search with university-based didactic investigations. In F. Rauch, A. Schuster, T. Stern, M. Pribila & A. Townsend (Eds.), Promoting change through action research (pp. 101–107). Rotterdam: Sense Publishers. Lunetta, V. N., Hofstein, A., & Clough, M. P. (2007). Learning and teaching in the school science laboratory: An analysis of research, theory and practice. In S. K. Abell (Ed.), Handbook of research on science education (pp. 393–441). Mahwah, NJ: LEA Lawrence Erlbaum Assoc. Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical con- tent knowledge for science teaching. In J. Guess-Newsome & N. Lederman (Eds.), Examining pedago- c e p s Journal | Vol.8 | No1 | Year 2018 33 gical content knowledge (pp. 95–132). Dordrecht: Kluewer. Maltese, A. V., Tai, R. H., & Sadler, P. M. (2010). The effect of high school physics laboratories on per- formance in introductory college physics. The Physics Teacher, 48(5), 333–3337. Mayring, P. (2014). Qualitative content analysis – theoretical foundation, basic procedures and software solution. Klagenfurt: Open Access Repository. Retrieved from www.ssoar.info Millar, R., & Abrahams, I. (2009). Practical work: Making it more effective. School Science Review, 91(334), 59–64. Minner, D. D., Levy, A. J. & Century, J. (2010). Inquiry‐based science instruction—what is it and does it matter? Results from a research synthesis years 1984 to 2002. Journal of Research in Science Teaching, 47(4), 474–496. National Research Council (1996). National science education standards: Washington, DC: National Academy Press. National Research Council (NRC) (2000). Inquiry and the national science education standards. Was- hington, DC: National Academy Press. OECD (2004). PISA Learning for tomorrow‘s world: First results from PISA 2003. Retrieved from http:// www.oecd.org/education/school/programmeforinternationalstudentassessmentpisa/34002216.pdf. Pathway UK (2013). Pathway UK: Science education through inquiry in schools, museums and infor- mal learning settings. Retrieved from http://www.pathwayuk.org.uk/uploads/9/3/2/1/9321680/_the_ features_of_inquiry_learning_theory_research_and_practice_eusubmitted.pdf Schecker, H., Nawrath, D., Elvers, H., Borgstädt, J., & Einfeldt, S. Maiseyenka, V. (Eds). (2013). Na- turwissenschaften: Modelle und Lernarrangements für die Förderung naturwissenschaftlicher Kom- petenzen. Komdif [Science: Models and learning paths to support the acquisition of competences]. Hamburg: Eigenverlag. Schreiber, N., Theyßen, H., & Schecker, H. (2016). Process-oriented and product-oriented assessment of experimental skills in physics: A comparison. In N. Papadouris, A. Hadjigeorgiou & C. Constanti- nou (Eds.), Insights from research in science teaching and learning (pp. 29–43). Bern: Springer Interna- tional Publishing. Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–23. Singer, S. R., Hilton, M. L., & Schweingruber, H. A. (Eds) (2006). America‘s Lab Report: Investigations in High School Science. Washington, WA: National Academies Press. Weiglhofer, H. (2007). Austria at the beginning of the way to standards in science. In D. Waddington, P. Nentwig & S. Schanze (Eds.), Making it comparable, standards in science education (pp. 61–70). Münster: Waxmann. Weinert, F. E. (2001). Concept of competence: A conceptual clarification. In D. Rychen & L. Saganick (Eds.), Defining and selecting key competences (pp. 45–65). Ashland, OR: Hogrefe & Huber Publishers.