Effects of German Language Teacher Professional Development on Pupils' Learning Outcomes in Intercultural Competence
Ana Šenjug Golub1
^^ The development of intercultural competence is increasingly being perceived as a key goal in today's education. As a result of a strong emphasis on that competence in curricular documents, teachers are faced with demanding tasks. Confirming this, recent research in the field of intercultural competence in Croatian schools indicates the numerous difficulties teachers face in accomplishing the goal of developing their pupils' inter-cultural competence. Teacher professional development can be crucial for a systematic approach to the development of learners' intercultural competence. Therefore, the aim of this study was to determine whether teacher participation in a further professional development programme on the topic of intercultural competence can significantly affect the achievement of goals set in the current curricular documents on foreign language learning related to intercultural competence. The effects of the professional development programme were assessed by means of analysing learner outcomes in intercultural competence prior to and after teacher participation in the programme. A quasi-experimental pre-test-post-test design was used, and the achievements of 752 primary and lower secondary level pupils attending and grades were evaluated. The results confirmed that foreign language teacher professional development in the field of in-tercultural competence had positive effects on pupils' learning outcomes, especially in the cognitive component of intercultural competence. This leads to the conclusion that it is necessary to motivate foreign language teachers to participate in such programmes. The results also indicate the necessity of placing particular emphasis on the problem of stereotypes and prejudices when designing such programmes.
Keywords: evaluation of learning outcomes, German language teaching and learning, intercultural competence, teacher professional development
1 Faculty of Teacher Education, University of Zagreb, Croatia; ana.senjug@ufzg.hr
Učinki profesionalnega razvoja učiteljev nemščine na učne dosežke učencev s področja medkulturnih kompetenc
Ana Šenjug Golub
^^ Razvijanje medkulturnih kompetenc je vedno bolj v ospredju kot ključen cilj v današnjem izobraževanju. Glede na to, da je ta kompe-tenca močno prisotna v kurikularnih dokumentih, se učitelji srečujejo z zahtevnimi nalogami. To potrjujejo tudi novejše raziskave na področju medkulturnih kompetenc v hrvaških šolah, ki kažejo na številne težave, s katerimi se učitelji spopadajo pri doseganju cilja - pri učencih razviti medkulturne kompetence. Učiteljev profesionalni razvoj je lahko ključnega pomena pri sistematičnem pristopu k razvijanju medkulturnih kompetenc učencev. Zato je namen te raziskave ugotoviti, ali lahko udeležba učiteljev v programih profesionalnega razvoja na temo medkulturnih kompetenc pomembno vpliva na doseganje ciljev, postavljenih v obstoječem kurikulumu za poučevanje tujega jezika, povezanih z medkulturnimi kompetencami. Učinki programa profesionalnega razvoja so bili merjeni s pomočjo analiziranja učnih dosežkov s področja medkulturnih kompetenc, preden so se učitelji udeležili programa in po udeležbi. Uporabljena je bila metoda kvazieksperimenta (predtest/ potest). Analizirani so bili dosežki 752 učencev, ki so obiskovali četrti in osmi razred osnovne šole. Rezultati so potrdili, da je imel profesionalni razvoj učiteljev tujega jezika s področja medkulturnih kompetenc pozitiven vpliv na dosežke učencev, še posebej glede kognitivne komponente medkulturnih kompetenc. To vodi do sklepa, da je učitelje tujih jezikov nujno treba motivirati, da se udeležujejo tovrstnih programov. Rezultati kažejo tudi na to, da je pri snovanju tovrstnih programov treba še posebno pozornost nameniti težavam, povezanim s stereotipi in predsodki.
Ključne besede: analiza učnih dosežkov, učenje in poučevanje nemščine, medkulturne kompetence, profesionalni razvoj učiteljev
Introduction
The global economy, the establishment of worldwide connections between people and or institutions, as well as economically and socially motivated migration, are characteristic occurrences of contemporary societies and contribute to the increasing heterogeneity of various aspects of human life. In order to sustain oneself, thrive and actively participate in such a society, individuals need a set of different competences. Intercultural competence is often singled out as an important component of that set, its purpose being to establish and improve relationships and bring about mutual understanding and respect among members of different cultural and social groups. The development of this competence is increasingly being perceived as a key goal in today's education. Official recognition of its importance in education at the European level was confirmed by its inclusion in the Recommendation 2006/962/EC of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning (Official Journal L 394 of 30.12.2006).
The development of key competences is regarded as a relevant goal both in the European Union's education policy and in the national education policies of European countries. Therefore, key competences are increasingly being included in the national curricula throughout Europe. Similarly, intercultural competence is incorporated in the curricula of most European countries today, with foreign language teaching representing a significant area of its development (see Languages and Cultures in Europe, 2006).
The Republic of Croatia is not an exception in following this trend, as the document on Key competences for lifelong learning - a European reference framework has exerted significant influence on the design of the Croatian National Curriculum Framework for Preschool Education, General Compulsory and Secondary School Education (2010) (hereafter: National Curriculum Framework). Intercultural competence is integrated into the National Curriculum Framework and is thus recognized by the Croatian education policy and language teaching methodology as an essential key competence. While this competence was defined as a foreign language teaching goal in the Primary School Curriculum of the Republic of Croatia as early as 2006, its achievement is more elaborately defined in the subsequently adopted National Curriculum Framework. Therefore, the competence has been given a significant position in the Croatian curricular documents. The development of intercultural competence is precisely defined and outlined in the foreign language teaching framework, and the competence is also - implicitly or explicitly - cited as a goal in other education areas and subject-integrated fields of study.
As a result of the placement of a strong emphasis on intercultural competence in curricular documents, teachers are faced with demanding tasks. In the Croatian education system, foreign language teachers are affected by the changes the most significantly, because intercultural competence is regarded as an essential component of foreign language competences. The accomplishment of the learning outcomes, such as the ability to reflect on the phenomena of 'the culture of the interlocutor without making value judgments' (National Curriculum Framework, 2010, p. 89) or the ability to reflect on the influence of 'one's own culture on the perception of both the culture of the interlocutor and one's own, on the ways in which culture affects communication and cultural patterns, and on heterogeneity, change and development as important aspects of culture' (National Curriculum Framework, 2010, p. 89) can present teachers with challenges. Recent research in that field confirms these assumptions.
Research results in the field of intercultural competence in Croatian schools point to the numerous difficulties teachers face in accomplishing learner outcomes in this area, i.e. they indicate that the approach to the development of this competence has not been sufficiently systematic (see Breka, 2012; Filipan-Žignič, 2008; Legac, Mikulan, & Siročič, 2007; Šenjug, 2008). The results of research conducted in Croatian elementary schools, i.e. among pupils at primary and lower secondary school levels,2 show evident shortcomings in relation to different aspects of pupils' intercultural competence. Filipan-Žig-nič (2008) identifies significant deficiencies in the area of specific cultural and civilizational knowledge and concludes that such topics are neglected in foreign language teaching. Šenjug's (2008) research carried out among German language learners shows that the harbouring of prejudice toward German language speakers is not necessarily significantly lessened with increased durations of target language study. Such results indicate that cultural aspects are not given sufficient attention in foreign language teaching, i.e. that the changes introduced in the curricular documents are not sufficiently considered in the teaching practice (see Legac, Mikulan, & Siročič, 2007). Although future and in-service teachers frequently express a positive attitude toward the development of intercultural competence and consider it to be an important goal in foreign language teaching (Piršl, 2011; Breka, 2012), they often do not cover intercultural topics in their own classes. This is confirmed by some findings by Breka (2012), according to whom most of the teaching time in the foreign language classroom is dedicated to the development of language competences (Breka,
2 In the Croatian education system, compulsory education comprises eight years of schooling and is organized in elementary schools. Grades 1-4 make up the primary, and grades 5-8 of elementary schools the lower secondary level, as referred to in this paper.
2012, p. 249). In this respect, teachers frequently report on their shortcomings in the field of intercultural competence as an obstacle to its development among their pupils (Breka, 2012, p. 251). While teacher professional development in the given field could be very useful, Breka's (Breka, 2012, p. 213) findings show that more than two thirds of teachers have never attended seminars on intercultural competence development. It is important to note that those teachers who have participated in such seminars and workshops consider such forms of training very useful. In addition, the majority of teachers are of the opinion that the development of intercultural competence should be given more attention than is currently the case (Breka, 2012, pp. 217-218).
With regards to the presented research results, it is evident that teachers face difficulties in achieving the aims related to intercultural competence in the classroom. One possible reason for this is surely the fact that the relevant topics are not covered in teacher initial education. Further professional development can be the key to improving the teaching of those teachers who are already in service and are attempting to cope with education policy changes. Teachers themselves also consider continuing professional development to be crucial for the systematic development of learners' intercultural competence (Breka, 2012, p. 219).
The development of teacher competences, teaching quality improvement and the improvement of the learning outcomes are the central goals of teacher continuing professional development (Croatian Education and Teacher Training Agency, 2013). Forms of teacher education and professional development are varied and range from those directed at developing work-specific competences to individual learning and different formal education courses in higher education institutions. Teacher professional development comprises "talks and discussions with colleagues, study of specialist literature, participation at symposia, working visits, participation in distance-learning courses, in scholarship programmes and in research" (Petljak Zekic, Rukljač, & Urek, 2013, p. 18). Forms of teacher professional development have changed over time. As opposed to the traditional model, in which teachers usually attended symposia and listened to plenary talks in large groups, modular training courses are more frequently offered today. Such courses enable participants to work in small teams and engage in an in-depth study of specific topics in the course of a few meetings or modules (Petljak Zekic, Rukljač, & Urek, 2013, p. 18). In this way, single in-service training courses are abandoned in favour of extended, more comprehensive, study-oriented programmes (Vujičic, 2007).
Given that the effects of teacher professional development are reflected in the learning outcomes of pupils, such outcomes are indicative of the quality of
80 effects of professional development on learning outcomes in intercültüral ...
teacher professional development programmes. For this reason, learner achievements are frequently the subject of research aimed at assessing the quality of teacher education and continuing professional development (see e.g. Angrist & Lavy, 2001; Domovic & Godler, 2005; Jacob & Lefgren, 2004; Shah & Alam, 2002). A great number of such studies indicate the weak spots of in-service teacher training (Kennedy, 1998) and identify low intensity or small amounts of input and a lack of continuity of professional development programmes as the main problems (Corcoran, 1995; Little, 1993). In contrast, some studies show that high quality in-service teacher training positively affects students' learning outcomes (Angrist & Lavy, 2001; Little, 1993).
In view of the difficulties faced by foreign language teachers in relation to achieving intercultural competence as a learning outcome and given the relevance of teacher's continuing professional development, the aim of this study was to determine whether teacher participation in professional development courses on the topic of intercultural competence can significantly affect the achievement of goals set in the current curricular documents on foreign language learning. Therefore, the purpose of this study was to ascertain the influence of German language teacher professional development in the field of intercultural competence on the outcomes defined as goals in the foreign language curricular documents. The effects of teacher professional development were assessed by examining learner outcomes in intercultural competence, prior to and after teacher participation in a specially designed teacher professional development programme. In order to define the subject area of teacher further professional development and set the guidelines for learner outcome evaluation, a definition of intercultural competence in the framework of foreign language teaching shall be given first.
Intercultural competence in foreign language teaching
The complexity of the construct of intercultural competence currently lacks a uniform definition. Attempts to convey the complex aspects of the concept are exemplified in a range of various terms currently in use. While no overlapping of terms was noted in the German language sources, studies written in English promote a variety of synonymously used expressions, such as cross-cultural awareness, global competitive intelligence, cultural competence, cultural sensitivity, ethno-relativity, international competence, intercultural interaction, biculturalism, multiculturalism, etc. (Fantini, 2006, p. 81; Mertesacker, 2010, p. 26; Yussefi, 2011, p. 16). Emphasis is laid on global knowledge in one term, sensitivity is in the focus of another, specific skills are singled out in yet another,
and so on. In spite of the focal differences of the said concepts, a consensus across disciplines exists on the three essential components of intercultural competence: the cognitive (knowledge), the affective (attitudes and beliefs) and the pragmatic (skills) component (Gehrmann, Petravic, & Šenjug Golub, 2013).
These components form part of Byram's (1997) model of intercultural communication competence. His model is very well accepted in foreign language learning and teaching, and has significantly influenced the way in which intercultural competence is defined in both the Common European Framework of Reference for Languages (Council of Europe, 2001) and in the Croatian National Framework Curriculum (see Petravic, 2011). This competence is regarded as an important aspect of Byram's model of intercultural communication competence, which consists of intercultural, linguistic, sociolinguistic and discursive competence. Intercultural competence itself comprises five components:
•	knowledge (fr. savoirs),
•	attitudes (fr. savoir etre),
•	skills of discovery and interaction (fr. savoir apprendre/faire),
•	skills of interpreting and relating (fr. savoir comprendre), and
•	critical cultural awareness/political education (fr. savoir sengager).
The knowledge one brings into interaction with individuals from another country, i.e. with interlocutors perceived as different from oneself, comprises two categories. One is the category of the knowledge of social groups and culture in one's own as well as one's interlocutor's country; the other is the knowledge of the processes of interaction, referring to the ways in which social identity is acquired and to the awareness of that identity's acting as a prism through which one is perceived by others. The prism also determines the way in which members of other groups are perceived by us (Byram, 1997, pp. 35-37).
Skills of interpreting and relating originate in the knowledge of one's own environment and the environment of another, and are not limited to interaction with others, but are also applied when faced with any kind of document from another culture. Skills of discovery and interaction are activated when one's knowledge in interaction with another is limited or entirely lacking. This dimension relates to the ability of acquiring specific knowledge and the understanding of beliefs and behaviour behind a given phenomenon, be it a document or a person in interaction (Byram, 1997, pp. 37-38).
Critical cultural awareness partially covers the remaining dimensions of Byram's model. It refers to the ability to critically reflect on the perspective, the customs and the products of one's own and one's interlocutor's culture. The aim of such reflection is not exclusively limited to achieving more efficient
communication, but also involves bringing to light the ideological side of one's own perspective (Byram, 1997, pp. 63-64).
Attitudes are a prerequisite of successful intercultural competence, with desirable attitudes being those marked with curiosity and openness toward others, readiness to suspend disbelief and respect for others' beliefs and behaviour. Furthermore, it is important that one's attitude expresses a readiness to observe one's own culture from the viewpoint of the interlocutor and critically reflect on one's own values (Byram, 1997, pp. 34-35).
Byram's (1997) model of intercultural competence is often applied in the design of instruments for the evaluation of foreign language teaching (see Äijälä, 2009; Eberhardt, 2009; Feng & Fleming, 2009; Sercu, 2000). While all but one component of intercultural competence can, according to Byram, be assessed directly, critical cultural awareness (fr. savoir sengager) is assessed indirectly via other components. Byram does not regard holistic evaluation as possible, because each component requires a particular mode of assessment. He cites four such assessment modes: portfolio, test, simulation and continuous assessment. Some of these procedures can also be applied when evaluating more than one dimension of intercultural competence. For instance, knowledge (fr. savoirs) of the target culture in terms of the knowledge of the products, events and phenomena, can be assessed via tests and continuous assessment (Byram, 1997, p. 98). The most appropriate means of assessment for attitudes marked with openness, curiosity and readiness to suspend disbelief related to one's own and one's interlocutor's culture (savoir etre) is the portfolio (Byram, 1997, p. 95), while skills of discovery and interaction (fr. savoir apprendre/faire), which enable students to act as intermediaries between cultures and independently look for and evaluate information on a foreign culture, can be assessed via tests, simulations and portfolios. Skills of interpreting and relating (fr. savoir comprendre), at work in interaction and in contact with any kind of foreign content, can be assessed through tests and continuous assessment (Byram, 1997, p. 102). While Byram names the procedures applicable for evaluation, he does not tackle the questions of actual tests, the kinds of tasks needed and the particular target groups of respondents, except in the case of portfolio. Furthermore, there are other aspects that make the carrying out of certain assessment modes difficult and need to be considered in the evaluation. They may include the specifics of the school environment, such as the cognitive-affective level of child development, individual pupil assessment in groups and the time and staff constraints. Nevertheless, Byram's framework outline enables the design of actual evaluation instruments adapted to the specifics of school context and evaluation purpose and aimed at assessing particular groups of respondents (see Äijälä, 2009).
Considering the usefulness of Byram's model of intercultural competence in foreign language teaching, its widespread acceptance and the possibilities of its implementation in the field of intercultural competence assessment, it serves as the basis of the research presented here. The following part of the paper deals with the individual steps and the results of the study.
Method
The research was conducted in three phases:3
1.	An initial evaluation was conducted at the beginning of the school year in all treatment and control groups.
2.	Teachers teaching treatment groups participated in three day-long teacher professional development modules with the topic of intercul-tural competence during one school year.
3.	Final evaluation was conducted at the end of the school year in all respondent groups.
Teacher professional development was carried out in the form of modules over the course of one school year. Five teachers of German as a foreign language took part in the programme during the 2009/2010 school year, and six during 2010/2011. The programme was conducted in three modules: at the start of the school year, at the end of the first semester and during the second semester. The first module focused on two aspects the theoretical background and practical exercises in the form of workshops. The aims of the first module were to gain awareness of the importance of intercultural competence, to deepen the knowledge of the concept and the models of intercultural competence, to become acquainted with the outcomes of teaching aimed at the development of intercultural competence, and to learn and carry out the activities for the development of the competence among pupils. The activities4 were specially designed in the framework of the aforementioned project or modified according to the relevant literature and adapted to the pupils' ages and the Croatian school context. In addition to deepening acquired knowledge, the next two modules focused on the teachers' reflection on the effects of their work based on the classroom examples and the exchange of participants' experiences. In this way,
Research was conducted in the framework of the research project Development of intercultural competence in primary foreign language teaching (MZOŠ RH, 227-2271168-0726) under the supervision of prof. dr. sc. Ana Petravic).
The activities were published as a collection of materials on IC as a learning outcome of young and teenage learners, under the title Auf dem Weg zum interkulturellen Sprecher (On the way to the intercultural speaker) (Petravic & Šenjug Golub, 2012).
3
the teachers kept a record of the intercultural activities they conducted during the school year, of their observations on the individual steps in the implementation of activities, of their pupils' reactions and the difficulties encountered, as well as of the possible solutions. Some difficulties were noted regarding the level of pupils' language competence, the discussions of sensitive issues, the complexity of some issues as exemplified in the concepts of value, cliche, stereotype and the like, the problem of perspective change, etc. Reports of pupils' positive reactions were frequently made and refer to the pupils' awareness of particular problems, their perception and acceptance of mutual differences, their abilities to connect certain topics with their own experiences, their openness and curiosity in learning about a foreign culture and positive changes in the relationships between pupils in class.
Assessment of the effect of the teacher professional development programme on the learning outcomes of pupils was conducted in fourth and eighth grades, i.e. at the end of primary and lower secondary education, respectively. Evaluation of learner outcomes (a paper-and-pencil procedure) in the duration of one school period was deemed to be an appropriate mode of assessment. tte main shortcomings of paper-and-pencil evaluation procedures are their failure to take into account the pupils' state of mind at the time of testing and the lack of the possibility to ask additional questions. ttis makes it quite difficult to gain better insight into pupils' reasoning. Despite these shortcomings, written evaluation methods are often deemed the most appropriate type of evaluation in the school context. They have the highest objectivity level due to minimal interaction between evaluators and respondents. In addition, they are the most practical, as the assessment time is minimal, and group testing can be conducted with a large number of pupils. Furthermore, as the financial costs are minimal, they can be conducted by classroom teachers (see Bärenfänger & Stevener, 2009.).
A pre-test-post-test intergroup quasi-experimental design was applied, and distinct emphasis was placed on the similarity between the control and treatment group in the dependent variable. This means that in order to successfully ascertain the effects of teacher professional development at the end of the experiment, it was vital that the pupils in both the treatment and the control groups accomplished similar pre-test results in their level of intercultural competence.
The aim and hypothesis
The aim of this survey was to determine the effects of German language teacher professional development in the field of intercultural competence development on the pupil intercultural competence learning outcomes through
the assessment before and after the teachers' participation in a particular professional development programme. The following problem is derived from the stated aim:
To evaluate the differences in the level of fourth (primary) and eighth grade (lower secondary) pupils' intercultural competence in relation to their teachers' participation in the professional development programme on intercultural competence development.
The following hypothesis was proposed:
H: Teacher professional development will have positive effects on pupils' learning outcomes in intercultural competence.
Pupils in the treatment group were exposed to a particular process of intercultural competence development in such a way that their teachers carried out certain actions and activities and covered the topics studied in the professional development modules. The topics covered were dealing with everyday life from the intercultural viewpoint. Also included were specific topics, such as intercultural misunderstandings, preconceptions and stereotypes, culturally influenced communication, etc. The choice of activities comprised specific tasks related to intercultural learning, such as tasks developing perception, the abilities of estimating culturally conditioned patterns and the ability of inter-cultural comparison.
This by no means implies that the pupils from the control groups did not receive any treatment at all, i.e. that their teachers did not cover the topics and activities aimed at the development of intercultural competence as defined in the teaching curriculum. However, it is assumed that such topics are usually not given a significant amount of time or focused on significantly in class and that they are frequently narrowed down to factual knowledge of culture and civilization of the countries of the German-speaking area (see Petravic, 2010). It was, therefore, expected that the pupils in the treatment groups would, due to their more intensive and structured exposure to the said topics, achieve higher post-test scores as a direct result of their teachers' professional development. The hypothesis comprises the following sub-hypotheses: Hi: Pupils in the treatment groups will achieve higher scores in the area of knowledge about the culture and civilization of the German-speaking area. H2: Pupils in the treatment groups will achieve higher scores in the area of
specific intercultural skills. H3: Pupils in the treatment groups will express more desirable attitudes toward individuals from the German-speaking area.
Respondents
Respondents in this survey were fourth and eighth grade pupils of primary and lower secondary school. tte sample comprised a total of 37 classes of pupils learning German as the first foreign language. A total of 13 schools from 6 counties took part in the survey (see Figure 1); 28% of the total number of respondents come from rural areas, 6% from small urban areas (up to 10,000 inhabitants), 6% from mid-sized towns (10,000 to 100,000 inhabitants) and 60% from large urban areas (more than 100,000 inhabitants).
Figure 1. Number of respondents in the initial and the final evaluation according to county, shown in percentages
tte initial evaluation was conducted at the beginning of the 2009/2010 school year in the fourth grade and at the beginning of 2010/2011 in the eighth grade. A total of 751 respondents, of which 397 (52.86%) were girls and 349 (46.47%) boys, participated in the pre-test. Five (0.66%) respondents left their gender unmarked on the pre-test. There were 323 (43%) respondents in the treatment group and 428 (56.99%) in the control group; 355 respondents attended the fourth and 396 the eighth grade. tte final evaluation in the fourth grade was conducted at the end of the 2009/2010 school year, and in the eighth grade at the end of 2010/2011. A total of 752 respondents (386 (51.33%) girls and 366 (48.67%) boys) took part in the post-test. The treatment group was represented by 321 (42.69%) and the control group by 431 (57.31%) respondents; 360 respondents were fourth grade and 392 eighth grade pupils.
Instruments
An intercultural competence evaluation instrument was designed for the pupils in the fourth grade and the eighth grade. tte design was based on Byram's (1997) model of intercultural competence, which was adapted for the specifics of the Croatian school environment, the current curricular documents and the level of pupils' cognitive-affective development. tte components of intercultural competence relevant for the target group were integrated into a threefold structure of the questionnaire. It represents a battery of instruments assessing the cognitive, affective and pragmatic aspects of intercultural competence.
The first part of the instrument assesses the knowledge of the culture and civilization of the German-speaking area, i.e. culturally specific knowledge (By-ram, 1997). The content of the test questions was adapted to the German language curriculum. Respondents were given a set of open questions in which they were asked to name the facts and the phenomena of the target culture such as information in relation to geography, history, science, humanities, arts, behaviour, customs and ways of communicating and behaving.
The second part of the instrument evaluates skills, primarily the skill of relativizing the foreign culture as well as one's own, the skill of gathering information and the skill of interpreting and interacting. Therefore, this part of the instrument covers individual competence areas of the skill of discovery and interaction (fr. savoir apprendre/faire) and the skill of interpreting and relating (fr. savoir comprende) (Byram, 1997). These specific skills were investigated by means of the completion of associative sequence tasks and the tasks of describing reactions to problem situations similar to some probable real situations. This part of the instrument was designed on the basis of previous research and the instruments used therein, primarily those related to the school context (see Äijälä, 2009; Brym, 2004; Hesse & Göbel, 2006; Pachevska, 2003), as well as the instruments of the INCA project (www.incaproject.org).
The third part of the instrument examines pupils' attitudes by means of a five-point scale measuring three sub-dimensions of attitudes toward members of the culture of the German-speaking area: curiosity/disinterest, openness/ lack of openness and inclination/disclination to stereotypes and preconceptions (see Byram, 1997, p. 35, 50, 91). The items were based on the instrument of the previously conducted survey of learner attitudes toward Germans (see Šenjug, 2008). Although the affective dimension of intercultural competence comprises the positive attitudes toward all people perceived as different from ourselves, this study focused on the specific context of German language teaching and was
88 effects of professional development on learning outcomes in intercultural ...
based on the curriculum of German as a foreign language (see Primary School Curriculum, 2006). For this reason, only the attitudes related to the members of the target culture were investigated.
Data processing
Metric characteristics of the instrument were assessed upon conducting the research. To test the validity, an item discrimination index was used. Mutual relatedness between the three parts of the instrument (knowledge (1), skills (2) and attitudes (3)) and the correlation of success on each part of the instrument with external variables of school success was measured using the Pearson correlation coefficient. In the attitudes part of the instrument, an exploratory factor analysis using the principal component method was conducted to test the assumption of the three sub-dimensions of attitudes toward members of the foreign culture. tte reliability of the instrument was examined by determining the internal consistency using Cronbach's alpha. In order to examine the sensitivity, the acquired range and the item's difficulty index were analysed.
It can be stated that the metric characteristics of all three parts of the instrument (knowledge, skills and attitudes) are satisfactory as a whole. The acquired range in all parts of the instrument greatly correlates with the possible theoretical range. All the questions in the parts of the instrument covering knowledge and skills have good item discrimination (r.u> 0.35). Factor analysis of the third part of the instrument confirmed the assumption of the three sub-dimensions of attitudes. Cronbach's alpha coefficient is in the range between 0.668 and 0.879 in individual parts of the instrument. All three parts of the instrument are mutually well correlated and in line with pupils' grades in the subject of German language, which points to the instrument's validity.
A T-test and Mann-Whitney U test were applied to test the differences in the level of pupils' intercultural competence development in relation to their participation in the experimental programme, i.e. in order to test the hypotheses. Data analysis was conducted using SPSS for Windows.
Results
The results are shown in two segments. The first part deals with the pretest results, where the focus lies on the assessment of similarities between the control and the treatment group in the dependent variable. The second part shows the post-test results and the differences between the control and the treatment group in relation to the level of intercultural competence.
Initial evaluation
As already stated, the similarity between the treatment and the control group in the dependent variable is essential in order to be able to reach viable conclusions after the experiment completion. For the purpose of identifying possible differences between the fourth grade control and treatment group in relation to the results accomplished in the area of knowledge and skills, an independent samples t-test was conducted. As the data in the area of attitudes do not follow the normal distribution, a nonparametric Mann-Whitney U test was conducted. The results obtained indicated the absence of statistically significant differences between the control and the treatment group regarding the results obtained in the area of knowledge (t=0.702, p=0.483) and skills (t=-1.669, p=0.096). ttis confirms that the two groups were equal with regard to their knowledge of culture and civilization of the countries of the German-speaking area and that the members of both groups were equal in their achievement of intercultural skills. Therefore, upon completion of the experimental programme, conclusions can be drawn on the programme's influence on the treatment group.
The absence of a statistically significant difference between the treatment and the control group in view of the respondents' lack of openness toward individuals from the German-speaking area and in view of their inclination to stereotypes was confirmed in the area of attitudes5 (see Table 1). However, the groups differed in the level of curiosity, with the treatment group having a higher mean rank in this respect than the control group (marked bold in Table 1).
Table 1. Mann-Whitney U test for independent samples - pre-test results of 4th grade pupils' attitudes
	Group	N	Mean Rank	U	z	p
Curiosity	Treatment Control	116 158	153.94 125.43	7257.500	-2.951	.003
Lack of openness	Treatment Control	100 140	112.11 126.49	6161.000	-1.596	.110
Inclination to stereotypes	Treatment Control	117 158	140.50 136.15	8951.000	-.449	.653
N - number of participants, U - Mann-Whitney U value, z - z value, p - statistical significance
5 ^ree aspects characterising attitudes toward members of the target culture (curiosity, lack of openness and inclination to stereotypes) were identified by submitting the part of the instrument relating to attitudes to factor analysis. ^ese findings are in line with the theoretical framework (see also Byram, 1997).
For this reason, it would not be possible to compare the groups in view of the said component of attitudes after post-test. Nevertheless, it can generally be said that in relation to the result of the pre-test, the treatment and the control group are quite similar, with a slight difference in one segment of attitudes.
In spite of the lack of normal distribution of data in the intercultural competence in the eighth grade pre-test, the analysis of kurtosis, skewness and modality of distribution shows that it was possible to use the parametric procedure for the comparison of the treatment and the control group in all parts of the instrument. An independent sample t-test was applied, and the results obtained showed the absence of a statistically significant difference between the control and the treatment group in the results achieved in the area of knowledge (t=0.243, p=0.808), skills (t=1.542, p=0.124) and attitudes - curiosity (t=1.077, p=0.282), lack of openness (t=-0.977, p=0.329) and inclination to stereotypes (t=-0.195, p=0.845). ttis shows that the two groups are equal in all the evaluated segments of intercultural competence. Therefore, both the conducting of the experiment and the conclusion drawing upon post-test are valid procedures.
Final evaluation
tte aim of the final evaluation was to ascertain the differences between the treatment and the control group in view of respondents' level of individual components of intercultural competence and thus determine the effects of teacher further professional development in the field of intercultural competence. It was expected that the treatment group respondents would, due to their more structured and more intense exposure to intercultural topics than that of the control group respondents, achieve better results in all dimensions of intercultural competence as a direct result of their teachers' professional development in that field.
In comparing the fourth grade posttest results, significant differences were identified between the treatment and the control group in the mean results obtained in individual parts of the instrument (Table 2). It can be seen that the treatment group pupils achieved a mean score of 22.1 points in the knowledge of culture and civilization, while the control group respondents only achieved a score of 13.7 points in that field. The treatment group also scored on average three points higher than the control group in the area of skills. As already stated, the pretest results for attitudes showed differences between respondents in the factor of curiosity, with the treatment group having higher scores. While the control group scored 22.18, the treatment group earned a score of 24.31 points. The control group acquired virtually the same result on
the post-test, while the treatment group achieved a higher post-test score. Furthermore, the treatment group pupils with a lower mean result were found to be more open toward foreign cultures. It is interesting to note that the treatment group respondents scored higher in their inclination to stereotypes than those in the control group. However, in order to determine the statistical significance of the stated differences between the groups, it is necessary to conduct appropriate statistical procedures.
Table 2. Descriptive parameters of overall 4th grade post-test results
	Group	N	Mean	Std. Dev	Std. Error Mean
Knowledge	treatment control	148 202	22.11 13.71	4.818 5.457	0.396 0.384
Skills	treatment control	148 202	25.11 22.12	6.942 7.491	0.571 0.527
Curiosity	treatment control	137 190	25.55 22.74	5.485 6.522	0.469 0.473
Lack of openness	treatment control	139 186	9.41 12.69	5.127 6.204	0.435 0.455
Inclination to	treatment	133	24.57	4.031	0.350
stereotypes	control	183	23.16	4.090	0.302
With the aim of determining a statistically significant difference between respondents' results in the area of knowledge and skills, an independent sample t-test was conducted. Table 3 shows that the above described differences between the groups are statistically significant, i.e. that the treatment group scored better in the area of specific knowledge and skills.
Table 3. Independent-samples t-test for4thgrade respondents - the comparison of the treatment and the control group in the post-test results for knowledge and skills
	t	df	p
Knowledge	14.939	348	.000
Skills	3.781	348	.000
df - degrees of freedom, p -statistical significance
Because data in the segment of attitudes did not follow normal distribution, a nonparametric rank-sum Mann-Whitney U test was applied to determine the significance of the differences between the treatment and the control
group. tte test was not conducted for the factor of curiosity due to the already mentioned difference between the control and the treatment group in the pretest. The results confirm that the above mentioned differences are statistically significant (see Table 4), which means that the treatment group pupils are more open toward the members of the German-speaking area, but they are also more inclined to stereotypes in relation to the members of other cultures. There are a few possible explanations for this interesting finding, one of which is that stereotypes may have been unconsciously transferred from the treatment group teachers in their lessons with a stronger emphasis on the target language culture and civilization. Teachers may have inadvertently transferred stereotyped images to pupils, who are open to accepting the attitudes from their environment, this especially being true for primary school pupils (Aboud, 1988, p. 128; Leopold-Mudrack, 1998, p. 67). Another possible explanation of this finding lies in the specifics of pupils' cognitive and affective maturation. Stereotyped notions are developed in pupils even before the beginning of schooling and are processed until the end of lower secondary education. tte focus shift from external to internal characteristics in the assessment of people occurs with age, so that it is only after ten years of age that pupils more readily accept the similarities between themselves and the persons from other cultures (Aboud, 1988; Selman, 1980, in Berk, 2008). It is, therefore, possible that the pupils in the treatment group, who were more intensively exposed to cultural contents of the target language, did not filter their acquired knowledge, but used it to develop stereotypes, i.e. to make generalizations. It is, thus, for example, possible for the pupils who learn that Germany is known for its beer production, to transfer the acquired knowledge to the area of their own beliefs and come to believe that all Germans drink beer.
Table 4. Mann-Whitney U test for 4th grade respondents - the comparison of the treatment and the control group in the post-test results for attitudes
	U	z	P
Lack of openness	8496.500	-5.399	.000
Inclination to stereotypes	9335.000	-3.546	.000
tte results of the post-test for eighth grade respondents significantly differ from those obtained in the fourth grades. If the mean results for groups in the post-test are compared (see Table 5), it can be seen that the control and the treatment group only differ in the section of knowledge. This difference is statistically significant (see Table 6).
Table 5. Descriptive parameters of overall post-test results for 8th grade respondents
	Group	N	Mean	Std. Dev	Std. Error Mean
Knowledge	treatment control	160 210	9.06 7.91	5.507 5.241	0.435 0.362
Skills	treatment control	160 210	22.45 21.50	7.781 9.119	0.615 0.629
Curiosity	treatment control	141 182	23.21 22.47	6.169 6.285	0.520 0.466
Lack of openness	treatment control	140 178	9.69 10.13	4.850 4.865	0.410 0.365
Inclination to	treatment	140	22.36	4.251	0.359
stereotypes	control	177	22.76	4.723	0.355
Although it was expected that all treatment group respondents would achieve better post-test results in all the tested segments of intercultural competence, this was not the case with the respondents from eighth grades. Slight differences between the control and the treatment group can be noted in the aspect of skills and attitudes (Table 5). However, these are not statistically significant (see Table 6). In other words, the treatment group pupils did not score better than the control group pupils in the area of skills and attitudes.
Table 6. Independent-samples t-test and Mann-Whitney U test for 8th grade respondents - the comparison of the treatment and the control group for the post-test results in the dependent variable
t-test	t	df	p
knowledge	2.042	368	.042
skills	1.080	368	.281
Mann-Whitney	U	Z	P
curiosity	11875.500	-1.152	.249
lack of openness	11708.500	-.934	.351
inclination to stereotypes	11280.000	-1.373	.170
The results show that the effect of teacher further professional development on the development of intercultural competence among eighth grade pupils was only evident in the cognitive aspect, i.e. in the field of specific knowledge of culture and civilization. It can be concluded that in the case of teenage
pupils, specific intercultural skills and attitudes are developed in a more complex way and over a longer period than among younger pupils. ttis conclusion can be based on the assumption that the attitudes of children up to a certain age are marked with more openness toward the world. Therefore, pupils under ten years of age do not possess a firmly developed system of values and attitudes, which is acquired in the process of socialization in one's own culture (Leopold-Mudrack, 1998, p. 65). ttis means that the attitudes of children under ten years of age toward other cultures and their members will be positive or neutral at the least. Further development is marked with stronger acceptance of norms and ways of behaving typical of one's own social circle, i.e. one's country, so that the openness toward the other and that which is different from one's own culture is diminished. ttis makes it somewhat more difficult to develop specific intercul-tural competences among lower-secondary pupils.
Nevertheless, if we take a look at the results for fourth grade respondents, certain similarities with the results described for eighth grade respondents are evident. Although the fourth grade pupils in the experiment group achieved better results in almost all the tested components of intercultural competence, the results show that the effect of intensive and more structured dealing with cultural topics is the greatest in the field of knowledge. An overall conclusion can be given that in the teaching of intercultural competence it is more difficult to influence complex skills and attitudes. Furthermore, this process becomes more complex and develops more slowly with age. It is, therefore, possible that the post-primary stage is not the best time to start dealing intensely with the contents aimed at developing intercultural competence.
Conclusion
In view of the results obtained it can be concluded that the first sub-hypothesis (H1) was confirmed for both fourth and eighth grade pupils and that the teacher professional development programme had a significantly positive effect on the learning outcomes in the cognitive aspect of intercultural competence. Furthermore, it can be said that the second sub-hypothesis (H2) was confirmed for fourth grade pupils, i.e. teacher professional development had a positive influence on the pupil achievement in the area of skills. tte second sub-hypothesis was not confirmed for respondents in eighth grade. Sub-hypothesis H3 was partially confirmed for fourth grade pupils. Teacher professional development influenced one segment of attitudes, i.e. pupils in the treatment group showed more desirable attitudes in view of openness toward individuals from the German-speaking area. This sub-hypothesis was not confirmed for the eighth grades.
In view of all the given results, it can be concluded that foreign language teacher professional development in the field of intercultural competence has positive effects on the pupil's learning outcomes, especially with younger learners, as with this group of respondents the participation in the experimental programme yeilded better results in all the aspects of intercultural competence with the exception of one aspect of attitudes. Positive outcomes of the experimental programme were less evident among eighth grade pupils. While this can be a consequence of certain limitations of short-term teacher development programmes, it should also be noted that the development of intercultural competence is a lengthy and slow process, especially with regards to complex skills and attitudes. This is also one of the possible reasons the effects of the experimental programme were limited.
In view of the complexity of the development of intercultural competence at the primary and lower secondary school levels, it should be noted that the further professional education course provided for the needs of this study was limited in its duration and comprised three day-long modules. The experimental programme was conducted in the course of only one year. Despite this, teacher participation in the programme and their subsequent application of the knowledge and skills acquired, i.e. their teaching in experimental classes, had positive effects on the pupil intercultural competence development. This is an argument in favour of such form of further professional education programmes. Nevertheless, it should be said that such programmes with a longer duration might give better results.
All the results presented thus far lead to the conclusion that the effects of short-term teacher professional development in the field of intercultural competence are very useful in spite of their limitations and that it is important to motivate teachers to participate in them. However, bearing in mind that the development of this competence is a complex and lengthy process, it is especially important to motivate teachers for continued activities related to intercultural competence in their foreign language teaching and preferably start with the pupils of the youngest school age. Regarding the design of teacher professional development programmes in the field of intercultural competence, particular emphasis should be placed on the problem of stereotypes and prejudices. While the outcomes of this study refer to the school system of the country in which it was conducted, they also point to the importance of teacher education in the development of learners' intercultural competence outside of the Croatian education system.
References
Aboud, F. E. (1988). Children and prejudice. Cambridge, MA: Blackwell.
Äijälä, H. (2009). Interkulturelle Kompetenz und ihr Testen im finnischen Fremdsprachenunterricht.
Am Beispiel der Abiturprüfung im Leistungskurs Deutsch vom Frühjahr 2008. Tampere: Universität
Tampere. Retrieved 20.1.2010 from http://tutklelmat.uta.fi/tutkielma.php?id=19984
Angrist, J. D., & Lavy, V. (2001). Does Teacher Training Affect Pupil Learning? Evidence from
Matched Comparisons in Jerusalem Public Schools. Journal of Labor Economics, 19(2), 343-369.
Berk, L. E. (2008). Psihologija cjeloživotnog razvoja. Zagreb: Naklada Slap.
Breka, O. (2012). Uloga udžbenika i nastavnika u razvoju interkulturalne kompetencije u nastavi
stranih jezika. Doctoral thesis. Zagreb, Filozofski fakultet.
Brym, A. (2004). Das Bild des fremden Landes in Aufsätzen von irischen und deutschen Grundschülern. Frühes Deutsch, Fachzeitschrift für Deutsch als Fremd- und Zweitsprache, 1(2), 39-43. Byram, M. (1997). Teaching and Assessing Intercultural Communicative Competence. London: Multilingual Matters.
Council of Europe. (2001). Common European Framework of Reference for Languages: Learning, Teaching, Assessment. Cambridge: Cambridge University Press.
Corcoran, T. B. (1995). Helping Teachers Teach Well: Transforming Professional Development.
CPRE Policy Briefs. Consortium for Policy Research in Education. Philadelphia, PA: University of Pennsylvania.
Croatian Education and Teacher Training Agency (Agencija za odgoj i obrazovanje). (2013). Strategija stručnog usavršavanja za profesionalni razvoj odgojno - obrazovnih radnika (2014 - 2020). Retrieved 10.12.2013 from http://www.azoo.hr/images/IPA/Nacrt_Strategije_IPA.pdf Dildy, P. (1982). Improving Student Achievement by Appropriate Teacher In-Service Training: Utilizing Program for Effective Teaching (PET). Education, 102(2), 132-138.
Domovic, v., & Godler, Z. (2005). Educational Systems' Efficiency Evaluation on the Basis of Student Performance: Comparison Finland - Germany. Društvena istraživanja, 14(3), 439-458. Gehrmann, S., Petravic, A., & Šenjug Golub, A. (2013). Interkulturelle Kompetenz und ihre Evaluation im Fremdsprachenunterricht - Schwerpunkt Primarstufe. Zagreber Germanistische Beiträge, 21, 301-327.
Eberhardt, J.-O. (2009). „Flaggen, Baguettes, auch wenn's komisch klingt, das Aussehen der Leute erinnert an Frankreich". Von der Herausforderungen, interkulturelle Kompetenz im Kontext von Fremdsprachenunterricht zu evaluieren. In A. Hu & M. Byram (Eds.), Interkulturelle Kompetenz und fremdsprachliches Lernen. Modelle, Empirie, Evaluation (pp. 253-272). Tübingen: Gunter Narr Verlag. Fantini, A. (2006). 87 Assessment Tools of Intercultural Communicative Competence. Brattleboro, VT: American Council on the Teaching of Foreign Languages. Retrieved 3.10.2010 from http://www.sit. edu/SITOccasionalPapers/feil_appendix_f.pdf
Filipan-Žignic, B. (2008). Das Konzept der Mehrsprachigkeit und Interkulturalität in der Grundschule - Stand und Perspektiven. In H. Sarter (Ed.), Lehrerkompetenzen und Lernerfolge im
frühen Fremdsprachenunterricht (pp. 1-13). Aachen: Shaker Verlag.
Feng, A., & Fleming, M. (2009). Assessing intercultural competence for purpose - the SAILSA project. In A. Hu & M. Byram (Eds.), Interkulturelle Kompetenz und fremdsprachliches Lernen. Modelle, Empirie, Evaluation (pp. 235-252). Tübingen: Gunter Narr Verlag. Hesse, H.-G., & Göbel, K. (2007). Interkulturelle Kompetenz. In B. Beck & E. Klieme (Eds.), DESI-Sammelband I. Sprachliche Kompetenzen. Konzepte und Messung (pp. 253-269). Weinheim: Beltz. INCA. Intercultural competence assessment. Retrieved 24.7.2008 from http://www.incaproject.org/
index.htm
Jacob, B. A., & Lefgren, L. (2004). ^e Impact of Teacher Training on Student Achievement: Quasi-Experimental Evidence from School Reform Efforts in Chicago. Journal of Human Resources, 39(1),
50-79.
Kennedy, M. M. (1998). Form and Substance in In-service Teacher Education. Research Report from the National Institute for Science Education. Madison, WI: University of Wisconsin. Little, J. W. (1993). Teacher's Professional Development in a Climate of Educational Reform. Educational Evaluation and Policy Analysis, 15(2), 129-151.
Languages and Cultures in Europe (LACE) (2007). ^e Intercultural Competences Developed in Compulsory Foreign Language Education in the European Union. Retrieved 6.9.2011 from http:// ec.europa.eu/education/policies/lang/doc/lace_en.pdf
Legac, v., Mikulan, K., & Siročic, D. (2007). Razvoj interkulturalne svjesnosti kod učenika mlade školske dobi. In Dijete i jezik danas. Zavičajnost u nastavi hrvatskog jezika. Interkulturalnost u nastavi stranih jezika (pp. 139-159). Čakovec - Osijek: Učiteljski fakultet sveučilišta u Zagrebu - Središte u Čakovcu i Učiteljski fakultet u Osijeku.
Leopold-Mudrack, A. (1998). Fremdsprachenerziehung in der Primarstufe. Voraussetzungen, Konzept, Realisierung. Münster and New York: Waxmann Verlag.
Mertesacker, M. (2012). Die Interkulturelle Kompetenz im Internationalen Human Resource Management. Eine konfirmatorische Evaluation. Lohmar - Köln: Josef Eul Verlag. National Curriculum Framework for Preschool Education, General Compulsory and Secondary School Education (Nacionalni okvirni kurikulum za predškolski odgoj i obrazovanje te opce obvezno i srednjoškolsko obrazovanje) (2010). Zagreb: Ministarstvo znanosti, obrazovanja i športa. Primary School Curiculum (Nastavni plan i program za osnovnu školu) (2006). Zagreb: Ministarstvo znanosti, obrazovanja i športa.
Pachevska, M. (2003). Interkulturelle Erziehung im früh beginnenden Fremdsprachenunterricht. Berlin: Logos Verlag.
Petljak Zekic, B., Rukljač, I., & Urek, S. (2013). Planiranje, organizacija i provedba modularno organiziranog stručnog usavršavanj učitelja razredne nastave grada Zagreba i Zagrebačke županije (zapad) - priručnik o stručnom usavršavanju. Zagreb: Agencija za odgoj i obrazovanje. Petravic, A. (2010). Udžbenik stranog jezika kao mjesto susreta kultura. Slike stranoga i vlastitoga u hrvatskim udžbenicima njemačkog jezika. Zagreb: Učiteljski fakultet Sveučilišta u Zagrebu & Školska knjiga.
Petravic, A. (2011). Interkulturelle Kompetenz im Fremdsprachenunterricht - Probleme und Perspektiven der Curriculumentwicklung. In V. Domovic, S. Gehrmann, M. Krüger-Potratz, & A. Petravic (Eds.), Europäische Bildung. Konzepte und Perspektiven aus fünf Ländern (pp. 105-128). Münster: Waxmann.
Petravic, A., & Šenjug Golub, A. (2012). Auf dem Weg zum interkulturellen Sprecher. Unterrichtsaktivitäten zur Entwicklung der interkulturellen Kompetenz im Unterricht Deutsch als Fremdsprache. Zagreb: Školska knjiga.
Piršl, E. (2011). Odgoj i obrazovanje za interkulturalnu kompetenciju. Pedagogijska istraživanja, 8(1), 53-70.
Official Journal of the European Union L394/10-18 (30.12.2006). Recommendation of the European Parliament and of the Council of 18 December 2006 on key competences for lifelong learning (2006/962/EC). Retrieved 11.4.2007 from http://eurlex.europa.eu/LexUriServ/site/en/oj/2006/l_394/ l_39420061230en00100018.pdf
Shah, M., & Alam, M. T. (2002). In-service teacher education (Concepts, Significance, Duration and Evaluation). Pakistan Journal of Education, 19(1).
Sercu, L. (2000). Acquiring Intercultural Competence from Textbooks. The Case of flemisch Adolescent Pupils Learning German. Leuven: Leuven University Press.
Šenjug, A. (2008). Stav učenika viših razreda osnovne škole prema Nijemcima. Metodika. Časopis za teoriju i praksu metodika u predškolskom odgoju, školskoj i visokoškolskoj izobrazbi, 9(2), 310-317. Vujičic, L. (2007). Kultura odgojno-obrazovne ustanove i stručno usavršavanje učitelja. Magistra Iadertina, 2(2), 91-106.
Yussefi, S. (2011). Interkulturelle Attributionskompetenz. Konzeptualisierungen, Operationalisierungen und empirische Testkonstruktion. München i Mering: Rainer Hampp Verlag.
Biographical note
Ana Šenjug Golub, dr., is Research Assistant at the Department of German Teacher Education at the Faculty of Teacher Education in Zagreb. Her research interests are related with foreign language teacher education, inter-cultural learning, early language teaching and evaluation of learning outcomes. She has participated in scientific research projects related with assessment of student learning outcomes and development of intercultural competence in compulsory education. She has presented her work in international conferences and has published in several scientific journals.