Ina Ferbežar	UDK 811.163.6,243:37.016(4)
Nataša Pirih Svetina	DOI: 10.4312/linguistica.54.L277-291 Mateja Lutar
University of Ljubljana*
THE COMMON EUROPEAN FRAMEWORK OF REFERENCE: A REFERENCE FOR SLOVENE
1 INTRODUCTION
The document Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEFR), which was published in 2001 by the Council of Europe and translated into Slovene in 20111, "provides a common basis for elaboration of language syllabuses, curriculum guidelines, examinations, textbooks, etc. across Europe" (CEFR 2001: 1). It therefore facilitates the transparency and international comparability of language curricula, as well as of testing and the issuing of certificates in foreign and second languages. For this reason, in recent years the majority of European educational institutions have been trying to link their language curricula, and particularly their examinations, to the CEFR scales (see for example Martyniuk/Noijons 2007). Slovenia is no exception to this2. Thus, from 2010 to 2013 at the Centre for Slovene as a Second/Foreign Language at the Faculty of Arts, University of Ljubljana (Centre for Slovene)3, steps were taken to ensure that acquiring Slovene language proficiency, as well as the system of testing and certifying it, were aligned with the CEFR and consequently comparable with systems in other European countries. The ultimate goal was a new educational programme for adults in Slovene as a second and foreign language which would replace, after 14 years, the previous officially recognised programme Slovene for Foreign Learners4. This programme was the basis for the official testing of Slo-
Authors' addresses: Filozofska fakulteta, Univerza v Ljubljani, Center za slovenščino kot drugi/tuji jezik, Kongresni trg 12, 1000 Ljubljana, Slovenia. E-mails: ina.ferbezar@ff.uni-lj.si, n.pirih@ffuni-lj.si, mateja.lutar@ff.um-lj.si.
Skupni evropski jezikovni okvir: učenje, poučevanje, ocenjevanje (2011). Ljubljana: Ministrstvo RS za šolstvo in šport, Urad za razvoj šolstva. 22 May 2014. http://www.mizks.gov.si/fileadmin/ mizks.gov.si/pageuploads/podrocje/razvoj_solstva/Jeziki/Publikacija_SEJO_komplet.pdf. The Slovene National Examination Centre (30 June 2014, www.ric.si) has already linked English tests at the final school examinations (national assessment of knowledge and the matura) and English examinations for adults to the CEFR (Bitenc Peharc/Tratnik 2014); examinations for other languages are now also in the linking procedure.
The Centre for Slovene operates within the Department of Slovene Studies. Its main objective is to develop infrastructure for attaining, examining and certifying Slovene language proficiency; since 1994 the Centre for Slovene has been the government-appointed commission for these activities. 23 September 2014. http://www.centerslo.net/.
Slovenščina za tujce. 22 May 2014. http://programoteka.acs.si/PDF/slo_za_tujce.pdf.
*
1
2
3
4
277
vene language proficiency and for the issuing of the related certificates. The first steps were taken at the Centre for Slovene in 2010, when the language proficiency descriptors in the existing Slovene for Foreign Learners programme were compared with those in the CEFR (cf. Centre for Slovene Annual Report 2010)5. Further activities connected with linking Slovene examinations and coursebooks to the CEFR, and the issues that arose, are described in detail below. Throughout, the procedure relied heavily on the Manual for Relating Language Examinations to the Common European Framework of Reference for Languages (Manual).6 This included familiarisation with the CEFR and standard setting, as well as benchmarking examples of written and spoken language production collected at Slovene examinations in line with the methods set out in the Manual. At different stages of this process, a total of 15 experts were involved.
2 LINKING SLOVENE LANGUAGE EXAMINATIONS TO THE CEFR 2.1 Receptive skills
In Slovenia, Slovene as a non-native language is tested and certified according to a three-level system, which is determined by the current officially recognised programme for adult learners, Slovene for Foreign Learners. The three levels are basic, intermediate and advanced. The examinations consist of four or five components (subtests) of equal value: listening comprehension, reading comprehension, writing and speaking, while at the intermediate and advanced levels grammar in use is also tested (cf. Ferbezar/Pirih Svetina 2004a; Ferbezar 2009a and 2012).
Because the Centre for Slovene would like to ensure that the system for testing and certifying Slovene language proficiency is comparable with those elsewhere in Europe, in 2011 the trial linking of examinations against the CEFR began7. This was first applied to the tasks in the Slovene examination at the basic level. In the case of tasks that test reading and listening comprehension, it was necessary to align the following features: the rubrics, the text and individual items connected with it; in short answer tasks, input questions as well as the expected answers or output also had to be linked8. Example 1 shows a short answer listening task.
5	Annual report. 1 July 2014. http://www.centerslo.net/l2.asp?Ll_ID=7&L2_ID=95&LANG=slo.
6	Manual for Relating Language Examinations to the Common European Framework of Reference for Languages. 22 May 2014. http://www.coe.miyt/dg4/linguistic/Source/ManualRevision-proofread-FINAL_en.pdf.
7	The Centre for Slovene was also encouraged to do this by its membership of the Association of Language Testers in Europe, ALTE. 29 May 2014. http://www.alte.org/.
8	In this paper, vocabulary of language testing is used according to Ferbezar et al. 2004b.
278
Example 1
Rubrics:
Pozorno poslušajte anketo o gledanju televizije in glede na informacije, ki jih boste slišali, v izpitni poli na kratko odgovorite na vprašanja, kot kaže primer. Anketo boste slišali dvakrat.
Text (extract)9:
Ne vem, bi rekel da televizijo gledam kakšne tri do štiri ure na dan, predvsem ponoči. A se vam zdi to veliko? V glavnem gledam tuje programe, ker na slovenskih televizijah bolj redko prikazujejo tisto, kar rad gledam. Zakaj rad gledam televizijo? Da se kaj naučim. Zelo rad imam dokumentarne oddaje, o naravi, pa tiste o popotovanjih v daljne dežele. Te oddaje so zelo poučne pa še občutek dobim, da sem bil tudi sam že tam.
Items:
1.	Koliko časa na dan Aleš posveti gledanju TV?
2.	Zakaj gleda TV?
The experts involved in the procedure of linking (panellists)10 had to place each of the features on a scale (the selected scales in the above example were Overall listening comprehension and Listening to audio media and recordings) and, finally, to link the whole task to the appropriate CEFR level. Where panellists disagreed, discussion was followed by a further attempt to align the task, while the final alignment was determined as the average of all the alignments (intermediate alignments were also possible, e.g. between A2 and B1).
Even at the trial phase, certain shortcomings of the examination tasks were noted, both in the receptive and the productive parts of the test. Thus, for instance, a discrepancy arose between the alignment for the rubrics (which were often very comprehensive and more complex than the tasks themselves), the texts (in listening, for example, the alignment was dependent on the speed of speech, not just the language structures and vocabulary in the text), input questions (how demanding in terms of language and content) and expected output (for instance, whether it was necessary to write only a specific piece of information or, more demandingly, to summarise the text or a part of it in the answer). In the case of the example 1, this is shown in the diagram:
9	The task involves four texts in all, spoken by four speakers, and for each text there are two questions.
10	They were all experts in foreign language teaching and testing, in particular in Slovene.
279
Diagram: The linking of examination tasks and their average alignment from Example 111
Trial linking encouraged the staff at the Centre for Slovene to reconsider the existing testing of Slovene language proficiency according to the state of the art in language testing and, consequently, to revise the examinations12. Revision towards greater congruity with the CEFR was also probably one of the reasons that, in linking new examinations, no great discrepancies arose in the alignment of the individual characteristics of tasks. In developing new examination tasks item-writers relied primarily on, in addition to previous experience and sample tests in other languages, descriptors (can do statements) from the relevant CEFR scales. In the case of listening, for example, the scales for Overall listening comprehension, Understanding conversation between native speakers, Listening to audio media and recordings, and Listening to announcements and instructions were used. In choosing texts, the item-writers relied a great deal on the content analysis grids offered by the Manual (2009: 29-30, 153-179), in which texts are classified according to source, authenticity, discourse type, length, abstract-ness of content, vocabulary, grammatical structures and so on, as well as the level needed for comprehension.
After developing new tests, the tasks within them were aligned with the CEFR according to the above-described procedures and piloted on a representative population13. After piloting the receptive parts of tests (indirect subtests with a numerical score:
11	Similar procedures were applied in the linking of coursebooks and so similar questions arose there - see section 3.
12	This included leaving out tasks in the receptive part of the examination involving short answer questions, since lengthy experience had shown that the level of errors in grading answers was very high, regardless of the fact that markers were systematically trained for such grading.
13	One (free of charge) exam administration (March 2013) was completely devoted to piloting. If participants passed the examination in line with the criteria of the existing programme (Slovene for Foreign Learners), then the Centre for Slovene issued them with the appropriate certificate. The testees' answers and their results were used for further analysis.
280
listening and reading), a statistical analysis was carried out on the results using the classical test theory method, through which one ascertains the difficulty of individual test items, their discrimination (i.e. the characteristic of a test item which shows how well it discriminates between different candidates within the task, in each sub-test and the whole test) and the reliability of the tasks and the whole test. Test items for listening and reading were then sequenced in the ordered item booklet (from the easiest to the most difficult) and the panellists had to determine the cut-off score which should determine the performance standard and thus the pass/fail score. Because the number of panellists was rather small (10) and because the ultimate goal of the procedure was the development of a new programme and the revision of existing examinations rather than linking tests with the CEFR itself, it was decided to make use of a somewhat simplified method14: on the basis of how individual tasks and tests as a whole were aligned with the CEFR, a borderline or minimally acceptable person was defined15. The panellists marked in the ordered item booklet the border or test item that two out of three borderline persons would not be expected to answer correctly. The first standard setting was followed by discussion and after the second standard setting the cut-off score was finally set as a simple ratio; no specific statistical analysis was used. In this process there was a high level of agreement among the panellists, which was a consequence of close familiarity with the CEFR and above all consistent interpretation of the descriptors (achieved at the familiarisation stage).
It is worth mentioning that in the Slovene context establishing such standards is rather problematic: the test population is very specific, with the great majority of test takers being speakers of one of the South Slavic languages closely related to Slovene16. In receptive parts of the test at the basic level, this population achieves very good results (cf. the Centre for Slovene Annual Reports), considerably higher on average than the minimally acceptable person defined according to CEFR descriptors. The question thus arises as to how reliable the perception of the borderline person among panellists with experience of this (real) population is and what their expectations are with regard to the performance of the minimally acceptable person and whether, as a consequence - in spite of standardisation - they lower or raise the cut-off score.
14	The standard setting methods are precisely described in the Manual (2009: Ch. 6).
15	If, for example, a test is aligned at level B1, we imagine that the minimally acceptable person is someone whose performance is at the lower B1 level, or on the border between A2 and B1 (this is a person who, according to CEFR descriptors, is on the border between understanding "sentences and frequently used expressions related to areas of most immediate relevance", as well as "the main points of clear standard input on familiar matters regularly encountered in work, school, leisure, etc."). This question remains unanswered: whether it is better for panellists to imagine a real, specific person that they know well, or to take an abstract description as their starting point (cf. Manual 2009: 62).
16	Especially Bosnian, Croatian and Serbian.
281
2.2 Productive skills
During the preparation of a new learning programme, experience of benchmarking examples of written and spoken production in relation to the CEFR proved particularly valuable, since this provided a basis for the development of new rating scales and cutoff scores for test performance in productive skills. Examples of written and spoken production were collected at the pilot examination and special checklists were developed for their alignment with the CEFR. In this way, the panellists were able to analyse specific production according to different language categories and to benchmark them with the appropriate scales.
Example 2
Example of written production at advanced level17:
Po javnih raziskavah, morebitni uvod davka na mastno hrano in sladke pijače bi imel tako pozitivne, kot negativne posledice. Komentarije posameznikov so zelo različne. Po eni strani, z uvodom davka se bo polnila državna blagajna, ter, z višjo ceno na tako produkcijo, se bo zmanjšala uporaba masne hrane in sladke pijače in z tem poboljšanje načina prehrane prebivalstva.
1. Res je da mastna hrana in sladke pijače negativno vplivajo na zdravje uporabnikov take hrane. Z uživanjem teh produktov se povečuje problem prekomerne teže prebivalstva, ter različne bolezni. Vse to ima negativne posledice tudi na sistem zdravstva v Sloveniji.
Ne glede na to, mislim, da davka na tako produkcijo nebi bilo potrebno uvajati. Hrana v Sloveniji že tako predraga, in za ljudje z nizko stopnjo dohodkov nekatere produkte (kot sladke pijače, meso, sir) postale luksuz. Z uvodom davka se bo zmanjšala že tako slaba kupna moč prebivalstva. Tudi za podjetja, ki proizvajajo take produkte, morebitni uvod davka bi imel negativne posledice. Dvig cen na svoje izdelke - pomeni zmanjšanje prodaj.
Moje mnenje, da glavni cilj uvoda davka na mastno hrano in sladke pijače je polnenje državne blagajne z denarjem davkoplačevaljcev, ne pa zaskrbljenost o negativnem vplivu na zdravje prebivalstva.
17 The example is an exact transcription of the text. The task included the following rubrics: Ali bi morali uvesti davek na mastno hrano in sladke pijače ali ne?
DA: Mastna hrana in sladke pijače negativno vplivajo na zdravje in njihova uporaba ima dolgoročno velike posledice ne samo za posameznike, ampak tudi za državo. NE: Raven maščobe bi morali določati v laboratorijih, to pa bi bilo za državo predrago. Tudi mleko je mastno, pa nihče ne pravi, da ni zdravo. Vsak se mora sam odločiti o načinu svoje prehrane. Informacij o možnih posledicah je dovolj.
Komentirajte oba navedena argumenta, napišite svoje mnenje o izbrani temi in ga utemeljite ter ponudite rešitev. Besedilo naj bo dolgo od 220 do 250 besed.
Ocenjuje se:
•	upoštevanje vsebine in izpolnitev naloge (komentiranje, izražanje mnenja in utemeljevanje),
•	jezikovna pravilnost,
•	povezanost besedila v celoto.
282
Ljudje zelo dobro obveščeni o posledicah na zdravje mastne hrane in pijače. V vsakem zdravstvenem domu /ustanovi lahko najdemo informacije o tem. Zato, mislim, da bi tudi sami mogli odločati (illegible) načinu prehrane.
The written production from Example 2 was benchmarked by seven panellists with individual categories or scales, as shown in the table, which also shows the average of all seven panellists for individual language categories:
Category/ scale	General linguistic range	Coherence and cohesion	Grammatical accuracy	Creative writing	Reports and essays	Together
Range of alignments18	between A2 and B2	between A2 and B2	between lower A2 and B1	between A2 and C1	between A2+ and B2	between A2 and B2
Average	B1+	lower B2	A2+	B1	lower B2	B1
Table: Benchmarking of written production from Example 2 by language category
This example involves the typical written production of someone taking the Slovene examination who is a speaker of a language closely related to Slovene. It demonstrates significant deviation with regard to the alignment of different categories with the CEFR; particularly noticeable is grammatical accuracy, which, as can be seen from the table, is on average aligned lower than the other categories. During the linking process, this kind of uneven ("unflat") profile has proven to be unsuitable for a benchmark19, but is typical for Slovene as a second/foreign language test performance. It is worth emphasising here that the CEFR scales were specifically developed so that different skills and language categories within them could be bench-marked at different levels, which is usually in line with real language use. But this leaves open the question how to give an overall alignment for production that is on the one hand marked by quite elaborate cognitive processes, but on the other strongly deviates from explicit norms (see also Ferbezar and Stabej 2014) and which is typical for speakers of languages closely related to Slovene who do not learn Slovene systematically (that is, the majority of those sitting the examination in Slovene). This also applies to examinations as a whole: alignment with the CEFR facilitates the processing of "flexible speaker profiles" (in reality, the different communicative skills of the average language user, i.e. listening, reading, speaking and writing, are rarely at
18	As can be seen from the table, the lowest alignment in all categories is A2 or lower A2; this involved one panellist who benchmarked systematically lower and who was therefore excluded from the linking process. Through discussion, panellists decided that written production in certain parts of Example 2 perhaps exceeded the Threshold Level (B1), but that the average alignment at this level did not do so.
19	Examples to be used as benchmarks should be "flat", i.e. aligned at roughly the same level in all categories.
283
the same level). The problem appears with regard to the final score and certification: those who make use of the results of the examination expect language proficiency to be assessed with a single score, preferably numerical, whereas a flexible speaker profile calls for interpretation in the shape of a detailed supplement.
3 THE LINKING OF COURSEBOOKS
The Centre for Slovene also decided to label coursebooks for learning Slovene as a non-native language in line with the CEFR20. This process was initiated by the need for the transparency and comparability of language courses and the actual observed demands of users. Coursebook users want to know at which CEFR level the coursebook can be placed, or to which language level it will lead them. Since 2011, procedures have been carried out to link the coursebooks published by the Centre for Slovene with the CEFR21.
The linking process was applied to seven of the Centre for Slovene's most frequently used coursebooks and three that were in preparation. It involved seven panellists, each of whom closely analysed individual tasks in the same way, as described above in relation to examinations (see 2.1 Receptive skills). In contrast to the linking of examination tasks, in the case of coursebooks the panellists decided independently which of the 57 CEFR scales they would use. The reason why the scales were not defined prior to the alignment is that, unlike the examination tasks (as described in section 2.1, above), coursebook activities have no specifications based on the CEFR descriptors. Moreover, the experts involved in the alignment of the coursebooks were not guided by a document such as the Manual. This brings a new dimension to the linking process for coursebooks or the individual tasks within them: panellists could make use of different scales. This phase of the process could have been steered, so that as a first step consensus was reached about which scales to use for aligning individual tasks. But this was not done, meaning that tasks were aligned at a given level from different perspectives. In spite of the fact that the final identified level of individual tasks was the result of consensus among all the panellists, doubt remains about the methodological appropriateness of the procedure. And for some of the coursebook tasks there is no suitable scale in the CEFR with which they can be aligned: for instance, tasks where users are asked to practise, reinforce or repeat certain language structures.
Example 3
Primer: Laura dobro govori (govoriti) angleško.
a)	Jaz_(govoriti) slovensko, ne hrvaško.
b)	Ana, ali_(delati) kot natakarica? (Slovenska beseda v živo 1a, p. 25)
In the case of tasks that call for spoken or written production from users, various issues arose, and these are presented below.
20	As is the case for coursebooks of other languages, published by Langenscheidt, Oxford University Press, etc.
21	See Centre for Slovene Annual Report 2012. 26 September 2014. http://www.centerslo.net/files/ file/Zalo%C5%BEni%C5%A1tvo/LP%202012_za%20web(1).pdf.
284
Example 4
Pogovarjajte se.
Za katere praznike pišeš voščilnice? Za katere praznike kupiš darila? Za koga kupiš
darilo? Kaj kupiš? Katere praznike praznuješ? Kateri je tvoj najljubši praznik? Kdaj
in kako ga praznuješ? (Gremo naprej, task 8, p. 16)
This task requires that users first read the instructions and then engage in conversation. For aligning the instructions, the scale for Reading instructions were used and for its expected performance those for Overall spoken interaction, Conversation and Goal-oriented discussion. The consensus of the panellists was that the task prompted by the questions was at the lower B1 level. In the case of this and similar tasks, the question arose as to what is the lowest level of Slovene language proficiency that learners need to achieve in order to be able to perform the task22. In Example 4 very simple answers suffice (for example, to the first question: "Za božič."), which do not go beyond level A2. These kinds of tasks are open "upwards", in the sense that the answers can be more complex and thus aligned at a higher level.
Determining the lowest level of language proficiency required for task performance is one of the basic differences between the processes of linking coursebooks and examinations, for, in the case of the latter, examples of actual language production are available (see 2.2 Productive skills, above). In the case of coursebooks, observing language practice during courses as well as interaction between the coursebook, its user and the language teacher would be necessary.
A particular challenge with regard to linking was represented by tasks with comprehensive instructions that on the scale for Reading instructions were aligned much higher than the level expected for task performance, as well as tasks that demanded of users a range of different activities and the application of different skills. This is the case with Example 5.
Example 5
Situation:
Odigrajte sestanek prebivalcev Emonske ulice. Poskusite poiskati možne rešitve,
kako bi preprečili uničevanje zelenice v njihovi ulici.
1.	Sestavite vabilo na sestanek prebivalcev Emonske ulice.
2.	Razdelite si vloge in se pripravite na sestanek.
3.	Odigrajte sestanek, na katerem pojasnite in argumentirajte svoje mnenje.
4.	Na koncu glasujte za najboljši predlog. (Slovenska beseda v živo 2, p. 105)
This elaborate two-step instruction is followed by descriptions of seven possible roles:
22 This is done in a way similar to the definition of the minimally acceptable person when linking examinations.
285
1. Predsednik hišnega sveta, Janez Kmet, upokojeni učitelj: vodi sestanek.
6. Gospa Potokar je mlada mamica. Njen mož je pilot in ga pogosto ni doma. Ima dojenčka, ki je pogosto bolan, zato potrebuje avto blizu bloka. Vseeno ima rada naravo v mestu. Nekaj površine zelenice bi ohranila in tja namestila opozorilne table, nekaj površine pa bi uredila v parkirna mesta za stanovalce. Če bi bil njen parkirni prostor zaseden, bi poklicala pajka.
The descriptions of roles are followed by eight proposals for how to act in the given situation, such as:
1. zavarovati zelenico z železnimi količki 2 prerezati gume vsem avtomobilom, parkiranim na zelenici
The task demands a great deal of reading from the learner, although the actual aim is not reading, but rather role play, discussion, the exchange of views, argumentation and so on. For this reason, the problem is the complexity of the activity as well as the instructions. The instructions should present the context or situation as simply as possible; at lower levels, this is rather difficult to achieve. Of course, in such cases the teacher helps to interpret them by using other (learner) languages or other strategies.
The panellists used different scales to align these tasks: Overall reading comprehension, Reading instructions, Overall spoken interaction, Formal discussion and meetings, Goal-oriented co-operation, Information exchange, Taking the floor (turn taking) and Co-operating. Such a wide range of scales was used because of the great complexity of the given task. It is worth mentioning that among all the different didactic approaches, this kind of task-based learning is recognised as one of the most authentic since it simulates real-life communication (in this case writing an invitation, organising a meeting, participating in a discussion, developing an argument, etc.).
Each panellist independently aligned the tasks in the coursebook, the individual units within it, and the coursebook as a whole. At joint meetings, panellists reported which scales they had used for alignment, and consensus was then reached regarding the level of each task. The level of separate coursebook units was determined and, finally, the level of the coursebook as a whole. At the same time, the level of individual communicative skills (reading, listening, speaking, writing) throughout the coursebook was determined; the main focus was on whether or not these were consistent throughout, that is, whether or not reading tasks would lead the coursebook user to the same level as speaking tasks, etc. After the meeting, each panellist would write a final report on the alignment of a specific coursebook. The level of Slovene language proficiency to which, based on the CEFR, a coursebook leads the user is now shown on the covers of the new editions.
The linking process largely confirmed expectations with regard to the level of Centre for Slovene coursebooks, which seem to "cover" almost all CEFR levels, from A1 to C1. Some coursebooks showed a systematic increase in level of difficulty as they
286
progressed (for instance, within B1 level: from lower B1 to B1+; or between two levels, such as A2 and B1). In some cases, however, it emerged that units towards the end of the coursebook were at a lower level than those at the beginning. From the pedagogical point of view such a decrease is fully acceptable if it is intentional. If it is not, this might be avoided by suitable planning in the early stages of the coursebook development. In this process the CEFR seems to be a very useful toolkit to refer to. Whether or not the level of proficiency foreseen has been achieved by using particular coursebook might be evaluated by testing. This means that appropriate tests should be developed, for determining the level of coursebooks without appropriate feedback (e.g. using appropriate testing procedures) is to a large extent still a matter of inspired guesswork.
4 CONCLUSION
The main purpose of linking language examinations and coursebooks for Slovene as a second and foreign language to the CEFR was not to offer a statistically verifiable alignment following procedures recommended in the Manual - the Centre for Slovene has neither the financial nor the human resources for this. Above all, the linking activities offered an opportunity for more precise analysis of examinations and coursebooks, as well as pointing the way forward for their further development; another tangible result of the project was, as already indicated, the development of a new educational programme and the revision of the existing system of testing and certifying Slovene language proficiency, which will facilitate testing speakers' proficiency in a more flexible and transparent way. Consequently, this will enable the Centre for Slovene to issue certificates showing speakers' language profiles.
A number of interesting questions arose during the procedures described here, and we have already tried to address some of these (e.g. how to align production of speakers of languages closely related to the target language, how to define the minimally acceptable person, how to align complex tasks demanding the most diverse language operations, how in this context to anticipate the performance level required, and so on). By way of conclusion, there are certain other issues we would like to consider. Certainly, each linking of language curricula and examinations needs to be treated cautiously: the CEFR represents a recommendation, not a prescription, which should be used only as a resource (CEFR 2001: xi, xiii) and only within the contexts for which it was developed23. Numerous questions regarding its universality were raised at the very beginning of its use, which is why profiles based on the CEFR have been developed for specific languages24. Furthermore, the name of the highest C2 level (Mastery) implies an ideal - or at least idealised - speaker, which is merely a theoretical construct. This "absolutism" is also apparent from the descriptors, with which even native speakers can find it hard to identify: although language skills formulated with "can do" are at first sight defined as variable, the
23	Linking language curricula and tests for children and young people to the CEFR can also mean its abuse: the CEFR was developed for adults whose cognitive skills are completely different from those of children and teenagers.
24	E.g. Profile Deutsch, English Profile, Profilo della lingua italiana, etc.
287
content of the descriptors are more a kind of expectation of what a C2 user should be able to do, rather than what the language user can do (cf. Widdowson 2014).
But the main problem regarding the (uncritical) use of the CEFR lies in its "narrowing of viewpoint": in principle, the CEFR promotes multilingualism as a social phenomenon and plurilingualism as an individual ability (CEFR 2001: Ch. 1.3) - a reality of the contemporary globalised world and of the individual speaker within it. Thus, in this context, the large-scale and frequently uncritical linking of curricula and examinations in national and other languages as autonomous and closed systems are often seen as a step towards the (re-)establishing or strengthening of monolingualism, which in the case of national languages in EU countries is currently being strongly promoted by the authorities (as well as being expected by language societies and shown in their practices). This is clearly paradoxical, since from a contemporary perspective monolingualism is an outmoded concept. Although the creation of plurilingual individuals involves the adding of new languages, according to Heller (1999, cited in García and Li Wei 2014: 43), it is really a case of "parallel monolingualism". In the globalised world, language speakers react against monolingualism by the application of "fluid" language practices, marked by the use of all the available language resources that have been integrated into the individual's repertoire, i.e. translanguaging (cf. García and Li Wei 2014). Being aware of both principles - monolingualism as social and political practice, as well as plurilingualism as an individual person's experience - language pedagogy (which includes language teaching and testing) should create its own way to approach them efficiently. And if concepts such as plurilingualism and translanguaging are already established in the context of language teaching25, in the case of language testing, which at first sight seems by its very nature to exclude interlanguage dynamics (rather than something positive, this is seen as a deficiency and is penalised as an error), they remain a challenge.
Primary sources
Slovene language exam papers on the basic and advanced levels (archive of the Center
for Slovene as Second/Foreign Language). MARKOVIČ, Andreja et al. (2004) Slovenska beseda v živo 2. Ljubljana: Center za
slovenščino kot drugi/tuji jezik. PETRIC LASNIK, Ivana/Nataša PIRIH SVETINA/Andreja PONIKVAR (2009) Gremo naprej. Ljubljana: Center za slovenščino kot drugi/tuji jezik.
Bibliography
Annual reports of the Centre for Slovene as a Second/Foreign Language. Letna poročila.
15 July 2014. http://www.centerslo.net/l2.asp?L1_ID=7&L2_ID=95&LANG=slo. BITENC PEHARC, Suzana/Alenka TRATNIK (2014) Umestitev nacionalnih izpitov iz angleščine v Skupni evropski jezikovni okvir. Zaključno poročilo o izvedbi projekta. Ljubljana: Državni izpitni center.
25 More in García and Li Wei 2014. See also the Framework of Reference for Pluralistic Approaches to Languages and Cultures (FREPA). 3 July 2014. http://carap.ecml.at/.
288
Common European Framework of Reference for Languages: Learning, Teaching, Assessment (2001). Cambridge: Cambridge University Press. 15 July 2014. http:// www.coe.int/t/dg4/linguistic/cadre1_en.asp.
English Profile. 15 July 2014. http://www.englishprofile.org/.
FERBEŽAR, Ina (2009a) "»Je moj B1 tvoj B1?« Ali kaj je novega na področju testiranja in certificiranja slovenščine kot drugega/tujega jezika." In: M. Stabej (ed.), Infrastruktura slovenščine in slovenistike. Simpozij Obdobja 28. Ljubljana: Znanstvena založba Filozofske fakultete, 129-136.
FERBEŽAR, Ina (2009b) "Med dvema jezikoma: med razumljivostjo in sprejemljivostjo." In: V. Požgaj-Hadži et al. (eds), Med politiko in stvarnostjo: jezikovna situacija v novonastalih državah bivše Jugoslavije (Kultura sožitja). Ljubljana: Znanstvena založba Filozofske fakultete, 197-211.
FERBEŽAR, Ina (2012) "»Izrekam zvestobo moji novi domovini Republiki Sloveniji ... «: testiranje znanja slovenščine kot drugega in tujega jezika v Sloveniji." Jezik in slovstvo 57/3-4, 29-45.
FERBEŽAR, Ina/Nataša PIRIH SVETINA (2004a) "Certificiranje slovenščine kot drugega/tujega jezika - zgodovina in perspektive." Jezik in slovstvo 49/3-4, 17-33.
FERBEŽAR, Ina/Nataša PIRIH SVETINA/Matija SVETINA/Anja PODLESEK (2004b) English-Polish-Slovene glossary of language testing terms. Angielsko-pol-sko-slowenski glosariust terminow z zakresu testowanina bieglosci jqzykowej. An-gleško-poljsko-slovenski pojmovnik s področja jezikovnega testiranja. Krakow: TAiWPN Universitas.
FERBEŽAR, Ina/Marko STABEJ (2014) "Slovene or not Slovene? Issues on assessing speakers of closely related languages." Strani jezici (accepted for publication).
GARCIA, Ofelia/Li WEI (2014) Translanguaging. Language, Bilingualism and Education. New York: Palgrave Macmillan.
GLABONIAT, Manuela/Martin MÜLLER et al. (2005) Profile Deutsch. Berlin: Langenscheidt.
Manual for Relating Language Examinations to the Common European Framework of Reference for Languages. 22 May 2014. http://www.coe.int/t/dg4/linguistic/Source/ ManualRevision-proofread-FINAL_en.pdf.
MARTYNIUK, Waldemar/Jose NOIJONS (2007) Executive summary of results of a survey on the use of the CEFR at national level in the Council of Europe Member States. Policy Forum: The Common European Framework of Reference for Languages (CEFR) and the development of language policies: challenges and responsibilities. Strasbourg: Intergovernmental Language Policy Division. 23 September 2014. http://www.coe.int/t/dg4/linguistic/Cadre1_en.asp.
Skupni evropski jezikovni okvir: učenje, poučevanje, ocenjevanje (2011). Ljubljana: Ministrstvo RS za šolstvo in šport, Urad za razvoj šolstva. 22 May 2014. http:// www.mizks.gov.si/fileadmin/mizks.gov.si/pageuploads/podrocje/razvoj_solstva/ Jeziki/Publikacija_SEJO_komplet.pdf.
Slovenščina za tujce. Program. 22 May 2014. http://programoteka.acs.si/PDF/slo_za_ tujce.pdf.
289
SPINELLI, Barbara/Francesca PARIZZI (2010) Profilo della lingua italiana. Milano: La Nuova Italia.
WIDDOWSON, Henry (2014) Sprachenbildung neu gedacht: Traditionen auf dem Prüfstand. Conference Quality education and language competences for 21st century society: traditions, challenges and visions. Graz: ECML. 7 July 2014. http://www.youtube.com/watch?v=1jSpA37ENrc.
Summary
THE COMMON EUROPEAN FRAMEWORK OF REFERENCE: A REFERENCE FOR SLOVENE
In recent years the experts at the Centre for Slovene as a Second/Foreign Language have been linking Slovene language coursebooks and examinations to the CEFR. This paper describes the complex process of aligning texts, tasks and examples of language production (the spoken and written production of examination participants). A number of questions arose during this process, including: how to align tasks where the rubrics are at a higher level than the accompanying activities and the expected task performance? How to define the minimally acceptable person, on the basis of which the cutoff score is determined in receptive skills? How to align tasks where the performance calls for the use of different language skills? A particular challenge is represented by the benchmarking of the written and spoken production of the speakers of languages closely related to Slovene who make up the great majority of test takers. The final result of the linking process is a new language programme for teaching/learning, testing and certifying Slovene as a second and foreign language.
Keywords: Common European Framework of Reference for Languages, alignment, benchmarking, Slovene language exams, Slovene language coursebooks
Povzetek
SKUPNI EVROPSKI JEZIKOVNI OKVIR: REFERENCA ZA SLOVENŠČINO
V zadnjih letih smo v Centru za slovenščino kot drugi/tuji jezik uvrščali učbenike za slovenščino in izpite iz znanja slovenščine kot drugega in tujega jezika na lestvice Skupnega evropskega jezikovnega okvira. V prispevku predstavljamo kompleksne postopke uvrščanja besedil, nalog in jezikovne produkcije (govorne in pisne produkcije udeležencev izpitov) na lestvice SEJO. V procesu uvrščanja sta se nam odprli dve temeljni vprašanji: prvo se nanaša na uvrščanje nalog, pri katerih so vhodno besedilo in navodila na višji ravni kot pripadajoča dejavnost in pričakovana jezikovna raba/performanca. Drugo vprašanje pa je povezano z izdelavo lestvic za ocenjevanje produkcije govorcev slovenščini sorodnih jezikov, ki predstavljajo večino udeležencev izpitov.
290
Končni rezultat procesa uvrščanja je nov jezikovni program, ki pomeni izhodišče za poučevanje/učenje, testiranje in certificiranje slovenščine kot drugega in tujega jezika.
Ključne besede: Skupni evropski jezikovni okvir, uvrščanje, izpiti iz znanja slovenščine, učbeniki slovenščine kot drugega/tujega jezika
291