Finding advanced characteristics of student population participating in the study of knowledge: case of clustering students from Slovene TIMSS Advanced study on learning mathematics Barbara Japelj Pavesic Introduction With the constant development of information technology in last 50 years, there is more cooperation between people in the world every day. Ideas are shared amongst many interested groups with the general aim to be informed, to improve work, to be better. 50 years ago the idea of cooperation between countries on the field of education provided the start of the international comparative studies of student achievement and other outcomes of school systems. Within years, international comparative studies developed and learnt many lessons. In 1988, Postlethwaite (Postlethwaite, 1988) discriminated four major aims of comparative education: "identifying what is happening elsewhere that might help improve our own system of education"; "describing similarities and differences in educational phenomena between systems of education and interpreting why these exist"; "estimating the relative effects of variables (...determinants) on outcomes (both within and between systems of education)" and "identifying general principles concerning educational effects" (relationship between variables within an educational system and an outcome)". The first two were easily achieved and reflected in a series of national and international reports on research findings (i.e. TIMSS, PIRLS) while the second two were found to be more demanding. One of the largest and still remaining problems in educational research is finding causal effects of background factors to the achievement or other outcomes. As recognised by the research (Gustaffson, 2006; Kodelja, 2005), as pure scientific experiments are not applicable in educational studies and it is not possible to control all influential variables or set the control group, therefore the significant causal relations are difficult to discover. As unreliable causal relations could lead to worse decisions about changes in educational systems, in the few last years, educational studies mainly report only primary results of the measurement of knowledge in the form of national means by background variables or indices on national level. Comparisons between countries or between groups of students inside a country are therefore not precise enough to directly serve as proposals for educational changes in a country. The need for additional analyses to reveal deeper links between factors and outcomes encouraged researchers to find and try to use new types of analyses and report findings in other non-traditional forms. The ideas for analyses comes also from other research areas, one of them is the fast developing area of social network analyses and data mining tools. In this work, the use and results of one such approach to discover advanced information about the students of gymnasia participating in the international measurement of mathematics knowledge and supporting factors will be shown. The problem Slovenia has participated in TIMSS Advanced study, international measurement of trends in mathematics and science amongst the students in their last year of advanced mathematics program before entering university (Mullis, Martin, Robitaille, Foy, 2009). There were 10 countries participating in the measurement, each with specific characteristics of the population tested, one of them being the coverage index, defined as the size of the included population over the size of whole appropriate age cohort of young people in each country. The coverage index tells the size of population in the country, which is taught advanced mathematics and which should be taken into consideration while comparing mean achievements. The results of the TIMSS Advanced study show the largest mathematical knowledge in Russian Federation. In general, more specialized populations or lower coverage indices had higher mathematics achievement (Mullis, Martin, Robitaille, Foy, 2009). Slovene mathematical knowledge is about average on the international scale but the difference in the coverage indices from the best is high: for Russian Federation, this is 1.5 and for Slovenia, this is 40. In Slovenia, the school system requires from applicants for any university study to finish the general secondary school, called gymnasium and pass the final examination, called matura, at the end of it. The gymnasia program is the most advanced program in the country for all subjects, also for mathematics and the same for all despite the fact that for matura, students may decide to take the basic or the advanced level mathematics exam. Over recent years, almost 40 % of all elementary school students have decided to study at universities leading them to choose gymnasia as their secondary school and consequently, they have enrolled in the most advanced mathematics course in the country. The percentage of future students is still rising. Around a quarter of gymnasia students take advanced mathematics matura. Figure 1 shows the mean achievements of the participating countries and the mean achievements of Slovene students who take basic or advanced mathematics matura examination along with coverage indices for all participants. Figure 1: Mean Mathematics Achievement of countries in TIMSS Advanced with coverage index. TIMSS Advanced findings and trends from 1995 in Slovenia started the discussions between policy makers about the advantages and disadvantages of having the largest part of student population "forced" to study the most advanced and demanding mathematics. The problem is linked to the system of entrance to university studies. Students are accepted into the studies with a limited number of positions according to their final grade from matura, which is the sum of the grades from all five matura subject examinations. The advanced level of mathematics matura examination can bring students the maximum grade 8 and the basic level maximum 5. Choosing the advanced level of final mathematics examination therefore increases students' chances to be accepted into university studies with limitations on the number of students independent of the area of study. However, the student's decision for the advanced level of mathematics examination is also connected to motivation and self-confidence in mathematics. The policy makers, preparing new White book of Education, started to discuss whether it would be better to keep all population in the same demanding mathemat- ics program or provide more than one mathematics programs with different levels of difficulties to choose from during all four years of gymnasia. Before the final decision from the Ministry, the government changed and the discussion has once again become alive with new proposals from the new members of the Ministry to make changes also to the definition and meaning of the matura examination. The observed problems of advanced mathematics in Slovene secondary schools are low motivation for learning the most advanced mathematics amongst students who are able but do not need high grades to enter university because of no limitations for their chosen studies in the areas of science, mathematics and engineering, and low motivation for general mathematics in schools amongst students who do not intend to choose studies connected with or requiring mathematical knowledge, such as social science or philosophy. Mathematics teachers have expressed complaints that with the increasing number of gymnasia students every year, mathematical knowledge has decreased. Teachers have to adapt teaching mathematics more and more to less able and less motivated students because they need to help them pass the final mathematics examination. More able students feel, in such a system, less attention from teachers and have less opportunity to achieve a higher level of knowledge. Support for the new, most advanced mathematics program has also come from research showing that programs for mathematically oriented students should have specially defined curriculum with a focus on important mathematics concepts for future development of mathematics knowledge rather than just mathematical puzzles and challenging problems added to the regular mathematics curriculum (Gavin, Casa, Adelson, Carrol, Sheffield, 2009). To develop new advanced mathematics courses, a subpopulation of students who could be the applicants for them should be recognisable in the school system in advance. This study intended to help in discussions by finding and defining the characteristics of groups of Slovene students who reach higher mathematical knowledge and could be candidates for different mathematics programs in pre-university secondary schools. We tried to find the answers to the research questions whether groups of successful mathematics students have common recognisable characteristics, which are these characteristics and which characteristics of teaching process lead to the highest mathematical knowledge of students. Methodology The research was done in two parts. In the first part, the influential variables were found amongst all variables measured with TIMSS Advanced questionnaire for students. Basic statistical analysis of the Slovene student mathematics achievement data and students, teacher and school background data from TIMSS Advanced 2008 was used. The achievement of the more specialized sub-population inside the population of Slovene TIMSS Advanced students, defined by students' chosen level of mathematics final examination, school grades from mathematics, planned area of study and higher motivation for learning mathematics was compared. On the basis of significant differences in achievement dependent on variable values, variables were accepted or not in the second part of research. The aim of the analysis in the second part was to define groups of students with similar characteristics and test whether these groups show differences in their TIMSS achievement, by the method of hierarchical clustering for units described by symbolic data (Batagelj, Japelj Pavesic, Korenjak-Cerne, 2011). In comparison to usual clustering, the method used for clustering symbolic data assigned students to groups regarding real values of variables without calculating the "average" value of variables with nominal scales. The method of clustering takes into account all variables at the same time and assigns students who choose similar answers to questions from their questionnaires to give the number of clusters or groups. The groups are then studied and described with the help of additional analyses. First step analyses From all background variables available in TIMSS database, we selected the variables, which show statistically significant differences in achievement between more (achievement of 500 points or more) or less successful (achievement of less than 500 points) Slovene students in TIMSS advanced. As shown in table 1, in Slovenia, 28.5 % of TIMSS Advanced students achieved 500 points or more. This is 10.9 % of students from the whole age cohort and is closer to the coverage indices of other countries in TIMSS Advanced study. The mean achievement of students over 500 points was 553 points, similar to the mean achievement in The Netherlands (Mullis, Martin, Robitaille, Foy, 2009). Table 1: Achievement ofSlovene students over and under the internation- al average National Percent Percent (s.e.) Mathematics Mathematics benchmark Achievement Achievement (s.e.) under 500 points 71.5 1.9 418.9 3.7 over 500 points 28.5 1.9 553.8 2.1 Selected variables were divided into two content categories, the background of the students (such as attitudes of students toward mathematics and students' background, home environment, education of parents, socio-economic status and perception of future education) and the background of learning mathematics (such as descriptions of real learning in school, lessons, teacher's report about realisation of teaching in class, the student's view of the teacher characteristics and the school climate). Two databases were formed with variables from each list together with the student's identification. Both data files contained no variables describing student achievement. Second step clustering With the heuristic clustering algorithm implemented into the computer software (Clamix, Korenjak-Cerne, Batagelj, 2002), many solutions of clustering of students were obtained using two data bases prepared in the first step. The two solutions in which students were most similar inside all groups, for the clustering of student by background characteristics and for the clustering of students by characteristics of learning mathematics were chosen as final results. The values of variables that were chosen by the high percentages of students in each cluster helped us to describe the nature of the two sets of clusters. At the end, the assignments to clusters were added as a new variable to the initial database of students with all variables, including achievement. The usual statistic analyses to compare achievement over groups were performed with the program IDB Analyser to take care of the sampling and weighting issues of TIMSS database. Comparisons of achievement between clusters were discussed together with characteristics of each cluster. Results of the first step analysis The first step analysis showed large differences in achievement of students regarding many factors. As motivation for mathematics, intended area of study and grades from mathematics could be used as criteria for a student making a decision to take the more or less advanced mathematics program, those variables were studied carefully. But the highest achievement differences were found to be linked to student motivation for mathematics, intended area of study and student gender. Students who choose the advanced level of the mathematics examination achieved higher scores. Furthermore, 80 % of these students almost always or very much like mathematics. The opposite is 73 % of students who choose the basic level of mathematics matura examination and do not like mathematics at all or only sometimes. Table 2: TIMSS Mathematics achievement ofstudents according to their liking ofmathematics How much do you like mathematics Percent Percent (s.e.) Mathematics Achievement Mathematics Achievement (s.e.) Advanced level of final math examination Not at all 6 1.2 454 17.2 Sometimes 26 2.1 4 CO 8.3 Almost always 43 2.1 535 6.0 Very much 25 2.0 555 9.0 Basic level offinal math examination Not at all 27 1.7 411 5.2 Sometimes 46 1.8 430 4.7 Almost always 20 1.4 465 5.9 Very much 7 0.6 495 8.0 Slovene students who plan to study mathematics, science, computer science and engineering cover 43 % of all gymnasia students or 17 % of age cohort. Those who choose the advanced level of the final math examination, show a higher achievement than students who choose the basic level independently of the area of study as shown in table 3. Students who plan to study mathematics, health science, science, computer science and engineering (69 % in TIMSS sample) made up 27.9 % of age cohort. The mean mathematics achievement of their subgroup of students taking the advanced level of the final mathematics examination, covering 7.3 % of age cohort, was calculated to be 539 points, enough for the fourth place on TIMSS Advanced country ranking. Table 3: Achievement of students according to their intended area of study INTENDED AREA OF STUDY Percent Percent (s.e.) Mathematics achievement Mathematics achievement (s.e.) Advanced level of final math exam science 20.01 2.47 533.35 11.44 health sciences 20.24 1.96 536.49 8. engineering 16.42 1.90 533.24 12.18 business 4.49 1.27 503.41 13.19 computer and information sciences 5.52 1.38 541.74 17.72 INTENDED AREA OF STUDY Percent Percent (s.e.) Mathematics achievement Mathematics achievement (s.e.) mathematics 6.86 1.33 573.88 11.47 social sciences 21.00 1.80 484.44 9.17 other field of study 5.46 1.22 500.76 17.55 Basic level of final math exam science 11.35 1.02 456.69 8.17 health sciences 4.45 0.63 473.26 17.07 engineering 12.06 1.34 462.72 7.03 business 14.35 1.44 420.23 9.72 computer and information sciences 8 CO 5 0.66 457.32 8.22 mathematics 1.89 0.33 491.12 16.27 social sciences 38.75 1.90 426.32 6.07 other field of study 11.78 1.05 414.87 8.04 Grades are an important and complex problem in Slovene gymnasia, impacting on a student's chance to enter specific studies at university (Ivanus Grmek, Javornik Krecic at al., 2008). A comparison of grades at the final mathematics examination and achievement in TIMSS (Cankar, Japelj, 2010) has shown that grades from mathematics and TIMSS achievement are not strongly connected. Girls with similar achievement in TIMSS Advanced as boys obtained in school mathematics almost one grade higher than boys. School grades obviously contain an additional view of mathematics achievement, which favours girls. Students, who have the two best grades from mathematics in school, very good or excellent (about 40 % of gymnasia students or 16 % of age cohort) reached more TIMSS score points than students with lower grades, but excellent-graded girls have achievement closer to the achievement of boys graded with very good rather than excellent. The results are shown in table 4. Boys with excellent grades from mathematics have achievement similar to the mean achievement of the Russian Federation's TIMSS Advanced student population. Their coverage index is 2.1 %, a little higher than the coverage index of the Russian Federation and confirms the belief that small specific groups of students in the huge Slovene TIMSS Advanced population reach a very high TIMSS achievement. Table 4A: Achievement ofstudents by level ofmath examination according to their grades from mathematics in schools Grade for mathematics in grade 12 Percent Percent (s.e.) Mathematics Achievement Mathematics Achievement (s.e.) Advanced level of final math examination Sufficient 11.43 2.30 449.72 11.87 Good 20.03 2.13 479.78 5.82 Very good 33.81 2.37 534.19 5.94 Excellent 34.73 3.46 561.96 7.72 Basic level offinal math examination Unsufficient 0.79 0.36 377.72 20.24 Sufficient 41.15 1.67 411.07 5.27 Good 35.80 1.47 441.77 3.87 Very good 17.44 1.43 468.99 8.32 Excellent 4.81 0.77 493.13 12.02 Table 4B: Achievement of students by gender according to their grades from mathematics in schools Grade for mathematics in grade 12 Percent Percent (s.e.) Mathematics Achievement Mathematics Achievement (s.e.) Girls Insufficient 0.50 0.23 366.30 33.58 Sufficient 32.40 2.08 405.72 6.71 Good 32.08 1.44 438.27 4.89 Very good 23.16 1.48 481.13 8.00 Excellent 11.86 1.48 528.63 8.90 Boys Insufficient 0.72 0.58 389.64 18.55 Sufficient 35.45 2.05 426.03 6.87 Good 31.52 1.69 461.32 5.49 Very good 19.12 1.74 520.19 7.68 Excellent 13.19 1.93 560.37 8.80 The results of the analysis in the first step has revealed that there is no simple solution to define the group of potential students of more advanced mathematics program in gymnasia because each individual influential factor is linked to another important effect. Results of clustering on the basis of student background factors The best clustering solution consists of ten clusters of students. From the cluster characteristics, the general descriptions for groups of students assigned to each cluster were drawn and summaries for the most interesting clusters are given in table 5.1 Figure 2: The distribution of students into 10 clusters by two sets ofvariables. The largest differences amongst clusters were made by classroom practices, student grades, motivation, students' reasons for deciding to choose the advanced or the basic mathematics matura examination, a student's opinion of what makes a good mathematics teacher and parental support to students. The last two were collected by questions developed for national purposes and then added to the international version of the questionnaire. Students were asked how important are, in their opinion, the specific teacher's characteristics to be valued as a good mathematics teacher and whether their teachers posses each of the given characteristics (i.e. strong, reliable, adapts the speed of teaching to students' needs, gives homework, listens to students, is recognised as very good mathematician, explains content well...). Almost all students in all clusters highly appreciate good explanation of content from teachers. Together with teachers' report that their teaching does not rely strongly on mathematics textbooks, it supports the belief that learning mathematics in Slovenia is based on individual teacher explanations in classes. Additionally, teachers who give additional explanations and examples or adapt the speed of their explanation to suit the students' needs were found to be more appreciated by students in clusters with less successful students. 1 Extended reports on characteristics of clusters are in Appendix. Parental support to students is often discussed and promoted in the Slovene school system. We asked students about their relationship with their parents with questions traditionally used in some other national studies. From the agreement of students to the statement "Parents like me very much", we assessed the general support of parents for students, the agreement with the statement "Parents encourage my school work" measured parental support for student education and the agreement with "Parents think I am smart" tells us about the general perception of student success by parents. Agreement with all three statements was measured using four categories, totally agree, agree, disagree, and totally disagree. Table 5: Clusters of students based on student background Cluster 3: Motivated girls for mathematics with strong parental support almost all girls, with excellent grades for mathematics and physics in G8 2/3 take the advanced level of the mathematics examination and not physics strong support from parents enjoy mathematics problems, positive attitudes toward mathematics 60% always like mathematics and worked hard on TIMSS test. Cluster 7: Successful physics students boys who are good at physics but not at mathematics. 91% take physics as an optional subject in the final examination 76% do not take the advanced level of the mathematics examination. 58% had excellent grades from physics and 50% from math in G8 78% of students have their own computer. Cluster 8: Most successful students • Over 90% had excellent grades for mathematics and physics in G8 • Take the advanced level of the final mathematics examination in grade 12 • They recognise (94%) good teacher as someone who explains content well (94%) adapts speed of explanation to students' need (67%). • They choose the advanced level of the mathematics examination because they are doing well in mathematics and have positive attitudes toward mathematics. • 73% also take advanced physics program. • Almost 70% of students have their own computer. • 70% report that their parents think they are smart. • Two thirds are boys. Cluster 10: Students with high expectations of a good teacher with strong support from parents and high self-confidence. • high expectations of a good mathematics teacher • more than 80% of students report: good teacher gives additional explanations, adapts the speed of explanation to students' needs, is fair, has authority and provides clear grading criteria • having a good teacher was a very important reason for choosing the level of the mathematics examination • only 33% take the advanced level of the final mathematics examination. • 80% students said that parents like them very much • 60% of students' parents encourage their work for school. • 70% of students strongly agree that their parents think they are smart. In the final step, we linked clustering to achievement in TIMSS. As shown in table 6, students in cluster 8 have the mean achievement much higher than the mean achievements of students from other clusters. Table 6: TIMSS Advanced Mathematics Achievement by clusters based on student characteristics CLUSTERS Percent Percent (s.e.) Mathematics Achievement Mathematics Achievement (s.e.) 1 1.44 0.42 418.19 17.78 2 12.60 0.68 445.96 6.20 3 10.69 1.01 506.36 6.68 4 10.02 0.66 447.71 6.97 5 15.84 1.00 429.70 5.01 6 15.84 1.20 419.98 5.51 7 8.71 1.10 483.93 7.29 8 5 m 8 0.80 567.68 6.59 9 8.63 0.78 394.22 8. 10 7.89 0.64 482.29 9.77 Cluster 1 contains a small percentage of all TIMSS Advanced students who had missing values for variables of student background. By multiplying the percentage of students in a cluster with the mathematics coverage index for sample of TIMSS Advanced, 40.5 %, the coverage index of the group of students forming a cluster in the whole age cohort can be estimated. Therefore, Cluster 8 covers 3.38 % percentages of all Slovene students of the appropriate age cohort, which is comparable to the Netherlands' coverage index of 3.5 % and larger than the coverage index of the Russian Federation, 1.4 %. According to our expectations, students in cluster 8, described as higher achievers, reached the largest score of 567 points on TIMSS Advanced achievement scale. Achievement is statistically similar to the mean achievement of the first country on TIMSS Advanced scale, the Russian Federation and significantly higher than the mean achievement of the second country on the international scale, the Netherlands. Cluster 8 students overlapped significantly with the set of students who chose the advanced level of mathematics and physics as an optional subject in the final examination at the end of grade 12. The percentage of the latest students in the TIMSS Advanced population was 9.8 % and 4.0 % in the general population of students of the appropriate age (or coverage index). They achieved 548 score points on TIMSS Advanced test for mathematics, which would place them between the Netherlands and Lebanon on the TIMSS scale and this score not being significantly different from both. Students from cluster 8 differ from these students in their high motivation for advanced mathematics so higher achievement is expected. Figure 3: Mathematics Achievement by clusters of Slovene students. If the more advanced mathematics program in secondary school is going to be provided for students with characteristics from cluster 8, this program should address specific areas of the students' future university studies. Table 8 shows percentages and achievement of students in cluster 8 by their intended areas of future university study. Table 7: Percentages and achievement of students in cluster 8 by their intended areas of university study INTENDED AREA OF STUDY Percent Percent (s.e.) Mathematics achievement Mathematics achievement (s.e.) science 22.47 3.79 579.75 14.60 health sciences 22.51 4.02 567.73 12.92 engineering 24.77 3.01 565.30 9.76 business 2.08 1.56 561.36 28.84 computer and information sciences 6.28 2.34 592.05 16.97 mathematics 11.47 2.75 607.34 14.10 social sciences 7.52 1.84 522.39 17.85 other field of study 2.89 1.78 564.52 35.28 A high percentage of students from cluster 8 want to study science, health science and engineering as well as mathematics but neither business nor social science. The future mathematicians and computer specialists have the highest achievement. For health studies, high grades from mathematics at matura examination raise the chance for students to be accepted into the study. Other popular studies for students from cluster 8 have no entrance limitations but have at least a one-year mathematics course - so future students need strong mathematical knowledge from gymnasia for success at university. From the intention for study, it may be concluded that students from cluster 8 would, by and large, benefit from a more advanced mathematics course in secondary school for their future study at university. Results of clustering of students by their learning environment Accepted clustering solutions for grouping students by their actual learning environment also gave good results. Ten clusters of students differ mostly by the level of preparation of teachers for teaching and by characteristics that students reported teachers have from a set of characteristics defining a good teacher in students' view. Again, the cluster 8 turned out to have most wanted characteristics of effective learning and teaching as seen in table 9. Other clusters are described in Appendix B. Table 8: Description ofcharacteristics ofthe cluster with best characteristics by learning environment factors Cluster 8: the best teaching practice All students have very well prepared teachers who explain the content well, have authority, are fair and have clear grading criteria. appreciate student's work for mathematics outside school. always give homework, always as a set of exercises, never ask students to find examples of the use of mathematics or data collection and analyses. participated in training programs about mathematical content and the use ofl CT More than 2/3 of students never use computers or calculators for modelling, solving equations or algebraic expressions. More than 2/3 students • every lesson listen to teacher explanations • never have to read a textbook in school or for homework. More than 1/2 of students agree that the teacher • is preparing them well for final examination, • makes students like to work on mathematical problems, • makes students feel successful. Teachers have very high expectations for student achievement. Mean achievements of students in clusters by learning environment also show significant differences. The highest achievement was observed in cluster 8 as shown in table 9. Table 9: Mathematics Achievement of students in clusters by learning environments CLUSTERS Percent Percent (s.e.) Mathematics Achievement Mathematics Achievement (s.e.) 1 14.63 4.37 448.36 9.74 2 18.00 2.77 431.96 7.45 3 15.39 4.59 453.86 13.14 4 6.27 2.31 452.59 11.76 5 13.60 3.33 478.60 11.82 6 2.91 1.74 412.13 7.88 7 3.56 2.20 469.20 27.45 8 12.46 3.56 504.16 12.06 9 12.06 3.22 448.63 16.82 10 1.12 0.05 455.12 27.00 The highest mathematics achievement of students in cluster 8 supports the high importance of teacher background together with approaches to teaching for students' knowledge. Teachers in cluster 8 demonstrate hard and devoted work for students and students obviously appreciate good and very demanding teachers. Conclusion The general mathematics curriculum in Slovenia is demanding. However, at the end of secondary school, students can decide to pass only the basic level of mathematics examination and are therefore required to demonstrate mostly the applications of mathematical content only (i.e. calculate the limit of a function). The whole coverage of mathematics theory is required only from students taking the advanced level of the final mathematics examination (i.e. understanding the "epsilon" definition of the limit). In spite of the fact that the year by year defined mathematics curriculum is required to be covered by teaching in classes over the school year, the majority of teachers admitted that their basis for their list of required contents to be taught is the Catalogue of standards of knowledge required for basic or advanced mathematics matura examination, chosen according to the average decision about the level of the final mathematics examination taken by students in the class. Since the Catalogue of standards lists less contents and requirements than the curriculum itself, that means that many students do not get the opportunity to learn all planned theoretical mathematics for secondary school. As discussed, a more advanced mathematics program should therefore be offered to motivated students and to give them the opportunity to systematically learn theoretical mathematics for more years during their everyday lessons, at least to the extent of the required standards of knowledge for the advanced level of mathematics matura examination. By clustering, we were successful in finding the group of candidates for more advanced mathematics in Slovenia. They are students, who decide to take the advance level of the final mathematics examination, choose physics as one of two compulsory optional subjects for the final examination, are recognized as smart and are motivated for learning mathematics. They also intend to choose mathematically demanding university studies. As such, they could be recognised by their teachers and peers. The achievement of this specific group was found to be very high inside Slovenia but also the highest amongst other countries on the international TIMSS Advanced scale. It can be assumed that these students would choose a more advanced program of mathematics in school if such a program existed. With the use of non-traditional methodology of clustering, it was possible to reveal some characteristics of the learning and teaching in specific student populations that could be not found using standard methods. By clustering, we also discovered some characteristics of student learning environment in Slovenia, which most likely support achieving higher mathematical knowledge. Well educated, demanding but devoted teachers with high expectations of student knowledge, who explain the content well enough that students do not need to learn it from other sources such as textbooks are most important factors. The use of ICT does not seem to be important for better students results in mathematics as much as regular short homework and tests. The message from the students for teachers is clear, they do not need to fear being characterised as bad teachers by their students if they expect and ask a lot from students or avoid the daily use of ICT. Finally, but more importantly for our school system, we also confirmed that a special population of Slovene future students of science, engineering, mathematics and computer science achieved very high results on TIMSS tests, comparable to the achievement of leading countries on TIMSS Advanced scale. With clustering itself, we are aware that we cannot find or explain casual effects between factors in education but its results/clusters are homogenous groups of students or other people. Inside those groups, traditional statistics may more reliably study links and causal effects of specific background than in the whole population. The search for possibilities of comparing spe- cific effects of different factors inside subgroups over the whole populations is going to further the work in the development of new approaches to large data collection in international comparative studies of education. Appendix A: TIMSS Advanced students from Slovenia: Description of characteristics ofclusters by student background factors Cluster Summary of cluster characteristics Description of cluster Cluster 1 Students with missing values for majority ofvariables. Missing values Cluster 2 Students who take the basic level of final mathematics exam and who do not take physics as an optional subject at the final exam. They require from good teacher to give examples of items which they should know how to solve. Students report that the very important reasons for choosing the level ofthe mathematics exam were positive attitudes toward mathematics, doing well in mathematics and enjoy solving mathematical problems. Since they choose the basic level, it can be assumed that they do not enjoy mathematics or have very positive attitudes toward mathematics. Half of students were not motivated to work hard on TIMSS test, they only sometimes like mathematics. The proportion of girls and boys is similar to the whole population. Less motivated student for learning mathematics. Cluster 3 Students in cluster 3 are almost all girls. High percentages of them have excellent grades from mathematics and physics in elementary school (grade 8). They do not take physics for an optional subject in the final examination but almost two thirds take the advanced level ofthe mathematics exam. They are strongly supported by parents. They enjoy solving mathematical problems and have positive attitudes toward mathematics, as these were the main reasons to choose the advanced level ofthe mathematics exam. 60% of students in this cluster always like mathematics and admitted that they worked hard on TIMSS test. Highly successful and motivated girls for doing mathematics with strong support from parents. Cluster Summary of cluster characteristics Description of cluster Cluster 4 Students in cluster 4 are similar in their expectations of good mathematics teachers. More than 90% ofthem thought that it is very important that the teacher gives good additional explanations and examples, is fair and provides clear grading criteria. The teacher should also adapt the speed of their explanations to suit the students' needs and provide a list of examples of items students should know how to solve. In addition, for 87% of students (the highest percentage of all clusters) it is very important for good teacher to have authority. A large majority of students take the basic level of the final mathematics exam but not physics also because this take less time and they have a good teacher for preparing them for basic level of math exam. Students have active support from parents. More than 60% agree that parents like them very much and also that parents encourage them to study. It seems that students in this cluster are passive learners as they rely more on the teacher's capabilities to teach them than on their own learning. They shift responsibility for their mathematics knowledge onto the teachers and, to some extent, onto their parents. Passive learners with high demands from teachers and support from parents. Cluster 5 Students in cluster 5 who didn't take physics or the advanced level of the final mathematics exam exceeded 90 %. Over 90% of students also required from good teacher to give additional explanations and examples and adapt the speed oftheir instructions to suit the students' needs. 60% of students chose the basic level of the final mathematics exam because oftheir (lower) grades. More than 75 % admitted that they could try harder on TIMSS test and 60 % disagree that they tried hard. Around half ofthem reported that they put the same effort in the TIMSS tests as they do for tests in school. Therefore, these students were not prepared to work hard for their mathematics education. Less successful students, with neutral attitudes to mathematics who are not prepared to work hard for mathematics. Cluster 6 Students in cluster 6 were not taking the advanced level ofthe final mathematics exam or physics. They reported that in choosing the level of final exam, interest in mathematics, attitudes toward mathematics or having a good teacher were not important - the only important reasons for half of these students were grades and the possibility to easily pass the test. Their grades for mathematics from elementary school were not the highest and 62 % of these students only partially agreed that their parents think they are smart. Students not interested in mathematics, who choose the basic level of the final mathematics exam just to easily pass the test. Cluster Summary of cluster characteristics Description of cluster Cluster 7 Cluster 7 consists of a clearly defined population of interested boys who are good at physics but not at mathematics. 91% take physics as an optional subject in the final examination but 76% do not take the advanced level ofthe mathematics exam. 58% had excellent grades from physics in elementary school and half of them from mathematics. 78% of students has his own computer. Successful physics students. Cluster 8 Students in this cluster are overall successful. More than 90% had excellent grades for mathematics and physics in elementary school (grade 8) are decided to take the advance level ofthe final mathematics exam in grade 12 and 73% also take physics as an optional subject in the final examination. Almost 70% of students have their own computer and are totally convinced that their parents think they are smart boys and girls. Two thirds are boys. Two thirds admit that they choose the advanced level ofthe mathematics exam because they are doing well in mathematics and have positive attitude toward mathematics. Most successful mathematics students. Cluster 9 Students in cluster 9 are weak mathematics students who admit that they do not like mathematics - 75% ofthem report that they totally disagree with the statement "I like mathematics." 73% of students expect to get the lowest positive grade, sufficient, for mathematics in grade 12. 96% of students do not take the advanced level ofthe final mathematics exam and 94% do not take physics. From good teachers, 87% of students expect they are fair and adapt the speed of their explanation to suit the students' needs. Students describe that having an interest in mathematics is not at all important for choosing the advanced or the basic level of the final exam. Students with the lowest grades for mathematics. Cluster Summary of cluster characteristics Description of cluster Cluster 10 Students in cluster 10 are most similar to each other by their expectations from a good mathematics teacher. For more than 90% ofthe students, for a teacher to be rated as good, it is very important that the teacher gives additional explanations and adapt the speed of their explanation to suit the students' needs. The teacher should also be fair and for more than 80% of the students, it is very important that he/she has authority and provides clear grading criteria. Being taught by a good teacher was a very important reason for choosing the level of final the mathematics exam independent from the chosen level. In this cluster, the highest percentage of students from all other clusters reported high parental support. 80% of students said that parents like them very much and almost 60% of student said that their parents actively encourage them in their work for school. Only one third of students take the advance level of the final math exam and one quarter choose physics as optional subject at matura exam. The main difference between these students and those from cluster 4 is higher perception of students success by parents. ence. Almost 70% of students strongly agree that their parents think they are smart while in cluster 4, only 61% only agree that their parents think they are smart. Students with high expectations from a good teacher with strong support from parents and high self-confidence. Appendix B: TIMSS Advanced students from Slovenia: Description of characteristics of clusters by learning environment factors Cluster Summary of cluster characteristics Cluster 1: Teachers are less well prepared for teaching. Schools do not encourage students to take advanced level of mathematics final exam. Cluster 2: Students never use any computer technology and report math lessons are boring. Cluster 3: Teacher are very well prepared for teaching. More than 90 % of students have teacher explaining contents well and give a lot of emphasis to informal assessment of students progress. Teacher expectations for student knowledge are high for more than 60 % of students. Students use computers and need 1 hour to do homework in comparison to average 3 0 minutes in other groups. Half of students are not encouraged by school to take advanced level of math exam. Cluster 4: Half of students said teacher has no authority, and 45 % students reported that teachers have no clear grading criteria or are fair. Half of students in half of lesson are asked to read theory from textbooks. Half of them use computers at some lessons. Teachers of students in this cluster reported being limited in teaching by different student factors: different academic abilities of students, high student-teacher proportion, student who skip classes and tests as well as student who are not interested in mathematics. But half students have teachers who are very satisfied with their work. Cluster 5: Almost all students have very well prepared teachers to teach and 70 % ofthem have teachers satisfied with their work. More then 80 % of students have fair teachers with authority. More than 90 % of students never use computers or read theory from textbooks. Almost two thirds of students are never asked to learn facts or procedure by heart in lessons or for homework. Almost 80 % of students are only at some lessons asked to decide by themselves how to solve problems. Cluster 6: All students have extremely well prepared teachers who participated in professional development courses about math curriculum. Half of students do use the most demanding textbook available for secondary school programs. All students are not encouraged to take advanced level of math exam by their schools. Students never use computers and are never asked to learn something by heart. 94 % students reported that their teacher has authority but almost one third of students report that teachers do not explain the content well. Half of students report that teachers do not adapt speed of explanation to students' needs. Cluster 7: Students with mostly missing values. Cluster Summary of cluster characteristics Cluster 8: Well prepared teacher with all characteristics of good teachers; Homework every lesson, rare computer use, not boring lessons. Valued student work for mathematics outside the school. All students reported to have very good prepared teachers for teaching and confirmed that their teacher explains the content well, has authority, is fair and has clear grading criteria. Teachers of more than 90 % of students takes into account the student work for mathematics outside school lessons, always gives homework, always as set ofexercises and never to find the use of mathematics or work with data. 73 % students finish homework in less then 30 minutes and have teachers who adapt speed of explanation to students' needs. Teachers of more then 80 % of students participated in training programs about math content and use of ICT but only half of students in some lessons use computer. More than two thirds of students never use computer or calculator for modeling, solving equations or algebraic expressions. More than 65 % students every lesson listen to teacher explanations and never have to read textbook in lessons or for homework. 57 % of students admitted that they have not the best relationship with teachers, but for half students mathematics lesons are not boring. More than half of students agree that their teacher is preparing them well for final exam, get students like to work on math problems, and make student feel successful. Teachers ofhalf of students have very high expectations to students achievement. Cluster 9: Teachers of all students are very well prepared for teaching but their satisfaction with work is not very high. Teachers of more than 80 % of students explain content well, has authority and gives additional explanations. But teachers of 91 % students feel very limited in teaching because students do not have homework done, teachers of 78 % of students feel very limited by not interested students and more than 60 % of students have teacher reporting to be limited in teaching by large number of students in class and their different academic abilities. 87 % of students never use computers and never read textbooks. For homework they are never asked to do something different from exercises or problem sets. Teachers of more than half of students do not participate in professional development programs. Cluster 10: All students in this cluster have teachers with the highest self-confidence. All teachers said they are extremely well prepared for teaching all contents and do not participate in any professional development program. Their satisfaction with work is not high. More than 90 % of students confirm that teachers have authority. 80 % of students said teachers explain content well and have clear grading criteria. No student ever get homework, 66 % of them never need to memorize formulas and teacher do not test student knowledge regularly. Half ofthe students are never asked by teacher to learn something by heart. For half of the students, teachers have low expectations of student knowledge. References Batagelj, V., Japelj Pavešic, B., Korenjak Cerne, S. (2011). Clustering large data sets described with discrete distributions and its application on TIMSS data set. Statistical analysis and data mining, Volume 4, Number 2, 199-215. Gavin, M.K., Casa, T.M., Adelson, J.L., Carroll, S.R., Sheffield, L.J. (2009). The Impact of Advanced Curriculum on the Achievement of Mathematically Promising Elementary Students. Gifted Child Quarterly: 53; 188. Sage. http://www.sagepublications.com. Gustafsson, J. E. (2006). Understanding causal influences on educational achievement through analysis of within-country differences over time. Invited paper presented at the 2nd IEA Research Conference, Washington, 1-39. Ivanuš Grmek, M., Javornik Krečič, M., Vršnik Perše, T., Leban Rutar, T. (2008). Zahteve gimnazijskih učiteljev pri ocenjevanju znanja. Šolsko polje, letnik XIX, 1/2, 119-130. Japelj Pavešič, B., Cankar, G. (2010). Dosežki TIMSS in ocene matematike na maturi (TIMSS Achievement and grades from mathematics in matura exam). Sodobna Pedagogika, 2/2010, 118-140. Japelj Pavešič, B., Svetlik, K., Kozina, A., Rožman, M. (2009). Znanje matematike in fizike med maturanti v Sloveniji in po svetu. (Slovene national TIMSS Advanced Report) Ljubljana: Pedagoški inštitut. Kodelja, Z., (2005) Komparativne edukacijske raziskave in šolska politika, Šolsko polje, letnik XVI, 3/4, 212-226. Korenjak-Černe S., Batagelj, V. (2002). Symbolic clustering of large datasets. In Classification, clustering and data analysis: recent advances and applications, (Studies in classification, data analysis, and knowledge organization). Berlin; Heidelberg; New York: Springer, 319-327. Korenjak-Černe, S. (2002) Symbolic Data Analysis Approach to Clustering Large Ego-Centered Networks. Metodološki zvezki 17: Developments in Statistics, FDV Ljubljana, 45-53. Mullis, I.V.S., Martin, M.O., Robitaille, D.F., Foy, P. (2009). TIMSS Advanced International Report. TIMSS & PIRLS International Study Center, Lynch School of Education, Boston College. Postlethwaite, T.N. (1988) The Encyclopeadia of Comparative Education and National Systems of Education, Oxford, Pergamon. Preckel, F., Goetz, T., Pekrun, R., Kleine, M. (2008). Gender Differences in Gifted and Average-Ability Students: Comparing Girls' and Boys' Achievement, Self-Concept, Interest, and Motivation in Mathematics. Gifted Child Quarterly 2008; 52; 146. Sage. http://gcq.sagepub.com. Barbara Japelj Pavešic Finding advanced characteristics of student population participating in the study of knowledge: case of clustering students from Slovene TIMSS Advanced study on learning mathematics In recent years, comparative studies in education have developed also in reporting their results. Some limitations to the analysis and interpretations have been found and discussed amongst researchers leading them to find new methodology, which can reliably explain collected data. The most problematic are causal effects, which are hard to find and complex to interpret. However, since the policy makers need explicit ideas for changes in the educational system, new methodology has entered into the field of education from other areas. One of them is clustering, specifically adapted to symbolic or nominal data from educational studies, which can serve to help explain complex links amongst the factors and, as an additional step, to narrow the finding of reliable causal relations. In this work, the application of such clustering on the Slovene data from the international study of the measurement of mathematical knowledge amongst students before starting university, TIMSS Advanced has been shown. Key words: advanced mathematics, coverage index, clustering, student background, learning environment Barbara Japelj Pavešic Iskanje nadaljnjih značilnosti populacij dijakov v raziskavah znanja: primer razvrščanja slovenskih dijakov iz raziskave TIMSS za maturante V zadnjih letih so se primerjalne raziskave v izobraževanju razvile tudi na področju poročanja o rezultatih. Raziskovalci odkrivajo in razpravljajo o omejitvah pri analizah in interpretacijah podatkov o izobraževanju ter ob tem razvijajo nove metodologije, ki lahko bolj zanesljivo razlagajo zbrane meritve. Med večjimi problemi se izkazuje študij vzročnih povezav med znanjem in družbenimi dejavniki, ki jih je težko odkriti in zahtevno pojasniti. Ker kljub temu načrtovalci sprememb v izobraževalnih sistemih potrebujejo eksplicitne ideje za izboljšave, so v polje raziskovanja izobraževanja vstopile metodologije iz drugih področij. Ena med njimi je razvrščanje v socialnih omrežjih, prilagojeno simboličnim ali nominalnim podatkom iz raziskav izobraževalnih učinkov. Izkazalo se je, da lahko pomaga razložiti kompleksne zveze med dejavniki in v nadaljnjem koraku ožiti področje iskanja povezav med dejavniki na zanesljivejše vzročne zveze. V prispevku je prikazana metoda in primer takšnega razvrščanja na podatkih o Sloveniji iz mednarodne raziskave merjenja matematičnega znanja med dijaki pred vstopom na univerzo, TIMSS za maturante. Ključne besede: zahtevnejši program preduniverzitetne matematike, indeks pokritja, razvrščanje, dejavniki dijakovega okolja, učno okolje Danuta Urbaniak-Zajqc Empirical studies in Polish pedagogy - between quantitative and qualitative research The first textbooks in the field of methodology of educational research appeared in Poland in the late 1960's and 1970's, and by the end of this century, they did not have any competition. Only at the beginning of the new millennium were new studies published (but those previously issued are still widely used, especially by students). It can be assumed that the general 'methodological approach' of several generations of educators was formed by the research scheme postulated by the authors of those first textbooks. From today's perspective, it is clear that the concept of methodology of educational research contained in the first textbooks was placed in the positiv-istic model of science. This designation is supported by the structure of the research process, valuation of methods and research techniques, the formal purpose of the research, the position assigned to a researcher (as well as mentioning the name of A. Comte). Key words: methodology, educators, positivism, textbooks, research Danuta Urbaniak-Zajqc Empirične študije v poljski pedagogiki - med kvantitativnimi in kvalitativnimi raziskavami Prvi učbeniki s področja metodologije pedagoškega raziskovanja so se na Poljskem pojavili v poznih šestdesetih in sedemdesetih letih in so bili do konca tega stoletja brez konkurence. Šele na začetku novega tisočletja so bile objavljene nove študije (vendar že prej izdane še vedno predvsem študentje pogosto uporabljajo). Lahko domnevamo, da je splošni 'metodološki pristop' več generacij pedagogov je oblikovala raziskovalna shema, ki so jo postuli-rali avtorji prvih priročnikov. Z današnjega vidika je jasno, da je bil koncept metodologije pedagoškega raziskovanja, vsebovan v prvih priročnikih, umeščen v pozitivistični model znanosti. To ugotovitev podpira struktura raziskovalnega procesa, vrednotenje metod in tehnik raziskovanja, formalni smoter raziskovanja in položaj, ki je pri tem dodeljen raziskovalcu (kot tudi navedbe imena A. Comta).