Informática 29 (2005) 477-481 477 Comparative Analysis of Educational Networks Alenka Žibert The National Education Institute of the Republic of Slovenia, Poljanska 28, 1000 Ljubljana, Slovenia. E-mail: alenka.zibert@zrss.si Vladimir Batagelj University of Ljubljana, FMF, dept. of Mathematics, Jadranska 19, 1000 Ljubljana, Slovenia. E-mail: vladimir.batagelj@fmf.uni-lj.si Vladislav Rajkovič University of Maribor, Faculty of organizational sciences, Kidričeva 55a, 4000 Kranj, Slovenia. E-mail: vladislav.rajkovic@fov.uni-mb.si Keywords: networking, educational networks, portals, comparative analysis Received: May 30, 2005 Abstract: Educational networks and portals are a formation of thematically gathered data on the web. Structure of national educational networks and portals depends on environment of their origin. In the paper, selected educational networks are analysed according to the following criteria: content-services, navigation, search, user interface, help, credibility, validity and target groups. The criteria were identified by a group of experts and final users (students and teachers) on the basis of web survey. The importance of criteria was articulated by using the Analytical Hierarchical Method and program Saaty. For the evaluation was selected Slovene National Educational Network (SIO) as well as educational networks of Canada, Ireland, United Kingdom, Europe, Germany, Africa, Australia and America. Based on the results of the comparative analysis a concept and guidelines for improvements of our national educational network SIO were prepared. Povzetek: Podana je primerjava izobraževalnih mrež v več državah. 1 Introduction Recently we decided to evaluate the current status and position of SIO - the Slovenian Education Network (http://sio.edus.si/), and to prepare some guidelines for its improvements and future development. For this purpose we first made and overview and comparisons of selected educational networks (EN). In this paper we present the main results of this analysis. SIO was founded in 1995 with the aim of providing access to individual educational servers and the material they offer. Educational users need a safe online environment they can trust and we strive to create one. Its main goals were: • to connect educational servers in Slovenia; • to collect and organize the information about educational resources and events in Slovenia and world-wide; • to support collaboration among students, teachers and parents; • to facilitate the distribution of educational materials and products; • to provide support for solving common problems (FAQs, recommendations, manuals, dictionaries, libraries of templates, ...); • to support distance learning; • to provide access to official documents on education (curricula, projects, announcements, ...). To automatize most of the SlO's functions we developed our own support system Trubar - a system of programs for Windows to build, search and maintain the catalogs -collections of units described by list of properties (dictionaries, directories, lexicons, catalogues, inventories, glossaries ...). It is freely available at: http://www.educa.fmf.uni-lj.si/trubar/. Tools, like Trubar, support the idea that every user should also contribute to the growth of a network. At the very heart of SIO are its catalogues of information - different collections of data: interesting websites, educational resources, educational institutions, educational events and more. Besides this SIO offers some additional services such as: Ask the experts - services that help users to solve any problem related to teaching and learning with ICT; Bulletin board; Forum; Distance learning support- a collection of educational materials: articles, online textbooks and manuals; Electronic journal List SIO. SIO is a member of EUN Schoolnet (www.eun.org) and we are collaborating on different projects on national and international level. Schools and individuals are encouraged to take part in several actions and projects. 478 Informatica 29 (2005) 477-481 A. Žibert et al. strani samo v slovenščini obvestila stihi iskanja H Pomladni dan 2005 ■ MIRK2005 BPom*nki Soglasna deska 'vprašanji Li ulenje na daljavo elektronski naslovi isti ■ poliči ■»dodaj — aidnjl teden učenci namen in zgradba M navodila in priporočil* ^ statistik » učitelji, vzgojitelji program Ro J izobraževanje S vodstvo iol« y uganke * ^priporočen» oprema ® natečaji 0 projekti starši in ostali stroke Edus J u re ü Pnovoiti priporočeno J novice B dogodki Učno okolje LOGU H vrtač 3osnovna tola H sr« dnj a čela y vUje In visoke iol« QUI ' ^L - - *PrcJ»kïûKO Figure 1: SIO - Slovenian Educational Network entry page 1 Overview and Comparisons Educational networks selected for the overview and comparisons are listed in Table 1: Table 1: Selected educational networks Name URL Type SIO - Slovensko izobraževalno omrežje, Slovenia http://sio.edus.si EN EUN Schoolnet http://www.eun.org EN Schoolnet Africa http://www.schoolnetafrica .net/ EN Canada's Schoolnet http://www.schoolnet.ca/ EN Scoilnet, Irland http://www.scoilnet.ie/Sco ilnet/ EN EDNA Education Network Australia http://www.edna.edu.au EN NGfL - National Grid of Learning, UK http://www.ngfl.gov.uk/ EN Ask ERIC -Educational Resource Information Center http://www.askeric.org/ portal SAN - Schulen ans Netz, Germany http://www.schulen-ans-netz.de/ EN First we identified their basic characteristics - criteria for comparison and possible directions of improvements of SIO. They are presented in Table 2. Table 2: Basic characteristics SIO EUN Schoolnet Canada's Schoolnet Schoolnet Africa Scoilnet EDNA NGfL Ask ERIC SAN BASIC DATA I C C C I C C I C CREDIBILITY founder missing y y y y y y y y VALIDITY OF LINKS not up to date y y y y y y y y d -d D d d -d d d l d l d l d d NAVIGATION y- P y- p y y y- p y+ p y+ p y y- p USER INTERFACE y y y y y y y y y SUPPORT ? ? y y NA y y NA NA SEARCHING BASE BASE BASE BASE BASE BASE+S BASE BASE+S BASE TARGET GROUPS y no parents y y y y y y y C - contact addresses; I - user instructions; D - dead links; d - description; l - label; P - personalization support; NA - not available; BASE - database; BASE+S - database and servers; According to Table 2 the most complete EN is EDNA, followed by NGfL, Schoolnet Africa and Canada's Schoolnet. Our SIO is at the end of the list with several options to be considered for implementation or improvement. 3 Criteria Analysis was performed according to the criteria presented in Figure 2. COMPARATIVE ANALYSIS OF EDUCATIONAL NETWORKS Informatica 29 (2005) 477-481 479 Figure 2: Characteristics of educational network Table 3 summarizes pair wise comparison scores for selected criteria assigned by a group of experts. According to Saaty's AHP method these scores are integers from 1 to 9 and their inverses. The interpretation of main values are: 1 -criteria i and j are of equal importance; 3 - criterion i is weakly more important than criterion j ; 5 - criterion i is strongly more important than criterion j ; 7 - ... The product of symmetric entries equals to 1. In ideal case a kind of 'transitivity' should hold for the scores: aik . akj = ay - we say that the scores are consistent. The real life comparison matrices are usually inconsistent. A special consistency coefficient K was introduced. It is assumed that a comparison matrix is consistent enough if K < 0.10. Table 3: Pairwise comparisons cont/serv navigation searching user I support credibility validity target G cont/serv 1 3 3 3 5 5 1/2 4 Navigation 1/3 1 2 1 2 3 1/4 2 Searching 1/3 1 /2 1 1 3 3 1/4 2 user I 1/3 1 1 1 3 2 1/3 1 Support 1/5 1 /2 1/3 1/3 1 2 1/5 1/2 Credibility 1/5 1 /3 1/3 1/2 1/2 1 1/5 1/2 Validity 2 4 4 3 5 5 1 3 target G 1/4 1/2 1/2 1 2 2 1/3 1 From the comparison matrix we get a vector of relative 8.30594, K = 0.0311 and from the eigen-vector the importance of criteria as the eigen-vector corresponding criteria ranking presented in Table 4. to its largest eigen-value. In our case we get X = 480 Informatica 29 (2005) 477-481 A. Zibert et al. Table 4: Criteria ranking EIGEN CRITERION RANK VECTOR validity 1 0.30416494 content/services 2 0.24085495 navigation 3 0.10880157 searching 4 0.09626298 user interface 5 0.09251457 target groups 6 0.07215293 support 7 0.04651902 credibility 8 0.03872904 After we obtained the experts' opinion about the importance of criteria we tried to get the users' opinion. Therefore, we conducted a survey. 4 Survey To get an insight to usability of networks and their comparison from the point of view of final users a web poll was constructed. In the construction the findings of J. Spool were considered. In his research about Web Site Usability he found out that web site users usually do not make use of it like web designers have planned for them. The skills and knowledge ob web designer usually do not ensure a useful web page. Web poll was planned to be filled in by various groups of users. Unfortunately only 59 users answered the questions, 61% of them were teachers. Due to a very small budget allocated to this research and due to a very small number of users who took part in the survey, we got only the teachers' point of view on usability of web sites. Users had to visit 3 foreign education networks and answered 2 questions about each of them. Then an opinion about Slovenian Education network had to be written together with the comparison of chosen networks. The chosen education networks are rich on various learning resources and interactive activities. The networks had been chosen because of: English According to users' evaluation the NGfL is the best choice limited to 3 chosen networks. The layout and user interface are best evaluated for NGfL. The connection of the type of users and their efficiency was not explored. In order to carry out such language, a long time of existence (Canadian Network) and because SIO is a member of EUN Schoolnet. Majority of activities of both educational networks are closely linked. NGfL is one of most extensive European EN. The users had to visit networks first and using them answer to very simple questions. The answers were placed on 1st or 2nd level. The users were very successful with searching the answers at English education network - NGfL and the least successful with EUSchoolnet. The answers to question about their latest visit on Network were as follows. Users wished to explore what was offered (47%), a quarter of them did that because of poll award, others have been searching for new learning resources and information for their work at school, some of them (13%) were not successful. The last group mostly made a remark like "I was unable to find anything" or "Not useful". This was their profile: almost half (48%) of users check the SIO portal a few times per year, for a quarter of them it was their first visit, almost the same size of group check the SIO portal once a month. They had five levels to express their opinion (1 - very poor, 2 - poor, 3 - medium 4 - good, 5 - very good) on content and users experience. The majority (68%) evaluate the content with 3-medium and the user-experience with 3-medium too (63%). They express their opinion as follows: the content has to be changed (44%), the design has to be changed (29%), name and title has to be changed (11%). The navigation is simple (58%). The answers to question " What should be changed about SIO in order to achieve the portal to become a valuable information source for education?" were as follows: more programs, more teacher education, up to date and solid information, promotion, better design and more activities to convince additional teachers to use SIO. As you can see in the table below CSF was evaluated as least useful (4 out of 6 criteria are low), NGfL was evaluated as most useful (4 out of 6 were marked high), EUN Schoolnet was evaluated as useful (2 out of 6 were marked high). analysis a larger number of users from all target groups have to take part in the poll. Table 5: Percentage answers above level 3 searching navigation layout content user interface graphics CSN 77% 67% 72% 81% 56% 53% NFGL 77% 78% 81% 83% 71% 65% EU 79% 74% 92% 81% 67% 62% COMPARATIVE ANALYSIS OF EDUCATIONAL NETWORKS Informatica 29 (2005) 477-481 481 5 Conclusions - Improvements to SIO and Future Development SIO was renewed for the first time in 1999. In the survey, described in the previous section, the users were satisfied with the design and content of its pages. SIO has a good position in the Slovenian web - according to the internet research study SIO was in 2002 on the 56th place among all Slovenian web sites (Petric, 2003) with respect to betweenness (Freeman, 1979). It contains also a critical mass of contents. Their downsides are the problems with maintenance of the content. The main reason for this is that SIO is not institutionally appropriately integrated and supported by the Slovenian educational system. From our overview of ENs and results of the survey we can conclude that SIO plays an important role in Slovenian education, but it should be renewed in technological (new tools), functional (additional services) and organizational (collection of materials, maintenance) sense. In the development of an EN there are two basic options: to establish a central institution that provides most of educational web services; or to establish only an 'index' to educational services distributed across several institutions. We believe that the limit situation is the second option. For this reason the primary function of SIO is to collect, maintain and provide information about educational resources and services. As we already explained in the first section, SIO is based on catalogue system Trubar in which also different materials (photos, drawings, texts, maps, programs, data, sounds ...) are collected. Because of SIO's financial and man-power limitations these materials are mainly contributed by users. Systematic approaches have to be developed to provide 'complete' collections of materials. During the last months we have checked several content management systems to find an appropriate replacement for Trubar. We will probably base the new SIO on the open source Zope connected with Python and MySql on the Apache server. These tools provide an up-to-date and platform independent development environment. We do not expect special problems in transferring the services from Trubar to the new environment. In the new solutions we intend to provide several new options: active link control, access statistics, voting evaluation system, editing, personalization, ... They will support semantic web (RDF, OWL) and educational (SCORM) standards. Since an EN is used by several types of users with different needs, we decided to develop the new SIO as a multi-entry site - each entry providing different, user/goal-oriented view of the content stored in catalogues: portal (entry dveri), e-journal (entry list), entertainment (entry zabava), for non-Slovenian guests (entry english), . For example, the portal entry will provide fresh information (last contributions in catalogues, news, surveys, events, . ) and information sources (addresses and basic data about schools and other educational institutions, manuals, dictionaries, templates, ...). A special challenge is the kindergarten entry. Here we will try to produce an environment adapted to the capabilities of kids - use of picture language, sound (audio) output ... A big problem on the web is non permanent contents -they are appearing, changing and disappearing. An additional service of SIO could be an Archive of selected educational materials. 6 References [1] Batagelj V & Brodnik A & Lokar M1996. Slovensko izobraževalno omrežje, Kongres Ro/2, Ljubljana, 26. - 27.9. [2] Batagelj V& Žibert A & Rajkovič V & Čampelj B 1999. Educational Networks Vision and Reality, IFIP WG 3.1 and 3.5 Open Conference Aulanko -Hameenlinna - Finland, 20-23. [3] Butcher N: Best Practice in Education Portals. Research document prepared for the Commonwealth of Learning and School Net Africa., October 2002. http://www.col.org/Consultancies/02EducationPort als_Report.pdf [4] Commission of the European Communities 2000. Designing tomorrow's education promoting innovation with new technologies, Report from the commission to the council and the european parliament, Brussels. [5] Freeman LC 1979. Centrality in social networks: Conceptual clarification. Social Networks, 215-239. [6] Petrič G 2003. Družbeno delovanje v omrežju svetovnega spleta: individualni in strukturni vidik. Phd thesis, FDV, Ljubljana. [7] Saaty TL 1980. The Analytic Hierarchy Process. McGraw Hill, New York [8] Spool J 1999. Web Site Usability: A Designer's Guide, M. Kaufmann Publishers, Inc. [9] Žibert A & Rajkovič V (mentor) 1998. Kritična analiza šolskih računalniških omrežij: diplomsko delo univerzitetnega študija, IMPRESUM: Kranj. [10] Žibert A & Rajkovič V (mentor) & Batagelj V (komentor) 2004. Critical analysis of usability on educational networks (in Slovene), Master thesis, IMPRESUM: Kranj. [11] "Complete e-learning introduction on the national scale in Slovenia", project within National Target Research Program "Slovenian Competitiveness 2001 - 2006", funded by Ministry for Education.