Elektrotehniški vestnik 77(5): 281-286, 2010 Electrotechnical Review: Ljubljana, Slovenija A game-based mobile-learning platform – description and evaluation Matevž Pogačnik 1 , Jože Guna 1 , Janez Bešter 1 1 Univerza v Ljubljani, Fakulteta za elektrotehniko, Tržaška 25, SI-1000 Ljubljana, Slovenia E-mail:matevz.pogacnik@fe.uni-lj.si Abstract. This paper presents a platform for mobile educational games, named "Quest for Knowledge." The platform allows for creation of interactive educational games that include a multimedia content (video, audio, documents, maps) and playing of games outside classrooms. Students access the games using personal computers and mobile terminals (phones, PDAs). The usual game scenarios include team-based solving of specific tasks and missions. The games are remotely monitored and managed by the teacher who interacts with teams, gives hints and guidance which lead to the final result. The concept of development and playing of mobile educational games has been developed under the project eMAPPS, 6 EU Framework Programme and tested in groups of more than 200 students and teachers in Slovenia and other European countries. The evaluation of games played in Slovenia has indicated a very strong acceptance of the concept, especially by elementary-school children. Keywords: game-based mobile learning, e-Learning, Quest for knowledge, multimedia. Platforma za izobraževanje skozi igranje mobilnih iger - opis in evalvacija Povzetek. V prispevku je predstavljena platforma za izvedbo mobilnih izobraževalnih iger, imenovana "Quest for knowledge". Platforma omogoča izdelavo interaktivnih izobraževalnih iger, ki vključujejo multimedijske vsebine (video, avdio, dokumenti, zemljevidi) in igranje iger tudi zunaj učilnic. Učenci dostopajo do iger prek osebnih računalnikov in mobilnih terminalov (telefoni, dlančniki). Običajni scenariji igre vključujejo timsko reševanje posameznih nalog in misij. Igre na daljavo spremlja in upravlja učitelj, ki komunicira z ekipami, daje namige in navodila, ki vodijo do končnega rezultata. Koncept razvoja in igranja mobilnih izobraževalnih iger je bil razvit v okviru projekta eMAPPS, 6. okvirni program EU, in preizkušen v skupinah, ki so vključevale več kot 200 študentov in učiteljev v Sloveniji in drugih evropskih državah. Evalvacija iger izpeljanih Sloveniji kaže na zelo dobro sprejetje koncepta platforme, zlasti pri osnovnošolskih otrocih. Ključne besede: učenje na podlagi mobilnih iger, e-Učenje, Quest for knowledge, multimedija 1 Introduction Rapid development and introduction of advanced broadband access technologies, multimedia and mobile telecommunications services, are influencing the characteristics of contemporary students. These are now mostly information-literate people, which are always connected and mobile. Participation in virtual communities and the tendency to explore them seems quite natural to modern students, as they have grown up using Internet and mobile devices, through which they participate in standard communication channels such as SMS, e-mail, chat rooms, social communities, etc. They are fonder of multimedia and other Internet content than of traditional text books. Modern technology represents a way of life to them, rather than separate activities, which can be used for work and learning [1, 2, 3]. The described characteristics of the contemporary students require changes and adoption of innovative methods of teaching and learning. One possible approach is based on learning through digital games (Game-Based Learning - GBL). By adding a mobility component, we are moving into the field of mobile educational games (Game Based Mobile Learning - GBML). In the past 25 years, a number of sociological, educational and technological research initiatives have been undertaken, related to the usage of ICT-supported games in education [1, 2, 4]. Analysis and reports on these initiatives have discussed the positive impact on the educational process with enthusiasm, but the practice has shown that commercial providers of ICT- supported games do not exploit their full potential and were mostly not very successful in the educational market. These research projects and analysis have led to Received 27 August 2010 Accepted 6 September 2010 282 Pogačnik, Guna a variety of guidelines aimed at a better integration of the educational content in ICT-based games – the GBL. For the topic of this paper one of the guidelines is very important; namely the advocation of the active role of students in the educational process. Unlike the traditional education process, which is mostly passive (frontal lectures), the educational content in GBL is presented in the form of game requirements. It requires constant interaction and immersion of participants in a virtual game world that is changing during the playing time and requires response activities of the participant. The advantage of this method of teaching is very simple: learning through cognitive activities that require active research, analysis, interpretation of events, problem solving, and often even physical activity. The added value of teaching does not come from the game itself, but from a creative combination of the educational content (educational media) and active involvement of participants in a meaningful process, which results in active knowledge acquisition. This paper describes a concept, technological overview and evaluation of an award-winning platform named Quest for Knowledge (QFK), developed within the 6FP EU project called eMAPPS (028051) [5]. QFK supports the game-based mobile learning (GBML) concept, and enables an advanced and efficient type of education. Section 2 includes a discussion about general premises of game-based learning, followed by a presentation of the QFK platform in Section 3. The platform presentation includes a functional description, game-play description as well as a short technical description. Section 4 presents an evaluation based on real life testing of the platform, while Section 5 concludes with a discussion. 2 Learning and games Having described the unambiguous benefits of GBL, we should note that experts in the field of educational games have also indicated concerns that prevent a wider deployment of such teaching methods. Some teachers perceive the process of playing as a frivolous activity. At the same time there is the problem of players spending too much time in front of computer screens. In addition, there is the concern of too heavy immersion of players and losing of touch with reality, which may happen during game playing. Finally, the development of digital games is complicated from both the technical as well as the conceptual standpoint. An average user of the technology (eg. a teacher) can not develop digital games, because the development requires a lot of programming, creativity and experience in the field of play. Those concerns are among the main reasons for the slow introduction of educational games in the regular educational process. However, most concerns can be overcome with the use of appropriate technologies for the development of educational games, the concept of game structure and the introduction of mobility. Mobility allows participants to play educational games anywhere, also in the real world (and not only in the virtual world game) with the help of mobile terminals. By moving from classrooms into real space, the participants do not spend too much time in front of their computers and are able to combine information from digital and real world. The tendency towards optimal experience during (educational) game playing, brings us to a number of important game components, also called “flow” [6]: - Clear goals and immediate feedback - Personal skills well suited to given challenges - Merger of action and awareness - Concentration on the task - Sense of control - Loss of self-consciousness The relevance of these ideas to game design is that these components of flow are the basic components of a good game. Process-intensive educational games are able to facilitate learning both because they engage their players in intrinsically motivating activities and because the process of interacting with a game world leads the player to internalise the game world's rules. Such games are enjoyable to play so long as they present the player with clear and desirable goals, they are interactive and immersive, their educational content is integrated within their structure, provide a challenge to both beginners and experts and create such learning situations, which motivates students to persist far in excess of any externally imposed requirements [6, 7]. These guidelines were followed during creation of the QFK requirements. 3 QFK platform In order to better understand the opportunities offered by the QFK platform, this section describes the concept and structure of the QFK educational games, followed by a typical game-play procedure and a short technical description. The fundamental premise at the time of development was that the platform should support any type of game, regardless of the educational content or topics it may cover. 3.1 QFK functional description Basic entities of each game are the so-called missions and tasks. Each game consists of one or more missions, which are further composed of one or more tasks. The simplest game would contain one mission with one task in it, but typical educational games that were developed within the platform, contain one to five missions, each A game-based mobile-learning platform – description and evaluation 283 containing three or more tasks. An example of missions and tasks is shown in Figure 1. Figure 1: Visual layout of a game mission in a QFK game : Below is a list of tasks and their statuses, on the right-hand side are chat messages.. Slika 1: Grafična podoba misije v igri QFK: Spodaj je seznam nalog, na desni strani so kratka sporočila (chat) A task is the basic entity of each game. In essence, it contains a textual description and one or more questions that players must solve and answer. Additional instructions and tips are presented by means of a related multimedia content (images, video, audio, documents), or location information - GPS coordinates, added to a task by the author of the game. Appearance of a typical task in the user interface is presented in Figure 2. All multimedia content used in game tasks is automatically stored in the platform repository and can be searched, browsed and accessed during the creation of new games or modifications of the existing ones. In addition, all the content is described using the LOM (Learning Object Model) standard [8] and can be exchanged between repositories supporting this state-of- the-art standard for educational content. The platform also allows for creation of interactive maps, presenting geographical locations, which are relevant for the game and its tasks and which should be visited during game play. Individual games stored in the platform are called game templates. This concept is important, because the platform supports numerous occurrences of playing of each game, during which the content of the played game might change (added multimedia content, added descriptions and location hints, etc.). Each occurrence of a game play is called a game session. The QFK platform saves every game session and its structure for later analysis of the results and overview of content generated during the game. The templates remain unchanged during game play. Figure 2: Visual layout of a task. It contains a task description, GPS coordinates as a link to a map and related multimedia objects (images, videos). Slika 2: Grafična podoba naloge: Vsebuje tekstovni opis naloge, koordinate GPS s povezavo na zemljevid ter pripadajoče multimedijske objekte (slike, video). 3.2 QFK game play To understand the procedure of game play we should first present the roles of the participants. There are three different roles in the QFK mobile educational games: - Game master - Game base - Game avatars The game master is usually the person who is the author of the game (e.g. a teacher) and knows the game best. Alternatively, the game master can be anyone who is qualified to conduct mobile educational games and has detailed instructions on how to run and end the game. Tasks of the game master are mainly control, support and checking of the results submitted by the playing teams during the game. The game master communicates with players through chat and is informed by the system of any incoming results, chat messages or uploaded content. Tasks are marked as completed by the game master and players can be awarded extra points for task completion. Players are divided into teams. Each team has two groups: the game base and game avatars. The game base are one or more players who play the game using personal computers in the classroom, while game avatars consist of one or more players playing the game outside the classroom using one or more mobile devices, GPS terminals, and other optional equipment. The base group is focused on problem solving (or parts of it) with the help of online resources and the guidance and feedback of avatars outside the classroom. Avatars, on the other hand, provide field information and data relevant to problem solving. In some sense they represent the eyes of the team. Their task is to watch for hints and signs, take photos, video and audio clips and upload them to the platform as a solution to the task or 284 Pogačnik, Guna as a hint for the base group. Base and avatar players of the same team communicate through chat; all messages are also visible to the game master. Games can also include other elements, avatars can search for physical items, talk to people, or in any other way address the given problem. Mobile user interface of the QFK platform, used by avatars, is presented in Figure 3. Figure 3: Visual layout of a task on a mobile device. It contains a task description and related multimedia objects (images, videos). Slika 3: Grafični izgled naloge na mobilnem terminalu: Vsebuje tekstovni opis naloge, ter pripadajoče multimedijske objekte (slike, video). 3.3 QFK technical description The QFK platform is a web-based application running in browsers. The reason for the web-based solution is the requirement for the functional usage on a large number of operating systems and devices. The platform is supported by a number of browsers like Mozilla Firefox 2+ or Internet Explorer 6+. The mobile platforms that are currently supported (tested and verified) are the ones using Symbian operating system with the Opera Web browser Kit. In practice this means Nokia S60 versions (N70+ N90+), Sony Ericsson P1i and the like. In addition, the system is also running on the Android mobile platform (version 2.2). QFK is currently being tested on a number of mobile terminals from other manufacturers, which will be fully supported in the future. The QFK platform is implemented based on open- source software. It is using an Apache server providing the PHP functionality and access to the MySQL database. The database contains information about users and their status, game templates, game sessions and multimedia repository modules. Graphical representation is implemented by means of XSL templates and XSL transformations. The client side functionality is supported through Ajax, JQuery and JScript scripting technologies. Those are required for chat rooms, status updates and event handling during the game. 4 Platform evaluation and results During the final stages of the project, the platform was first tested and evaluated by a group of internal experts, trying to identify design flaws, bugs or any other problems that may occur during the platform usage. Some issues were identified and the platform was updated and optimised. As we wanted to evaluate it in real-life scenarios, we organised four separate events in 2009 with different age groups and asked them to perform one or more educational games. The technical equipment used consisted of MS windows-based PCs and two types of mobile phones: Sony-Ericcsson P1i and Nokia N95. The educational games that were played were devised and developed by leading members of the playing groups (teachers, team-building leaders), who had some initial support by the platform development team. Games scenarios included tasks that needed to be performed in a city (Ljubljana or Maribor) within a 3 km radius from the base. At the beginning, all game participants were given a 5 min demonstration of the system and some instructions. The instructions were very general and related only to the system functionality, without any instructions concerning the game scenarios. The participants were also able to tryout some basic operations (open a task, open a mission, send a chat message, upload a result) before they started playing the games. Table 1 shows some basic facts about the evaluation groups and games played. Group number Group type (city) Age of particip. Num. of playing teams Game duration (hours) 1 Elementary school (Ljubljana) 11-12 2 4 2 Elementary school (Maribor) 10 4 5 3 Secondary school (Ljubljana) 16-19 3 6 4 Company team-building environment (Ljubljana) 26-50 2 4 Table 1: Evaluation groups description Tabela 1: Opis skupin, ki so sodelovale v evaluaciji A game-based mobile-learning platform – description and evaluation 285 The evaluation results are given in the form of four evaluation criteria, with ratings given on the Likert scale (1-very poor, 2-poor, 3–neutral, 4-good, 5-very good): The evaluation criteria used are explained below and the summary is presented in Table 2. • User interface intuitivity: Describes the difficulty of finding any target functionality without instructions. Recognition is favoured instead of recall, consistency of menus in different screens is required. • User control and freedom: Difficulty of moving between different elements (tasks, missions, chat) without much effort or without being interrupted/redirected by the system. The possibility of easily reversing actions, visibility of occurring events during game play. • System reliability/responsiveness: Duration of timeouts during actions, system errors and failures. • General impression and usability: Evaluation of the platform concept, immersiveness during game play, suitability for different fields of use. The evaluation results are given for each of the evaluation groups and represent average values given by the teams. Each of the playing teams evaluated the platform using the evaluation criteria. As the evaluation groups number 1 and 2 included children from elementary schools, they had to be supervised by their teachers during game play. The evaluation was adapted to their age level and was performed in the form of discussion about the platform and its characteristics with children and their teachers. Group number Intuitivity User control and freedom Reliability and responsiveness General impression 1 5.0 4.0 3.0 5.0 2 4.0 3.25 3.0 5.0 3 5.0 4.0 3.7 5.0 4 3.0 2.5 3.0 4.0 Table 2: Evaluation results Tabela 2: Rezultati evaluacije Additional observations obtained during game play and evaluations were very interesting: In group no.1 the children’s knowledge about the game topic was tested by their teachers and the results, according to the teachers, were high above the expectations. Moreover, the children demonstrated some organisational skills as they distributed tasks within teams in order to complete the game. In group no.2, which had the most teams playing at the same time, some drawbacks of the team-progress monitoring system became obvious, as teachers found it difficult to monitor four teams at the same time. In group no.3 the educational games were accepted with great enthusiasm, despite the duration of the game (6 hours). As their games included task solving in an area with weak GPS reception, they had to resort to other means, such as the use of standard maps, which, according to players, further increased their motivation. Group no.4 was the least enthusiastic about the game, as they expected better user experience. Most of complaints were related to the number and size of action icons on the mobile-device screen and consequently need to scroll up and down (too many small icons on a single screen). They also expected more visual clues about incoming events and notifications given to the game master. Nevertheless, the general impression was positive. 5 Discussion and conclusions The evaluation performed in real life scenarios showed that the platform has a lot of potential and that it is suitable for creation and playing of very different educational games, with many topics and goals. However, some flaws and problems like small icons, scrolling issues and lack of visual clues were identified and these should be taken care of. Interestingly enough, most of the complaints came from the group with the oldest users. One reason for this might be the design of the mobile phone user interface, which is targeting elementary-school children. The action buttons and icons are indeed relatively small and thus easier to access with small fingers on touch screen devices. Furthermore, the monitoring of the team progress needs to be improved as the game masters need a better overview of incoming events such as chat messages, solutions given by teams and uploaded content. One possibility would be inclusion of pop-up menus, which were excluded in the early stages of design for being too intrusive. Reliability and performance of the platform should also be optimised. Finally, the platform support should be extended over a larger number of mobile devices. The good news concerning this issue is the fact that the next generation devices already support the technologies required by the platform (AJAX, JQuery, etc.). 286 Pogačnik, Guna An interesting project would be creation of educational games for the university-level students. Due to the nature of the platform, which enables outdoor learning and activities, the most suitable educational topics would probably be the ones involving location-based topics and might be more suitable for the social sciences curriculum. The platform QFK was presented at the EDUCA 2008 conference, where it raised a lot of interest. A year later it was awarded the ComeniusEduMedia 2009 award [9], given to educational products based on the ICT technology. The primary goal of the platform and games developped within this project was to create an environment that will engage a broad audience of players by creating complex social networks, and interactive stories that tap into a broad range of player experiences. Understanding how learning occurs through game play, examining how game play can be used to support learning in formal learning environments, and designing games explicitly to support learning are the areas that educational research can contribute to game studies. 6 Acknowledgements The platform implementation and associated mobile educational games would not have been possible without the support of the project eMAPPS (028 051), which was financed by the European Commission under the 6th Framework Programme. Furthermore, we also wish to thank all partners in the project for support, ideas and a comprehensive evaluation and validation. 7 References [1] Gee, J.P. What video games have to teach us about learning and literacy. New York: Palgrave MacMillan., 2003 [2] Kafai, Y. B. The Educational Potential of Electronic Games: From Games-To-Teach to Games-To-Learn. 2001. [3] Kafai, Y.B., Ching, C.C., Marshall, S. Children as designers of educational multimedia software. Computers in Education, 29(2/3). p.117-126. 1997. [4] Prensky, M. Simulations: Are they games? http://www.marcprensky.com/writing/Prensky - Simulations-Are They Games.pdf., 2001, (last accessed July 2010). [5] Project eMAPPS. http://www.emapps.com. (last accessed July 2010). [6] Csikszentmihalyi, M. Flow: The Psychology of Optimal Experience. New York: Harper and Row, 1990. [7] Malone, T. W., Lepper, M. R. Making Learning Fun: A Taxonomy of Intrinsic Motivations for Learning. Aptitute, Learning and Instruction: III. Conative and affective process analyses. Hilsdale, NJ Erlbaum, 1987. [8] LOM, SCORM standard. http://en.wikipedia.org/wiki/LOM, http://en.wikipedia.org/wiki/SCORM (last accessed July 2010). [9] Gesellschaft für Pädagogik und Information e.V. http://www.gpi-online.de/front_content.php (last accessed August 2010). Matevž Pogačnik graduated in 1997 and defended his Ph. D. thesis in 2004 in the field of telecommunication and informatics at the University of Ljubljana. His research and scientific work is focused on development of interactive multimedia services for different devices with a special emphasis on development of systems for user-adapted content choice (personalization). Lately he has been involved in development of interactive services on all devices in the fields of e-learning and IPTV systems using different interaction modalities. Matevž Pogačnik has participated in numerous European projects in the field of interactive digital television, e-learning, e-tourism and P2P systems. He is also a member of the international organization IEEE. Jože Guna received his B.Sc. and M.Sc. degrees from the Faculty of Electrical Engineering, University of Ljubljana, in 2002 and 2005, respectively. He is currently with the Laboratory for Telecommunications as an assistant and researcher. His research focuses on IP multimedia services, human-computer interaction, networking and Internet technologies, digital-rights management, and smart home technologies. Janez Bešter, Ph.D., is a professor and Head of the Laboratory for Telecommunications at the Faculty of Electrical Engineering, University of Ljubljana, Slovenia. His work focuses on planning, realization and management of telecommunication systems and services, implementation and application of information technologies into education, and economic opportunities for knowledge-based societies. He is a member of several national committees, i.e. AAATE, IEEE, IFIP, ACM, and IEICE.