Informatica 30 (2006) 439-445 439
Surfing Hypertexts with a Metacognition Tool
Giuseppe Chiazzese, Simona Ottaviano, Gianluca Merlo, Antonella Chiari, Mario Allegra, Luciano Seta and Giovanni Todaro
Italian National Research Council, Institute for Educational Technologies (ITD),
Department of Palermo, C.N.R.
Via Ugo La Malfa, 153 - Palermo, Italy
E-mail: [giuseppe.chiazzese,simona.ottaviano,gianluca.merlo,antonella.chifari]@itd.cnr.it E-mail: [mario.allegra,luciano. seta,giovanni.todaro]@itd.cnr.it
Keywords: metacognition, hypertext, web surfing
Received: July 24, 2006
This paper presents a pilot study for the testing of the Did@browser system, a tool developed for supporting the knowledge construction process of students during their studies on educational hypermedia. The system adopts a strategy of posing metacognitive questions and receiving answers from students in order to improve their awareness in their surfing abilities and cognitive strategies used during the learning process on the Net. The paper describs the tools used, the experimentation carried out with the students and the relative evaluation of the results. Povzetek: Predstavljena je analiza koristnosti metaorodja za spletno delo.
1 Metacognition and hypertext
The new technologies for distance learning have, in the last ten years, offered new opportunities to create innovative models for teaching and learning which can be integrated into traditional educational contexts. In developing these models web technologies have contributed to the use of hypermedia as learning tools [12, 15]. During the use of the hypertexts the learner controls his own process of knowledge construction independently, even if he is often unaware of his surfing behaviour and of the cognitive strategies he adopts in order to surf and learn at the same time [11].
We believe that this lack of awareness can effect the surfing and learning behaviour of contents.
Basing on the considerations of studies by [7, 8], we define a student's surfing behaviour as a dynamic process in which he/she adopts cognitive strategies both for learning the web contents and for improving his/her surfing skills for interacting with the interface of a web site.
By surfing skill we mean a complex set of actions which the learner adopts for checking and deciding how to proceed in the surfing process in order to satisfy the learning goal. During surfing activity users often unconsciously ask themselves questions and take decisions: What key word shall I choose? Shall I select this link or that one? Shall I go on? How Shall I move in this page? Should I modify the search? Shall I read or not? Does this information meet my needs?
Some of the skills that the pupil adopts while he/she is surfing the Net are knowing how to distinguish a need for information, how to surf within a site, how to carry out a search, which key words to insert, which categories to choose, how to make hypotheses, how to integrate previous experiences of surfing with new ones, how to use skimming and scanning mechanisms, how to interpret meaningfully what he reads or hears[4].
These surfing skills are used during the educational hypermedia interaction but may produce a cognitive overload for the user [18, 3] and consequently a sense of being lost in hyperspace [5].
Some research [1, 7, 16] has explored this perception of disorientation on the web, defining it as a cognitive problem in which the user lacks a clear conception of the relationship between the elements of the system, is unable to identify his/her position within the hypermedia structure and finds it difficult to continue surfing in a way which is coherent with his goals.
Therefore to take advantage of the hypermedia with respect to the traditional learning activities the users must be able to exploit their functionalities and to apply cognitive strategies for the acquisition of knowledge through online study.
Besides, to reduce the risk of the user losing himself, surfing randomly or not reaching his learning goals effectively, it is necessary for him to be trained in the effective use of the tools in order to support learning and stimulate the improvement of cognitive strategies involved in the learning processes.
Various scholars have studied the use of hypertexts in education, examining their effectiveness in facilitating students' learning. To do this, a variety of theoretical frameworks were employed and several cognitive issues related to learning were analysed. In general, these studies underline the relevance of metacognitive and self-regulatory skills to explain differences in students' performance[2].
As a result of these considerations some researchers have used computer based learning environments both to deliver information and to strengthen students' control over their individual learning processes. In this respect the research has proposed different tools to support student learning during web activities. The common
440 Informatica 30 (2006) 439-445
G. Chiazzese et al.
starting point of these studies is the development of tools to support the acquisition of metacognitive skills and stimulate students to reflect on their learning processes.
In the literature [9, 10, 12, 13, 17, 19] there are some examples of metacognitive tools which aim at facilitating the surfing activities and at reaching the learning goals.
In some cases the metacognitive aids, or cues, are provided at the end of the activity, at other times tools assist the students during the search activity, providing them with some suggestions about the actions to take when surfing; other systems provide interactive web maps to create graphical representations of surfing paths, and tools for adding notes, for creating new connections between visited links and for sharing documents.
Some of these researchers [9, 10, 12] were interested in analysing the effect of the metacognitive prompts on hypertext comprehension and in testing the transfer of the knowledge acquired to problems in the real world. Others [17] considered whether the effects of metacognitive prompting are mediated by metacognitive or motivational processes.
Very rarely [13, 19] a research has been aimed at studying the effects of metacognitive prompting on the surfing behaviour of students involved in a learning activity.
Moreover, the majority of existing tools enable the students to reflect both on their interaction with the system and on their cognitive strategies for learning. We have found that some systems do not support the subject during browsing when the process of knowledge acquisition is taking place but in general they stimulate the metacognitive skills before and/or after the surfing activity.
Believing that the cognitive strategies used when studying hypertexts change dynamically during the surfing process and the knowledge construction according to the assigned task, we argue that it is useful to support the user during these processes from the beginning to the end of surfing.
In order to create this situation where the student is able to monitor his surfing behaviour, we have developed a system that stimulates metacognitive reflection on both surfing and cognitive strategies for learning during the whole surfing process.
Starting from these considerations, we introduce a pilot study which aims at experimenting the use of the Did@browser system with middle school students.
2 Did@browser, a metacognitive tool
The Did@browser system is a new technological solution developed by the Institute for Educational Technology of the CNR in Palermo in order to support students during surfing and learning on the Net.
The system is based on a client-server architecture, and it is composed of the server and two client components: the student and teacher client which are both available in Internet Explorer.
The system stimulates the self-monitoring of the cognitive processes used by students for surfing and learning by posing metacognitive questions during web
surfing. Moreover, the teacher can customize a set of questions for each student in order to evaluate their activities and to stimulate their surfing skills, thus facilitating the effective achievement of learning goals.
The set of questions was planned according to the didactic activities that the students are required to carry out and the web site selected by the teacher. The sets of metacognitive questions used during the experimentation are given below:
-Why have I clicked on this link?
-What information do I expect to find?
-What other surfing tools were there on the page?
-Why have I selected this link rather than the others on the page?
-Have I already explored the other objects on the page (images, links, text)? If not, do I expect to do so?
-Do I intend to return to this page? Why/Why not?
-Why have you returned to this page?
-Has the image which I've seen helped me to understand better?
-Have I found the information I expected on this page?
-What has interested me most on this page?
The questions were selected by the researchers and associated to specific nodes of hypertext. The association between the nodes and questions was made so that the students could improve their awareness of strategies employed during surfing. When the student clicked on the link the system showed the question in a window. The student's surfing was interrupted and the system invited him to answer the question.
The system recorded in the log file the information related to the pages visited, the duration of the visits and the student's answers.
3 The pilot study
The pilot study of the Did@browser system took place during the year 2004-05 with the collaboration of the Alberigo Gentile School in Palermo. The work sessions were organized in the computer laboratory of the school for a total of 24 hours. The 27 students involved in the research experiment were attending the second class at the middle school and they were divided into two groups, experimental (EG) and control (CG).
The subjects of the two groups were divided up, balancing the level of scholastic competences, gender and familiarity with the PC.
3.1 The tools used
To investigate the cognitive strategies employed during surfing, a Surfing Behaviour Questionnaire was used to obtain considerations about surfing the web, as well as the Conceptual Maps to assess learning and comprehension of the structure of the websites used for the experimentation.
To analyse the surfing behaviours, we used the data stored in log files and graphical representations for the nodes of the hypertexts visited by the students during the surfing activity.
SURFING HYPERTEXTS WITH...
Informatica 30 (2006) 439-445 441
only the experimental group was presented with the metacognitive questions.
Each group worked for 6 sessions of 2 hours each during which the students in pairs used the PC for surfing and studying the sites that we structured ad hoc.
In each session a specific didactic aim was established. The activities in every session were structured in the following way: at the beginning of each session we introduced the task, the time and the activities to be developed. At the end of the experimental activities we organized two other sessions for discussion with the students and to give them feedback about the experience; in these sessions the above mentioned tools, the Surfing Behaviour Questionnaire and the Conceptual Maps, were also administered.
The contents of the activities were previously agreed on by researchers and the teacher. In particular, we created two didactic sites, one focused on the circulation of the blood (Table 1) and the other one on Genetically Modified Organisms (Table 2). The topics illustrated in the hypertexts had never been dealt with before by the teacher in the traditional class setting.
Concerning the second tool, students were asked to reproduce conceptual maps which describe the links between the different areas of the hypertexts studied.
In this way it was possible to assess whether the students' conceptual map corresponded to the real structure of the two sites and whether some fundamental concepts essential for surfing (links, nodes, toolbar, etc.) had been understood.
Group	Task	Assessments
CG	1.	Free surfing (10') 2.	Seeking principal topics (10') 3.	Find information about a specific topic, save and copy in a text file	Text file
EG	See above	Text file
CG	1.Use	the hypermedia to study the topic (15') 2.Answer	questions in the course book	Test
EG	See above	Test
CG	1.Use	the hypermedia to study the topic (15') 2.Use	the hypermedia to study a specific aspect	Test and conceptual map
EG	See above	Test and conceptual map
Table 1. The sessions of the activities on the circulation of the blood		
Group	Task	Assessments
CG	1.	Free surfing of the sites (20') 2.	Find information about a specific topic	
EG	See above	
CG	1.Use	the hypermedia to study the topic (15') 2.Find	information about a specific topic and copy the most relevant in a text file	Text file
EG	See above	Text file
CG	1.Use	the hypermedia to study the topic (15') 2.Use	the hypermedia to study a specific aspect (20') 3.Create	a document summarizing the topic studied	Document Conceptual maps
EG	See above	Document Conceptual maps
Table 2. The sessions of the activities on the GMO
3.1.1	The Surfing behaviour questionnaire
The aim of the questionnaire was to evaluate how the subject perceives his surfing behaviour.
The questionnaire consisted of 9 questions concerning:
-the self-examination of surfing behaviour; -the subjective evaluation of the efficacy of content presentation in the hypermedia;
-the subject's awareness of strategies used during surfing.
3.1.2	Conceptual maps
The Conceptual Maps, as a tool for knowledge representation, allow students to construct the network of the relationship between concepts and to highlight the key ideas. The construction of the conceptual maps of a hypertext is useful for evaluating whether the student:
-	has understood the structure of the hypertext studied;
-	has identified the navigational areas relevant to the didactic aims;
-	has organized the information correctly and completely.
3.2 Experimental procedure
In order to verify if and how Did@browser facilitated monitoring of the surfing behaviour of the subjects involved, we assigned the same tasks to both groups, but
442 Informatica 30 (2006) 439-445
G. Chiazzese et al.
4 Results
The evaluation of the system was finalized to asses whether the metacognitive questions activated during the surfing process had been a useful tool for the self-monitoring of the student.
In particular, the evaluation was based on the analysis of the answers given by students and on the surfing pattern analysis where the following parameters were considered: number of visited links, number of transitions between the pages, duration of visits
Table 3. Some evidence from the log files
The nodes were classified according to topic areas of the hypertext, identifying the visited nodes and relative frequency of the visited links for each area. This information was evaluated considering the relevance and coherence with reference to the assigned task.
Moreover, the focus of attention was on the number, frequency, typology, position of metacognitive questions and the relative answers provided by students.
The analysis of the answers indicated that on the whole the use of metacognitive questions was perceived by students as a useful monitoring tool of their activity.
The analyzed data show that this way of posing questions to the students creates two problems: the first is the inability to describe their cognitive strategies and the second is the functional limitation of the system in associating questions to links. In the first case some students gave content focused answers to questions concerning metacognitive processes, while others had problems describing the procedures.
In the second case, this version of the system is unable to pose questions related to a specific element of the page and this can be misleading for the students. In fact, the questions on surfing, apparently unconnected to the context of study and to their task, created irritation or confusion in the subjects when they were unable to understand their relevance.
The quality evaluation of surfing paths was carried out using the visualization software GraphViz [6]. This tool was chosen thanks to its versatility and the possibility to modify the visualization parameters and due to the clarity with which the graphics are represented. It was used to assess the structure and the relationships between the visited links of a hypertext. Besides, to find temporal data all the log files created during the Did@browser experimentation sessions were analyzed.
In particular, during every surfing session, for each user and for each group, a calculation was made of the
according to the assigned task of the surfing session. For example, in Table 3 we present some statistics for two sessions; the results for the other sessions are very similar.
From these data some differences between the two groups emerge: while the total number of pages visited is the same for the two groups, the subjects in the experimental group performed fewer transitions and their surfing behaviour appears more uniform (smaller standard deviation).
real time spent on the visited pages and of the absolute and average time spent on the pages of a specific area of the site.
To assess the paths, a distinction was made between the episodic structure of the hypertexts, that is the choices that a single user made during surfing, and the emergent structure, or rather all the accumulated episodic structures of the whole group of users [14]. This method provides a generalized and representative surfing pattern of a whole user group. The top two graphs in Figure 1 represent the entire surfing of the two groups as a whole during a single session. The nodes are the pages visited and the arcs are the transitions performed by the subjects. Each arc represents a set of transitions, and we can weight each arc with respect to the number of times this transition was performed by one member of the groups. We have not indicated these weights in the figure.
The following graphs in Figure 1 were obtained by eliminating the arcs with lower weights one at a time.
The analysis indicates that in the EG the surfing activity was more goal-oriented. In fact, it appears that the members of the EG were able to recognize the structure of the web site after fewer transitions; they surfed between the pages in detail and a pattern emerged sooner.
The CG surfing behavior appears to be more erratic. In fact, at the same level, the emergent structure for the CG is more complex with more transitions and the surfing activity does not look as if it is clearly goal-oriented.
It was observed that the emergent structures of the users' surfing were relevant to the assigned tasks. In fact, the subjects spent the majority of their time on the nodes of the assigned study area. However, the number of subjects involved was too small to affirm that this result is an effect of the metacognitive prompts, but we can conclude that the presence of metacognitive prompts did
		Control Group		Ex	perimental Group
		Mean	Standard deviation	Mean	Standard deviation
II session	n. of transitions	64	±39.25	48	± 21.14
	n. of pages visited	18	±10.41	21	± 7.6
III session	n. of transitions	95	± 38	53	± 13.55
	n. of pages visited	21	± 11.35	23	± 5.34
SURFING HYPERTEXTS WITH...
Informatica 30 (2006) 439-445 443
Figure 1. The emergent structures for the two groups during a session
444 Informatica 30 (2006) 439-445
G. Chiazzese et al.
not disturb the students' surfing or their learning processes.
From the answers to the Surfing Behaviour Questionnaire the following observations emerged.
The visual components are the most attractive parts of the hypermedia. In fact, the images are a means to facilitate the comprehension of the contents; they make the presentation more attractive and exciting. Students' answers were supported by the tracking data which showed that the pages with images were visited more frequently.
The pages containing a lot of links and written text proved to be less attractive and less visited, and the pages containing many technical terms were particularly uninteresting for the students.
The questionnaire also revealed the principal cognitive learning strategies adopted by the subjects. The subject read the whole text carefully before beginning another activity. If he/she had difficulty in understanding, he/she tended to reread the text several times, using didactic tools such as glossaries and dictionaries to increase his/her comprehension.
The analysis of the answers to the question: "If you had to surf the websites again, which actions would you repeat and which would you not repeat?" revealed that the subjects recognized that they would modify the strategies they had adopted previously.
In conclusion, the results of the questionnaire allowed us to study the students' surfing behaviour and suggest some ways to improve the system.
The analysis of the Conceptual Maps revealed notable differences between the two research groups. It was carried out by evaluating the representation and the interconnections among the nodes of the maps, considering how many strategic points were reproduced and the depth of the level of the graphic representation. Besides, the experimental group produced maps in greater detail compared to the control group, which frequently omitted some essential nodes. On average, the experimental group proposed a higher number of links which correctly described the hierarchical structure of the areas of the site studied. So, the metacognitive stimulus may have influenced the attention and the mnemonic performance of the subjects involved.
5 Conclusions and forthcomings
The pilot study of the Did@browser system showed that the users' surfing was pertinent to the assigned tasks and that the metacognitive questions did not disturb the students' surfing and learning processes. In fact, the students in the experimental group improved their comprehension of the intrinsic structure of the hypermedia and created more accurate conceptual maps.
Finally, the students considered the system to be a useful self-monitoring tool.
The system we have proposed can be improved in a number of different ways: by integrating into the system other tools related to particular metacognitive skills which are useful for surfing didactic hypermedia;
developing tools within the system for monitoring and assessing student behaviour and permitting teachers to adapt the system according to the class and the learning activity planned.
We intend to implement some of these improvements in the next version of the Did@browser system.
Acknowledgments. This work is based on our paper presented at the Third International Workshop on Developing Creativity and Broad Mental Outlook in the Computer Age - CBMO-2006 in conjunction with the Tenth Conference of the International Society for the Study of the European Ideas (ISSEI 2006) "The European Mind: Narrative and Identity", held at the University of Malta, July 24 - 29, 2006. We are grateful to the CBMO chair Vladimir Fomichov for the opportunity to show and debate our researches.
References
[1]	J. S. Ahuja and J. Webster. Perceived disorientation: an examination of a new measure to assess web design effectiveness. Interacting with Computers, 14(1): 15-29, 2001.
[2]	R. Azevedo. Using Hypermedia as a Metacognitive Tool for Enhancing Student Learning? The Role of Self-Regulated Learning. Educational Psychologist, 40(4): 199-209, 2005.
[3]	J.M. van Bruggen and P. A. Kirschner and W. Jochems. External representation of argumentation in CSCL and the management of cognitive load. Learning and Instruction, 12: 121-138, 2002.
[4]	S. L. Chang and K. Ley. A Learning Strategy to Compensate for Cognitive Overload in Online Learning: Learner Use of Printed Online Materials. Journal of Interactive Online Learning, 5(1): 104116, 2006.
[5]	J. Conklin. Hypertext: An Introduction and Survey. Computer, 20(9): 17-41, 1987.
[6]	J. Ellson, E. R. Gansner, E. Koutsofios, S. C. North and G. Woodhull. Graphviz and Dynagraph - Static and Dynamic Graph Drawing Tools. Technical report AT&T Labs - Research, Florham Park NJ 07932, USA, 2003.
[7]	E, Herder. Modeling User Navigation. In P. Brusilovsky and A. Corbet and F. de Rosis, editors, User Modeling 2003. Springer, , 2003.
[8]	C, Hoelscher and G, Strube. The role of knowledge in WWW search: differences in IR skills and domain expertise. In P. Lenca, editor, Proc. of the Human Centered Processes Conference, pp. 425430, 1999.
[9]	D. Kauffman. Self-regulated learning in web-based environments: Instructional tools designed to facilitate cognitive strategy use, metacognitive processing, and motivational beliefs. Paper
SURFING HYPERTEXTS WITH...
Informatica 30 (2006) 439-445 445
presented at the annual meeting of the American Educational Research Association, 2002.
[10]	B. Kramarski and R. Ritkof. The effects of metacognition and email interactions on learning graphing. Journal of Computer Assisted Learning, 18(1): 33-43, 2002.
[11]	M. Lee and A. L. Baylor. Designing Metacognitive Maps for Web-Based Learning. Educational Technology & Society, 9(1): 344-348, 2006.
[12]	X. Lin, T. J. Newby and W. T. Foster. Embedding Metacognitive Cues Into Hypermedia Systems To Promote Far Transfer Problem Solving. In Proc. of Selected Research and Development, pp. 463-482, 1994.
[13]	A. Lloyd. A Software Tool for Supporting the Acquisition of Metacognitive Skills for Web Searching. In Proc. 10th International Conference on Artificial Intelligence in Education, 2001.
[14]	J. E. McEneaney. A Transactional Theory of Hypertext Structure. In Proc. National Reading Conference, 2002.
[15]	J. B. Nunes and S. P. Fowell. Hypermedia as an experimental tool: a theoretical model. Information Research, 2(1), 1996.
[16]	M. Otter and H. Johnson. Lost in hyperspace: metrics and mental models. Interacting with Computers, 13(1): 1-40, 2000.
[17]	R. Stark, M. Tyroller and U.-M. Krause. Effects of a Metacognitive Prompting Procedure in the Context of a Computer-Based Learning Environment. Pratical Relevance and Explanation by Metacognitive and Motivational Processes. In P. Geijets, P.A. Kirschner, J. Elen and R. Joiner, editors, Instructional design for effective and enjoyable computer-supported learning (CD-ROM), 2004.
[18]	J. Sweller, J.J.G. Marrienboer and F.G.W.C. Paas. Cognitive architecture and instructional design.
Educational Psychology Review, 10(3): 251-296, 1998.
[19]	J. B. Watson and B. S. Allen. The Effect of Metacognitive Prompts on Student Navigation, Comprehension, and Metacognitive Awareness in a Multimedia Science Tutorial. .A paper presented at
the Annual Meeting of the American Educational Research Association, New Orleans, 2002.