Informatica 40 (2016) 291–302 291 Using a Natural User Interface to Enhance the Ability to Interact with Reconstructed Virtual Heritage Environments Silviu Butnariu, Alexandru Georgescu and Florin Gîrbacia Department of Automotive and Transport Engineering, Transilvania University of Brasov 29 Eroilor Blvd, RO-500036, Brasov, Romania E-mail: butnariu@unitbv.ro acgeorgescu@yahoo.com, garbacia@unitbv.ro Keywords: 3D reconstruction, virtual heritage, kinect sensor, games engine, interaction, navigation Received: June 28, 2016 Virtual Reality (VR) techniques are used to create computer-simulated environments which enable new forms of cultural entertainment. One critical problem is how to design the 3D virtual environments (VE) and what methods should be used for interaction, in order to offer an entertaining experience in an open approach. This paper focuses on the added value of gaming methods for 3D reconstruction and interaction with 3D representations of cultural heritage assets using a NUI designed for games. The impact of game mechanics and interaction was investigated by developing a VE of a medieval burg (Brasov, Transylvania). The VE development involves high quality 3D reconstruction of the burg, creating a collection of gestures to interact with the VE, and their implementation and testing within the VE. The paper also includes a qualitative and quantitative evaluation of the implemented system, as tested by subjects who carried out a navigation session in the VE. Povzetek: Prispevek obravnava interakcijo kot v računalniških igrah, a v sistemih elektronske dediščine s 3D virtualno realnostjo. 1 Introduction Technological breakthroughs in real-time computer graphics, virtual and augmented reality gave us a way to create 3D virtual environments (VE) that allow users to explore cultural heritage. Virtual Heritage (VH) supports the reconstruction, preservation, transfer and display of cultural heritage information using virtual reality. Virtual Reality (VR) refers to a system of concepts, methods and techniques used to develop simulated environments by means of modern computer systems (computers and specialized equipment). 3D historic site reconstruction is one of the most VR - supported cultural dissemination form. A critical problem is how to design the VH environments in order to offer an entertaining experience in an ubiquitous and personalized manner. Most literature has argued that interactive engagement in a computer-based environment is best demonstrated by games [1]. Game mechanics and interfaces represent an exceptional opportunity providing innovative approaches to interact with 3D representations of cultural heritage assets. In this paper we examine the potential of using computer game engines and user interface techniques, to intuitively create and explore a VE. This article attempts to find answers to the following questions: 1. What is the added value of the game mechanics for the development of VH environments? 2. What is the value and utility of this approach in cultural heritage interactive exploration, using a Natural User Interface (NUI) device? 3. Are NUI technologies useful for the navigation in a 3D VH environment? The impact of the game mechanics and NUI interaction techniques were investigated by implementing a virtual environment of the medieval burg of Brasov. 2 Experimental section Virtual Heritage is a system built to create visual representations of cultural heritage facets like historic sites, towns, monuments or artifacts. This concept can become a platform for understanding and learning certain events and historical elements by researchers and visitors. The use of VR technology in architectural heritage facilitates access to cultural heritage sites by mitigating the preservation and dissemination difficulties. To create a VH system, one must address two basic issues: virtual environment reconstruction and interaction. The implementation of these two concepts, which is more than a mere reproduction of an archaeological site, represents a simulation process where objects, behaviors, ecosystems are combined [2, 3, 4]. The goal is to analyze the cultural artifacts, to preserve and share a record of their geometry and appearance, then to acquire detailed shape information [3, 5, 6, 7]. 2.1 Virtual environment reconstruction The procedure of building virtual environments starts with a virtual construction of an archaeological site. Firstly, the potential ancient landscape (geomorphology, water system, soil, vegetation) will be rebuilt and 292 Informatica 40 (2016) 291–302 S. Butnariu et al. secondly, the anthropic layer of the landscape (sites, monuments, road system, settlements). In addition, the relationships people / environment and people / people are to be defined. However, one issue is raised regarding 3D reconstruction: how can we build a realistic virtual archaeological model based on fragmentary data, in order to obtain the highest reliability and accuracy? Finally, the 3D reconstruction is an interpretation based on the complex analysis of various types of sources, where the “vision” concept has a basic role [3,8]. In literature, there are various archaeological reconstructed sites, from small artifacts to entire cities, from computer games to academic studies, using various technologies. The following are some examples: games – Second Life [9], Virtual Egyptian Temple, Roma Nova [10], Battle of Thermopylae, Rome 3D, Assassin’s Creed video games universe[3]; research - Poseidonia-Paestum [22], Aphrodisias Odeon [8], Santa Maria di Cerrate church [12], Suleymaniye mosque [6], city of Tomis [7]; details, reliefs - Byzantine Crypt of Santa Cristina in Carpignano Salentino, Italy [5], an Aeolus (the ruler of the winds in Greek mythology) mask [13], damaged byzantine icons [14], a boxwood prayer nut [15]. A 3D reconstruction of a totally or partially destroyed building requires a 3D CAD model that observes the historical construction details. The concept of "cultural heritage" involves several actions (acquiring, processing, presenting and recording data) in order to determine the position, shape, structure, size and other characteristics of a monument or historical site in 3D space at a given time [16]. 3D reconstruction involves the use of continuous surfaces, so it must use global scanning techniques. With traditional surveying techniques it can only get information of discrete lines characteristic of surfaces or edges [17]. Photogrammetry or laser scanning, combined with conventional types of data (historical chronicles, drawings, engravings, lithographs, historical photos) provides different information about the building, other than its geometry. There are several documentation techniques available: traditional manual methods (using very simple equipment: tape measure, plumbs, manual laser distance measurement), topographic methods, photogrammetric methods and scanning methods [2, 5, 12, 18, 19] or new methods, such as computer tomography (CT) technology [15]. Moreover, an important issue, besides the 3D reconstruction of buildings, is the reconstruction of facade details or smaller items, such as statues, paintings reliefs. There are a lot of applications using image processing methods [10, 12, 13, 14, 21]. At the same time, in the real world, such places are populated with people [22]. In the last years, high-resolution recording of historical sites or cultural artifacts has stimulated many researchers in different research areas: computer graphics and computer vision, reconstruction based on photogrammetry [12]. 2.2 Interaction Immersive virtual environments offer users a natural setting for educational and instructive experiences, while the game engine technology offers an efficient solution for their development [11]. The virtual environment represents, on one hand, a static world: geomorphology, road system, vegetation, buildings, and other items that do not move and have no behavior and, on the other hand, entities represent dynamic items, some of them affected by physics, or with some form of artificial intelligence (people, rivers, fire and smoke). Every entity has a position and an orientation that can be manipulated by the engine. There is often some crossover, so that parts may be manipulated like entities. For the real-time 3D rendering procedure, 3D Game Engines represent the most appropriate tool for using detailed environments on personal computers [23]. The interaction between elements of the virtual environment is an important challenge, compared to the implementation of static elements. While reconstructions used in research do not require special functionalities for navigation, in case of games, the use of real time fast running applications is required. In accordance with the increasing presence of VR based software in computer applications, there has been a growing need for more realistic virtual worlds with ever increasing complexity (the size of the scenes, the complexity of details) compelling developers to invent ever more complex solutions to address the challenges created by such requirements [24]. Using 3D game engines to create immersive environments was rarely used technique despite the commercial success and the high level of photorealism achieved by current software / hardware technology (called Serious Games–SG) [20]. 3D Game Engines are large and complex, but very efficient in creating huge, artificial worlds for entertainment. As they mature, they improve and mimic the real world equally realistic as their competing Virtual Reality (VR) systems counterparts [23]. Examples of these VR platforms can be seen in projects such as: For Virtual Reality:  Oculus Rift games developed by RiftTime (a Romanian-Moldavian collaboration)  The VOID (or the Vision of Infinite Dimensions, developed by a company in Lindon Utah. The project includes combination between haptic feedback and VR)  SteamVR (developed by Steam, subsidiary of Valve inc.)  Star Trek: Bridge Crew (developed by Nintendo and Sony)  Obstruction (developed by Cyan Worlds/ Ubisoft)  Eagle Flight (developed by the Ubisoft Montreal studio)  For Augmented Reality:  Minecraft (developed by Microsoft)  Pokemon GO (developed by Niantic, from San Francisco)  Drop Dead for GearVR (developed by PixelToys)  other technologies introduced by Alienware, Thrustmaster and Razer. Using a Natural User Interface to Enhance the... Informatica 40(2016) 291–302 293 The concept of “modern game engines” represents complex parallel systems that are competing for limited computing resources [10]. The technology used for developing virtual environments (games for entertainment, serious games or simulations) is limited by the development budget. Modern entertainment computer games frequently require high budget. Some of these costs can be reduced by using procedural modelling techniques for generating assets, including terrain, vegetation or whole urban environments [23]. Interaction in the virtual environment has two major components: navigation and contact between elements. Here are some examples from the researched literature. The Ancient Paestum project [11] uses UnrealEngine 2 (UE2) Runtime Edition from Epic Games that provides an integrated development environment through a content creation tool called UnrealEd, able to export objects modelled with 3D editors. Within this tool editing actions of behavior objects (such as triggers, players, non-player-character) are possible. In reference work [4] a virtual heritage is presented, where the reconstruction procedure is based on Autodesk software. Using the map editor of the game engine from Bethesda Softworks® (http://bethsoft.com/age), the virtual model was imported. All entities can be adjusted and modified. The game runs on Gamebryo from Emergent Game Technologies Inc. (http://www.gamebryo.com/) , which is based on a high-quality graphics engine and a physical engine (AI). This game engine supports scientific visualization, allows the avatar creation, and scripting. Another game engine is Crystal Space engine (http://www.crystalspace3d.org), detailed in reference work [24]. This software is an open-source software development kit for modern video games and virtual environments. Based on a few modules, this engine forms a complete open-source middleware solution covering most of the needs for the development of modern video games and VR environments. Anderson et al. [10] offers a comprehensive state-of- the-art review, which covers a lot of serious games, used in various applications: online virtual environments (e.g. Second Life), cultural heritage (e.g. Roma Nova, Ancient Pompeii, Parthenon Project), virtual museums (e.g. Virtual Egyptian Temple, The Ancient Olympic Games, Virtual Priory Undercroft Coventry), commercial historical video games (e.g. History Line: 1914–1918, Great Battles of Rome, Age of Empires series, and Total War series). 2.3 Game engines for virtual reconstruction of medieval cities The use of game methods to develop virtual heritage applications is known, and a number of projects have used this approach to reconstruct medieval cities Table 1). The ratings of dedicated websites Gamespot (http://www.gamespot.com) and IGN (http://www.ign.com) are dated back to May 2013. For some projects (especially from universities) we have little information, because they have never been released to a wider public. It was also revealed that none of the projects surveyed (online virtual tours, videos, university projects and games developed in the last 10 years) used NUI, excepting other entertainment games from platforms such as Wii, Playstation Move, Xbox 360 and Xbox ONE. However the compatibility of the surveyed projects with NUI sensors has not been formally tested and evaluated. In the case of new games and computer graphics engine sites, although it is speculated that the tests have started, they have never been made public. 3 Test case: The medieval burg of Brasov Virtual Heritage Environment The virtual environment development of the medieval Brasov burg, based on game mechanics consisted of the following steps (Figure 1): high quality 3D reconstruction of the medieval burg of Brasov, creating a collection of gestures to interact with the virtual environment, implementing and testing gestures in the virtual environment, verifying and testing on human subjects, studying results and drawing conclusions. The graphics engine used in the virtual reconstruction of the medieval burg is CryEngine 3.4.5, (http://www.crytek.com/cryengine) using the Sandbox Editor within. The main reasons behind this choice are: realistic graphics, freedom of control, movement and interaction, detailed weather simulation, A.I. scripting, openness for a wide variety of 3D model types. Also, the interfaces and the buttons are similar to 3DstudioMax or MotionBuilder (http://www. autodesk.com/products/). These tools are provided in order to facilitate animation, scripting, and object creation. CryEngine uses many parameters in order to define the distribution of different textures or types of vegetation. Also, an algorithmic form of painting textures is used. This is the same engine used by famous games producers in developing their own games and has similar power as graphic technologies based on C++: opportunities for import-export techniques, scripting and Figure 1: The algorithm of a test case. 3D reconstruction of the medieval burg of Brasov. 294 Informatica 40 (2016) 291–302 S. Butnariu et al. graphics to create a virtual environment from scratch. Today, elements of the new DirectX 11 have been implemented furthermore. We also observed back in the used version that the Kinect sensor is under development but not yet compatible with this gestures technology. Project name C am er a G ra p h ic s In te ra ct io n in te rf ac e N U I R ea li sm In te ra ct iv it y R at in g s jo u rn al s P er so n al fe ed b ac k P er sp ec ti v e P er so n al R at in g In te ra ct io n d ev ic es T y p e o f in te rf ac e A v ai la b il it y P er so n al R at in g A v ai la b il it y R at in g G am es p o t R at in g IG N R at in g R at in g Tour Virtual : Google Art Project1 first person 10 mouse and keyboard WIMP NO 10 No 1 NA NA 7 Tour Virtual2 kinematics 7 mouse WIMP NO 10 No 1 NA NA 6 Google Earth3 total 9 mouse and keyboard WIMP NO 10 partially 5 NA NA 8 Tour Virtual : “Real and Virtual”4 first person 9 mouse and keyboard WIMP NO 10 partially 5 NA NA 7 Tour Virtual : Panoramic Earth5 first person 8 mouse and keyboard WIMP NO 10 No 1 NA NA 7 Tour Virtual – Paris Medieval6 kinematics 8 mouse WIMP NO 10 No 1 NA NA 5 Battle Castle Dover7 kinematics 10 mouse WIMP NO 10 No 1 NA NA 5 Medieval City of Bologna8 kinematics 10 mouse and keyboard WIMP NO 10 No 1 NA NA 7 City Engine9 kinematics 8 mouse and keyboard WIMP NO 8 partially 5 NA NA 7 Castel Almodovar10 kinematics 7 mouse WIMP NO 10 No 1 NA NA 5 Nürnberg 3D Second Life11 third person 8 mouse and keyboard WIMP NO 9 Yes 10 NA NA 8 Assassins’s Creed I12 third person 9 mouse and keyboard WIMP NO 9 Yes 10 9,00 7,70 8 II13 10 10 third person 10 9,00 9,20 10 Brotherhood14 10 10 10 9,00 8,00 9 Revelations15 10 10 8,00 8,50 9 Mount and Blade16 third and first person 8 mouse and keyboard WIMP NO 10 Yes 10 7,50 8,00 8 Chivalry: Medieval Warfare17 first person 10 mouse and keyboard WIMP NO 9 Yes 10 NA 7,90 8 War of the Roses18 first person 10 mouse and keyboard WIMP NO 10 Yes 10 7,50 7,30 9 Medieval: Total War I şi II19 total 10 mouse and keyboard WIMP NO 10 Yes 10 8,80 8,80 9 London Project20 total 10 mouse and keyboard WIMP NO 10 Yes 10 NA NA 9 Table 1: Examples of reconstructed medieval burgs [29]. Using a Natural User Interface to Enhance the... Informatica 40(2016) 291–302 295 new DirectX 11 have been mplemented furthermore.. In order to achieve the 3D reconstructed model of Brasov’s medieval burg (Ger.: Kronstadt, Hun.: Brasso, Rou.: Corona – in year 1235, Middle Ages), several sources of information were used: old maps (Fig. 2,a) [25], satellite photos - Google Earth (http://www.google.com/earth/) (Fig. 2,b), photos of present fortifications (Fig. 2,c), and other historical sources, such as information from The Carpenter Tower Museum, The National Museum of History and The First Romanian School of Brasov (situated within St. Nicholas Church courtyard), The Virtual Museum inside The Drapers’ Bastion, documents from The National Heritage Centre, information from The City Hall Museum, monographs [26, 27]. The southern defense wall of the burg which also includes some major towers like The Drapers' Bastion and The Carpenters’ Tower has been recently rebuilt in detail (orange in Fig. 2,b).. We used different 3D reconstruction methods: image-based reconstruction, 3D modeling, 3D reconstruction from multiple images (especially for details) and multiplication. Implementation of the 3D reconstructed environment into the virtual medium (engine interface) and interaction tasks, has been achieved using CryEngine software in collaboration with other software applications listed below. First we focused on rebuilding the southern part of the burg, near Mount Tâmpa. Later we added the specific elements for the Carpenters Tower interior, using other application software: ARC 3D (http://www.arc3d.be/) Google Sketch-up (http://www.sketchup.com/), 3D StudioMax2011 (http://www.autodesk.com/products). The additional free plug-in Play-Up needed to be installed so that the import-export operations between platforms be valid and accurately interpreted by the engine models (http://www.playuptools.com/). The original purpose of the project was to reconstruct the burg only partially. Later, the project grew significantly, elements such as walls, streets and buildings appearing throughout the stage. The first step was to reconstruct the site geomorphology. The land is almost unchanged since the Middle Ages, taken as point of reference. In the second stage, we rebuilt in detail the southern part of the defense wall of the medieval burg of Brasov. Furthermore Fig. 3 illustrates the result of a 3D reconstruction of the old burg of Brasov: land, gateway to the settlement, comparison with the old map, and reconstructed south- eastern and northern defense walls in high detail then added the remaining walls and buildings to complete Brașov’s medieval burg. The major monuments (towers, gates, The Black Church, etc.) were reproduced and located according to plans and historical documents (Josephine map, Radu Oltean’s map etc). In the beginning, our intention was to create a virtual panorama using photos and textures acquired on site. Later, due to high consumption of resources (processing and graphics), we chose to use a random reconstruction of buildings between the virtual walls of the virtual burg of Brasov, by multiplying models for atmosphere, most of which are similar also in reality. The advantages of using the game engines technologies for the reconstruction of Brasov medieval burg were: easy integration of day-night cycle in order to emphasize the mountain climate and to simulate the specific atmosphere of the real city, Brasov; the use of specific medieval and natural elements (torches on fire, the sounds of birds and other animals, a few square stacks of straw, wood furniture items, etc.) with the aim to re-create a realistic atmosphere; modeling a realistic water channel area (the medieval “defense moat”) in order to obtain a more accurate representation; building the town’s borders (walls, fences, etc.) in order to separate the accessible area in the virtual environment from the rest of the city region, to increase the sensation of immersion; modeling the wooden pillars on the wooden architecture of the buildings (stairs, walkways, bridges, squares, etc.); generating random copies of virtual buildings; possibility to easily adjust the virtual medieval burg items (scaling the dimensions of virtual doors in all the reconstructed buildings, in order to facilitate access inside the building of the controlled avatar; the slope of the virtual mountain Tâmpa, near the wall of defense, in order to facilitate climbing, should the user want; adjustments to the water channel areas). Figure 2: (a) Brasov map, 17th century [25], (b) satellite view (Google earth), (c) photos taken from ground level on site (by C. Postelnicu), on the T. Bradiceanu St., showing The Carpenters’ Tower in the foreground and Drapers' Bastion in the background along the fortified wall. [28]. 296 Informatica 40 (2016) 291–302 S. Butnariu et al. 3.1 Interaction techniques description In recent years, the most effective implemented technologies have been based on keyboard-mouse controls, joystick or other controllers. All these devices operate on the simple unidirectional model: up, down, left, right. These interfaces and hardware components can be wired or wireless. The virtual environment perspective has evolved from two to three dimensions, and thus intuitive human-computer interfaces that allow interaction based on complex commands have become necessary. Interfaces that recognize user’s gestures seem to be suitable and more intuitive for interaction with virtual environments. In order to control the virtual environment using gestures, human body position, configuration and movements need to be tracked by different sensing technologies. Mainly, these technologies are attached to the user as body suits and are based on magnetic field or optical trackers. These solutions offer suitable accuracy for gesture recognition, but they are expensive and uncomfortable to wear. Recently, ubiquitous tracking sensors suitable for controller-free gestures interfaces have been released. Among these sensors, used especially in the latest video games, we can mention the following: Microsoft Kinect®, Nintendo Wii®, Xsens, Lukotronic, Leap Motion. To achieve the stated purpose of this study, we chose the Microsoft Kinect® (http://www.microsoft.com/en- us/kinectforwindows/) sensor because it is inexpensive, off-the-shelf and widely available on the market for end- users. Kinect has a RGB camera, depth sensor and multi- array microphone thus being able to detect human natural movement and sound. The software technology enables advanced gesture recognition, facial recognition and voice recognition. A dedicated application can locate the joints of the user in space and track their movements over time, based on the virtual skeleton superimposed over the user's body profile. The software used to create a control interface and communication between user and sensor, as well as the interface between the sensor and computer commands, is Flexible Action and Articulated Skeleton Toolkit (FAAST), developed by MxR, South California University (http://projects.ict.usc.edu/mxr/). FAAST is a middleware software used to facilitate the integration of gesture control interfaces based on game Figure 3: The 3D virtual reconstruction of ancient buildings. (a) the reconstructed old saxon town streets; (b) the Main Gateway; (c) the same gateway, detail from ancient map [25]; (d) the south-eastern defense wall with the Carpenters’ Tower exterior (e) and interior (f), photos taken from Cryengine 3.4.5; (g) the medieval burg of Brasov, XVII century [28]. Using a Natural User Interface to Enhance the... Informatica 40(2016) 291–302 297 control devices in virtual environments. For this purpose, it uses OpenNI and the Microsoft Kinect technology for Windows, both software systems dedicated to tracking human movement through a virtual skeleton. FAAST is free and can be redistributed for commercial or non- commercial purposes. Figure 4: Used nodes of human skeleton in FAAST application software. The reason for choosing this software is simple: unlike other Kinect-computer interfaces, FAAST acts promptly on calibration and can easily send commands to the computer. Besides, response times are lower, increasing the accuracy. This interface is much easier to manage than Kinect drivers and other dedicated products such as Brekel Kinect or Microsoft Kinect for Windows. The program translates the user’s skeleton over a VRPN server (Virtual-Reality Peripheral Network - a device- independent and network-transparent system for accessing virtual reality peripherals in VR applications) identified as Tracker @ ip address if it’s run over the same computer as the client. The server starts automatically when the feature set of the program connects to the sensor. According to the OpenNI rules, used joints (nodes) are presented in Fig. 4. The user’s body consists of line segments linked by joints. The system continuously checks the user’s stick figure and the motion of the joints are used to recognize the interaction gestures. In order to navigate in the virtual environment using just a Kinect sensor, we propose a simple and intuitive model, based on the main types of movements and choices in a virtual environment: change of perspective and/or plan of interaction with the environment, activation of certain objects and commands - such as grabbing and/or throwing objects in the scene, their manipulation in a virtual environment, pushing or pulling certain objects in the scene. A series of gestures were developed based on skeleton joints tracking, which were mapped to the interaction commands with the virtual environment: moving forward & back, swimming, panning left and right (strafing), grabbing, jumping, and squatting (or crouching). The following graphic (Fig. 5) illustrates an example of gesture sensing in the real environment, interpretation in FAAST application and the result in virtual environment. Figure 6 below lists the interaction commands mapped to gestures, based on the above presented procedure. 3.2 Experimental data The aim of the evaluation study presented in this section is to assess the way game interface can be used for virtual heritage applications. In the conducted experiment we compared the proposed game natural interaction interface with a traditional WIMP interface. For our test case, we decided to use the old burg of Brasov virtual heritage environment previously presented. Figure 5: The act of opening a door and its effect in the virtual environment. Man on the left-hand side of photo: Alexandru Constantin Georgescu – author - after which the motions and map were scaled. 298 Informatica 40 (2016) 291–302 S. Butnariu et al. We carried out a usability test on a number of 24 subjects aged 19 to 57 (mean= 29,80). Each subject filled in a questionnaire providing information about age and sex, previous experience in playing games and using VR technologies, experience with virtual environments and computer skills. A few of the subjects, usually PhD students, had previous experience in using natural user interfaces. However, all the users were proficient in at least one game (as virtual environment) and had good computer skills. At the beginning, each subject was informed about the purpose of the experiment and specific instructions were given. They were asked to perform two tests: first, to navigate to the Carpenters’ Tower from the old burg of Brasov and find some objects using traditional desktop tools with 2D input (keyboard and mouse). The same task was then performed using the proposed game-like natural user interface. Before the test, each participant was allowed to understand and familiarize with the game interface settings. The users had 20 minutes, prior to the experiment, to practice the natural game interface functions of navigation and interaction with the virtual environment. Half of the users first used the traditional desktop WIMP Figure 6: Used gestures commands. Using a Natural User Interface to Enhance the... Informatica 40(2016) 291–302 299 interface, then, after a 20 minute break, they were asked to conduct the same task using the game interface newly created. The other half interacted with the virtual environment using the proposed game interface in first instance, and then the desktop system. Each time value was recorded for the assessment of the results. The application ran on a Windows 7 PC with Intel Xeon 3,6 GHz Processor and 16 GB RAM with a NVIDIA Quadro 6000 GPU. 3.3 Results and discussion The task completion time was measured for each participant. Fig. 7 displays the average time needed by the subjects to complete the assigned tasks using both user interfaces. Concerning the performance evaluation, it is to be observed that natural game interface does not considerably improve the navigation in the virtual environment, compared to conventional WIMP interface. Figure 7: Comparative tests. At the end of the experiment the participants were asked to rate the game interfaces on a 5-point Likert scale: Strongly agree (5), Agree (4), Neutral (3), Disagree (2) or Strongly disagree (1) (Fig. 8). The post experiment questionnaire was meant to evaluate ease of use, satisfaction level, and intuitiveness of the game-like interface. Figure 8. Results of the subjective questionnaire In the first phase, the mean value of responses by age participants was calculated. The results are shown in Table 2. We can observe a decrease of the mean value with age, which can be interpreted, on the first hand, as reluctance to new devices used to interact with machines, and on the other hand, as reservation of the older subjects to run ample moves in front of special devices (Microsoft Kinect). Age of respond ents First impres sion Inter acti on Funct ionali ty Acc ura cy Easy to learn Easy to underst and Av era ge 19-24 (12 pers) 3,83 3,50 3,50 3,42 4,17 4,17 3,76 25-34 (6 pers) 3,50 3,33 3,50 2,83 4,33 4,67 3,69 35-44 (3 pers) 3,33 3,33 3,67 3,67 4,67 4,67 3,89 45-60 (3 pers) 3,00 2,67 3,00 3,00 4,33 4,67 3,44 Table 2: Mean values of responses by age of respondents. In the second phase, we tried to establish a correlation with the gender of the subjects. The results, presented in Table 3, show a significant tendency in women of not being impressed by the new equipment. Gender of responde nts First impres sion Inte racti on Funct ionali ty Acc ura cy Easy to learn Easy to underst and Av era ge M (17 pers) 3,82 3,41 3,59 3,29 4,41 4,47 3,83 F (7 pers) 3,00 3,14 3,14 3,14 4,00 4,29 3,45 Table 3: The mean values of the responses by gender. One aspect that all the studied groups have in common is that this new method of interacting with virtual environments is very easy to learn and understand. Another important element is the accuracy of the interaction, which did not receive a good rating. It was determined that more training is required in order to obtain good accuracy, which might be a natural factor also in regard to any type of new technology. It can be observed that the time values for completing the task were higher when the game-based interface was used, as compared with the 2D traditional devices. This can be due to the existing depth camera tracking limitation which does not allow obtaining a precise and stable tracking. In comparison with the traditional WIMP interface, the subjects expressed great interest in using game technologies for virtual heritage. The stated reasons relate to the more intuitive virtual environment and the simplification of the series of windows and menus needed for various operations. The users appreciated the gestures interaction functionalities which offered the opportunity of a natural communication of commands. The subjects considered the game interface commands easy to learn and understand, thanks to the natural and intuitive communication paradigm. 4 Conclusions This paper discusses the development of a VH application based on NUI game methods. The test case developed for this purpose was the 3D complex reconstruction of the medieval burg of Brasov, Transylvania, using an available game engine. The 300 Informatica 40 (2016) 291–302 S. Butnariu et al. advantage of using game technology for development of cultural heritage virtual environment resides in the possibility to achieve better quality, faster development, complexity and fidelity by using high quality real-time graphics, powerful AI to handle character behavior, advanced algorithms for character movement system. We also proposed and developed a simple and intuitive interface for virtual environment exploration based on the main types of movement gestures as a next step in helping the ever expanding progress of future generations of communication interfaces. The navigation includes forward/backward movement, left/right turn and jump/squat. In the interactive 3D reconstructed medieval burg of Brasov, the user can choose which buildings and rooms to visit, interact with and manipulate virtual objects, experience near-photorealism in indoor and wide-open outdoor environments and extraordinary real- time special effects (like swimming) or the weather system. The experiments and trials conducted on subjects showed the feasibility of hand gesture-based navigation techniques and the fact that the natural interaction approach presented herein is preferred by the users in an unconstrained 3D navigation. In comparison with the traditional navigation methods using keyboard and mouse, the participants perceive the NUI navigation interface intuitive and easy to use. It is our firm belief that NUI interfaces enable new forms of virtualized cultural heritage which amplify the user’s experience and, for example, can attract a larger number of visitors in museums. In future works we will focus on improving movement gestures by filtering the data or using other more performant new NUI devices in order to obtain a more accurate and stable user tracking. 5 Acknowledgement Author Contributions: Silviu Butnariu contributed to the theoretical research and the state-of-the art. Alexandru Constantin Georgescu contributed to the 3D reconstruction, linking the NUI, calibration through devices and drivers and simulation. Florin Girbacia contributed to testing solutions, recording and processing data. 6 Conflicts of Interest The authors declare no conflict of interest. 7 References [1] Champion E. (2008). Otherness of Place: Game- based Interaction and Learning in Virtual Heritage Projects. International Journal of Heritage Studies, 14, 210 – 228. [2] Noh Z. ; Shahrizal Sunar M. ; Pan Z. (2009). A Review on Augmented Reality for Virtual Heritage System, Learning by Playing. Game-based Education System Design and Development, Lecture Notes in Computer Science, 5670, 50-61 [3] Pescarin S. (2012). Ancient Landscape Reconstruction, in 1st Digital Cultural Heritage Workshop, The Cyprus Institute, Nicosia, Cyprus. [4] Rua H.; Alvito P. (2011). Living the Past: 3D Models, Virtual Reality and Game Engines as Tools for Supporting Archaeology and the Reconstruction of Cultural Heritage – the Case-Study of the Roman Villa of Casal De Freiria, Journal of Archaeological Science, 38, 3296-3308. [5] Beraldin J.-A.; Picard M.; El-Hakim S.; Godin G., Borgeat L., et al. (2005). Virtual Reconstruction of Heritage Sites: Opportunities and Challenges Created by 3d Technologies, in The International Workshop on Recording, Modeling and Visualization of Cultural Heritage, Ascona, Switzerland. [6] Mansor I.; Puteri S.J.; Maisarah A.; Nurul S. A. L.; Nor Zalifa Z. A. (2007). Virtual Reality in Heritage Studies and Historical Reconstruction through Animation – a Case Study of a 16th Century University Complex in the Ottoman World, in 7th International Conference on Construction Applications of Virtual Reality, Penn State University, USA. [7] Popovici D. M.; Querrec R.; Bogdan C. M.; Popovici N. (2010). A Behavioral Perspective of Virtual Heritage Reconstruction, Int. J. of Computers, Communications & Control, V, 8. [8] de Heras Ciechomski P.; Ulicny B.; Cetre R.; Thalmann D. (2004). A Case Study of a Virtual Audience in a Reconstruction of an Ancient Roman Odeon in Aphrodisias, in VAST-04The 5th International Symposium on Virtual Reality, Archaeology and Cultural Heritage,. [9] 3D Virtual World Second Life (accesed on 13 November 2013). Available online: http://secondlife.com/ [10] Anderson E. F.; McLoughlin L.; Liarokapis F.; Peters C.; Petridis P. et al. (2010). Developing Serious Games for Cultural Heritage: A State-of- the-Art Review, Virtual Reality, 14, 255-275. [11] de Chiara R. (2006). Real Positioning in Virtual Environments Using Game Engines, in Eurographics Italian Chapter Conference ed. by R. De Amicis and G. Conti, The Eurographics Association, p. 6. [12] Gabellone F. (2009). Ancient Contexts and Virtual Reality: From Reconstructive Study to the Construction of Knowledge Models, Journal of Cultural Heritage, 10, e112-e117. [13] Bianco G.; Gallo A.; Bruno F.; Muzzupappa M. (2013). A Comparative Analysis between Active and Passive Techniques for Underwater 3D Reconstruction of Close-Range Objects, Sensors, 13, 11007-11031. [14] Lanitis A.; Stylianou G. (2009). e-Restoration of Faces Appearing In Cultural Heritage Artefacts, in Proceedings of 15th International Conference on Virtual Systems and Multimedia, VSMM '09, Vienna, 15-20. Using a Natural User Interface to Enhance the... Informatica 40(2016) 291–302 301 [15] Zhang X.; Blaas J.; Botha C.; Reischig P.; Bravin A., et al. (2012). Process for the 3d Virtual Reconstruction of a Microcultural Heritage Artifact Obtained by Synchrotron Radiation CT Technology Using Open Source and Free Software, Journal of Cultural Heritage, 13, 5. [16] Yilmaz, H. M.; Yakar M.; Yildiz F. (2008). Documentation of historical caravansaries by digital close range photogrammetry. Automation in Construction, 17(4), 489-498. [17] Núñez Andrés, A.; Buill Pozuelo, F.; Regot Marimón, J.; de Mesa Gisbert. (2012). A. Generation of virtual models of cultural heritage. Journal of Cultural Heritage, 13(1), 103-106. [18] de Mesa A.; Regot J.; Núñez M. A.; Buill F. (2009). Métodos y procesos para el levantamiento de reconstrucción tridimensional gráfica de elementos del patrimonio cultural. La iglesia de Sant Sever de Barcelona, Revista EGA, 14,82-89. [19] Styliadis A. D. (2007). CAAD for Demolished Buildings Modeling with Reverse Engineering Functionality, Cadastre Journal RevCAD, 7, 41-74. [20] Carrozzino M.; Evangelista C.; Brondi R.; Lorenzini C.; Bergamasco M. (2012) Social Networks and Web-Based Serious Games as Novel Educational Tools. Procedia Computer Science, 15, 303-306. [21] Chevrier C.; Charbonneau N.; Grussenmeyer P.; Perrin J-P. (2010). Parametric Documenting of Built Heritage: 3D Virtual Reconstruction of Arhitectural Details. International Journal of Arhitectural Computing, 8 (2), 131-147. [22] Ulicny B.; Thalmann D. (2002). Crowd Simulation for Virtual Heritage, Proceedings of First International Workshop on 3D Virtual Heritage, 28- 32. [23] Johnston D. (2004). 3D Games Engines as a New Reality, in 4th Annual CM316 , Conference on Multimedia Systems http://mms.ecs.soton.ac.uk/ mms2004/abstracts.php , Southampton University, UK, p. 8. [24] Tyberghein J.; Sunshine E.; Richter F.; Gist M.; van Brussel C. (2010). Developing Video Games and Virtual Environments with the Crystal Space Engine, (www.crystalspace3d.org) [25] Brasov map, accessed on 24 October 2013 from http://art-historia.blogspot.ro/2009/04/afisele-art- historia.html [26] Oltean V.; Radu M.; Marin E.; Nussbächer G.Brașov (2004). Brașov Monografie Comercialã, Camera de Comerț și Industrie Brașov, ISBN: 973- 0-03514-8, Brașov, 40-52. [27] Oltean V. (2010). Configurația Istorică și Bisericească a Brașovului, (Sec. XIII – XX), Ed. Andreiana, ISBN 978-606-8106-19-9, Sibiu, 14-88. [28] Georgescu A. C.; (2013). Navigarea și interacționarea prin gesturi în oraşe medievale reconstruite virtual, Master Dissertation thesis at Universitatea Transilvania Brașov, Brașov, 1-82. [29] (1)http://www.googleartproject.com/museums/ altesnational; (2) http://www.viagensimagens.com/; (3) http://www.google.com/earth/index.html; (4) http://www.learn.columbia.edu/ha/html/medieval.ht ml; (5) http://www.panoramicearth.com/; (6) http://www.secretsofparis.com/heathers-secret- blog/virtual-tour-of-medieval-paris.html; (7) http://www.youtube.com/watch?v= _DeqEnsme 7c&feature=html5_3d; (8) http://www.youtube.com/ watch?v=5U5mSluzZzc; (9) http://www.esri.com/ software/cityengine/ , http://www.procedural.com/ cityengine/ (10) http://www.youtube.com/ watch?v=z-qT5ye9O6Q, http://claudiomarr. blogspot.ro/; (11) http://www.youtube.com /watch?v=sLCQgmWkgI0; (12) https://www. ubisoft.com/en-US/game/assassins-creed/; (13) http://assassinscreed.ubi.com/en-us/games/assassins- creed-2.aspx; (14) http://assassinscreed.ubi.com /en- us/games/assassins-creed-brotherhood.aspx; (15) http://assassinscreed.ubi. com/en- us/games/assassins-creed-revelations.aspx; (16) https:// www.taleworlds.com; (17) http:// www.tornbanner.com/chivalry/esrb/; (18) http:// www.fatsharkgames.com/portfolio_page/war-of-the- roses/; (19) http://www.creative-assembly.com/ game/arena; (20) http://triumphantgoat. blogspot.co.uk/ [or https://www.cryengine.com/ community/viewtopic.php?f=328&t=109027 ]