Zbornik 20. mednarodne multikonference INFORMACIJSKA DRUŽBA - IS 2017 Zvezek E Proceedings of the 20th International Multiconference INFORMATION SOCIETY - IS 2017 Volume E Delavnica AS-IT-IC AS-IT-IC Workshop Uredila / Edited by Matjaž Gams, Jernej Zupančič http://is.ijs.si 9.–13. oktober 2017 / 9–13 October 2017 Ljubljana, Slovenia Zbornik 20. mednarodne multikonference INFORMACIJSKA DRUŽBA – IS 2017 Zvezek E Proceedings of the 20th International Multiconference INFORMATION SOCIETY – IS 2017 Volume E Delavnica AS-IT-IC AS-IT-IC Workshop Uredila / Edited by Matjaž Gams, Jernej Zupančič http://is.ijs.si 9. - 13. oktober 2017 / 9th – 13th October 2017 Ljubljana, Slovenia Urednika: Matjaž Gams Odsek za inteligentne sisteme Institut »Jožef Stefan«, Ljubljana Jernej Zupančič Odsek za inteligentne sisteme Institut »Jožef Stefan«, Ljubljana Založnik: Institut »Jožef Stefan«, Ljubljana Priprava zbornika: Mitja Lasič, Vesna Lasič, Lana Zemljak Oblikovanje naslovnice: Vesna Lasič Dostop do e-publikacije: http://library.ijs.si/Stacks/Proceedings/InformationSociety Ljubljana, oktober 2017 Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID=292475904 ISBN 978-961-264-116-0 (pdf) PREDGOVOR MULTIKONFERENCI INFORMACIJSKA DRUŽBA 2017 Multikonferenca Informacijska družba (http://is.ijs.si) je z dvajseto zaporedno prireditvijo osrednji srednjeevropski dogodek na področju informacijske družbe, računalništva in informatike. Letošnja prireditev je ponovno na več lokacijah, osrednji dogodki pa so na Institutu »Jožef Stefan«. Informacijska družba, znanje in umetna inteligenca so spet na razpotju tako same zase kot glede vpliva na človeški razvoj. Se bo eksponentna rast elektronike po Moorovem zakonu nadaljevala ali stagnirala? Bo umetna inteligenca nadaljevala svoj neverjetni razvoj in premagovala ljudi na čedalje več področjih in s tem omogočila razcvet civilizacije, ali pa bo eksponentna rast prebivalstva zlasti v Afriki povzročila zadušitev rasti? Čedalje več pokazateljev kaže v oba ekstrema – da prehajamo v naslednje civilizacijsko obdobje, hkrati pa so planetarni konflikti sodobne družbe čedalje težje obvladljivi. Letos smo v multikonferenco povezali dvanajst odličnih neodvisnih konferenc. Predstavljenih bo okoli 200 predstavitev, povzetkov in referatov v okviru samostojnih konferenc in delavnic. Prireditev bodo spremljale okrogle mize in razprave ter posebni dogodki, kot je svečana podelitev nagrad. Izbrani prispevki bodo izšli tudi v posebni številki revije Informatica, ki se ponaša s 40-letno tradicijo odlične znanstvene revije. Odlične obletnice! Multikonferenco Informacijska družba 2017 sestavljajo naslednje samostojne konference:  Slovenska konferenca o umetni inteligenci  Soočanje z demografskimi izzivi  Kognitivna znanost  Sodelovanje, programska oprema in storitve v informacijski družbi  Izkopavanje znanja in podatkovna skladišča  Vzgoja in izobraževanje v informacijski družbi  Četrta študentska računalniška konferenca  Delavnica »EM-zdravje«  Peta mednarodna konferenca kognitonike  Mednarodna konferenca za prenos tehnologij - ITTC  Delavnica »AS-IT-IC«  Robotika Soorganizatorji in podporniki konference so različne raziskovalne institucije in združenja, med njimi tudi ACM Slovenija, SLAIS, DKZ in druga slovenska nacionalna akademija, Inženirska akademija Slovenije (IAS). V imenu organizatorjev konference se zahvaljujemo združenjem in inštitucijam, še posebej pa udeležencem za njihove dragocene prispevke in priložnost, da z nami delijo svoje izkušnje o informacijski družbi. Zahvaljujemo se tudi recenzentom za njihovo pomoč pri recenziranju. V 2017 bomo petič podelili nagrado za življenjske dosežke v čast Donalda Michija in Alana Turinga. Nagrado Michie-Turing za izjemen življenjski prispevek k razvoju in promociji informacijske družbe bo prejel prof. dr. Marjan Krisper. Priznanje za dosežek leta bo pripadlo prof. dr. Andreju Brodniku. Že šestič podeljujemo nagradi »informacijska limona« in »informacijska jagoda« za najbolj (ne)uspešne poteze v zvezi z informacijsko družbo. Limono je dobilo padanje slovenskih sredstev za akademsko znanost, tako da smo sedaj tretji najslabši po tem kriteriju v Evropi, jagodo pa »e-recept«. Čestitke nagrajencem! Bojan Orel, predsednik programskega odbora Matjaž Gams, predsednik organizacijskega odbora i FOREWORD - INFORMATION SOCIETY 2017 In its 20th year, the Information Society Multiconference (http://is.ijs.si) remains one of the leading conferences in Central Europe devoted to information society, computer science and informatics. In 2017 it is organized at various locations, with the main events at the Jožef Stefan Institute. The pace of progress of information society, knowledge and artificial intelligence is speeding up, and it seems we are again at a turning point. Will the progress of electronics continue according to the Moore’s law or will it start stagnating? Will AI continue to outperform humans at more and more activities and in this way enable the predicted unseen human progress, or will the growth of human population in particular in Africa cause global decline? Both extremes seem more and more likely – fantastic human progress and planetary decline caused by humans destroying our environment and each other. The Multiconference is running in parallel sessions with 200 presentations of scientific papers at twelve conferences, round tables, workshops and award ceremonies. Selected papers will be published in the Informatica journal, which has 40 years of tradition of excellent research publication. These are remarkable achievements. The Information Society 2017 Multiconference consists of the following conferences:  Slovenian Conference on Artificial Intelligence  Facing Demographic Challenges  Cognitive Science  Collaboration, Software and Services in Information Society  Data Mining and Data Warehouses  Education in Information Society  4th Student Computer Science Research Conference  Workshop Electronic and Mobile Health  5th International Conference on Cognitonics  International Conference of Transfer of Technologies - ITTC  Workshop »AC-IT-IC«  Robotics The Multiconference is co-organized and supported by several major research institutions and societies, among them ACM Slovenia, i.e. the Slovenian chapter of the ACM, SLAIS, DKZ and the second national engineering academy, the Slovenian Engineering Academy. In the name of the conference organizers we thank all the societies and institutions, and particularly all the participants for their valuable contribution and their interest in this event, and the reviewers for their thorough reviews. For the fifth year, the award for life-long outstanding contributions will be delivered in memory of Donald Michie and Alan Turing. The Michie-Turing award will be given to Prof. Marjan Krisper for his life-long outstanding contribution to the development and promotion of information society in our country. In addition, an award for current achievements will be given to Prof. Andrej Brodnik. The information lemon goes to national funding of the academic science, which degrades Slovenia to the third worst position in Europe. The information strawberry is awarded for the medical e-recipe project. Congratulations! Bojan Orel, Programme Committee Chair Matjaž Gams, Organizing Committee Chair ii KONFERENČNI ODBORI CONFERENCE COMMITTEES International Programme Committee Organizing Committee Vladimir Bajic, South Africa Matjaž Gams, chair Heiner Benking, Germany Mitja Luštrek Se Woo Cheon, South Korea Lana Zemljak Howie Firth, UK Vesna Koricki Olga Fomichova, Russia Mitja Lasič Vladimir Fomichov, Russia Robert Blatnik Vesna Hljuz Dobric, Croatia Aleš Tavčar Alfred Inselberg, Israel Blaž Mahnič Jay Liebowitz, USA Jure Šorn Huan Liu, Singapore Mario Konecki Henz Martin, Germany Marcin Paprzycki, USA Karl Pribram, USA Claude Sammut, Australia Jiri Wiedermann, Czech Republic Xindong Wu, USA Yiming Ye, USA Ning Zhong, USA Wray Buntine, Australia Bezalel Gavish, USA Gal A. Kaminka, Israel Mike Bain, Australia Michela Milano, Italy Derong Liu, Chicago, USA Toby Walsh, Australia Programme Committee Bojan Orel, chair Mitja Luštrek Niko Schlamberger Franc Solina, co-chair Marko Grobelnik Stanko Strmčnik Viljan Mahnič, co-chair Nikola Guid Jurij Šilc Cene Bavec, co-chair Marjan Heričko Jurij Tasič Tomaž Kalin, co-chair Borka Jerman Blažič Džonova Denis Trček Jozsef Györkös, co-chair Gorazd Kandus Andrej Ule Tadej Bajd Urban Kordeš Tanja Urbančič Jaroslav Berce Marjan Krisper Boštjan Vilfan Mojca Bernik Andrej Kuščer Baldomir Zajc Marko Bohanec Jadran Lenarčič Blaž Zupan Ivan Bratko Borut Likar Boris Žemva Andrej Brodnik Janez Malačič Leon Žlajpah Dušan Caf Olga Markič Saša Divjak Dunja Mladenič Tomaž Erjavec Franc Novak Bogdan Filipič Vladislav Rajkovič Andrej Gams Grega Repovš Matjaž Gams Ivan Rozman iii Invited lecture AN UPDATE FROM THE AI & MUSIC FRONT Gerhard Widmer Institute for Computational Perception Johannes Kepler University Linz (JKU), and Austrian Research Institute for Artificial Intelligence (OFAI), Vienna Abstract Much of current research in Artificial Intelligence and Music, and particularly in the field of Music Information Retrieval (MIR), focuses on algorithms that interpret musical signals and recognize musically relevant objects and patterns at various levels -- from notes to beats and rhythm, to melodic and harmonic patterns and higher-level segment structure --, with the goal of supporting novel applications in the digital music world. This presentation will give the audience a glimpse of what musically "intelligent" systems can currently do with music, and what this is good for. However, we will also find that while some of these capabilities are quite impressive, they are still far from (and do not require) a deeper "understanding" of music. An ongoing project will be presented that aims to take AI & music research a bit closer to the "essence" of music, going beyond surface features and focusing on the expressive aspects of music, and how these are communicated in music. This raises a number of new research challenges for the field of AI and Music (discussed in much more detail in [Widmer, 2016]). As a first step, we will look at recent work on computational models of expressive music performance, and will show some examples of the state of the art (including the result of a recent musical 'Turing test'). References Widmer, G. (2016). Getting Closer to the Essence of Music: The Con Espressione Manifesto. ACM Transactions on Intelligent Systems and Technology 8(2), Article 19. iv KAZALO / TABLE OF CONTENTS Delavnica AS-IT-IC / AS-IT-IC Workshop .................................................................................................................. 1 PREDGOVOR / FOREWORD ................................................................................................................................. 3 PROGRAMSKI ODBORI / PROGRAMME COMMITTEES ..................................................................................... 5 Intel igent Services for Municipalities / Evseev Greg, Bizjak Jani, Grasselli Gregor, Tavčar Aleš, Mahnič Blaž, Gams Matjaž ............................................................................................................................................. 7 Tourist Information Center Slovenj Gradec as a Part Of The AS-IT-IC Interreg Project / Lah Marija, Zupančič Jernej ................................................................................................................................................ 11 Nadgradnja sistema e-turist in integracija z AS-IT-IC platformo / Mahnič Blaž, Koricki Vesna, Zemljak Lana .................................................................................................................................................................. 15 Open Questions of Technology Usage in the Field of Tourism / Peischl Bernhard, Tazl Oliver August, Wotawa Franz .................................................................................................................................................. 19 Testing of Artificial Intelligence Applications / Peischl Bernhard, Tazl Oliver August, Wotawa Franz ................. 23 Virtual Assistants for the Austrian-Slovenian Intelligent Tourist-Information Center / Zupančič Jernej, Grasselli Gregor, Tavčar Aleš, Gams Matjaž ................................................................................................... 27 Indeks avtorjev / Author index ................................................................................................................................ 31 v vi Zbornik 20. mednarodne multikonference INFORMACIJSKA DRUŽBA – IS 2017 Zvezek E Proceedings of the 20th International Multiconference INFORMATION SOCIETY – IS 2017 Volume E Delavnica AS-IT-IC AS-IT-IC Workshop Uredila / Edited by Matjaž Gams, Jernej Zupančič http://is.ijs.si 9. oktober 2017 / 9th October 2017 Ljubljana, Slovenia 1 2 PREDGOVOR Projekt Avstrijsko-Slovenski inteligentni turistčno-informacijski center (AS-IT-IC) je bil sprejet na razpisu Programa sodelovanja Interreg V-A Slovenija-Avstrija v obdobju 2014-2020. Glavni rezultat projekta bo delujoča mreža ljudi s podpornimi orodji, kot so: virtualni asistent (ta bo omogočal komunikacijo z uporabnikom v naravnem jeziku ter integracijo z zunanjimi storitvami), komunikacijske storitve (rešitve, ki bodo omogočale komunikacijo med turisti, virtualnimi asistenti, ponudniki turističnih informacij in lokalnimi skupnosti), turistične vsebine, priporočilni sistem za načrtovanje izletov ter mreža turističnih ponudnikov in njihovih storitev. Delavnica AS-IT-IC je organizirana v sklopu Multikonference Informacijska Družba. Delavnica naslavlja naslednje teme na področju turizma: raziskovalne aktivnosti, inženirske aplikacije in pogled na informacijsko komunikacijske rešitve z vidika ponudnikov turističnih informacij. Matjaž Gams, Jernej Zupančič 3 FOREWORD Austrian-Slovenian Intelligent Tourist-Information Center project (AS-IT-IC project) was approved in the Cooperation Programme Interreg V-A Slovenia-Austria in the period 2014-2020. The main project output will be the operational center with humans involved, having support of the following tools: Virtual assistant (providing automatic answering in natural language to the questions and performing services), Communication service (web-based solution that will enable conversation between the tourists, virtual assistants, tourist information officers and local communities), Information sources of tourism-oriented data, Recommender system for tour planning, Network of tourist services and services from local communities. The AS-IT-IC Workshop is organized within the Multiconference Information Society. It covers research activities, engineering applications, as well as tourist-information providers view of the information-communication technologies solutions in the field of tourism. Matjaž Gams, Jernej Zupančič 4 PROGRAMSKI ODBOR / PROGRAMME COMMITTEE Matjaž Gams, IJS (chair) Franz Wotawa, IST (co-chair) Bernhard Peischl, IST Jernej Zupančič, IJS Tomaž Šef, IJS Oliver August Tazl, IST Dieter Hardt-Stremayr, GRAZ Katarina Čoklc, ZOS Marija Lah, SPOTUR ORGANIZACIJSKI ODBOR / ORGANIZING COMMITTEE Matjaž Gams, chair Lana Zemljak Vesna Koricki Mitja Lasič Robert Blatnik Blaž Mahnič Jure Šorn 5 6 Intelligent Society Greg Evseev Jani Bizjak Gregor Grasselli “Jožef Stefan” Institute “Jožef Stefan” Institute “Jožef Stefan” Institute Jamova cesta 39 Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia Ljubljana, Slovenia greg.evseev@ijs.si jani.bizjak@ijs.si gregor.grasselli@ijs.si Aleš Tavčar Blaž Mahnič Matjaž Gams “Jožef Stefan” Institute “Jožef Stefan” Institute “Jožef Stefan” Institute Jamova cesta 39 Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia Ljubljana, Slovenia ales.tavcar@ijs.si blaz.mahnic@ijs.si matjaz.gams@ijs.is ABSTRACT In this paper we will present some of our systems and so- lutions that will allow local societies and municipalities to become well-suited in the modern world. By providing ser- vices such as: Live Television Interactive 3D Virtual Reality navigation Intelligent virtual assistant with natural language in- terface for municipalities, societies, tourists and tourism organizations Health-related solutions Figure 1: Ecosystem Keywords virtual assistant; streaming service; smart tourism; health- care system To bring plain and straightforward services for people that need digestible information about municipality oers, touris- tic sightseeing destinations, accommodation and health. 1. INTRODUCTION There are many challenges for societies to be known and keep Articial intelligence as well as Information and Commu- their members informed about recent actions and events. nication Technologies are getting better every day and the Most of the time they need some instruments to deliver rich Department of Intelligent Systems is a part of this arduous information in the most simple and user-friendly way. The process. best ways are classic newspapers delivered to members and interested people and accessible television shows or even a In addition to researching new ways of understanding infor- channel. mation we are exploring ways of not only presenting but also maintaining information. While having all content it is still complicated to deliver it to all curious individuals more so it is still a challenge to make it easy to understand and simple to get. 2. TELEVISION By adopting our internet television solution any municipal- That is why research prototypes of Institute \Jozef Stefan" ity could be enabled to broadcast their festivals, events, news are coming to municipalities, retirees and other societies. and general information blocks live 24 hours per day with ease. This solution is prepared to be deployed without pro- fessional help nor special equipment since we already have a public web page with a handbook for setting up your own broadcasting server. The simplest live streaming solution is well-suited for the most popular use-case - a PC with modern OS like Linux with GUI (tested on X and Wayland), macOS or Windows. It has a rich interface and is easy to use by inexperienced users (it is also well documented). 7 to desired masterpiece or historical item. Also it open pos- sibility to have a virtual tour (with a chosen virtual guide) for people who cannot access the museum in person, Gov- ernment facilities - to guide visitors to desired oce, person before or during the visit, It is also possible to enrich and combine 3D Assistant with other solutions like Television and our Text-to-Speech solu- tion `Govorec'. 4. ASISTENT Figure 2: IJS TV interface Figure 4: Asistent view There's also a headless solution with an intuitive web inter- face (Figure 2). It is advised to deploy it with help of the There are about 200 municipalities in Slovenia and every specialist since it requires some special knowledge like Linux single one of them1 already has it's own \Asistent" which administrating and the Docker container system. helps the public to interact with their local government and guides tourists to desired destinations. 3. 3D ASISTENT The system [2] has prepared answers for a variety of ques- To provide eortless navigation in public locations like muse- tions starting from simple ones like renting a bicycle to com- ums, parks or even government facilities our team prepared plex ones like getting a list of documents needed for a land a virtual reality solution. use certicate. The \Asistent" is made up by combining systems like: 1. Cloud-based service that oers API calls and has it's own web interface. It is built on MAS (multi-agent system) with several agents communicating in various environments depending on the intent of the user. 2. Administrative tools for maintaining databases and smaller services, it has it's own separate web interface. 3. A modern client-side application with a rich graphic interface that allows the end-user to have a meaning- ful conversation with multiple services and APIs in a Figure 3: 3D Asistent VR comfortable and convenient way. There are ways to access the virtual helper via special ap- 4. Mobile applications that are easy to download via ap- plications but there's an option where a web browser is suf- propriate mobile application store. Currently among cient. supported mobile OSes there are iOS, Android, Black- Berry an Windows Phone. This product can be used for helping to navigate facilities like: Hospitals - to help patients navigating in a maze of departments and rooms, Museums - to help visitors to get 8 In recent years the Institute \Jozef Stefan" was developing solutions and researching algorithms for identifying types and measuring levels of stress, detecting heart problems as well as several solutions tted for elders. There are vari- ous modules that are already available for no price - some of them were developed or are still in development at our Department. The electronic health system has numerous components - First Aid assistant, information from the Na- tional Institute of Public Health, Self Care advice suggester, modules that work in collaboration with EkoSMART sub- projects like \e-Health and Mobile Health", the Repository of Domains and Prototypes, Stress detectors and system for the care of elderly. Here are the projects involved: 1. IN LIFE3: aims to prolong and support the indepen- Figure 5: e-Turist Explorer dent living of seniors with cognitive impairment, through interoperable, open, personalized and seamless ICT solutions that support such everyday tasks as home 5. TOURISM activities, communication, health maintenance, travel, mobility, socialization tasks. Project \e-Tourist"2 [1] is a service for planning itineraries in 2. E-gibalec4: Smartphone game for children that will en-Slovenia. The project was founded in 2013 within the \Call courage them to do exercises, empowered with statis- for proposals for co-funding of projects developing e-services tics for supervisors. and mobile applications for public and private non-prot organizations" funded by the European Union and Slovenian 3. ASPO5: An online application for identifying and in- Ministry of Education, Science and Sport. forming about sexually transmitted infections 4. Stress detector: Students project for identifying stress It allows to plan trips within selected regions it also tailors type and level via text web interface the route to user's dened preferences like gastronomy trip or a hiking tour. 5. EkoSmart, EMZ6: One of the important features of the program is the integration of the solutions in dierent This project is an important step in developing Austrian- areas into a common ecosystem. Too often the practice Slovenian Intelligent Tourist Information Center (AS-IT-IC) of introduction of smart cities shows limited focus on project. The project addresses the problem of not getting certain areas and lacks connection with others. One the desired information about natural and cultural heritage of the important objectives of the EkoSmart program sites in the Slovenian-Austrian cross-border area in an inte- is therefore the development of the platform with the grated way. same name (EkoSmart platform) which will allow easy integration of sector-specic solutions into a common ecosystem (featured in the program, as well as others) 6. HEALTH and will, as such, facilitate the identication and sup- port of inter-sectoral value chains. This platform will be compatible with global solutions and will include concepts such as the Internet of Things (IoT).Within EMZ (e-Health and Mobile Health) we collect domain repositories and prototypes, so you can see who stores what data and what prototypes in Slovenia. We are developing an EMZ assistant here. 7. INTEGRATION WITH AS-IT-IC To help the AS-IT-IC project to create a joint Austrian- Slovenian center. A network of services that help to navigate and deliver useful information will be necessary to collabo- rate with service providers and tourist oces, municipalities, Figure 6: IN LIFE smart watches tourists and citizens to enhance continuous cooperation be- tween them. These projects are going to be part of the touristic ecosystem that is still developing. Most of the medical professionals understand that Articial 3 Intelligence is going to revolutionize the healthcare system. http://www.inlife-project.eu/ 4https://www.e-gibalec.si/ 1http://projekt-asistent.si/meta-asistent 5https://aspo.mf.uni-lj.si/static/ASPO_new/#/ 2http://turist.ijs.si 6http://ekosmart.net/sl/ekosmart/ 9 8. CONCLUSION In this paper we presented dierent systems that are going to help societies, municipalities and organizations to become more visible in the modern world, become more accessible and transparent for interested parties and collaborate with each other. With help of Articial Intelligence and Natural Language Processing societies could build modern informa- tion systems now with no need of special knowledge and deliver services for more customers in the nearest future. 9. ACKNOWLEDGMENTS Part of the work was co-funded by Cooperation Programme Interreg V-A Slovenia-Austria 2014-2020, project AS-IT-IC. 10. REFERENCES [1] B. Cvetkovic, H. Gjoreski, V. Janko, B. Kaluza, A. Gradisek, M. Lustrek, I. Jurincic, A. Gosar, S. Kerma, and G. Balazic. e-turist: An intelligent personalised trip guide. Informatica, 40(4):447, 2016. [2] D. Kuznar, A. Tavcar, J. Zupancic, and M. Duguleana. Virtual assistant platform. Informatica, 40(3):285, 2016. 10 Tourist Information Center Slovenj Gradec as a Part of the AS-IT-IC Interreg Project Marija Lah Jernej Zupančič SPOTUR Slovenj Gradec “Jožef Stefan” Institute and Glavni trg 1 Jožef Stefan International Slovenj Gradec, Slovenia Postgraduate School marija.lah@slovenjgradec.si Jamova cesta 39 Ljubljana, Slovenia jernej.zupancic@ijs.com ABSTRACT 6. Developing of new tourist products. We present the Tourist Information Center Slovenj Gradec and the organization within which it operates. Additionally 7. Encouraging the development and regulation of tourist we provide a short description of the Austrian-Slovenian In- infrastructure facilities in the area of the founder’s mu- telligent Tourist-Information Center (a project within the nicipality. Cooperation Programme Interreg V-A Slovenia-Austria, pe- riod 2014-2020) and the role of the Tourist Information Cen- 8. Organizing and marketing public events. ter within the project. We also provide our view of Infor- mation and Communication Technologies impact on tourism 9. Graphic design and editing of the web presence. and discuss the need for a nation/region-wide platform for services and tools for tourist services providers and tourists. 10. Providing tourist information. Keywords 2. AS-IT-IC PROJECT tourism; information society; AS-IT-IC project 2.1 About the Project 1. INTRODUCTION Austrian-Slovenian Intelligent Tourist Information Center (AS-IT-IC)1 is a project that was accepted in the cross- SPOTUR Slovenj Gradec (Slovene: ”Javni zavod za turizem, border Cooperation Programme Interreg V-A Slovenia-Austria šport, mladinske in socialne programe SPOTUR Slovenj Gradec”) in the programme period 2014-2020. The project has two is a public institution, established in 2009, with a view to main goals: carry out various tasks of public interest in the field of tourism, sports, youth, and social programs in the munici- pality of Slovenj Gradec. SPOTUR cooperates with several 1. To develop information and communication technology similar institutions in the area of Koroška as well as in the (ICT) tools to support a tourist when he or she wants wider region and is an important operational partner in the to create a personalized itinerary for the visit of the Regional Development Agency for Koroška (which operates Slovenian-Austrian cross-border area. under the auspices of the Regional Development Agency). 2. To create a sustainable community that will support 1.1 SPOTUR Objectives the use and maintenance of the developed tools. Objectives of the SPOTUR Public Institution in the field of tourism are the following: The project consortia comprises 5 partners from the Slovenian- Austrian cross-border area. 1. Organized, constant and professional approach to the development of new integrated tourism programmes (excursions, weekend programs, holidays). 1. Jožef Stefan Institute, 2. Coordination an cooperation with all tourist providers 2. Graz University of Technology, Institute for Software in the municipality. Technology, 3. Promotion and marketing of Slovenj Gradec as a tourist 3. SPOTUR Slovenj Gradec, destination on the domestic and foreign markets. 4. Creating and ensuring the strategic development of 4. The Association of Municipalities of Slovenia, and tourism in the region. 5. Graz Tourismus und Stadtmarketing GmbH. 5. Linking the public and entrepreneurial interests and services. 1https://as-it-ic.ijs.si 11 2.2 Relation to the Project 3. Chat platform will allow the tourists to communicate The Tourist Information Center of Slovenj Gradec is a de- with the tourist information officer or the virtual assis- partment within the Public Institution SPOTUR, which is tant in the same familiar chat interface. This will pro- the main office involved with the AS-IT-IC project. With vide a single access point to the information on sights the participation to the project we want to improve and in the Slovenian-Austrian cross-border area. update the quality of our tourist information services and cooperate with Austrian and Slovenian partners in building the next generation tourism-oriented ICT tools. Within the scope of the project we will assist in building a modern interface to approach the tourists and the services 2.2.1 ICT Project Tools for continuous availability of information. This is of great importance to us and every other tourist office. By partici- Currently, a tourist cannot get the desired information about pating in the project and the advantage to be the first to add Slovenj Gradec in an integrated way from both the hu- new content to the project platform and use it we expect to mans and Web services, much less from the joint Austrian- increase the visibility of our tourist offer and thus increase Slovenian services. Our tourist office provides local informa- the number of tourists who decide to visit our tourist desti- tion in traditional ways through printed materials, brochures, nation. web sites and social media, and face-to-face communication with tourists when they visit our office. According to the world-wide trend, the tourists increasingly obtain more in- 2.2.2 Tourism Network formation about sights and attractions on the Internet or The most important goal of the project for us is to create through mobile applications. They do that either in ad- a joint cross border Tourist Information Center - an ICT vance, when planning their trip, or on the spot, when they supported network of service providers and tourist offices, find themselves in an unknown place with some time to to enhance continuous cooperation that is practically non- spend and do not know what to do. In both cases they do existing at the present time. In our opinion the cross-border not have the access to the human tourist information officer tourist exchange, collaboration and transfer of expertise be- or the printed brochures - all available at the tourist infor- tween providers is very important and can increase the visi- mation office. By relying on third party applications such as bility of the tourist attractions to a wider range of tourists. Google Search2, Google Maps3 and TripAdvisor4, they find Of particular interest are the possibilities to cooperate with only the most popular places and spend a big part of their tourist offices from the neighboring Austria, since the di- time searching and deciding on where to go. Consequently, verse tourist offer from both countries nicely complement tourists may miss locations they might be interested in and each other. have less time for sight-seeing. 3. SLOVENJ GRADEC TOURIST OFFER The lack of information and non-personalized trip planning The wider area of Slovenj Gradec has a lot to offer. Slovenj is two-fold: first, tourists may visit the places that receive Gradec is the cultural and economic centre of the Mislinja most publicity but skip the ones that they might really be valley. With its number of inhabitants, it is a small town, interested in, and second, some interesting smaller tourist but when you take its creative tradition and institutions into locations may get less visits, because they cannot afford the account, its importance extends over many borders. Numer- publicity of bigger attraction managers. ous exhibitions in the art gallery and events (some of them under the aegis of the UN) have brought the town closer to ICT tools that we help to develop will provide personal- its foreign neighbours - that is why in 1989 Slovenj Gradec ized recommendations to the tourist and allow him or her got the distinguished title of the Peace Messenger City. to communication with a human tourist information officer through the familiar chat-like interface. 3.1 Town History The ICT tools will consist of Virtual assistant, Chat plat- The historical image of Slovenj Gradec and its surround- form, and Tour planner. ing area stretches back to pre-historic times. This may be traced in the remains of Illyrian and Celtic settlement called Colatio. The medieval town was (like other oldest Slovene 1. Virtual assistant will provide automatic answering towns) founded in the 13th century. It has survived cen- in natural language and will be an important addition turies of turmoil but the town folk (most often artisans and to classic means of providing information, which nowa- merchants), together with foreign and native masters and days exist in the Tourist Information Centers. Virtual artists, have managed to care for the image of the town. assistant is available 24/7 and has the access to a vast The old town center has remained the focus of cultural and knowledge of tourist attractions available on the Inter- social life right up to the present day. net. 3.2 Cultural Heritage 2. Tour planner will enable better planning of cross- Slovenj Gradec has always been and still remains rooted border visits for the tourists - they will be able to within its historical and cultural tradition. The most im- discover less popular sites that would otherwise be portant artistic monuments in the town are to be found in missed, stay longer, and better satisfy their needs. the Gothic church of the Holy Spirit and in the church of 2https://www.google.si St. Elizabeth; it is also interesting to examine the old town 3https://maps.google.com center which has been preserved in its original design. In 4https://www.tripadvisor.com the nearby surroundings, there are quite a few cultural and 12 historical monuments, whose particular characteristics res- Slovenia lacks a common ICT platform (such as the one de- onate in a wider cultural context. The most important are scribed in [2]) with tools and services that can be used by any the church of St. George at Legen, the ruins of Vodriž castle, tourist information provider or a provider of a tourist ser- and the church of St. Pancras above the Stari trg (the Old vice. Some tourist-oriented points-of-interest such as sights, Square). There are also some sites of ethnological interest services, accommodation and activities are highlighted with that have been preserved. Among them we find many tra- a short description and a photo or a video on the main ditional Slovene hayracks (Slovene: “kozolci”), old peasant Slovenian tourist information site8 managed by the Slove- houses, chapels, and old watermills and sawmills. nian Tourist Board. According to the website: ”The Slove- nian Tourist Board (STB) is a national tourist organisation 3.3 Natural Heritage responsible for planning and carrying out marketing poli- Many diverse environments to be found around Uršlja gora cies in regard to Slovenia’s comprehensive tourist offerings. and Pohorje offer the visitors peace and its simple charms, Furthermore, this organisation is also entrusted with the which are the reasons on why they are worth discovering. task of developing Slovenian tourism.”9. While STB does a The town and surrounding area offer many different oppor- great job at promoting the Slovenian tourism destinations tunities for recreation: skiing on Kope, horse riding, biking, at fairs and social media and provides a great entry point gliding, and mountaineering, which make one appreciate the for a tourist that is yet to decide whether to visit Slove- nearby natural sights. nia, it lacks a platform that would be useful for providers of services (high quality sights entries, access to reservation 4. ICT TECHNOLOGIES system, tools for innovative sights presentations, high avail- ability access to potential customers over the Internet, a Slovenj Gradec with its surrounding area offers a wide range platform for establishing B2B contacts etc.) and the con- of smaller, attractive products. The promotion of the tourist sumers (tour planners, live chat support, dynamic informa- offer is mainly based on the printed materials, brochures, tion providers etc.). web sites5 and social media6. However, due to the small size of Slovenj Gradec and low budget, some attractions might Each municipality deals with the same problem - there is no remain undiscovered. sustainable solution that would help in creating innovative ICT tools that would be of use for all municipalities at af- In the last years we have tried to involve some modern tools fordable cost. A tourism platform would have to be a cloud of promotion, such as virtual tour 7 on Google maps and native ecosystem with: mobile applications. 4.1 Virtual Tour 1. Open application programming interfaces (APIs) that Virtual tour (Figure 1) includes the integration of 360deg would enable the solution providers to develop useful spherical images of points-of-interest in the Slovenj Gradec services for tourist service provider and tourists. area and a map with the locations of the chosen points. The application enables the user to interact with the spherical 2. Ready-to-use modules (provided as a service) with a images, zoom-in/out, share the view or like it using the so- pay-per-use subscription models that would enable tourist cial media ”Like” button. We provide 14 spherical images of service providers a one-stop shop to useful services. churches, squares and tourist infrastructure. 3. Tourist applications gallery that gives the access to the applications relevant for tourists, which integrate 4.2 Problems with ICT Tools into the platform and use the platform databases and Due to the lack of professional support and resources the services. novel ICT products have been more or less a short terms so- lutions that didn’t bring us many positive results, although the ICT tools are of great importance for successful promo- 5. CONCLUSION tion (as evident in [1]). We have presented the tourism oriented goals of the SPO- TUR public institution, our role in the AS-IT-IC project We see the same obstacles to innovative ICT supported solu- and our views of the ICT tools and services for tourism. We tions in other Tourist information centers and municipalities have also discussed the problems of smaller tourist points- as well. Municipality is able to reserve some resources for an of-interest in promoting the attractions, obtaining the vis- innovative ICT tool (web or mobile application or advanced itors and managing the destinations. To this end we pro- interactive content), however, due to the lack of sustainable posed a larger-scale platform with built in services available resources the tool is only developed to the prototype stage to tourists and tourist service providers. We are convinced where it remains unchanged for years. Lacking the mar- that the AS-IT-IC project is a step in the right direction of keting power even great ideas go unnoticed by the general enabling such a platform. public for which the solution was intended. This goes hand in hand with the problem of discoverability - each organiza- With the participation in the AS-IT-IC project and with tion hosts its web applications on its own web site that is the help of well-known institutions we expect to lay the usually not optimized for search engines. groundwork for good practice of a modern Tourist Infor- mation Center also for other Information centers in Slovenia 5http://www.spotur.si/ 6https://www.facebook.com/spotursg/ 8https://www.slovenia.info/ 7http://www.turizem-slovenjgradec.si/slovenj- 9https://www.slovenia.info/en/business/about-slovenian- gradec/virtualna-panorama tourist-board 13 Figure 1: Web-application with 3D views of points-of-interest in the Slovenj Gradec area and Austria, with whom we expect to build a fruitful and long term cooperation. 6. ACKNOWLEDGMENTS Project AS-IT-IC is co-funded by the Cooperation Programme Interreg V-A Slovenia-Austria 2014-2020. 7. REFERENCES [1] U. Gretzel, R. Law, and M. Fuchs. Information and communication technologies in tourism 2010. Springer, 2010. [2] U. Gretzel, M. Sigala, Z. Xiang, and C. Koo. Smart tourism: foundations and developments. Electronic Markets, 25(3):179–188, 2015. 14 Nadgradnja Sistema e-Turist in Integracija s platformo AS-IT-IC Blaž Mahnič Vesna Koricki Lana Zemljak Institut Jožef Stefan Institut Jožef Stefan Institut Jožef Stefan Jamova cesta 39 Jamova cesta 39 Jamova cesta 39 Ljubljana, Sloveija Ljubljana, Sloveija Ljubljana, Sloveija blaz.mahnic@ijs.si vesna.koricki@ijs.si lana.zemljak@ijs.si POVZETEK Z večanje števila turistov, se povečuje tudi število posameznikov Slovenski turizem in turizem v sosednjih državah beleži v zadnjih in manjših skupin, kateri se ne udeležujejo organiziranih izletov letih rekordne številke. Slovenija se uvršča nad povprečje Evrope. pod vodstvom strokovno usposobljenih turističnih vodičev in katere zanimajo tudi manj znane turistične znamenitosti. Ti turisti V letu 2016 je bilo v Sloveniji za 9,9% več prihodkov in 8,1% več prenočitev glede na leto 2015. [1] Z večanjem števila si po navadi sestavijo program ogleda sami, kar pa običajno ni turistov, se veča tudi število tistih turistov, kateri se ne lahka naloga, saj so na spletu podatki razdrobljeni in ne povezani udeležujejo organiziranih izletov. Taki turisti si po navadi v med sabo. kratkem času ogledajo veliko število turističnih znamenitosti. Sistem Turist sestavlja spletna aplikacija v katero uporabnik Kljub dostopnim informacijam na spletu pa je planiranje poti in vnese svoja zanimanja glede na lokacijo znamenitosti katere si ogledov za povprečnega turista težak zalogaj. želi ogledati, namen potovanja, čas za potovanje, ki ga ima na voljo, prevozno sredstvo s katerim potuje in čas katerega bo Sistem Turist pripravi za turista program izleta, prilagojen njegovim željam. Turistu so ponujene turistične znamenitost iz namenil za obrok, če tako želi. Na podlagi zbranih podatkov, program s pomočjo priporočilnega sistema organizira izlet izbranega območja, izriše pa se tudi pot ogleda turističnih znamenitosti na zemljevidu. Ponujeni so mu tudi pisni in govorni prilagojen turistovim željam. V ta namen, sistem uporablja opisi, fotografije in ocene drugih uporabnikov, kateri so že priporočilni sistem in metode za iskanje najkrajše poti z obiskali te turistične znamenitosti. najzanimivejšimi znamenitostmi. Vsako znamenitost je mogoče tudi oceniti, kar priporočilni sistem upošteva pri načrtovanju poti Sistem smo nadgradili z dodatnimi tri tisoč turističnimi v prihodnje. Del sistema Turist so tudi administrativne strani, ki znamenitostmi na Slovenskem. Znamenitosti so bile avtomatsko turističnim delavcem omogočajo vnos novih turističnih dodane v bazo podatkov s pomočjo modula za avtomatsko znamenitosti in pregled ocen obiskovalcev ter obiska Slika 5. dodajanje novih znamenitosti katerega bomo opisali v Obstoječi sistem smo nadgradili z dodatnimi tri tisoč nadaljevanju. turističnimi znamenitostmi na Slovenskem. Znamenitosti so bile avtomatsko Cilj dodajanja novih turističnih znamenitosti je povezava dodane v bazo podatkov s pomočjo modula za dodajanje novih obstoječega informacijskega sistema Turist z novim naprednejšim znamenitosti katerega bomo opisali v nadaljevanju. informacijskim sistemom imenovanim AS-IT-IC. Cilj dodajanja novih turističnih znamenitosti je povezava obstoječega informacijskega sistema Turist z novim naprednejšim 1. UVOD informacijskim sistemom imenovanim AS-IT-IC. Naloga Turizem je ena izmed najpomembnejših in hitro rastočih panog informacijskega sistema AS-IT-IC je pomoč turistom pri slovenskega gospodarstva. Slovensko turistično gospodarstvo načrtovanju njihovih čezmejnih obiskov, spodbujanje k prispeva 13% BDP, neposredno ali posredno zaposluje 12% odkrivanju manj znanih zanimivosti in omogočanje kvalitetnejših aktivne delovne populacije in predstavlja 40% izvoza storitev. izpolnjevanj želja. V nadaljevanju bomo opisali arhitekturo Slovenija je mednarodno prepoznana kot »zelena, aktivna in Sistema Turist, vnos novih turističnih znamenitosti, povezavo »zdrava« turistična destinacija. med sistemom Turist in priporočilnim sistemom za načrtovanje Slovenski turizem in turizem v sosednjih državah beleži v zadnjih poti glede na želje uporabnikov, kateri je del sistema AS-IT-IC. letih rekordne številke. Slovenija se uvršča nad povprečje Evrope. V letu 2016 je bilo v Sloveniji za 9,9% več prihodkov in 8,1% več prenočitev glede na leto 2015. [1] 2. PRIPOROČILNI SISTEM IN UPORABNIŠKI VMESNIK SISTEMA TURIST 2.1 Priporočilni sistem Sistem deluje kot spletna storitev, uporabniki lahko dostopajo do sistema preko spletnih brskalnikov. Priporočilni sistem v dveh korakih sestavi program ogleda. V prvem delu za vsako znamenitost izračuna primernost za danega turista. V ta namen uporablja kombinacijo priporočanja na podlagi znanja in Slika 1: Povečevanje obiska turistov v Sloveniji za leto 2016[1] skupinskega filtriranja (collaborative filtering). Priporočanje na 15 podlagi znanja primernost znamenitosti izračuna iz strokovnega mnenja o njeni pomembnosti in tega, katere znamenitosti so primerne za katere turiste na podlagi starosti, izobrazbe, narodnosti in finančnih sredstev turistov, ki jih ti lahko vnesejo v svoj profil. Če je profil na voljo, je prednost tega načina priporočanja, da deluje takoj – ne potrebuje nobenih predhodnih ocen znamenitosti ali turista. Skupinsko filtriranje pa primernost izračuna iz ocen, ki so jih znamenitosti dali drugi turisti, ki so v preteklosti izkazali podoben okus kot turist, za katerega se primernost računa. V drugem koraku se znamenitosti na podlagi primernosti, ki jih izračuna priporočilni sistem, in njihovih zemljepisnih položajev uvrstijo na program ogleda. [2] 2.2 Uporabniški vmesnik Načrtovanje ogleda (slika 2) se prične z izbiro regije katero si Slika 3: Program in zemljevid ogleda želimo ogledati. Če v regiji lahko obstaja več občin, te se nam prikažejo ob kliku na regijo. Uporabnik lahko na to izbere določene občine ali pa pusti izbrano celotno regijo. Trenutno pokriva sistem 13 regij v Sloveniji, preko administrativnega vmesnika pa je mogoče regije poljubno urejati ali dodajati. Potem, ko si uporabnik izbere regijo oz. kraj, si lahko po želji nastavi tudi čas kosila, pričetek izleta, čas katerega ima na voljo, število dni trajanja izleta in prevoz s katerim bo odšel na izlet. Ta je lahko peš ali pa z avtomobilom. Slika 4: Ogled podrobnosti turistične znamenitosti 2.3 Administrativne spletne strani Slika2: Načrtovanje ogleda Administrativne strani omogočajo uporabniku urejanje lokacij in Program ogleda (slika 3) je sestavljen iz znamenitosti in urejanje turističnih znamenitosti. podznamenitosti. Ob kliku na znamenitost se nam na zemljevidu Urejanje lokacij določita geografski položaj in okvirna velikost prikaže naslov znamenitosti in trajanje ogleda. Več o znamenitosti (polmer). si lahko uporabnik ogleda s klikom na gumb “več” (slika 4). Tam Urejanje znamenitosti med urejanjem znamenitosti se vnesejo je na voljo tudi govorni opis in slika znamenitosti. opis s slikami in bogati metapodatki. Med metapodatke spadajo: Zemljevid ogleda (slika 3) prikazuje pot ogleda, Z različnimi naziv, naslov, vrsta, strokovna ocena, geografski položaj, barvami so prikazane znamenitosti na programu, znamenitosti ob odpiralni čas, čas ogleda, dostopnost za osebe z gibalnimi poti, turistična infrastruktura (npr. informacijske točke) in omejitvami, raznovrstna dodatna ponudba, podatki o starših in ogledane znamenitosti. otrocih, ki sestavljajo hierarhijo podznamenitosti, ter podatek o 16 tem, ali znamenitost v resnici ni znamenitost, temveč del turistične infrastructure in podatek za kakšne profile turistov je znamenitost primerna. 3. POVEZAVA SISTEMOV TURIST IN AS- IT-IC AS-IT-IC sistem je še v fazi razvoja, naloga sistema bo pomoč turistom pri načrtovanju njihovih čezmejnih obiskov, spodbujanje k odkrivanju manj znanih zanimivosti in omogočanje kvalitetnejšega izpolnjevanja želja kot obstoječi sistemi. Lokalne skupnosti bodo obiskovalcem učinkoviteje ponujale lokalne storitve in informacije, npr. organizator lahko vključi obisk obrtnika/umetnika glede na želje in tako poveča prodajo. Slika 6: Delovanje JSON formata datoteke Glavna prednost sistema bo pogovor turista in sistema v naravnem jeziku npr. “Vse reke na dolenjskem”. Turistične znamenitosti so bile zapisane kot objekti, kateri so vsebovali trinajst atributov: ime atrakcije, povezava na spletno Cilj povezave sistemov je, enostavno načrtovanje krajše in stran, naslov atrakcije, telefonska številka, spletna stran, oznake, večdnevne poti, ki vključujejo obisk naravne in kulturne dediščine, z možnostjo primerne nastanitve. Pri načrtovanju tip atrakcije, opis, fotografija, ime regije, ime občine, gpsX in večdnevne poti, zlasti pri čezmejnem območju, je običajno gpsY. Znamenitosti je bilo sprva potrebno prebrati, jih filtrirati in potrebno veliko truda za zbiranje informacij od začetne točke, nastanitve, do najboljših poti grupirati po regijah. V drugem koraku smo podatke shranili v - informacije o ciljnih lokacijah so razpršene, opisi za mesta so na voljo v različnih jezikih, itd. podatkovno strukturo in naprej v obstoječo podatkovno bazo Računalniški programi so dobri pri vključevanju in analiziranju sistema Turist (slika 7). velikih količin podatkov, filtriranju informacij ter računanju optimalnih rešitev. Zato je potrebno orodje, ki omogoča uporabniku, da hitro ustvari pot v skladu z željami in parametri. 3.1 Potrebne nadgradnje Sistema Turist Za povezave sistemov bo potrebno razviti različne API (“Application Programming Interface”) vmesnike za namensko programiranje. [4] API je namenjem naprednejšim uporabnikom ter razvijalcem in omogoča dostop do storitev sistema preko HTTP zahtevkov. Storitve bodo vračale tekstovne odgovore v formatu, kot je npr. JSON. Do sedaj smo že razvili API za dodajanje novih znamenitosti v podatkovno bazo, kateri je opisan v nadeljevanju. Potrebno pa bo Slika 7: Potek vnosa novih znamenitosti v obstoječo še razviti API za priporočanje znamenitosti, in gradnjo načrta podatkovno bazo poti. 3.2 API za vnos novih turističnih znamenitosti API funkcija za branje novih turističnih znamenitosti iz V obdelavo smo dobili veliko količino podatkov o turističnih strežnika pošlje podatke o turistični znamenitosti v JSON obliki v znamenitosth v Sloveniji, katere smo predhodno na strežnik obdelavo modulu za vnos novih znamenitosti. zapisali v podatkovni obliki JSON. V ta namen smo razvili API Funkcija za filtriranje vstopnih podatkov omogoča kasnejšo preko katerega so nam bili podatki na voljo, ta nam je ob klicanju lažjo obdelavo. Ker vsi atributni niso potrebni za vnos v obstoječo vračal objekt tipa JSON. JavaScript Object Notation ali JSON, [3] podatkovno bazo Sistema Turist, je najprej potrebno pobrisati je odprtokodni format datoteke, ki uporablja človeško berljivo nerelevantne attribute. Po tem sledi filtriranje znamenitosti, katere besedilo za prenos podatkovnih objektov, sestavljenih iz parov ne vsebujejo atributa občina. Na podlagi atributa naslov, je na to atributnih vrednosti in podatkovnih tipov matrike (ali katerekoli potrebno istancam določiti občino kateri pripadajo, tako da lahko druge serijsko spremenljive vrednosti). To je zelo pogosta oblika kasneje instance grupiramo po regijah. podatkov, ki se uporablja za asinhronsko komunikacijo brskalnika Funkcija za grupiranje po regijah nam omogoča natančen zapis / strežnika. v obstoječo podatkovno bazo, tako da lahko dodamo nove znamenitosti v že obstoječe regije, pravtako pa regije katere še niso zapisane v podatkovni bazi vpišemo na novo. Začetnim trem regijam Slovenska Istra, South Holland in Srce Slovenije smo dodali še enajst novih: Gorenjska, Goriška - Smaragdna pot, Jugovzhodna Slovenija, Koroška, Notranjsko – kraška, Obalno – kraška, Osrednjeslovenska, Podravska, Pomurska, Savinjska, in Zasavska. Funkcija za zapis novih znamenitosti, se sprehodi po vseh prebranih turističnih znamentostih, katere smo predhodno obdelali (filtriranje, grupiranje, vpis novih regij) in jih sproti zapisuje v 17 obstoječo podatkovno bazo. Med obdelavo so instance shranjene v pomnilniku računalnika. Po končanem zapisovanju lahko s pomočjo administrativnih spletnih strain opisanih v razdelku 2.3 preverimo pravilnost in smiselnost podatkov, da med samim zapisovanjem ni prišlo do kakšnih napak. S pomočjo API-ja za vnos novih turističnih znamenitosti smo tako število turističnih znamenitosti iz 300 povečali na skoraj 3000. 4. ZAKLJUČEK V članku smo predstavili obstoječi sistem za načrtovanje izletov Turist in povezavo sistema z novim sistemom za pomoč turistom pri načrtovanju njihovih čezmejnih obiskov AS-IT-IC. Podrobneje je predstavljeno delovanje Sistema Turist, v nadeljevanju pa potrebne nadgradnje. Opisan je tudi API za vnos novih turističnih znamenitosti v obstoječo podatkovno bazo. Zaradi velikega števila novih neobdelanih turističnih znamenitosti in morebitne kasnejše uporabe tudi v drugih sistemih je bilo nujno potrebno razviti API vmesnik za vnos novih turističnih znamenitosti. Vmesnik nam omogoča vnos novih turističnih znamenitosti preko HTTP zahtevkov. Programerju tako ni potrebno podrobno poznati delovanja API-ja. Tako lahko API uporabimo v bodoče tudi za morebitne druge naloge. V prihodnje je potrebno zaradi povezave sistemov razviti še API kateri bo nudil funkcije za priporočanje znamenitosti, in gradnjo načrta poti. API bo preko HTTP zahtevkov klican iz strain sistema AS-IT-IC. 5. LITERATURA [1]Publikacija turizem v številkah - https://www.slovenia.info/uploads/dokumenti/raziskave/2017 _06_sto_tvs_2016_a4_slo_web_1.pdf [2] B. Cvetković, H. Gjoreski, V. Janko, B. Kaluža, A. Gradišek, M. Luštrek, I. Jurinčič, A. gosar, S. Kerma, and G. Balažič. E- turist: An intelligent personalized trip guide. Informatica, 40(4):447, 2016 [3] D. Peng, L. Cao, W. Xu. Using JSON for Data Exchanging in Web service Applications. Journal of Computational Information Systems 7,16 (2011) 5883-5890 [4] T. Grill, O. Polacek, M. Tscheligi. Methods towards API Usability: A Structural Analysis of Usability Problem Categories, International Conference on Human-Centred Software Engineering, HCSE 2012: Human-Centered Software Engineering pp 164-180 18 Open Questions of Technology Usage in the Field of Tourism∗ Bernhard Peischl Oliver A. Tazl Franz Wotawa Institute for Software Institute for Software Institute for Software Technology Technology Technology Graz University of Technology Graz University of Technology Graz University of Technology b.peischl@ist.tugraz.at oliver.tazl@ist.tugraz.at wotawa@ist.tugraz.at ABSTRACT In order to come up with a general tourism application, The focus of this paper is to identify and discuss require- we need to identify potential use case scenarios from which ments necessary for a tourism recommender that is capa- we identify the most general system requirements. In addi- ble to interact with customers naturally in order to identify tion, we need to know potential competing systems and their their wishes and needs in order to plan for the best jour- shortcomings in fulfilling the obtained requirements. From ney. For this purpose, we introduce a case study comprising the requirements, we also are able to extract new challenges a typical conversation between a customer planning a trip that serve as foundation for research in order to come up and the system. We further on use this case study when with methods and techniques that allow to develop the in- discussing available solutions in order to identify shortcom- dented general tourism application. One example of such ings. We summarize the findings and come up with open an open issue necessary to be closed, corresponds to the research questions to be tackled in order to provide methods question of how to deal with inconsistencies that naturally and techniques needed when developing a recommendation arise during a conversation when searching for the best cus- system for tourists. tomer solution. There might be wishes like always staying overnight in a five star hotel, which might be in contradic- tion with the available budget or the chosen route in cases of Keywords unavailability of requested hotels. Therefore, systems have to adapt constantly during a conversation which requires intelligent recommendation systems; trip planning; require- solving inconsistencies and also obtaining knowledge to fur- ments for tourist applications ther enhance future recommendation interactions with the same customer. 1. INTRODUCTION In this paper, we start describing a general use case for In the last years a lot of booking portals and other tourism a tourism recommendation system that interacts with cus- applications have been arising, which people use regularly. tomers in the context of the AS-IT-IC project1. This use Most of these applications provide specialized services and case is further on used for identify shortcomings in avail- functionality but do hardly really cover interactions occur- able tourism systems considering planing a whole journey. ring for example in a travel agency when customers plan for In particular, we discuss TripAdvisor2 and Google Trips3 in their vacation. Hence, more sophisticated tourism applica-detail. Afterwards, we summarize the findings and identify tions are required that allow interactions with customers research questions to be tackled in order to really come up to identify needs and wishes for recommending traveling with a general tourism recommendation system that is able plans considering requirements like the available budget, to naturally interact with users in order to find the best the customer’s interests, routes, and available dates. Such traveling solution based on availability of resources and the tourism systems need to provide and integrate chat func- customer’s needs and wishes. tionality, booking systems, travel planning, and recommen- dation technology to identify the best match between the customer’s requirements and available offers. 2. USE CASE In the following use case, we discuss a typical scenario occur- ∗Authors are listed in alphabetical order. ring during a recommendation session with a tourism appli- cation we have in mind. In this use case, we mainly focus on the interaction between an intelligent tourism recommender and ignore other means like human interventions into the process. The purpose of this use case is to identify require- ments and needs for a tourism recommendation system to be developed. 1See https://as-it-ic.ijs.si/ 2See https://www.tripadvisor.com 3See https://get.google.com/trips/ 19 Use case ‘Trip to Europe‘. John is an American business- 3. REQUIREMENTS man who is interested in modern architecture and art, nat- The use case shows many features and requirements to an ural heritage, and ethnic culinary. For his next trip, he and application which is important for a user experience we pro- his family would like to visit the central European region, pose. specifically Graz. He wants a truly customized package of experiences, not a standard tourist package which he would be offered to him in a local tourist office. He can also use the internet search to obtain the information on his own. But 3.1 Natural Language Interface & Chat bot John neither wants to a standard package nor search on his A natural language interface is a great way to interact with own. So he decided to use again an application which pro- a user in a more natural and interactive way. Therefore it is vides recommendations, based on his interests and previous necessary to ensure that the system can process many kinds visits, human-like, but fully automated, communication via of information via a natural language interface. Talking to a chat interface and an easy to use interface via his mobile a virtual assistant enhances the user experience and allows devices. new ways of helping the customer to achieve his goals. The technological challences lie in the problem that a conversa- Therefore he visits the application and chooses to log-in since tion could involve many topics and domains which is not he can use one of his many identification providers such as easy to cover (see also [1], [2], [3]). This level of complexitiy Google or Facebook. After log-in, he checks his previously drops to some extent by narrowing the domain to the touris- specified preferences and the basic information about him- tic sphere. Nevertheless stays the complexitiy on a very high self, like his age, gender, and his interests. Next, he clicks on level. a button to launch a new tourist session. Immediately, he is greeted by a virtual assistant, which asks him which places John would like to visit. The assistant takes into account the provided answer and John’s interests and recommends him some natural and the cultural sights in the Slovenian- 3.2 Recommendation Austrian cross-border area. The task to recommend hotels, sights, etc. to customers is a very important task, because the time they want to invest in The virtual assistant first asks John about the basic trip their trip planning seems decreases. There are also several dates, like start date, duration, the number of travelers, stakeholders in this process of recommendation. Hotels want travel radius and cause of the stay. After answering these to sell their product, attractions want visitors, customers questions, the assistant offers him several interesting sights want a perfect vacation experience. Here it is necessary to in Graz like the Kunsthaus, the Eggenberg palace, and the find a balance between each of these interests. Schlossberg. All these recommendations are based on his profile, which he defined, and his past trips. By accepting The recommending strategies of the available platforms and several sights, the system tries to find other fitting sights as applications did not consider the user’s interests in the first well. Although John’s travel radius is too small for some in- place. There is no profile and not enough information avail- teresting sights in Ljubljana, the assistant offers him some of able to ensure qualitative recommendations. The user could these because of his strong preferences to this sort of sight. choose from several packages or has to plan the days on his The assistant included an offer of a car-rent as well for the own. There is no really customized and intuitive way to plan travel from Graz to Ljubljana. John accepts this offering such trips. The process of trip planning has to become more because he really likes the pictures and information, that humanized. the assistant provides to him. After selecting the sights, he wants to visit, the assistant starts to show him some possi- The approach, shown in the use case, by using the profile bilities to eat lunch and dinner during his visit. In his past and past trips to recommend the customer also shows that travels, John prefers quite expensive restaurants, because on this could lead to problems when the parameters for the trip business trips he does not really care. The assistant shows are different to everything before. There the system has to him that kind of restaurants in Graz and Ljubljana, but adapt itself to the new situation and should be able to learn he denies them because of the estimated costs. The sys- from this new kind of information. Maybe the change in tem adapts to the new situation and shows him lower priced behaviour is triggered by the weather, the fellow travelers, restaurants. John browses through the provided information the purpose of the travel or other external factors. The of the restaurants and selects his preferred places. The last system should detect this kind of changes, adapt for the step is the selection of the hotel. The system knows that specific session and ask the customer where the change of the hotel has to be family friendly and needs to be located behaviour come from. next to the sights he wants to visit, because of the answers before. The system lowers also the price range of the hotels. The assistant now finishes the session with calculating the 3.3 Trip & route optimization optimal path for his travel and offers him the possibility to The trip and route optimization is the next and last step in a book all necessary hotels, attractions, transportation and trip planing procedure. The distribution of places should be sights in a convenient way. The travel information is now evenly throughout the stay addressing the duration every provided within the application and could be easily accessed sights need to consume. The route should be short, but from every device John uses. scenic. The tourist needs enough time to visit the sight and should not feel like in a hurry. 20 4. AVAILABLE APPLICATIONS AND PLAT- the other portal which implies more hassle for the booking FORMS tourist. In the following section we discuss the main and established contendors in trip planning as well as new and arising plat- 4.2 Google Trips forms. Trips is a service by Google which provides similar features than TripAdvisor. It allows marking sights which should be 4.1 TripAdvisor visited during the trip. It also provides several predefined TripAdvisor is a platform, owned by the identically named sights packages, which could be visited in a day or more. The American company, which provides information and recom- ways between are already calculated and time approximated. mendations of sights and hotels, as well as hotel booking Another way is to define your own custom day sightseeing. features and more. It offers customers to share their expe- There you could select some sights and pin them to your riences and opinions with a large user audience. Originally route. After that, the application could add sights on the founded as an aggregator of professional reviews, found in way to complete a full day. After saving the package it is newspapers, guidebooks, and magazines, it evolved into a available in the trip view. user-generated content platform. Figure 2: A trip view in Google Trips Figure 1: A trip view in TripAdvisor It works tightly together with the product Google Maps (see The platform aggregates now all kind of information, like [4]) . The ratings, top comments, and some pictures are pictures, texts, ratings or tagging. It includes recommen- shown in the Trips app, along with phone number and web- dation features to provide advices for traveling to a specific site. There is no possibility to comment or rate there. If you city. want to comment or rate the sight, it is necessary to switch to the Maps App to do so. It provides a feature to plan specific trips and book hotels and visits from the platform. After selecting a name for The app works well with Gmail. Tickets and Reservations the trip, the destination and the date of the trip, the plat- are extracted from Mails and put into a section of the trip form enables the user to select several sights and attractions. view. There they could be easily accessed in one place during These places could be planned on days. A map of and routes the actual trip. This integration improves the user experi- between them is also provided. ence, but shows the high level of data analysis and intercon- nection within the Google services.4 The integrations to book hotels or reserve tables in restau- rants exist but do not submit every needed information to 4See http://inbox.google.com 21 4.3 AS-IT-IC in at least two languages and these data sources are possi- The project AS-IT-IC will provide an application, which al- bly enriched with timely information (local tourism events lows user to interact with human and virtual assistants to etc.). Considering the high granularity and large diversity plan trips to Austria and Slovenia. The usage of chat bots of the collected data and the fact that NLP and chatbots in- enhances the possibilities of tourist centric applications in a troduce additional inconsistencies in terms of data obtained great way. It allows a 24/7 coverage of user requests. The from interaction with the user, it is of uttermost importance capabilities of chat bots are a bit restricted today but the to address these issues. That is, one of the crucial questions field is subject to a rapid development. The recommenda- is to provide recommendation systems [5] that can handle tions will come from members of tourist offices and other such inconsistencies such that the user is seamlessly guided involved people as well as virtual assistants. Aside from when planning his trips. recommendations, the platform will support booking hotels and restaurants in a convenient and easy way. In this article we motivated the AS-IT-IC project and il- lustrated a use case in the field of tourism. Afterwards we The knowledge base of the platform stores hotels, sights and briefly discussed the requirements wrt. natural language in- other points of interest. Aside of the typical information terfaces and chat bots. We listed the central features of two about a place, it stores also information about the estimated well-known products in this field and focused on the chal- duration to visit it. lenges in designing the features for natural language pro- cessing, chat bots and the chat interface. One of the major The trip is planned during a normal chat conversation. This issues is the handling of inconsistent information. These planning could be done with a member of a tourist office or kind of information arises due to the multi-language inter- with the virtual assistant as conversation partner. After de- face, the granularity of cross-boarder relevant data and the termining some basic information the platform shows several interaction with the user. recommendations and possibilities to visit. After selecting the desired sights, hotels and eating spots the platform au- Acknowledgement tomatically calculates a convenient way through the points Research presented in this paper was carried out as part of of interest. the AS-IT-IC project that is co-financed by the Coopera- tion Programme Interreg V-A Slovenia-Austria 2014-2020, Trip Google AS-IT- European Union, European Regional Development Fund. Advisor Trips IC Recommendations + + + Reviews + + + 6. REFERENCES Rating + + + [1] R. J. Moore, R. Arar, G. . Ren, and M. H. Szymanski. Manual planning + + + Conversational ux design. In Conference on Human Auto planning - + + Factors in Computing Systems - Proceedings, volume Preplanned packages - + + Part F127655, pages 492–497, 2017. NLP & chat bots - - + [2] Stuart J. Russell and Peter Norvig. Artificial Chat interface - - + Intelligence: A Modern Approach. Pearson Education, 2 edition, 2003. Table 1: Comparison of the platforms [3] H. Kahaduwa, D. Pathirana, P.L. Arachchi, V. Dias, S. Ranathunga, and U. Kohomban. Question answering system for the travel domain. pages 449–454. Institute 5. DISCUSSION AND CONCUSION of Electrical and Electronics Engineers Inc., 2017. The compared platforms show many possibilities and fea- [4] Personalized travel planner google trips gets better at tures which are useful for a good user experience. The fol- handling your reservations | techcrunch. lowing table (see Table 1) shows a quick overview of the https://techcrunch.com/2017/04/26/personalized- supported functionalities of every platform. travel-planner-google-trips-gets-better-at-handling- your-reservations/. (Accessed on Ratings, reviews and recommendations in any way are broadly 17/09/2017). used in this applications. There are differences between the [5] Dietmar Jannach, Markus Zanker, Alexander Felfernig, level of integration and automation of these functionalities. and Gerhard Friedrich. Recommender Systems: An Planning features are also differently implemented in the re- Introduction. Cambridge University Press, New York, viewed platforms. Trip Advisor lacks fully automatic plan- NY, USA, 1st edition, 2010. ning. As part of our research effort, natural language processing, chat bots and chat interfaces will be supported by the plat- form AS-IT-IC. It brings this technology in a new way to the field of tourism. That makes it unique to some extent throughout the reviewed products. However, in order to support the prototyping of such a product, several research challenges need to be addressed within the project. First, the fact that AS-IT-IC deals with local cross-boarder con- tent, there is the risk of inconsistencies among the various data sources. Data is retrieved from local tourism offices, 22 Testing of Artificial Intelligence Applications ∗ A State of the Art Survey Bernhard Peischl Oliver A. Tazl Franz Wotawa Institute for Software Institute for Software Institute for Software Technology Technology Technology Graz University of Technology Graz University of Technology Graz University of Technology bpeischl@ist.tugraz.at oliver.tazl@ist.tugraz.at wotawa@ist.tugraz.at ABSTRACT Artificial Intelligence (AI). It is often assumed that an im- Verification and validation are procedures that are used to- plementation of an algorithm works as expected because of gether for checking that a product, service, or system meets available correctness, termination, and completeness proofs. requirements and specifications and that it fulfills its in- However, often systems fail because of underlying bound- tended purpose. With the advent of artificial intelligence- aries like memory limitations, or the used data structure, enabled applications, there is an increased pressure to test causing a system to crash. Unfortunately, such bugs are such systems. In this article we present a survey on the hardly considered during formal verification based on math- state of the art in testing artificial intelligence applications. ematical proofs. The necessity to deal with testing is fur- We present the general publication population in this area ther supported considering a most recent example. In Tom and discuss the open challenges and issues when it comes to Simonite’s Wired article, Even Artificial Neural Networks verification and validation of artificial intelligence software. Can Have Exploitable ‘Backdoors‘1, the author mentioned the case where a neural network can be trained to behave Keywords differently in identifying a traffic sign in cases with or with- test, software test, verification, validation, adaptive systems, out attaching a post-it note. Such behavior can have severe software evolution consequences in real life and thus testing AI systems is re- quired. 1. INTRODUCTION In this paper, we focus on the current state of the art in test- Verification and validation is one of the most important ac- ing AI applications. In particular, we are interested in find- tivities carried out during system development to assure sys- ing out whether there are already testing approaches used tem quality. The purpose of verification is to assure that in the context of AI applications and also to discuss future a system fulfills its specification, whereas validation deals research directions and open challenges. The goal behind is with assuring that the system implements the functionality to prepare for testing an application in the area of tourism users are expecting. Hence, verification answers the question recommendation systems where we are interested in finding whether someone has built the system right, whereas vali- the right testing technique to be applied before deployment dation deals with the question: ”Have I developed the right in order to capture the most important bugs during develop- system?”. Testing (see e.g. Myers [6]) is one activity that ment. This paper is organized as follows: First, we discuss can be used for both validation but also verification. Quot- the research design behind the survey including the tackled ing Edsger W. Dijkstra ”Testing shows the presence, not the research questions, the analysis procedure and the study re- absence of bugs” it is obvious that the purpose of testing is sults. We further on discuss the obtained results and come to find faults in the system but there is no guarantee to find up with open research questions. Finally, we conclude the all of them before deployment. Nevertheless, testing is in- paper. evitable for quality assurance, which cannot be superseded by formal proofs as said by Donald Knuth: ”Beware of bugs in the above code; I have only proved it correct, not tried it.”. 2. RESEARCH DESIGN Unfortunately, testing as a necessary activity within the soft- Our objective is to capture the field of testing wrt. ap- ware engineering process has gained only little attention in plications that exhibit intelligent behavior, i.e., a software ∗Authors are listed in alphabetical order. that perceives its environment and takes actions to maxi- mize its chance to achieve a specific goal. In this context, the software mimics cognitive functions such as learning or problem solving. By analyzing the exiting pool of publica- tions, we provide a snapshot that further will be used to analyze trends or to identify gaps. Therefore, we identify the research questions first. 1See https://www.wired.com/story/machine-learning- backdoors/ 23 Table 1: Criteria for selecting relevant articles. Table 2: Research type facets for our survey Criteria (Wieringa et al. [13]). title, keyword list and abstract suggest that the paper is research type description related to testing and artificial intelligence an implementation has been carried paper presents test and AI-related topics, out, evaluation of implementation has evaluation research e.g., testing methods or tools, evolution of software systems been conducted, requires more than article is written in English language just one demonstrating case study paper belongs to the body of literature in the field of a solution of the problem is proposed, computer science or software engineering benefits/applicability is demon- full text of the paper is available strated by example, this includes solution proposal proposals complemented by a demon- strating case study, however, no Research questions: dissemination plan is obvious paper comes up with a new way of thinking or structuring a specific field, • RQ 1: What is the general publication population e.g., in the form of a taxonomy of con- when it comes to testing AI-enabled applications? This philosophical paper ceptual framework, secondary studies research question aims to structure the publication like systematic literature reviews or pool on and around testing AI-enabled applications. systematic mapping studies In particular we are interested in the research type captures a personal opinion, the work facets in this evolving field of research. opinion paper however, is not grounded in related • RQ 2: What types of research contributions are in work or research methodology place regarding software testing of AI-enabled appli- captures personal experiences and de- cations? This question deals with the addressed topics experience paper scribes how things are done in prac- and major contributions (e.g., models, theories, frame- tice works, guidelines, etc.). • RQ 3: Can we observe trends respectively are there al. [13]. Table 2 illustrates the proposed research types. In significant open challenges and issues? The last ques- order to characterize how a specific research type contributes tion will investigate the focus point and strive to iden- to the body of knowledge regarding the testing of AI-enabled tify gaps in order to sketch possible future research on applications we used contribution type facets as proposed by testing of AI-enabled applications. Shaw [8]. Table 3 lists the criteria for these specific facets. In this study we collected data from two sources. First we 3. STUDY RESULTS carried out a query via Scopus2 document search. Second In this section, we present and discuss the results of our we considered articles published in workshop series specifi- study. In particular we address the research questions raised cally dealing with realizing artificial intelligence synergies in in the previous section. software engineering [5, 4, 10, 9, 1]. 3.1 RQ1: General publication population Query construction: In a worksop we defined the key- To get an overview of the selected publications, we per- words that we are interested in. Since we are interested formed a categorization and defined the research type and in articles that investigate functional testing of of AI soft- the contribution type. Table 4 provides an integrated picture ware we looked for the keywords ’test’ or ’testing’ and the keywords ’functional’ or ’regression’ or ’acceptance’ and the keywords ’AI’ or ’artificial intelligence’ in the title or ab- stract. We performed an automated search that required us Table 3: Contribution types for our survey ([8]) to filter the result. For example, we found a number of pub- contribution type description lications that are not in software engineering or computer representation of observed science. We therefore cleaned the initial result by removing reality by concepts, these publications and we removed duplicates. model representation after Selection process: We classified the obtained papers as conceptualization relevant or irrelevant to build the final set of publications construction of a for further investigation. We applied the criteria listed in theory cause-effect relationship Table 1. framework of method framework related to testing 2.1 Analysis and classification of AI-enabled systems On the final set of publications we carried out the analysis guideline list of advices and classification. The classification has been carried out number of outcomes from in two dimensions. We classified every selected publications lessons learned obtained results according to the research type as proposed by Wieringa et a tool supporting testing of tool 2see www.scopus.com AI-enabled systems 24 Table 4: Integrated picture: type of research in the obtained result set (numbers in percent) evaluation solution philosophical opinion experience research proposal paper paper paper 11 50 31 4 4 Table 5: Integrated picture: type of research in the obtained result set (numbers in percent) fw./ less. model theo. guidel. adv. tool meth. lear. 11 4 60 0 8 3 15 that shows the papers in the different categories. Regarding the research type facet, our analysis reveals that the ma- jority of the contributions deal with solution proposals (50 percent) and philosophical papers (31 percent). Taking into account the fact, that most of the publications appeared in the last couple of years, the classification according to research types indicates a evolving research field. Only a minority (11 percent) of the research papers are classified as evaluation research. Table 5 aggregates the contribution type facet and shows a similar tendency. From the 53 papers in the result set, al- most 60 percent contribute frameworks or methods, followed by models (11 percent) and tools (15 percent). 3.2 RQ2: Research contributions From the analysis using the basic classification schemas, we see a clear trend towards solution proposals and the majority of the proposed solutions considers frameworks or methods. Figure 1: Systematic map over research- and con- A second trend is the appearance of philosophical papers. tribution types. Regarding the solution proposals, approx. 19 percent (10 out of 53 papers) are classified as framework/methods, i.e., solution proposals without any evaluations beyond a demon- the original software. They necessitated the development strating case study. In summary this indicates an emerging of repeatable ’intelligence tests’ that could be automated research field that is developing new approaches but the field to confirm that no functional changes occurred. Most no- yet lacks evaluated models and /or sound theories. Figure 1 tably, this article reports on lessons learned to smooth the illustrates a systematic map over research- and contribution transitions along the path from a research project to a com- types. mercially deployed artificial intelligence application. In [11] the authors describe their early experiences of using agile 3.3 RQ3: Trends and open challenges techniques while developing a solution to a specific, multi- As outlined previously, the majority of the contributions ad- objective real world problem called the United States Navy dress testing of AI-enabled software in terms of a framework Sailors’ Assignment Problem. Because the investigators are or method at the level of a solution proposal. The first pub- working in a research environment where the results pro- lications date back to 1991 and deal with testing of expert duced at intermediate stages cause the requirements to con- systems in the field of flight software [2] followed by the tinually change, an agile software development methodology testing of AI-applications for satellite command and control was deemed most appropriate. Although the research team [7] in 1996. In the recent past, the number of publications applied several agile practices, the paper emphasizes their dealing with verification and validation of AI-application has experiences when performing test-first or test-driven devel- risen. In [3] the authors present an analysis of the problems opment. Whereas the latter contribution focuses on test- and lessons learned from the deployment of an artificial intel- ing within an agile process when developing an AI-research ligence based financial application that was developed and prototype, at the other end of the spectrum, publications commercialized later. Regarding the research to develop- deal with automated learning of the behavior of evolving ment interface, the authors state the difference between an functionality. For example, the authors of [12] show that experimental AI, with its quirks and oddities; and a devel- model-based testing allows the creation of test cases from opment tool that must be a reliable and testable product. a model of the system under test. Often, such models are During this transition the authors encountered the prob- difficult to obtain, or even not available. Automata learn- lem of being able to confirm that the software, after opti- ing helps in inferring the model of a system by observing its mizations and modifications, was functionally equivalent to behavior. Under some assumptions, for dealing with nonde- 25 terminism, input-enabledness and equivalence checking, the [3] J. P. Gunderson and L. F. Gunderson. And then the authors prove that the algorithm produces a model whose phone rang .. In What Went Wrong and Why: Lessons behavior is equivalent to the one under learning. To what from AI Research and Applications, Papers from the extent this method is applicable to learn the behavior of 2006 AAAI Spring Symposium, Technical Report AI-enabled applications is an open issue. SS-06-08, Stanford, California, USA, March 27-29, 2006, pages 13–18. AAAI, 2006. 4. THREATS TO VALIDITY [4] R. Harrison, M. Mernik, P. Henriques, D. da Cruz, As a literature study in an emerging research field, this study T. Menzies, and D. Rodriguez. 2nd international may suffer from potential incompleteness of the obtained re- workshop on realizing artificial intelligence synergies sults. Also the study may exhibit a general publication bias, in software engineering (raise 2013). In 2013 35th i.e, positive results are more likely published than failed ap- International Conference on Software Engineering proaches. For example, to our best knowledge, the result (ICSE), pages 1543–1544, May 2013. set does not contain papers that report on failed attempts [5] R. Harrison, D. Rodriguez, and P. Henriques. to testing of AI-enabled applications. We counteract this Welcome to the first international workshop on risk by providing the result set to other researchers3 and en- realizing artificial intelligence synergies in software courage them to continue this line of research. Among the engineering (raise 2012). In 2012 First International major threats is also the threat regarding internal validity. Workshop on Realizing AI Synergies in Software Internal validity of the study could be biased by personal Engineering (RAISE), pages iii–iv, June 2012. ratings of the authors. To counteract this risk we used sup- [6] Glenford J. Myers. The Art of Software Testing. John porting tools in cleaning, study selection and classification Wiley & Sons, Inc., 2 edition, 2004. and in particular carried out this work in a peer setting. [7] GR Seftas. Testing artificial intelligence applications for satellite command and control. In AIAA, Space 5. DISCUSSION AND FUTURE WORK Programs and Technologies Conference, Huntsville, In this article we analyzed the publication flora in the emerg- AL, 1996. ing field of testing AI-enabled applications and structured [8] Mary Shaw. Writing good software engineering the general publication population in this field. We ana- research paper. In Lori A. Clarke, Laurie Dillon, and lyzed 53 contributions and conclude that the majority of Walter F. Tichy, editors, Proceedings of the 25th the papers are solution proposals, i.e, proposed methods of International Conference on Software Engineering, frameworks that are illustrated in terms of a single case May 3-10, 2003, Portland, Oregon, USA, pages study. There is lack of evaluation results and to our best 726–737. IEEE Computer Society, 2003. knowledge we cannot report on an article proposing a sound [9] B. Turhan, A. Bener, R. Harrison, A. Miransky, theory of testing AI-enabled applications. Although numer- C. Mericli, and L. Minku. 4th international workshop ous applications appeared in the recent past, we lack papers on realizing ai synergies in software engineering (raise that in particular deal with systematic testing of such appli- 2015). In 2015 IEEE/ACM 37th IEEE International cations. Most of the investigated articles address testing in Conference on Software Engineering, volume 2, pages a broader sense (namely verification and validation). Test- 991–992, May 2015. ing is about finding critical faults within the implemented [10] Burak Turhan, Ayse Basar Bener, Çetin Meriçli, software, such as mentioned in the introduction (memory Andriy V. Miranskyy, and Leandro L. Minku, editors. limitations, data structures and configuration issues). In de- 3rd International Workshop on Realizing Artificial ploying AI-enabled applications, testing will become a neces- Intelligence Synergies in Software Engineering, RAISE sity for professional software engineering of AI-applications. 2014, Hyderabad, India, June 3, 2014. ACM, 2014. Research in this direction, in particular contributions deal- [11] Pavan K. Vejandla and Linda B. Sherrell. Why an AI ing with conceptual and formal models, and sound theories research team adopted XP practices. In John D. alongside with evaluation research is thus highly desirable McGregor, editor, Proceedings of the 47th Annual in setting up future research. Southeast Regional Conference, 2009, Clemson, South Carolina, USA, March 19-21, 2009. ACM, 2009. Acknowledgement [12] Michele Volpato and Jan Tretmans. Active learning of Research presented in this paper was carried out as part of nondeterministic systems from an ioco perspective. In the AS-IT-IC project that is co-financed by the Coopera- Tiziana Margaria and Bernhard Steffen, editors, tion Programme Interreg V-A Slovenia-Austria 2014-2020, Leveraging Applications of Formal Methods, European Union, European Regional Development Fund. Verification and Validation. Technologies for Mastering Change - 6th International Symposium, 6. REFERENCES ISoLA 2014, Imperial, Corfu, Greece, October 8-11, [1] Proceedings of the 5th International Workshop on 2014, Proceedings, Part I, volume 8802 of Lecture Realizing Artificial Intelligence Synergies in Software Notes in Computer Science, pages 220–235. Springer, Engineering, RAISE@ICSE 2016, Austin, Texas, 2014. USA, May 14-22, 2016. ACM, 2016. [13] Roel Wieringa, Neil A. M. Maiden, Nancy R. Mead, [2] M. P. DeMasie and J. F. Muratore. Artificial and Colette Rolland. Requirements engineering paper intelligence and expert systems in-flight software classification and evaluation criteria: a proposal and a testing. In IEEE/AIAA 10th Digital Avionics Systems discussion. Requir. Eng., 11(1):102–107, 2006. Conference, pages 416–419, Oct 1991. 3see goo.gl/F4NvQ4 26 Virtual Assistants for the Austrian-Slovenian Intelligent Tourist-Information Center Jernej Zupančič Gregor Grasselli Aleš Tavčar “Jožef Stefan” Institute and “Jožef Stefan” Institute and “Jožef Stefan” Institute Jožef Stefan International Jožef Stefan International Jamova cesta 39 Postgraduate School Postgraduate School Ljubljana, Slovenia Jamova cesta 39 Jamova cesta 39 ales.tavcar@ijs.si Ljubljana, Slovenia Ljubljana, Slovenia jernej.zupancic@ijs.com gregor.grasselli@ijs.si Matjaž Gams “Jožef Stefan” Institute Jamova cesta 39 Ljubljana, Slovenia matjaz.gams@ijs.si ABSTRACT 1. Quicker and smarter booking. We present a virtual assistant for the Austrian-Slovenian Intelligent Tourist-Information Center (AS-IT-IC - project 2. Quicker and more entertaining travel options and des- within the Cooperation Programme Interreg V-A Slovenia- tinations searching process. Austria, period 2014-2020) that can answer questions and 3. Personalized city tour guides. hold a conversation on the topic of natural and cultural her- itage sights. The prototype is currently integrated into the 4. At least partial automation of call centers and infor- AS-IT-IC communication platform prototype and Slack ap- mation offices. plication and communicates in Slovene language. During the AS-IT-IC project the virtual assistant will be expanded to other languages (English and German) and other com- Existing travel bots usually specialize in one aspect of travel munication platforms. and can be sorted into one of the following categories: Keywords 1. Customer service/info bots. Usually very limited assis- virtual assistants; chatbots; chat platforms; tourism; natural tants that offer information about a certain business language understanding; AS-IT-IC project where the user talks to the VA instead of consulting complicated FAQ pages. Examples include Ana4 and Julie5. 1. INTRODUCTION Virtual assistants (VAs) or chatbots are software programs 2. Travel options searching and booking bots. These as- that can interact with the user through a vocal, textual or sistants allow one to search and book a flight, hotel, graphical user interface (or a combination of them) usually drive or restaurant through a conversation. This is mimicking the way humans converse. Continuous advances usually more time-consuming than using a graphical in artificial intelligence technologies, particularly text pro- user interface (GUI) with well-known and established cessing and natural language understanding, have greatly web forms. However, some search domains are easier increased the performance of VA systems. Easily accessi- to navigate using natural language, therefore with the ble tools for creating such systems (examples are API.ai1, advance of natural language understanding tools and Wit.ai2, and Microsoft’s bot framework3) together with the their easier integration into conversation platforms the widespread adoption of chatting platforms and mobile elec- options for innovation will increase. Examples of such tronic devices, have caused the number of VAs available to assistants are the Expedia Facebook Messenger Bot a user to increase significantly in the past few years. (hotel search) and the Skyscanner Facebook Messen- ger Bot (flight search). 1.1 Virtual Travel Assistants 3. Human assisted bots. Despite the advancements in Tourism is one of the industries where VAs can provide a natural language understanding research and the ris- significant added value as is evident by the increasing num- ing popularity of fully automated VAs the chatbot-only ber of tourism or travel chatbots. Travel chatbots usually response is rarely of good quality. Until enough us- aim to enable: age data is gathered and several conversational corner 1https://api.ai 4https://connectmiles.copaair.com/en/web/guest/ask-ana 2https://wit.ai 5https://www.amtrak.com/about-julie-amtrak-virtual- 3https://dev.botframework.com travel-assistant 27 cases are addressed, the human-in-the-loop approach logic and webhooks that will use a natural language under- seems to be the most effective assistance. The user standing toolkit on the backend, and several microservices may communicate through a familiar chat interface that will extend the functionality of Rocket.Chat for the pur- with a VA and when the VA is not sure about the pose of the AS-IT-IC project. Dashboards that will enable answer it forwards the question and the conversation access and modifications of the existing knowledge base will history to a human operator, who continues the con- have to be added, together with an ecosystem-wide user- versation with the user. Examples include Tradeshift management service that will allow for a seamless transition Go, Pana6, Lola7, and Mezi8. between applications, and a content system that will enable the ground truth knowledge base (multilingual tourist infor- 1.2 AS-IT-IC Project mation data) to be used by the various components. Austrian-Slovenian Intelligent Tourist-Information Center (AS- IT-IC) project was accepted in the cross-border Coopera- 3. AS-IT-IC VIRTUAL ASSISTANTS tion Programme Interreg V-A Slovenia-Austria in the pro- The AS-IT-IC VA will provide the integration of several ser- gramme period 2014-2020. vices and tools: Hubot - bot infrastructure for Rocket.Chat, Asistent, natural language understanding toolkit, content Currently the only way to obtain relevant information about system and third party services that will be integrated through cultural and natural heritage sights is through user-unfriendly the specialized bots (Figure 2). web search, less known information sites (usually managed on the government or local authority levels), and then plan the trip using itinerary planners such as Google Maps9 or more advanced tour planners such as e-Turist10 [1] and Tri- pHobo11. ICT tools will be built within the scope of the project that will enable integration of several solutions: tourism information search, tour itinerary planning, sight and tour recommendations, and live-chat with the tourist informa- tion providers (service providers and tourist offices, munic- ipalities, tourists and citizens) in one place - the AS-IT-IC Platform. 2. AS-IT-IC PLATFORM The goal of the AS-IT-IC project is to integrate and up- grade the existing tools to enable smart tourism. The ex- Figure 2: Example architecture for the AS-IT-IC isting components that will be integrated into the AS-IT-IC specialized bot ecosystem include (Figure 1): Rocket.Chat - a chat plat- form, e-Turist - a tour planner, Asistent - a rule based 3.1 AS-IT-IC VA Components question-answering and natural language understanding toolkit. The existing components will be upgraded with application 3.1.1 Hubot programming interface (API) implementations that will en- Hubot12 is an open source chatbot framework that has built able the use of their functionalities through third party ap- in support for the Rocket.Chat client. It provides a stan- plications and will be customized for the AS-IT-IC project. dardized way to create chatbots on several conversational platforms by matching patterns from user input and pro- ducing a response by calling custom JavaScript code that in most instances calls an external api. For the AS-IT-IC project, Hubot provides a way to call APIs exposed by the AS-IT-IC Virtual assistant and correctly displaying the re- sults in the Rocket.Chat client. 3.1.2 Asistent Asistent13 - slovenian for assistant ([2]) is a rule-based vir- tual assistant framework developed at the Jožef Stefan In- stitute, Department of Intelligent Systems. It enables the Figure 1: AS-IT-IC ecosystem components embedding of a floating window on a website, within which questions can be asked and the reply is presented, addition- Additionally, services that will enable the integration of ex- ally the background web-page is changed to a page relevant isting components will be developed: integrations that can to the answer. use the Asistent and e-Turist as a service, conversational Asistent provides a rudimentary API for posting questions 6https://pana.com and receiving answers, which will be upgraded during the 7https://www.lola.com/welcome project. The API exposes the endpoint /ask which takes 8https://mezi.com the question as a URL parameter and responds with a json 9https://www.google.si/maps 10http://e-turist.si 12https://hubot.github.com/ 11https://www.triphobo.com 13http://projekt-asistent.si/info/ 28 document that contains the answer and, optionally, the url about a sight. Since a microservice approach to implemen- of a web-page with relevant information to be displayed in tation will be used, the Content system will be accessible by the background. An example query to ask who is the mayor several AS-IT-IC ecosystem components. At the moment of Ljubljana with the question in Slovenian being “Kdo je the integration modules are developed for obtaining the in- župan?” is (after url encoding the question) formation about sights from the web and transforming the http://projekt-asistent.si/ljubljana data into a database schema used by the e-Turist. /ask?question=Kdo%20je%20%C5%BEupan%3F. The JSON re- sponse from the asistent service is: 3.1.5 Third Party Services The biggest advantage of a VA is the possibility to inte- grate some third party applications into the chat platform. { This greatly extends the functionality of the chat platform, "answer": "Župan Mestne občine Ljubljana je brings all the interaction into one place and enables a user-  Zoran Janković.
friendly natural text based conversation interface for con- Oglejte si tudi podatke o trolling applications. Several third party services will be in- tions (such as Asistent, e-Turist, full-text sights search) and dosedanjih ljubljanskih županih. outside services (such as Google places, Google maps, Tri-
", pAdvisor, restaurant bookings, hotel bookings, event ticket "url": "http://www.ljubljana.si/si/mol/zupan/", purchase). Some of these services already provide an API "id": 5495 that will facilitate access for the VA, for others, however, } wrapper methods will be implemented, which will then be used by the AS-IT-IC VA. Additionally, Asistent enables the information providers to 3.2 VA Prototype enter a custom knowledge base for the Asistent in the form of a triple: (question, answer, web-page), where the question In order to test the AS-IT-IC VA concept, we have imple- is presented in the form of a rule using keyword stems and mented a tourist VA that the tourist can use to ask questions logical operators. about a specific sight, a group of sights in a specific area, a specific group of sights, and to hold a conversation about a sight (preserving context and taking into account the con- 3.1.3 Natural Language Understanding Toolkit versation history). Currently, the prototype is implemented Natural language understanding (NLU) toolkit enables the for Slovenian language, since the used service for full-text parsing of entities, parameters and intents from a natural search is indexed only over the Slovenian content. language text. In a way NLU transforms non-structured text into a structured text that can then be used by other The prototype architecture is very similar to the example ar- programs and services to provide functionalities for the user. chitecture for the AS-IT-IC specialized bot (Figure 2). For Currently available NLU toolkits include API.ai, Wit.ai, Mi- the NLU toolkit we have chosen Api.ai, another third party crosoft bot framework, and Rasa NLU14. The AS-IT-IC VA service that we integrated into the prototype was full-text solutions will be designed in such a way that they will be as search over the sights, instead of the Content API we have much toolkit-agnostic as possible. This will enable higher accessed the sights database directly through the ORM (ob- sustainability of the project results and lower the damage in ject relational mapping) layer, conversation data was man- cases of unavailable external toolkit services. aged by the Api.ai chatbot framework, as well as the bot API. Additionally, we have provided rich message templates 3.1.4 Content System for bot integration into the Slack communication platform and custom Hubot scripts for integrating into the Rocket.Chat AS-IT-IC project partners have access to multiple databases client, which will form the basis for the AS-IT-IC plat- with information about sights in the cross-border area. How- form. Rich messages are only partially supported by the ever, each database is in its own format with its own special Rocket.Chat platform for now. To overcome this short- fields. In order to provide a unified data interface for all coming we will implement custom frontend modifications systems integrated into the AS-IT-IC ecosystem, a separate and API calls for Rocket.Chat, which will provide the re- Content system will be developed. quired functionalities. Rich messages are already supported by Slack, which was the reason we provided such integration. The goal of the Content system will be the integration of var- We will strive to provide services in a chat platform-agnostic ious information sources about sights into a database that way, which will allow us to easily transfer the bots to other will be easily updateable from the information sources and messaging platforms such as Facebook Messenger, Microsoft enriched with the data from third party content providers Cortana, Google Assistant, WeChat, Viber and others. from the Internet such as TripAdvisor, Google Places and relevant tourist information websites. The content system will therefore facilitate machine-human cooperation, where 3.2.1 VA Modeling and Integration some of the information may be pulled from third party web- The VA intelligence comprises NLU and a webhook part. sites automatically and a human may be able to view, edit or add sight entries. The REST API based microservice layer NLU part comprises model training, entity definitions, and will be responsible for providing all the relevant information context specifications. In order to train the model to adapt it to the tourism domain we had to come up with several pos- 14https://rasa.ai/products/rasa-nlu/ sible inputs from the user, when he/she communicates with 29 the VA. Each input was classified into one of the following categories: welcome, general sights query, sight query near specific place, sight query in the specific region, similar sight query near specific place, similar sight query in the specific region, adding the sight to ”must see list”, obtaining the path to sight, sight recommendation. Additionally we have added a few small talk categories, allowing for the VA to answer questions about itself - what does it do, who developed it, how old it is etc. Entities enable the VA to label a word or a set of conse- quent words and categorize it. We have added the following entities: attraction name, attraction type, place, and region. When enough training examples are given the NLU is able to infer the category on its own. The webhook integrates the logic of the VA transforming the intents and entities provided by the NLU toolkit to com- mands and method calls. Additionally, the webhook may use original input text from the user to call third party APIs. In the case of the VA prototype, we have devised different methods that correspond to the intent, determined by the NLU toolkit. The original input text is transformed to a query that was tested to yield better results and the trans- formed query is then sent to the sight description full-text search service. The obtained response is then transformed to a rich message representation which is forwarded to Api.ai, which in turn responds to the client in its own format. 3.2.2 VA Interaction The VA prototype supports the following types of questions by a user: 1. Show me some rivers in the Dolenjska region. 2. List the castles near Kranj. 3. (After asking about a sight in previous question) Rec- ommend me nearby sights. 4. (After asking about a sight in previous question) How do I get there? Figure 3: VA interface in the Slack client All the answers are provided in text form, with additional be based on the presented prototype, and the AS-IT-IC VA formatting when providing the description of a sight - url will be implemented. It will provide a unique access point link to the detailed description, category and a location of to all the specialized VAs. This will provide easy access to a sight, quick description, interactive buttons for quicker available bots and a seamless experience for tourists. interaction. An example is shown in Figure 3. 5. ACKNOWLEDGMENTS 4. CONCLUSION We thank student Jakob Šalej for his help in data integration The paper presents the architecture and functionalities of and tourist-information full-text search service implementa- the virtual assistant for the AS-IT-IC platform and its inte- tion. The work was co-funded by Cooperation Programme gration into the AS-IT-IC ecosystem. Additionally, a proto- Interreg V-A Slovenia-Austria 2014-2020, project AS-IT-IC. type VA was presented, which has already been integrated into the AS-IT-IC conversational platform and enables the 6. REFERENCES tourist to search a particular sight, a set of sights of partic- [1] B. Cvetković, H. Gjoreski, V. Janko, B. Kaluža, ular type, or obtain relevant information about a sight. A. Gradišek, M. Luštrek, I. Jurinčič, A. Gosar, S. Kerma, and G. Balažič. e-turist: An intelligent Future work will include the addition of other languages, pri- personalised trip guide. Informatica, 40(4):447, 2016. marily English and German and more thorough integration [2] D. Kužnar, A. Tavčar, J. Zupančič, and M. Duguleana. with the Rocket.Chat platform. Additionally, several other Virtual assistant platform. Informatica, 40(3):285, 2016. specialized VAs will be added, the integration of which will 30 Indeks avtorjev / Author index Bizjak Jani ...................................................................................................................................................................................... 7 Evseev Greg ................................................................................................................................................................................... 7 Gams Matjaž ............................................................................................................................................................................ 7, 27 Grasselli Gregor ....................................................................................................................................................................... 7, 27 Koricki Vesna ............................................................................................................................................................................... 15 Lah Marija .................................................................................................................................................................................... 11 Mahnič Blaž ............................................................................................................................................................................. 7, 15 Peischl Bernhard .................................................................................................................................................................... 19, 23 Tavčar Aleš .............................................................................................................................................................................. 7, 27 Tazl Oliver August ................................................................................................................................................................. 19, 23 Wotawa Franz ........................................................................................................................................................................ 19, 23 Zemljak Lana ............................................................................................................................................................................... 15 Zupančič Jernej ...................................................................................................................................................................... 11, 27 31 32 Konferenca / Conference Uredila / Edited by Delavnica AS-IT-IC / AS-IT-IC Workshop Matjaž Gams, Jernej Zupančič Document Outline A - Naslovnica-SPREDNJA - E B - Naslovnica - notranja - E C- Kolofon - E D-E - IS2017 - skupni zacetni del Blank Page F - Kazalo - E G - Naslovnica podkonference - E H - Predgovor - E I - Programski odbor - E J - PDF - E 1 - EVSEEV_intelligent-services INTRODUCTION TELEVISION 3D ASISTENT ASISTENT TOURISM HEALTH INTEGRATION WITH AS-IT-IC CONCLUSION ACKNOWLEDGMENTS References 2 - LAH_tourist-information-center 3 - MAHNIC_nadgradnja-sistema-eturist 4 - TAZL_open-questions Introduction Use Case Requirements Natural Language Interface & Chat bot Recommendation Trip & route optimization Available Applications and Platforms TripAdvisor Google Trips AS-IT-IC Discussion and Concusion References 5 - TAZL_testing-in-ai 6 - ZUPANCIC_virtual-assistants K - Index - E L - Naslovnica-ZADNJA - E Blank Page Blank Page Blank Page Blank Page Blank Page Blank Page