Zbornik 24. mednarodne multikonference • INFORMACIJSKA DRUZBA Zvezek D Proceedings of the 24th International Multiconference INFORMATION SOCIETY Volume D I S S 0 S I Delavnica projekta Insieme Insieme Project Workshop Uredniki • Editors: Matjaž Gams, Primož Kocuvan, Flavio Rizzolio 5. oktober 2021 Ljubljana, Slovenija • 5 October 2021 Ljubljana, Slovenia • http://is.ijs.si Zbornik 24. mednarodne multikonference INFORMACIJSKA DRUŽBA – IS 2021 Zvezek D Proceedings of the 24th International Multiconference INFORMATION SOCIETY – IS 2021 Volume D Delavnica projekta Insieme Insieme Project Workshop Uredniki / Editors Matjaž Gams, Primož Kocuvan, Flavio Rizzolio http://is.ijs.si 5. oktober 2021 / 5 October 2021 Ljubljana, Slovenia Uredniki: Matjaž Gams Department of Intelligent Systems Jožef Stefan Institute, Ljubljana Primož Kocuvan Department of Intelligent Systems Jožef Stefan Institute, Ljubljana Flavio Rizzolio Dipartimento di Scienze Molecolari e Nanosistemi Cà Foscari University, Italy Založnik: Institut »Jožef Stefan«, Ljubljana Priprava zbornika: Mitja Lasič, Vesna Lasič, Lana Zemljak Oblikovanje naslovnice: Vesna Lasič Dostop do e-publikacije: http://library.ijs.si/Stacks/Proceedings/InformationSociety Ljubljana, oktober 2021 Informacijska družba ISSN 2630-371X Kataložni zapis o publikaciji (CIP) pripravili v Narodni in univerzitetni knjižnici v Ljubljani COBISS.SI-ID 85865987 ISBN 978-961-264-217-4 (PDF) PREDGOVOR MULTIKONFERENCI INFORMACIJSKA DRUŽBA 2021 Štiriindvajseta multikonferenca Informacijska družba je preživela probleme zaradi korone v 2020. Odziv se povečuje, v 2021 imamo enajst konferenc, a pravo upanje je za 2022, ko naj bi dovolj velika precepljenost končno omogočila normalno delovanje. Tudi v 2021 gre zahvala za skoraj normalno delovanje konference tistim predsednikom konferenc, ki so kljub prvi pandemiji modernega sveta pogumno obdržali visok strokovni nivo. Stagnacija določenih aktivnosti v 2020 in 2021 pa skoraj v ničemer ni omejila neverjetne rasti IKTja, informacijske družbe, umetne inteligence in znanosti nasploh, ampak nasprotno – rast znanja, računalništva in umetne inteligence se nadaljuje z že kar običajno nesluteno hitrostjo. Po drugi strani se je pospešil razpad družbenih vrednot, zaupanje v znanost in razvoj. Se pa zavedanje večine ljudi, da je potrebno podpreti stroko, čedalje bolj krepi, kar je bistvena sprememba glede na 2020. Letos smo v multikonferenco povezali enajst odličnih neodvisnih konferenc. Zajema okoli 170 večinoma spletnih predstavitev, povzetkov in referatov v okviru samostojnih konferenc in delavnic ter 400 obiskovalcev. Prireditev so spremljale okrogle mize in razprave ter posebni dogodki, kot je svečana podelitev nagrad – seveda večinoma preko spleta. Izbrani prispevki bodo izšli tudi v posebni številki revije Informatica (http://www.informatica.si/), ki se ponaša s 45-letno tradicijo odlične znanstvene revije. Multikonferenco Informacijska družba 2021 sestavljajo naslednje samostojne konference: • Slovenska konferenca o umetni inteligenci • Odkrivanje znanja in podatkovna skladišča • Kognitivna znanost • Ljudje in okolje • 50-letnica poučevanja računalništva v slovenskih srednjih šolah • Delavnica projekta Batman • Delavnica projekta Insieme Interreg • Delavnica projekta Urbanite • Študentska konferenca o računalniškem raziskovanju 2021 • Mednarodna konferenca o prenosu tehnologij • Vzgoja in izobraževanje v informacijski družbi Soorganizatorji in podporniki multikonference so različne raziskovalne institucije in združenja, med njimi ACM Slovenija, SLAIS, DKZ in druga slovenska nacionalna akademija, Inženirska akademija Slovenije (IAS). V imenu organizatorjev konference se zahvaljujemo združenjem in institucijam, še posebej pa udeležencem za njihove dragocene prispevke in priložnost, da z nami delijo svoje izkušnje o informacijski družbi. Zahvaljujemo se tudi recenzentom za njihovo pomoč pri recenziranju. S podelitvijo nagrad, še posebej z nagrado Michie-Turing, se avtonomna stroka s področja opredeli do najbolj izstopajočih dosežkov. Nagrado Michie-Turing za izjemen življenjski prispevek k razvoju in promociji informacijske družbe je prejel prof. dr. Jernej Kozak. Priznanje za dosežek leta pripada ekipi Odseka za inteligentne sisteme Instituta ''Jožef Stefan'' za osvojeno drugo mesto na tekmovanju XPrize Pandemic Response Challenge za iskanje najboljših ukrepov proti koroni. »Informacijsko limono« za najmanj primerno informacijsko potezo je prejela trditev, da je aplikacija za sledenje stikom problematična za zasebnost, »informacijsko jagodo« kot najboljšo potezo pa COVID-19 Sledilnik, tj. sistem za zbiranje podatkov o koroni. Čestitke nagrajencem! Mojca Ciglarič, predsednik programskega odbora Matjaž Gams, predsednik organizacijskega odbora i FOREWORD - INFORMATION SOCIETY 2021 The 24th Information Society Multiconference survived the COVID-19 problems. In 2021, there are eleven conferences with a growing trend and real hopes that 2022 will be better due to successful vaccination. The multiconference survived due to the conference chairs who bravely decided to continue with their conferences despite the first pandemic in the modern era. The COVID-19 pandemic did not decrease the growth of ICT, information society, artificial intelligence and science overall, quite on the contrary – the progress of computers, knowledge and artificial intelligence continued with the fascinating growth rate. However, COVID-19 did increase the downfall of societal norms, trust in science and progress. On the other hand, the awareness of the majority, that science and development are the only perspectives for a prosperous future, substantially grows. The Multiconference is running parallel sessions with 170 presentations of scientific papers at eleven conferences, many round tables, workshops and award ceremonies, and 400 attendees. Selected papers will be published in the Informatica journal with its 45-years tradition of excellent research publishing. The Information Society 2021 Multiconference consists of the following conferences: • Slovenian Conference on Artificial Intelligence • Data Mining and Data Warehouses • Cognitive Science • People and Environment • 50-years of High-school Computer Education in Slovenia • Batman Project Workshop • Insieme Interreg Project Workshop • URBANITE Project Workshop • Student Computer Science Research Conference 2021 • International Conference of Transfer of Technologies • Education in Information Society The multiconference is co-organized and supported by several major research institutions and societies, among them ACM Slovenia, i.e. the Slovenian chapter of the ACM, SLAIS, DKZ and the second national academy, the Slovenian Engineering Academy. In the name of the conference organizers, we thank all the societies and institutions, and particularly all the participants for their valuable contribution and their interest in this event, and the reviewers for their thorough reviews. The award for lifelong outstanding contributions is presented in memory of Donald Michie and Alan Turing. The Michie-Turing award was given to Prof. Dr. Jernej Kozak for his lifelong outstanding contribution to the development and promotion of the information society in our country. In addition, the yearly recognition for current achievements was awarded to the team from the Department of Intelligent systems, Jožef Stefan Institute for the second place at the XPrize Pandemic Response Challenge for proposing best counter-measures against COVID-19. The information lemon goes to the claim that the mobile application for tracking COVID-19 contacts will harm information privacy. The information strawberry as the best information service last year went to COVID-19 Sledilnik, a program to regularly report all data related to COVID-19 in Slovenia. Congratulations! Mojca Ciglarič, Programme Committee Chair Matjaž Gams, Organizing Committee Chair ii KONFERENČNI ODBORI CONFERENCE COMMITTEES International Programme Committee Organizing Committee Vladimir Bajic, South Africa Matjaž Gams, chair Heiner Benking, Germany Mitja Luštrek Se Woo Cheon, South Korea Lana Zemljak Howie Firth, UK Vesna Koricki Olga Fomichova, Russia Mitja Lasič Vladimir Fomichov, Russia Blaž Mahnič Vesna Hljuz Dobric, Croatia Klara Vulikić Alfred Inselberg, Israel Jay Liebowitz, USA Huan Liu, Singapore Henz Martin, Germany Marcin Paprzycki, USA Claude Sammut, Australia Jiri Wiedermann, Czech Republic Xindong Wu, USA Yiming Ye, USA Ning Zhong, USA Wray Buntine, Australia Bezalel Gavish, USA Gal A. Kaminka, Israel Mike Bain, Australia Michela Milano, Italy Derong Liu, Chicago, USA Toby Walsh, Australia Sergio Campos-Cordobes, Spain Shabnam Farahmand, Finland Sergio Crovella, Italy Programme Committee Mojca Ciglarič, chair Bogdan Filipič Dunja Mladenič Niko Zimic Bojan Orel, Andrej Gams Franc Novak Rok Piltaver Franc Solina, Matjaž Gams Vladislav Rajkovič Toma Strle Viljan Mahnič, Mitja Luštrek Grega Repovš Tine Kolenik Cene Bavec, Marko Grobelnik Ivan Rozman Franci Pivec Tomaž Kalin, Nikola Guid Niko Schlamberger Uroš Rajkovič Jozsef Györkös, Marjan Heričko Stanko Strmčnik Borut Batagelj Tadej Bajd Borka Jerman Blažič Džonova Jurij Šilc Tomaž Ogrin Jaroslav Berce Gorazd Kandus Jurij Tasič Aleš Ude Mojca Bernik Urban Kordeš Denis Trček Bojan Blažica Marko Bohanec Marjan Krisper Andrej Ule Matjaž Kljun Ivan Bratko Andrej Kuščer Boštjan Vilfan Robert Blatnik Andrej Brodnik Jadran Lenarčič Baldomir Zajc Erik Dovgan Dušan Caf Borut Likar Blaž Zupan Špela Stres Saša Divjak Janez Malačič Boris Žemva Anton Gradišek Tomaž Erjavec Olga Markič Leon Žlajpah iii iv KAZALO / TABLE OF CONTENTS Delavnica projekta Insieme / Insieme Project Workshop ....................................................................................... 1 PREDGOVOR / FOREWORD ................................................................................................................................. 3 PROGRAMSKI ODBORI / PROGRAMME COMMITTEES ..................................................................................... 4 Platform for Multi-Omics Integration (PlatOMICs) applied to skin diseases with alterations in Notch signaling pathway / Brandão Lucas, Tricarico Paola Maura, Gratton Rossella, Moura Ronald, Crovella Sergio ............ 5 Implementing the INSIEME portal according to the patients and caregivers’ point of view / Truccolo Ivana, Gerlero Virginia, Rizzolio Flavio, Canzonieri Vincenzo ...................................................................................... 9 An Analytical and Empirical Comparison of Electronic and Mobile Health Platforms / Kocuvan Primož, Dovgan Erik, Gams Matjaž ............................................................................................................................................ 12 Android Application for Distance Monitoring of Elderly Parameters / Kocuvan Primož, Gams Matjaž, Valič Jakob .......................................................................................................................................................................... 16 Development and structural design of the frontend for unifying electronic and mobile health platform / Eržen Samo, Ilijaš Tomi .............................................................................................................................................. 20 Description of Health Service Selection and Structure of ISE-EMH Platform / Bele Klemen, Kocuvan Primož, Dovgan Erik, Gams Matjaž ............................................................................................................................... 23 Usability of smart home and home automation data / Palčič Devid, Ražman Simon, Strnad Marjan ................ 27 Intelligent cognitive assistant technology for (mental) health in the ISE-EMH project / Kolenik Tine, Klun Urša, Kocuvan Primož, Gams Matjaž ........................................................................................................................ 31 Analysis of a recommendation system used for predicting medical services / Noveski Gjorgji, Valič Jakob ...... 35 PlatformUptake Methodology for AHA Solution Assessment / Kolar Žiga, Gams Matjaž, Dovgan Erik, Vuk Zdenko .............................................................................................................................................................. 38 What-If Analysis of Countermeasures Against COVID-19 in November 2020 in Slovenia / Janko Vito, Reščič Nina, Tušar Tea, Luštrek Mitja, Gams Matjaž .................................................................................................. 42 Effectiveness of non-pharmaceutical interventions in handling the COVID-19 pandemic: review of related studies / Tomšič Janez, Susič David, Gams Matjaž ....................................................................................... 46 Napovedovanje trendov in optimiziranje ukrepov v boju proti pandemiji COVID-19: Tekmovanje XPRIZE in naslednji koraki / Luštrek Mitja, Reščič Nina, Janko Vito, Susič David, De Masi Carlo, Vodopija Aljoša, Marinko Matej, Tušar Tea, Dovgan Erik, Gradišek Anton, Cigale Matej, Gams Matjaž .................................. 52 Indeks avtorjev / Author index ................................................................................................................................ 57 v vi Zbornik 24. mednarodne multikonference INFORMACIJSKA DRUŽBA – IS 2021 Zvezek D Proceedings of the 24th International Multiconference INFORMATION SOCIETY – IS 2021 Volume D Delavnica projekta Insieme Insieme Project Workshop Uredniki / Editors Matjaž Gams, Primož Kocuvan, Flavio Rizzolio http://is.ijs.si 5. oktober 2021 / 5 October 2021 Ljubljana, Slovenia 1 2 FOREWORD The year 2021 is the first Insieme (ISE-EMH) Workshop since we have reached some achievement to present in the second year of the Insieme Italian-Slovenian Interreg project. Unfortunately, 2021 is also the second year of the Covid-19 pandemics, causing several problems. For one thing, we received less papers for the workshop than expected, although still quite a reasonable number. The second undesired Covid-19 problem is that the participants will not be able to meet in person and discuss the presentations and also the Insieme project progress. There is a certain difference when people meet alive and discuss issues also in the free time after the conference compared to the virtual official-time only events. However, that is the reality that we face these months. Nevertheless, the Insieme Worksop was proclaimed open to the broader area of Electronic and Mobile health (EMH). As a result, a couple of quite interesting papers were submitted, while on general the quality was extraordinarily high for a workshop. Some of the Insieme and EMH papers seem to bear a huge potential to progress towards decent SCI papers since they indeed tackle important issues and present novel research ideas and methods. Some of the additional papers deal with the research that yielded 2nd place in one of the top worldwide Xprize competitions for non-pharmaceutical measures against Covid-19. This research also resulted in some other awards, e.g. at the ETAI conference. There are 13 accepted papers for this year's workshop. Nine of them are related to JSI and the other four to the Insieme project partners. Each Insieme partner submitted at least one paper, which demonstrates the successful cooperation on this project. The workshop consists of two subsections. One is directly related to the Insieme project and the other is for general EMH topics, mainly to the Covid-19, and Comparison between health platforms – another European project Platform Uptake. Flavio Rizzolio, Primož Kocuvan Program Chairs 3 PROGRAMSKI ODBOR / PROGRAMME COMMITTEE Flavio Rizzolio (Chair) Rossella Gratton Diego Santaliana Matjaž Gams Primož Kocuvan Simon Ražman Samo Eržen 4 Platform for Multi-Omics Integration (PlatOMICs) applied to skin diseases with alterations in Notch signaling pathway. Lucas Brandão Paola Maura Tricarico Rossella Gratton Department of Medical Genetics, Department of Medical Genetics, Department of Medical Genetics, IRCCS Burlo Garofolo, IRCCS Burlo Garofolo, IRCCS Burlo Garofolo, Trieste, Italy Trieste, Italy Trieste, Italy lucabrand@gmail.com paolamaura.tricarico@burlo.trieste.it rossella.gratton@burlo.trieste.it Ronald Moura Sergio Crovella Department of Medical Genetics, Department of Biological and IRCCS Burlo Garofolo, Environmental Sciences, College of Trieste, Italy Arts and Sciences, University of ronald.rodriguesdemoura@burlo.trieste Qatar, Doha, Qatar sgrovella@qu.edu.qa ABSTRACT 1 INTRODUCTION Over the last years, a huge amount of information concerning Omics data have been produced and are of crucial Permission to make digital or hard copies of part or all of this work for significance for the understanding of the molecular personal or classroom use is granted without fee provided that copies are not mechanisms and for the identification of potential molecular made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party targets associated to many diseases. Indeed, Omics components of this work must be honored. For all other uses, contact the approaches allowed to initially decipher several biological owner/author(s). Information Society 2020, 5–9 October 2020, Ljubljana, processes found to be critically involved in the context of Slovenia © 2020 Copyright held by the owner/author(s). various pathologies. Despite these remarkable scientific advances, the majority of obtained results are disconnected We developed a Platform for Multi-Omics Integration and divergent, making their use limited. Thus, our team (PlatOMICs) that assembles a set of tools and bioinformatics started the deployment of PlatOMICs, a new Platform for applications that can allow the retrieval of scientific literature multi-omics integration, carrying an user-friendly interface. data (genomics, epigenomics, transcriptomics, proteomics Currently, PlatOMICs is under deployment in an and microbiomics) together with the analysis, deciphering, international cooperation including Brazil, Qatar and Italy interpretation and integration of all these set of information and has been divided into three phases. In the present work automatically, therefore building networks of molecular we report phase I in which multiple interactions and Omics meta-analysis. database/resource/repositories were interrogated to access Our goal is to refine the data available in scientific literature data from skin diseases presenting alterations in Notch and in Omics databases/resource/repositories relative to skin signaling pathway, as they constitute a cluster of disorders diseases that are characterized by defects in Notch signaling that were extensively studied during the Omics era, in order route, seeking to describe networks of molecular interactions to perform biological syntactical analysis to be implemented in the epithelial tissue potentially involved in the loss of in the next PlatOMICs phases. homeostasis in this district, event that may lead to the onset of different skin pathologies. 1.1 Multi-omics integration applied to skin diseases with KEYWORDS Notch signaling alterations Omics, genomics, transcriptomics, proteomics, network interaction, skin diseases. An aberrant progression of Notch signaling, either due to altered regulation or direct mutations, can induce skin diseases [1,2]. To date, molecular alterations in Notch signaling pathway have been reported for five human skin diseases including: Hidradenitis Suppurativa (HS), Dowling Degos Disease (DDD), Adams–Oliver Syndrome (AOS), Psoriasis (PS) and Atopic Dermatitis (AD) [1,3]. Therefore, a deep characterization of this cellular route seems to be of 5 Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia Brandão, et al. pivotal importance in order to clarify potential new performed, hence securing a more reliable and homogeneous pathogenic scenarios involved in these skin diseases. Indeed, investigation. Therefore, PlatOMICs will contain the results considering this critical aspect, in order to further restrict the obtained from literature and their integration, databases and search, we decided to consider in this study only skin new Omics studies. diseases possessing alterations in Notch pathway excluding malignancies. 2 RESULTS These skin disorders have been thoroughly studied in the last five years; indeed, 1555 articles regarding these five diseases As a validation model, phase I of PlatOMICs was executed and OMICs (genomics, transcriptomics, proteomics and on skin diseases presenting alterations in Notch signaling microbiomics) studies are available in PubMed [4]. pathway by examining the literature, thus providing Specifically, considering these five skin disorders possessing molecular insights to multi-omics integration approaches. alterations in Notch signaling, 821 articles about genome, 2.1 Results deriving from literature analysis: molecular 225 about transcriptome, 143 about proteome and 602 about insights to multi-omics integration microbiome, were published. The literature scan was accomplished by assessing the 1.2 Perspectives on multi-omics integration for skin following term "(Hidradenitis Suppurativa OR Dowling diseases with alterations in Notch signaling pathway Degos Disease OR Adams Oliver Syndrome OR Psoriasis OR Atopic Dermatitis) AND (Genome OR transcriptome OR Currently, PlatOMICs is under deployment in an proteome OR epigenome OR microbiome OR metagenome international cooperation including Brazil, Qatar and Italy. OR metabolome OR omic OR multi-omic) AND 'Homo PlatOMICs will be an online platform offering services to sapiens'[orgn:__txid9606]" using the DaVinci tool. access and analyze scientific literature and Omics data A DaVinci literature database (DaVinci Lit) was created automatically with great accuracy. The deployment was with 1252 articles retrieved from PubMed, and amongst all divided into three phases and in the present work, we report recovered papers 82 were excluded due to the absence of phase 1. Briefly, the various phases that constitute abstract/summary. Next, the remaining 1170 articles were PlatOMICs multi-omics analysis are given by: phase I, step analyzed, classified and categorized. The most cited words based on the interrogation and analysis of the whole available were 'skin' and 'patient'. The words ‘immune’, literature and Omics databases; phase II, stage regarding the ‘inflammatory’ and ‘inflammation’ were common. 742 analysis and questioning of previous and new Omics (or (63.4%) of articles cited, at least once, one of the indicated multi-omics) studies; phase III, part relative to the merge of words. Next, we seeked the context of each of these terms, findings deriving from phases I and II in order to finally revealing that they were mainly used to explain the immune compose the ultimate multi-omics integration in a meta- and inflammatory conditions of each disorder. 'Expression' multi-omics analysis (Figure 1). was cited along 333 (28.4%) articles to demonstrate molecular expression on experimental works of transcriptome (48 articles), epigenome or methylome (36 articles) and proteome (12 articles). The last word worth commenting is 'gut'. Gut was present in 158 articles and refers to the existing relationship between gut dysbiosis and the onset of allergic, the latter also represents a term included in the top cited words, disbalance. The overview of word atomization enabled us to understand what was the main focus of Omics literature for skin diseases with alterations in Notch signaling pathway. Next, we categorized the whole DaVinci Lit into five classes of Omics. Most of the articles were included as a genome or microbiome (metagenome) study, followed by transcriptome and multi-omics approaches (Table 1). Moreover, in the Figure 1. Workflow of OMICs Platform (PlatOMICs) for Omics multi-omics category, the most commonly employed integration. (1) The user informs the descriptors, categorical approaches included the combination between genome and terms and keywords in PlatOMICs. (2-3) Through DaVinci tool, transcriptome and genome and microbiome. the literature and the OMICs databases will be evaluated. (4) Selected omics studies and new ones are (re)analyzed and integrated by standard pipelines. (5) PlatOMICs produces the final meta-analysis multi-omics integration results in a friendly Category Number of article interface. Genome 245 The first analysis (phase I) outputted by PlatOMICs is Microbiome 241 performed by the new tool called DaVinci Literature and Transcriptome 97 Database Analysis (under submission and not publicly available) . Briefly, DaVinci is able to scan several databases, Proteome 32 such as PubMed, SRA, GEODatabase and GWAS Catalog, Metabolome 2 extracting multiple information from summary, abstracts and other meta-data information to report a syntax analysis and Multi-Omics 95 molecular panels (genes, variants, tissues, cells and drugs). Next, following the study and sample selection from the Table 1. Omic categorization of literature from Omics studies previous researches, raw data might be downloaded. The concerning skin diseases with alterations in Notch signaling analysis, including the new Omics (or multi-omics) studies, pathway. will be carried out by the same standard pipeline when 6 Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia Brandão, et al. The next step in PlatOMICs is to extract genes and variants networks, underlining both health and diseases states, using from the literature. The goal is to unravel genes/variants existing data. The presented platform for multi-omics previously established as involved with skin disorders integration constitutes a time-saving and cost-efficient characterised by alterations in Notch signaling route. In this approach that might surely guide researches in the circumstance, the gene atomization process retrieved 546 advancement of more elaborate and articulated hypothesis. genes. From obtained genes, we extracted each time the Indeed, PlatOMICs is able to refine, assemble and integrate context in which the gene was cited. In total, 465 articles and thousands of information spread around multiple 1308 gene contexts were analysed. Subsequently, four database/resource/repositories. In the future, PlatOMICs will researchers classified, independently, the gene relations as present an intuitive and automated friendly web-end interface associated or not associated with the disease. Of these, 80 with accessible tables, graphs and images. genes were excluded, and 426 genes were associated. The accumulation of scientific texts and Omics data settled in Next, PlatOMICs outputted the top 10 pathways and gene various databases may have never been correlated and ontology (GO) predicted by these genes (Table 2). analysed in conjunction. In this perspective, it is presumable Enrichment pathway and a GO analysis were conducted by that significant scientific responses may have been generated reactomePA, limma and topGO Bioconductor package. The but are still uncovered. In this critical context, PlatOMICs pathway reveals the role of interleukin (IL) signaling, mainly was developed in order to promote the analysis and driven by IL-4, IL-13 and IL-10. GO adds the defense integration of the available Omics data, and in the present response and interspecies interactions between organisms. study we applied PlatOMICs for the analysis of skin diseases Collectively, both descriptions point out that inflammation as a validation model. Our approach allowed us to further and skin microbial host defense are to be considered as key emphasize that our integrated strategy seeks to identify a outcomes from the global literature findings, suggesting that common link between skin diseases and deregulations in these pathways and GO should be included in future Omics homeostatic processes in epithelial tissues. studies. PlatOMICs also performed a gene atomization on DaVinci ACKNOWLEDGMENTS Omics. This analysis was assessed on 158 genes, most of which were found to be similar to the DaVinci Lit output. Equals enriched pathways and GO from Table 2 were found, This work was supported by a grant from the Institute for thereby ratifying the importance of these pathways and GO Maternal and Child Health IRCCS “Burlo Garofolo/Italian on multi-Omics integration. Ministry of Health” (BioHub 03/20), by the grant Interreg Italia-Slovenia, ISE-EMH 07/2019 and by CNPq 3 CONCLUSION (311415/2020-2). The scientific goal of PlatOMICs is to promote the understanding of biological mechanisms and molecular Reactome ID Pathway Description GO ID GO Description R-HSA-449147 Signaling by Interleukins GO:0034097 Response to cytokine R-HSA-6785807 Interleukin-4 and Interleukin-13 signaling GO:0019221 Cytokine-mediated signaling pathway R-HSA-6783783 Interleukin-10 signaling GO:0071345 Cellular response to cytokine stimulus R-HSA-877300 Interferon gamma signaling GO:0002376 Immune system process R-HSA-447115 Interleukin-12 family signaling GO:0006952 Defense response R-HSA-8854691 Interleukin-20 family signaling GO:0009605 Response to external stimulus R-HSA-913531 Interferon Signaling GO:0044419 Interspecies interaction between organisms R-HSA-380108 Chemokine receptors bind chemokines GO:0070887 Cellular response to chemical stimulus R-HSA-451927 Interleukin-2 family signaling GO:0010033 Response to organic substance R-HSA-1059683 Interleukin-6 signaling GO:0051707 Response to other organism Table 2: Top 10 Enriched pathway and a gene ontology of 426 genes associated with skin diseases with alterations in the Notch signaling pathway. 7 Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia Brandão, et al. REFERENCES [1] Gratton R., Tricarico P.M., Moltrasio C., Lima Estevão de Oliveira A.S., Brandão L., Marzano A.V., Zupin L., Crovella S. Pleiotropic Role of Notch Signaling in Human Skin Diseases. Int. J. Mol. Sci. (2020), 21(12):4214. doi: 10.3390/ijms21124214. [2] Siebel C. and Lendahl U. Notch Signaling in Development, Tissue Homeostaasis, and Disease. Physiol Rev. (2017), 97(4):1235-1294. doi: 10.1152/physrev.00005.2017. [3] Massi D. and Panelos J. Notch Signaling and the Developing Skin Epidermis. Adv Exp Med Biol. (2012), 727:131-41. doi: 10.1007/978-1-4614-0899-4_10. [4]https://pubmed.ncbi.nlm.nih.gov/?term=%28Hidradenitis+ Suppurativa+OR+Dowling+Degos+Disease+OR+Adams- Oliver+Syndrome+OR+Psoriasis+OR+Atopic+Dermatitis%2 9+AND+%28genome+OR+transcriptome+OR+proteome+O R+microbiome%29&filter=pubt.classicalarticle&filter=pubt. clinicalstudy&filter=pubt.clinicaltrial&filter=pubt.comment &filter=pubt.dataset&filter=pubt.journalarticle&filter=pubt.le tter&filter=pubt.observationalstudy&filter=pubt.technicalrep ort&filter=years.2016-2020, last accessed on November 18, 2020). 8 Implementing the INSIEME portal according to the patients and caregivers’ point of view Ivana Truccolo† Virginia Gerlero Flavio Rizzolio ANGOLO OdV, Italian Venetian Cluster Department of Molecular Sciences Association of Long-term Cancer virginia.gerlero@venetiancl and Nanosystems, Ca’ Foscari Survivors, President University of Venice uster.eu ivanatruccolo@gmail.com flavio.rizzolio@unive.it Vincenzo Canzonieri Anatomia Patologica, Centro di Riferimento Oncologico di Aviano (CRO) IRCCS, Department of Medical, Surgical and Health Sciences, University of Trieste vcanzonieri@cro.it Figure 1: Official logotype from the project In the health sector as well, technology can help people tracking ABSTRACT / POVZETEK and controlling their own health information and make informed decisions about their health but only at two conditions2: The 2030 Agenda for Sustainable Development highlights that the spread of information and communications technology and global interconnectedness has great potential to accelerate human progress, to bridge the digital divide and to develop - Technology must be really accessible knowledge societies1. - Health information must be really understandable, scientifically correct and aligned with the reality The project Interreg Italia Slovenia ISE-EMH aims to create a †Corresponding name health portal integrating useful information about health topics Permission to make digital or hard copies of part or all of this work for personal or and health facilities related to a specific interregional area both classroom use is granted without fee provided that copies are not made or distributed in Slovenian and Italian languages. INSIEME is the name of the for profit or commercial advantage and that copies bear this notice and the full portal. citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia© 2021 C opyright held by the owner/author(s). 1 WHO, Draft Global strategy on Digital Health 2020-2025. Available at http://bit.do/fLkpp 2 Australia’s health 2018, available at http://bit.do/fLkoA 9 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Ivana Truccolo et al. KEYWORDS - Social services: the individual municipalities of the province of Oncology, health topics, health facilities, patients, caregivers, Venice and the Region of Friuli Venezia Giulia offer social cancer survivors, digital health services to citizenship - Physiotherapy and rehabilitation services: as far as the province of Venice is concerned, the two main departments of Physical 1 Introduction Medicine and Rehabilitation of the San Giovanni e Paolo The contribution of the Ca' Foscari University, partner of the Hospital of Venice and the Angelo Hospital of Mestre have been project, was to integrate into this portal INSIEME some described. In addition, the local rehabilitation and physical categories of useful resources related to the cancer topic. The therapy services in the different health and social authorities have selection of these categories was made with the contribution of a been listed. About the Friuli Venezia Giulia Region, the two cancer survivors and caregivers association. It is a fact that very University & health agencies of Udine and Trieste have been often a patient relies on the search engines algorithms to find out described, in addition to health services of home rehabilitation the information he/she needs. But it happens that resources - Psychological services: the family counselors are available in ranked as not important by the search engines are actually very the metropolitan area of Venice and in the Friuli Venezia Giulia useful to him/her. So, the point of view of some individuals, both Region. In addition, the ANT Foundation 1978 Onlus has been cancer patients and caregivers, was the main guide to identify the described that offers psychological support at home to all resources to be listed in the portal. Not only the management of patients. the cancer disease is important to a patient but also his/her mental - Accommodation services: accommodation facilities near the condition, his/her wellness and practical things such as main hospitals have been described with the information of their accommodation and work related issues websites. Some of them are free, others offer low price 3 . There are good examples of this integration accommodation. In addition, the channels of Airbnb and 4 Booking.com have been reported; 2 Results - Information and advocacy services for the protection of patients' rights: for both areas were listed the main services A preliminary search was made to find out informational - Administrative services resources about cancer to widen the spectrum of categories and - Local health and social services - Screening: the cancer services. Then we tried to categorize the cancer services related prevention opportunities of the Friuli Region has been listed both to the area of the metropolitan city of Venice and the Region - Local social and health services - Palliative care: the Health Friuli Venezia Giulia. District of Mirano and Dolo has been reported with regard to palliative care services; In addition to basic services, the search has highlighted the - Specialized voluntary associations through the useful webtool following complementary categories: of Oncoguida 5 , the large number of voluntary associations operating in the area of Venice and Friuli can easily be identified. - Hospital services - websites with dealing with current information about cancer - Social services topics and tools to easily discover fake news were also described - Physiotherapy services - websites with patients stories or personal narrations were also - Physical activities opportunities and related opportunities indicated as they are very helpful to patients wellness. - Psychological services Last but not least the websites related to all the hospitals and - Accommodation services for patients and/or family members Health Comprehensive Centers of the Friuli Venezia Giulia - Information and counselling services about health topics and Region and Venice area were listed and described. patients' rights - Administrative services - Local social and health services – screening - Local social and health services - palliative care - Voluntary associations - Independent information on cancer - Information on fake news For each category, a minimum of 5 websites was listed if available. Below a brief summary of the research carried out through the consultation of a large number of websites. - Hospital health services: no 15 facilities were found in the province of Venice and no 24 in Friuli Venezia Giulia. 3 Truccolo I, Cipolat Mis, C, De Paoli P (eds), Insieme ai pazienti. Costruire la Stakeholder Co-Design. J Med Internet Res. 2019 Feb 11;21(2):e11371. doi: Patient Education nelle strutture sanitarie. Il Pensiero scientifico, 2016 10.2196/11371 4 Kildea J, Battista J, Cabral B, Hendren L, Herrera D, Hijal T, Joseph A. Design 5 Oncoguida, available at 5 http://www.oncoguida.it/html/home.asp and Development of a Person-Centered Patient Portal Using Participatory 10 Insert Your Title Here Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia 3 Conclusions The different categories of resources integrated into the INSIEME portal are not exhaustive of course. They just give a hint of the importance of taking into account both informational and practical resources and involving patients and caregivers in the building of digital health portals ACKNOWLEDGMENTS The paper was supported by the ISE-EMH project funded by the program: Interreg V-A Italy-Slovenia 2014-2020. We thank the patients and caregivers that help to identify the resources listed in the portal. REFERENCES [ 1 ] WHO, Draft Global strategy on Digital Health 2020-2025. Available at http://bit.do/fLkpp [2] Australia’s health 2018, available at http://bit.do/fLkoA [3] Truccolo I, Cipolat Mis, C, De Paoli P (eds), Insieme ai pazienti. Costruire la Patient Education nelle strutture sanitarie. Il Pensiero scientifico, 2016 [4] Kildea J, Battista J, Cabral B, Hendren L, Herrera D, Hijal T, Joseph A. Design and Development of a Person-Centered Patient Portal Using Participatory Stakeholder Co-Design. J Med Internet Res. 2019 Feb 11;21(2):e11371. doi: 10.2196/11371 [5] Oncoguida, available at http://www.oncoguida.it/html/home.asp Figure 2: View of the Oncology section in the INSIEME portal 11 An Analytical and Empirical Comparison of Electronic and Mobile Health Platforms Primož Kocuvan Erik Dovgan Matjaž Gams primoz.kocuvan@ijs.si erik.dovgan@ijs.si matjaz.gams@ijs.si Jožef Stefan Institute Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39 Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia Ljubljana, Slovenia ABSTRACT The rest of the paper is organized as follows: In Section 2, Electronic and mobile health (EMH) is a new way of deliver- we describe existing platforms. Section 3 presents the ISE-EMH ing health services to patients with the use of small portable platform, The results of analytical and empirical comparison devices like mobile phones or tablets. The term electronic indi- of the platforms are given in Sections 4–5. Finally, Section 6 cates that doctors and medical personnel use electronic health summarizes the paper with ideas for future work. records or electronic prescribing of medicine for patients. Some countries like Slovenia, use electronic prescriptions of medicine 2 AN OVERVIEW OF EXISTING EMH for many years now. According to World Health Organization PLATFORMS (WHO), mHealth has the ability to transform the delivery of health services all over the world and bring about a paradigm 2.1 Genoa shift in healthcare delivery processes [6]. By using technologi-Genoa is a platform that offers telepsychiatry services [5]. It is, cal innovations we can overall improve healthcare not only in for now, available only in the United States. It connects people developed countries but also in countries that are still in the (patient-doctor) through a system of video-conference technol- developing phase. In these countries, there is a lack of doctors, so ogy. Telepsychiatry is a branch of telemedicine where they only optimizing the process of delivering medicine and information provide help for psychiatric and mental disorders. to the patients is very desirable. In this paper, we describe some of the EMH online available platforms and compare them with 2.2 DigiGone the one which we developed within the ISE-EMH project. DigiGone is a packet of services, including a medical one called DigiMed [2]. Their philosophy is like that of Genoa. However, KEYWORDS they offer general medical services, not only for psychiatric prob- mHealth, eHealth, electronic and mobile health, EMH, compari- lems. For example, when applying as a patient, you get a technical son of EMH platforms, ISE-EMH kit. When you have an appointment with a doctor, your nurse or caregiver comes to your home and examines you, and via video 1 INTRODUCTION conference tells the information to a doctor. In this kit, you have From the early beginning of the web when there was only limited also an ultrasound device, so it can stream the data to the doctor information about key institutions, e.g., universities, libraries, in real-time. and organizations, available on the web and till this day where we can find practically anything including illegal organizations 2.3 Doxy.me and activities, the public need platforms or portals which will Doxy.me is a free telemedicine service, implemented as a web aggregate all useful information on one central place. While application, thus no installation is needed. It is also accessible anything can be found on the web, it is often difficult to find from various devices. All you need is a microphone and a web proper and useful information [9]. camera. All the data is encrypted and also no account is needed, To overcome the issues of disinformation especially in the in contrast to many other platforms [3]. field of medicine and products for the elderly, as part of the ISE- EMH project we implemented a unified EMH platform together 2.4 eVisit with an application for smart hardware devices. We described the application for smart devices in [9]. The EMH platform is eVisit is a telehealth service that is not free of charge. For the a central entity where the user can find key information about doctors and patients it offers virtual flexible scheduling [4]. For health and elderly, and where he can converse with other patients doctors it provides a list of medications for prescription to pa- and doctors via text- or video-based call centers, or exchange tients, which is a very convenient and helpful feature that saves information, e.g., x-ray images or photographs of the patient time. Other platform services are similar to the already described skin. We analytically and empirically compare the existing EMH ones in other telemedicine platforms. platforms with each other to learn more about the pros and cons, and to improve the ISE-EMH platform in the future. 2.5 iPath iPath is the oldest telemedicine platform from the early begin- Permission to make digital or hard copies of part or all of this work for personal ning of the development of protocols for the internet. iPath is a or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and case-based collaboration platform that is used in telemedicine the full citation on the first page. Copyrights for third-party components of this applications to share information within a distributed group of work must be honored. For all other uses, contact the owner/author(s). people. It is being used in the domains of consultation, teach- Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia © 2021 Copyright held by the owner/author(s). ing and research [8]. It is also multilingual like the ISE-EMH platform. 12 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Primož Kocuvan et al. 2.6 MedSymphony the patients can converse with each other in the public chan- The MedSymphony platform is build for telemedicine electronic nels, where they can exchange thoughts / opinions about their and mobile health, MedSymphony is directed to accelerate the diagnosis, disease or condition. Also, the platform includes a use of telemedicine as a key platform for providing health care. virtual assistant (bot) and connects it with the Rocketchat sys- The key feature is to provide better health care for millions of tem. The purpose of bot is to answer questions about medicine, patients anywhere and anytime. MedSymphony was created to partners, waiting queues, and similar. This is helpful when no qualify doctors, medical personnel, caretakers and health institu- doctor is available. There is also an advanced search mechanism, tions as well as patients with a complete electronic and mobile implemented with the versatile, fast, and efficient Elasticsearch. health technology platform. The platform includes everything The ISE-EMH platform development will result in an EMH you need to establish a video-based doctor’s office. – a completely ecosystem that will include/provide [7]: cloud-based compliant solution with integrated video conferenc- • A platform that connects products and services, i.e., the ing, online prescription ordering joined with SMS, MMS, and backbone of the ecosystem; email integration to facilitate the doctor-patient relationship, and • Integration and connection of existing products, services, automated billing for recurring and one-time fees [10]. and systems through the platform in a complete ecosys- tem. 2.7 Bodi Zdrav 4 ANALYTICAL COMPARISON Bodi Zdrav (in English "Be Healthy") is a Slovenian health-related platform [1] and it is only meant for patients in Slovenia. Its pur-The described platforms were compared with respect to a set of pose is to give information about the services and to connect selected features (see Section 4.1). The comparison was performed the patients and doctors. It only offers services that are not offi-analytically and empirically, where the results of the former cially recognized in medicine, e.g., homeopathy, bio-resonance, evaluation are presented in Section 4.2 and the results of the hypnotherapy, etc. Its main content is a search function through latter one are given in Section 5. regions in Slovenia and filtering of services from specific medical branch. 4.1 Choosing the Features for Comparison The set of features for the platform comparison consists of care- fully selected features, selected based on the state-of-the-art re- 2.8 EcoSmart search in the EMH domain. Demographic and social character- The EcoSmart project was a three-year project that included the istics were also carefully taken into account, e.g., if it is free of participation of 26 partners. It included smart cities as well as charge for using it and if it is available in more than one lan- eHealth and mHealth domains. Within the project, an electronic guage. We also included some of the other key features which and mobile health system was developed. The purpose of the are important for user experience, e.g., if it has a GUI and if it is system was to provide key information about the project partners responsive or not. The selected features are the following: and municipalities in Slovenia, as well as health domains and prototypes. It also included a smart bot for which the main task • Free: Is the EMH platform free of charge to use it? was to answer questions to users. • Graphical user interface: Does it have a GUI or is it just text-based? • Dynamic data: Can we insert new data and update the 3 THE ISE-EMH PLATFORM fields via a form? 3.1 Basic Information • Responsive: Is the web platform responsive, which means, The ISE-EMH platform is being developed within an Interreg does it automatically resize the website when viewing on Italy-Slovenia project, where the final goal is to develop a unified the different devices? telemedicine (EMH) platform for both Slovenian and Italian pub- • Virtual assistant: Does the platform offer the chance to lic and private institutions, with the aim of accelerating the coop-talk with a bot, e.g., about medicine, diagnosis, partners, eration between Italian and Slovenian stakeholders and transfer etc.? knowledge from academic field into practice. The platform in- • Use without registration: Can we use the EMH platform cludes new diagnostic approaches, advanced sensors, including without registration? devices that monitor vital signs, and also methods of Artificial • Multilingual: Is the EMH platform available in more than Intelligence (AI) that will help patients overcome anxiety, depres- one language? sion and sedate stress. By connecting various stakeholders, the • Official medicine: Does the EMH platform offer services platform also aims at overcoming the main problem of EMH that to the people from only official (recognized) medicine? is the lack of transfer of innovative services from laboratories • Call center: Does the EMH platform offer call centers (text- into practice, due to the lack of support services in terms of both based or video-based) for getting help? ICT systems and human partners, and their integration [9]. • General medicine: Does it offer services from different medicine practices or only one? 3.2 Detailed Description of the ISE-EMH 4.2 Results of the Analytical Comparison Platform The results of the analytical comparison of platforms are shown The purpose of the ISE-EMH is to connect different partners, in Table 1. Based on these results we constructed a histogram medical personnel, doctors, patients, and end-users. This is done of features, where each bar presents the percentage of included through different logic and programmatic mechanisms. The plat- features in a specific EMH platform (see Figure 1). This histogram form uses the Rocketchat text-based communication system to shows that the worst platforms with respect to the chosen fea- enable users, e.g., patients to send questions to doctors. Also, tures are Genoa and BodiZdrav. 13 An Analytical and Empirical Comparison of Electronic and Mobile Health Platforms Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia 5.2 EcoSmart User Experience The EcoSmart platform has a simple and sleek graphical user interface design. When the user first visits the page it has ten categories on the landing page. It does not provide a search tool, but it has a link to the EcoSmart bot and other bots on the page (see Figure 3). Figure 1: A histogram of platform feature percentages, where each bar represents the percentage of checkmarks from Table 1. For Genoa the cons are that it is not free, it does not have a virtual assistant, e.g., a bot that can answer your question at any Figure 3: EcoSmart Homepage. time of the day, you cannot use it without registration, it is not multilingual, and also the usage is limited to only psychiatric conditions, since they are specialized only in telepsychiatry. 5.3 ISE-EMH User Experience For BodiZdrav the cons are that it does not have a virtual as- The ISE-EMH platform has an original graphical user interface sistant, it is only available in the Slovenian language, the services design. A user visiting the page for the first time has all the com-which they offer are not from officially recognized medicine, it ponents on the landing page (see Figure 4). These components does not have call centers, and also they do not have services are a virtual assistant, services, search tool, and button for chang-from general medicine but only alternative medicine. ing the language. This is very crucial when users need to find The most versatile and useful system based on the selected specific information fast. The user interface is constructed and features is the ISE-EMH platform. designed in such a way that every person no matter the age can use the platform. 5 EMPIRICAL COMPARISON OF SIMPLY ACCESSIBLE PLATFORMS Among the evaluated platforms, the ones that are simply acces- sible, i.e., they do not require to create an account to use them, were further analysed. This analysis was empirical, from the user perspective usage, and it also included side features such as user story. Based on the simply accessible criterion, three systems were empirically analysed: Bodi Zdrav, EcoSmart and ISE-EMH. 5.1 Bodi Zdrav User Experience The Bodi Zdrav platform has a good graphical user interface design. When the user visits the page, it has only one component Figure 4: ISE-EMH Homepage. on the landing page, i.e., a search. The user can select a category and a region, which act as filters for search. As already mentioned, it does not have a virtual assistant nor call centers (see Figure 2). 6 CONCLUSION In this paper, we described and compared a set of most important EMH platforms that are available on the web. Also, we presented the developed ISE-EMH platform. We also analytically compared these platforms based on a selected set of key features. In addition, a subset of platforms was empirically compared by focusing on a user’s point of view. This analysis shows that the best-rated platform is the ISE-EMH platform. In our future work, we will test our hypothesis stating that a user can find specific information about health-related topics on the ISE-EMH platform in under 30 seconds, instead of search- ing for more than 30 minutes on other platforms or through the search engine like Google or Bing. We will conduct the experi- Figure 2: Bodi Zdrav Homepage. ment with the help of volunteers, where they will try to find 10 randomly selected services on the ISE-EMH platform and also using a general search engine. 14 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Primož Kocuvan et al. Genoa DigiGone Doxy.me eVisit iPath MedSymphony BodiZdrav EcoSmart Insieme Free ✗ ✗ ✓ ✗ ✓ ✗ ✓ ✓ ✓ User-friendly ✓ ✓ ✓ ✓ ✗ ✓ ✓ ✓ ✓ Graphical user interface ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✓ Dynamic data ✓ ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ Responsive ✓ ✓ ✓ ✓ ✗ ✓ ✓ ✓ ✓ Virtual assistant ✗ ✗ ✗ ✗ ✗ ✗ ✗ ✓ ✓ Use without registration ✗ ✗ ✗ ✗ ✓ ✗ ✓ ✓ ✓ Multilingual ✗ ✗ ✗ ✗ ✓ ✗ ✗ ✗ ✓ Official medicine ✓ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✓ Call center ✓ ✓ ✓ ✓ ✗ ✓ ✗ ✗ ✓ General medicine ✗ ✓ ✓ ✓ ✓ ✓ ✗ ✓ ✓ Table 1: Comparison between the analysed EMH platforms. ACKNOWLEDGMENTS [5] Genoa Telepsychiatry. 2021. https://genoatelepsychiatry. The paper was supported by the ISE-EMH project funded by com. the program Interreg V-A Italy-Slovenia 2014-2020. We thank [6] Varadraj P. Gurupur and Thomas T. H. Wan. 2017. Chal- students Urša Klun, Lan Sovinc and Jan Urankar for helping at lenges in implementing mhealth interventions: a technical the implementation of the ISE-EMH platform. perspective. mHealth, 3, 8. [7] Insieme: Full General Electronic and Mobile Health Plat- REFERENCES form. 2021. https://www.ita-slo.eu/en/ise-emh. [1] BodiZdrav: Free eHealth Solution and Search Engine for [8] iPath.me: Free EMH Solution. 2021. https://www.ipath- Ehealth. 2021. https://bodizdrav.net/. network.com/ipath. [2] DigiGone: Telemedicine and Other Services. 2021. https: [9] Primož Kocuvan, Matjaž Gams, and Jakob Valič. 2021. An- //www.digigone.com/home-health-care. droid application for distance monitoring of elderly pa- [3] Doxy.me: Free Telemedicine Solution. 2021. https://doxy. rameters. In Proceedings of the Information Society 2021. me/. Submitted for publication. [4] eVisit: Telemedicine Solution and Other Services. 2021. [10] MedSymphony: Advanced Mobile Health Platform. 2021. https://evisit.com/platform/overview. https://www.medsymphony.com/. 15 Android Application for Remote Monitoring of the Elderly’s Parameters Primož Kocuvan Jakob Valič Matjaž Gams primoz.kocuvan@ijs.si jakob.valic@ijs.si matjaz.gams@ijs.si Jožef Stefan Institute Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39 Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia Ljubljana, Slovenia ABSTRACT for smart devices. The platform includes new diagnostic ap- proaches, advanced sensors, including wearable devices that According to the latest predictions, the average age in Europe monitor vital signs, and sophisticated computer algorithms and in 2050 will be 49, whereas today it is only 39 years [3]. Europe, artificial intelligence methods that gain new knowledge from therefore, faces quite a significant population problem. With data. The main problem regarding the introduction of EMH re- insufficient numbers of young workers, we need technical, eco- mains the transfer of innovative services from laboratories into nomic, and political solutions to help the elderly maintain vitality practice, as there is a lack of support services in terms of both and independence. The aim of technical solutions is to delay the ICT systems and human partners and their integration. As a rule, departure of the elderly to a retirement home and to help main- researchers can not find commercial partners for even the most taining the economic stability of the country. In this paper, we excellent academic prototypes, while the prototypes are rejected describe an application that helps the elderly and relieves the due to inertia, despite the indisputable advantages of both ICT society and economy. We present the technical specifications and and knowledge. It is difficult to implement novel ICT solutions features of the Android application, which was developed as part to the elderly. The key purpose of this project is to accelerate of the project ISE-EMH (Insieme) with the collaboration of IPM the cooperation of Italian and Slovenian stakeholders in the field Digital within the project HoCare 2.0. The Android application of EMH and the transfer of knowledge, systems, and services is created and intended for usage for the pairs: one elder and one of EMH from the academic sphere to actual use. The other pur- caretaker (e.g relatives, nurses, paramedics). pose is to enable better connections between users and providers. KEYWORDS While anything can be found on web, it is often difficult to find proper information. Bearing these specifications in mind, sev- Android application, care for the elderly, elder, caretaker, fall eral applications are being included in the platform, one of them detection, easy to use, ISE-EMH, Insieme, HoCare 2.0 being the Android application presented here. 1 INTRODUCTION 1.2 Chosen Android OS and programming Due to the development and introduction of so-called MEMS language (Microelectromechanical systems) technologies in mobile devices, According to the global market share in Table 1, obtained in these became smarter in terms of tracking and perceiving the May 2021, the most used OS for smartphones is Android OS environment [1]. This is achieved by using accelerometer, GPS, with about 72% market share [2]. Therefore, Android OS was gyroscope, proximity sensor, and many other sensors. These chosen as the operating system for our application. Regarding sensors allow us to monitor the movement of a person, location the programming language, we were deciding between Kotlin and brightness of the room in which a person is located. This and Java. As Kotlin is a fairly new language, we had chosen Java. comes in handy in the research field of ambient intelligence. In Java is still a versatile and general programming language that our case, we developed an application for the elderly and their 1 runs inside Java Virtual Machine environment . caretakers which allows us to use these technologies. In this paper, we will describe the application and its technical features. The Table 1: Global market share for mobile phone’s OS paper is thus divided into two central sections. The first describes the functions that an elderly person can use and the second Operating system Market share in procent describes the functions caretaker can use. Finally, we describe the advantages and disadvantages of the developed application, Android OS 72.18% which were highlighted by the elderly in the performed focus iOS 26.96% group. Samsung 0.43% KaiOS 0.19% 1.1 Basic information about the project Unknown 0.14% Nokia Unknown 0.03% As part of the Insieme project, we implemented a unified EMH (Electronic and Mobile Health) platform together with software Permission to make digital or hard copies of part or all of this work for personal 1.3 Overview of the functions or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and Here is a list of all functions implemented. the full citation on the first page. Copyrights for third-party components of this The elder has access to these functions: work must be honored. For all other uses, contact the owner /author(s). Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia • reminders, © 2021 Copyright held by the owner/author(s). 1 The Kotlin also uses virtual environment 16 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Primož Kocuvan et al. • contacts, • SOS function, • settings, • fall detection, • pedometer, • alarm when searching for mobile phone, • alarm as a reminder to charge the battery. The caretaker has access to these functions: • overview of elder’s parameters, • history, • exact location of the elder, • sandbox settings, • enable/disable the settings for the elder. All functions for both the elder and the caretaker will be de- scribed in more details in the following sections. The application runs on two advanced mobile phones with Android OS. The general idea is that the elder needs help, support and monitor- ing of the caretaker, and the mobile phones enable the needed functionality. 2 THE INITIAL SCREEN First, user has to confirm and enable access to the services of the phone. The user has to grant application permissions of: • accessing the contacts, • accessing the location of the device, • accessing the multimedia and files, • recording and taking photography, • sending SMS messages, • making telephone calls, • audio recordings. Figure 1 presents the initial screen of the Android application. The elder and the caretaker enter their role. The selected role is set once for the application, and to change it, the application has to be installed again. On the initial screen, the user can select the language. The application is available in Italian, English, and Slovene. There is also a button on the bottom of the initial screen. By pressing it, a user can start or stop a function of searching the mobile phone by vocal call. 3 THE ELDER’S HOME VIEW AND FUNCTIONS 3.1 Sandbox Sandbox is a term that denotes the area of elder’s home, residence or a safe place defined by the elder during the initialization of Figure 1: The initial screen, shown during the first use of their profile. When the elder leaves the sandbox area, the care- the application. taker is notified via SMS message. The elder or caretaker can arbitrarily set the radius of a sandbox area from minimum of 0 meters and a maximum of 500 meters. Changing the distance by has no internet connection or GPS service turned on, the last elder is only possible when the caretaker allows it in the settings. location on the server is displayed to the caretaker. The elder can also enable the search for mobile phone function. 3.2 Battery If enabled, the mobile phone is constantly listening to its environ-The elder is alerted when the battery charge drops to 20%. In ment. In case the elder forgets the location of the mobile phone case the elder does not connect their phone to the charger, the and wants to find it, they should say the keyword "TSUNAMI" application warns them about it every 5 minutes. loudly and clearly. The mobile phone will start to vibrate and ring in order to reveal its location. 3.3 Mobile phone location and vocal search 3.4 Alarms and reminders The application is periodically sending the location of the mo- bile phone to the central server. From there, the information is The elder has an option of adding one-time or periodic alarms. transferred to caretaker’s mobile phone. In case the elder’s phone One-time alarms are designed for non-daily tasks such as visiting 17 Android Application for Remote Monitoring of the Elderly’s Parameters Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia response to the call of the first contact, it calls the second, and so on. The elder can add a maximum of three contacts to the SOS call list. The application has an implemented algorithm that detects the falls of the elder using the phone’s accelerometers. In order to do that, the elder must have a phone that has a built-in accelerometer. In case of false detection, the elder can press the "Cancel Fall" button. A SMS message is sent to the caretaker that either a fall or a false fall has occurred. 3.6 Phone book and pedometer The elder has the option of storing existing or new contacts in the app’s phone book. By arranging the contacts in the directory, priorities are assigned to the individual contacts for the SOS function. The app measures the number of steps that the elder has taken. Depending on the refresh interval, the application sends the data to the central server, from which the values can be read by the caretaker. 4 THE CARETAKER VIEW AND FUNCTIONS The motivation for the caretaker’s application is that it enables the caretaker to monitor and communicate with the elder. The elder’s application, on the other hand, has two modes of work: in case of elder’s inability to set technical functions, the elder has access to limited set of functions, such as calls, SOS button and similar. If the elder is still able to control the settings of the application, then all options are enabled. The caretaker has full control of all functions. 4.1 Sandbox In case of the caretaker, the sandbox area is denoting the home, residence, or safe place of each specific elder separately. The caretaker can arbitrarily change the radius of sandbox area from the minimum of 0 meters and the maximum of 500 meters. They do this by entering the more options extension of settings and using the slider to select the desired distance. In the menu, they can also enable access to the settings of a particular elder. 4.2 Battery At 15% charge of the elder’s battery, the application automatically sends a SMS message to the caretaker. The message contains a warning that the battery status of the elder’s mobile phone is Figure 2: The elder’s home view, after they enter their per- low. This gives the caretaker the chance to contact the elder and sonal data remind them about charging the phone. 4.3 Mobile phone location and vocal search a doctor. Periodic alarms are designed for daily tasks such as taking medications at a specific time. When the alarm goes off, The caretaker can see the last known location of the elder by the elder must confirm it. This sends a confirmation to the central pressing the "Show exact location" button. A Google map opens, server, from which the caretaker can check the status of alarms. where a blue dot indicates the last known location of the elder. If The elder’s access to the alarms can be seen on elder’s home page, the elder has internet connection and GPS location turned on, the depicted on Figure 2. last known location is also the current location of elder’s phone. The caretaker does not have the option to perform vocal search 3.5 SOS function and fall detection for their mobile phone. In the event of problems such as nausea or feeling unwell, the 4.4 Alarms and reminders elder can press the "SOS" button, which triggers a call for help by successively calling the contacts on the list. The application The caretaker sees the confirmation of specific elderly person’s calls contacts by the list order, i.e. priority. In case there is no alarms. 18 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Primož Kocuvan et al. 5 CONCLUSION We developed the Android application for the elderly and their caretakers. This article describes the features of the application. Sensors integrated into today’s smartphones and special soft- ware enable us to create applications which help the elderly live more independently. The drawbacks of the software which is now available on Google play market are: complicated usage, high price, lack of features. We designed the application bearing in mind ease of use for the elder and features that allow the care- takers to monitor the elderly anywhere anytime. Also, after the application is fully tested, it will be available for free. ACKNOWLEDGMENTS The paper was supported by the ISE-EMH project funded by the program: Interreg V-A Italy-Slovenia 2014-2020. We thank IPM Digital, which organized the focus group as part of the HoCare 2.0 project. We also thank students: Lukas Kranjc, Urša Klun and Jan Hrastnik for helping with implementation part of the project. REFERENCES [1] Abdullah Algamili, Mohd Haris, Md Khir, John Dennis, Abdelaziz Ahmed, Sami Omar, Saeed Ba Hashwan, and Muhammad Junaid. 2021. A review of actuation and sens- ing mechanisms in mems-based sensor devices. Nanoscale Research Letters, 16, (January 2021). doi: 10.1186/s11671- 021- 03481- 7. [2] 2021. Market share android vs ios and others. https : / / gs . statcounter . com / os - market - share / mobile / worldwide. (2021). [3] 2021. The european demographic deficit. https : / / www . europarl . europa . eu / sides / getDoc . do ? pubRef= - / / EP / /TEXT+ IM - PRESS + 20080414FCS26499 + 0 + DOC + XML + V0//EN. (2021). Figure 3: The caretaker’s home view 4.5 SOS function and fall detection The caretaker or elder’s relative or anyone on the elder’s SOS list receives a call for help. In case of a fall, the caretaker receives an SMS that there was a fall of the elderly. 4.6 Phone book and pedometer The caretaker does not have a phone book, but a list of the elderly they take care for. The caretaker has the option to call a specific elder by pressing their contact. For the caretaker, the application does not measure the number of steps made. However, the caretaker has the ability to review the number of steps for each elder they take care for. The caretaker can see all the key information of each elder they take care of, e.g. Figure 3. 19 Development and structural design of the frontend for unifying electronic and mobile health platform Samo Eržen Tomi Ilijaš Arctur d.o.o. Arctur d.o.o. Industrijska cesta 1a Industrijska cesta 1a Nova Gorica, Slovenia Nova Gorica, Slovenia samo.erzen@arctur.si tomi.ilijas@arctur.si ABSTRACT Eurostat on 27/03/2020 reported the survey on the use of ICT in 2 FRONTEND DESIGN households and by individuals, one in two EU citizens (53%) Anonymous visitors come to the site in order to find information aged 16-74 reported that they sought online health information about their disease or guidance on where to turn for advice. They related to injury, disease, nutrition, improving health or similar have different knowledge not only of disease causes, symptoms, [1]. On the other hand, an estimated 7 percent of Google's daily and health domain language, but also of various computer skills. searches are health-related, according to Google Health Vice Using user story mapping we isolate six stories: President David Feinberg, MD and Google's total daily health- related searches amount to 70,000 each minute, according to The 1. Browse – Visitor is knowledgeable and can navigate Telegraph report [2]. through the structured content to find information. 2. Search – Visitor can describe the main symptoms to search Nobody really enjoys going to a doctor, but there are many for possible results. reasons why you should avoid simply searching the internet for 3. Bot – Visitor can chat with the bot to narrow down the medical advice. Essentially this can go in two bad ways: either results and get the best information available. you overestimate your symptoms and end up taking the wrong 4. Chat – Visitor is having difficulties finding useful results medication or engaging in the wrong self-treatment, or you via navigation and he would like to chat with real person to underestimate your symptoms and let a condition worsen. get proper help. 5. HelpDesk – Call-center operator or doctor wants to In this paper, we describe a platform that helps an anonymous supervise many chat channels and answer messages from visitor to find verified health information or even chat with the the visitors. call-centre operator or health expert when available. We present 6. Administration – Administrator of platform wants to the technical specifications and features of the Insieme platform, manage the content and users (administrators, call-center which was developed as part of the project ISE-EMH. operators, and doctors). KEYWORDS The frontend is developed as a fully responsive webpage and is Frontend design, the structure of the frontend, health, EMH, also usable on mobile devices. medical advice, health-related searches At this point the platform supports content in three languages (Slovene, Italian, English) as this was the objective of the ISE-EMH project. 1 INTRODUCTION While Google certainly has a vast quantity of information, it 3 FRONTEND FEATURES lacks selectivity. Although it’s easy to find lists that sound like our symptoms, we don’t have the medical training to understand Application features implement the user stories described in the the other factors that go into making a medical diagnosis, like previous section. personal and family history. And neither does Google. 3.1 Browse Insieme platform content is organized by professionals and To assure to a site visitor effortless navigation through the managed by medical experts. The aim of the platform is not only platform content this was divided into two level services to publish verified content but also to promote other proven hierarchy. Visitor can select a hierarchy item on left vertical bar platforms and online content. And if the site visitor doesn’t find to get a list of all the services of the selected group each in few appropriate content, he can choose to speak with the available lines. By selecting a service from the list visitor gets detail page trained operator or medical expert. of that service. 20 Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia Samo Eržen et al. Figure 1: Platform content navigation 3.2 Search On the middle of the home page a visitor can find a search box to insert the search string. This is sent to the backend to execute the search on the main entities (services hierarchy, companies, user groups, experts) for presence of the search string in name or description. The result is presented as a list of entities grouped by type, each in few lines of description. By selecting a result item from the list, the visitor gets the detailed page of that entity. Using the Advanced search option, a visitor can filter the results using various parameters. Figure 3: Chat with an operator or expert 3.5 HelpDesk Figure 2: Search panel on the home page After logging in the user with the role of expert can select the HelpDesk button to open the chat dashboard with the list of 3.3 Bot active chat channels. When another visitor starts a new chat, a Visitor can at any moment decide to invoke bot and start a chat. channel is added to the list and the dashboard user is notified. He Bot backend tries to understand the intent of the visitor and offer can select the channel and chat with a visitor. There is also a the platform content, or some implemented functionality (e.g., possibility to use a shortcut to add a link pointing to the selected Waiting times and booking). platform content to the message he is writing. Adding more content and functionality to the bot backend, we don’t need to change the frontend to enable visitor to use them. 3.4 Chat On the home page an anonymous visitor sees active operators or experts. By selecting one chat, a pop-up window opens and he can write a message and chat with the selected person as we all do using many chat applications (e.g., WhatsApp, Skype, …). Figure 4: Operator chat dashboard 21 Development and structural design of the frontend for unifying Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia electronic and mobile health platform 3.6 Administration harmful information, people are searching the internet for health advice. Administrators have to login to gain the possibility to manage users and their properties as roles (call-center operator, expert, In this first phase of the project, we’ve learned how to organize …). Further he can define services, experts, companies, user health information and make it accessible to everyone to find groups and configure various parameters necessary for the answers to their questions or advise them on the best way to correct functioning of the platform. The content can be described proceed. The information can also be accessed by asking the bot in one or more languages to be visible to the visitors of platform or go into detailed chatting with a qualified operator or a doctor in a selected language. if available. During the next phases we will offer the visitors the possibility to register, so the operator can see his/her name and conversation history. We will improve the bot’s conversation recognition capabilities. By saving navigation and search history we can use Machine Learning to understand the visitor’s behaviour. With this understanding and adding seasonal (e.g., winter) or exceptional (e.g., Covid) health problems we can enrich search results and bot behaviour. ACKNOWLEDGMENTS The paper was supported by the ISE-EMH project funded by the program: Interreg V-A Italy-Slovenia 2014-2020. REFERENCES Figure 5: Platform content management [1] »53% of EU citizens sought health information online«, https://ec.europa.eu/eurostat/web/products-eurostat-news/-/ddn- 20200327-1: Accessed 31. August 2021. 4 CONCLUSION AND FUTURE WORK [2] »Dr Google will see you now: Search giant wants to crash in on your medical queries«, The quality and reliability of websites providing health https://www.telegraph.co.uk/technology/2019/03/10/google-sifting-one-knowledge can vary greatly. Yet despite the chances of billion-health-questions-day/: Accessed 31. August 2021. unnecessary stress and finding incorrect, or even potentially 22 primoz.kocuvan@ijs.si ABSTRACT Human population is getting older and more prone to diseases. In the information society, it is possible to find lots of helpful information on the web; however, the overload of information makes it hard to find information about a particular health issue. The ISE-EMH platform aims at providing the needed medical information faster and more context-specific than general search engines. The goal is to enable finding relevant services in few minutes instead of seeking for information with general search engine in web browser spending on average half an hour for one of common actions. To achieve such goal, the platform has to be direct and transparent and well structured, and these issues were studied as part of the project. The platform also includes an Future work is described at the end of the text, where future stakeholders are mentioned service providers and medical doctors. Their role will be significant for successful use of the platform, which can hopefully improve access to relevant information about healthcare services. 2 ISE-EMH PLATFORM Health service, patient, disease, improving healthcare system, 2.1 Concept Description ICD, ISE-EMH platform, workflow diagram, medical doctor 1 INTRODUCTION Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this work must be honored. For all other uses, contact the owner/author(s). Information Society 2021, 4 8 October 2021, Ljubljana, Slovenia 21 Copyright held by the owner/author(s). 23 Information Society 2021, 4 8 October 2021, Ljubljana, Slovenia Klemen Bele et al. 20 minutes search 2 minutes search Figure 1: Comparison between searching on ISE-EMH platform and web browser. 2.2 Concept Demonstration 3 THE ISE-EMH PLATFORM To demonstrate how the platform works and to emphasize its advantages, a simple demonstration is presented in Figure 1. We 3.1 Side Menu Layout compared time spent for a user to find an application for skin cancer detection. Firstly, the user visited ISE-EMH platform and tried to find a link to the application. That took him around 2 minutes and can be seen on the left side of Figure 1. After that we used Google search engine to find the same result. The first search engine could not find the preferred application within first cancer checking) in the search engine and preferred application was found as 38th result. That can be seen on the right side of the Figure 1. This demonstration shows the real added value of the platform. 2.3 Other Health Platforms 24 Description of Health Service Selection and Structure of ISE-EMH Information Society 2021, 4 8 October 2021, Ljubljana, Slovenia Platform 3.2 Disease Webpage Layout 4 WORKFLOW DIAGRAM 3.3 Figure 4: Workflow Diagram. International Statistical Classification of Diseases and Related Health Problems (ICD). Inside the team we agreed to first include around 75 most common diseases and add more of them later. That number 25 Information Society 2021, 4 8 October 2021, Ljubljana, Slovenia Klemen Bele et al. comes from fact that they are 15 sets of diseases included at the ISE-EMH platform and we agreed to include 5 diseases in each The paper was supported by the ISE-EMH project funded by the set [9]. program Interreg V-A Italy-Slovenia 2014-2020. We thank our When all the diseases were selected the team started to write project partners for great collaboration: ARCTUR., Institut definitions of diseases. They were formulated in few sentences , Robotina, Burlo Hospital Trieste, Polo, and include information about the cause of the disease, common University of Venezia [11].We thank for help at the symptoms, diagnostics and any special warnings about the discussions and his contribution to project. disease (whether it is contagious or needs immediate treatment). Next step was searching for any kind of relevant services on Web. For that purpose, we were mainly using Google as search engine and spend around one hour to go through each relevant service for patients. We arranged services in eight categories: clinic services, mobile phone applications, world leading experts, symptoms, diets and special products that helps diagnosing or monitoring the disease. The relevant and checked services were added to platform. Main criteria in evaluating such services were very good reviews different clinics. For example, University Clinical Centers in Ljubljana and Maribor are part of tertiary healthcare system in [4] zVEM. 2021. https://zvem.ezdrav.si/domov Slovenia and as such have the best available equipment and also [5] Mediately. 2021. https://mediately.co/si medical staff in the state [11]. [6 , Mediately. 2021. http://www.healthday.si/novice-news/2020/3/18/intervju-z- 5 CONCLUSION blaem-triglavom-mediately [7] NHS App. 2021. https://digital.nhs.uk/services/nhs-app [8] Insieme: Full General Electronic and Mobile Health Platform. 2021. https://www.ita-slo.eu/en/ise-emh 26 Usability of smart home and home automation data Devid Palčič Simon Ražman Marjan Strnad Director Ma rketing specia list Product ma na ger for Sma rt Robotina d.o.o. Robotina d.o.o. Spa ces OIC - Hrpelje 38, 8333 Kozina , OIC - Hrpelje 38, 8333 Kozina, Robotina d.o.o. Slovenija Slovenija OIC - Hrpelje 38, 8333 Kozina, devid.pa lcic@robotina .com simon.ra zma n@robotina.com Slovenija ma rja n.strnad@robotina .com ABSTRACT enabling a variety of services and strategies for assisted living and detection of behavior change. In this case HEMS and 4G Technology and electrical energy are as a matter of fact closely platform act as an enabler for third party assisted living services. interconnected. The electric revolution and adopted technologies that assure a better standard of living enabled a longer life span. The demographic prediction is that by 2050, one in four persons KEYWORDS living in Europe and Northern America could be aged 65 or over. HEMS (Home Energy Management System), Smart Home, care In 2018, for the first time in history, persons aged 65 or above for elderly, elderly, change behavior detection. ISE-EMH, outnumbered children under five years of age globally. The Insieme number of persons aged 80 years or over is projected to triple, from 143 million in 2019 to 426 million in 2050. Growing at a slower pace, world population is expected to reach 9.7 billion in 2050 and could peak at nearly 11 billion around 2100 | UN DESA | United Nations Department of Economic and Social Affairs ) 1 INTRODUCTION With the more user friendly and affordable technology, the In harmony with sustainable energy use, interconnected standard of living and the quality of life for the past 60 years was devices and solutions for energy management in homes (HEMS), rising, and it enabled to adopt many technologies from the the possibilities of using the captured data for a variety of simplest as lighting, heating to more complex as home secondary use open up. Technology of connectable devices, automation technologies and solutions. The use of this sensors, cloud data technologies and energy monitoring and technology, however, has become part of everyday life, which is energy management technologies, such as Robotina's HEMS reflected in the routine management or use of them. solution, enable real-time monitoring of energy consumption and The future demographic structure and longer life expectancy based on this, generation of events and patterns of user behavior. also bring challenges for society. With the increase in life expectancy and the lack of young people who opt for nursing professions, it will be necessary to find appropriate technological solutions that will help maintain the quality of life and assistance of the elderly. To provide an adequate general care for the elderly, remotely assist them, predict health issues and to alert the caregivers appropriate technological solutions must be in place. In this paper, we describe the way how through energy management we enable to create events based on the behavior or use of devices in homes, apartments, or spatialized caregiving rooms. These (1) events are the foundation for triggering actions for assistance or prevention and safety in the relevant behavior change. In this paper we describe the role of HEMS (Home Energy Management System) solution and 4G cloud platform in Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed Figure 2: HEMS system operation and for profit or commercial advantage and that copies bear this notice and the full functionalities scheme. citation on the first page. Copyrights for third -party components of this work must be honored. For all other uses, contact the owner/author(s). Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia © 2020 Copyright held by the owner/author(s). 1.1 Basic information about the project 27 Robotina 2021, August 2021, Kozina, Slovenia D. Palčič et al. Data sets As a part of Insieme project we developed the features for data Time of the event acquisition, safe data transfer, models of event generation and event logging. For the purpose of third-party data use we Duration of the event developed a standardized Data API on 4S platform based on Type of the device or system SMIP cloud platform API service. For a better understanding of the topic the paper describes the Additional parameters to trigger actions are: possibilities and the technology behind it. Age of the person 1.2 Home energy management system as a source Current health status of data 2 DATA AND CONNECTIVITY The HEMS system monitors all energy production systems and all energy consumers in an individual home and, in the case of a Data collecting and data transfer to the cloud are crucial for the hospital or nursing home, energy consumption at the level of an interoperability and inter-usability of data by third-party individual floor or individual room. The feature of the Energy providers of ambient assisted living application information Management System (HEMS) works locally, and at the same solutions and home care assistance solutions. At the level of the time the connection to the cloud allows it to take advantage of all HEMS (Home Energy Management System) solution, the benefits that cloud technologies allow. To comprehensively connectivity is ensured through the edge technology of the cover all business models and services of energy management Robotinas IoT Linker product, which ensures that data is solutions, Robotina has built the 4S cloud platform, which securely transmitted to the cloud xEMS platform. enables data capture and analytical data services. Event detection, event generation and consequently as the main functionality 3 EVENT GENERATION activity triggering. In case of assisted living as an example triggering a warning to caregivers. Events once generated may Events are generated out of available data sets and are time be used for other purposes. stamped. Event can be generated through result of a simple equation. Let’s take temperature as an example. Equation, which would generate variable high_temperature as (TRUE): if (t1)>38 then high_temperature. More complex events may be result of an algorithm, like counting number of bathroom light_ON events between 22 and 6’ o clock. Finally, events may be result of a complex combination of algorithms, machine learning and artificial intelligence. Events are defined/described by healthcare specialists and translated into formulas or algorithms. Artificial Intelligence is used, when patterns are not clearly known upfront or when relations are too complex. 3.1 Lighting scenarios Figure 3: Sankey diagram from HEMS energy management Lighting and its use in rooms is the simplest, but on the other application. hand the most used device or set of devices in daily life of every person. The data of lighting usage provides us a variety of information. Use at a certain time of the day, the frequency of its 1.3 Overview of available data sets use, use in an individual, specific space provides us with valuable data in generating events and patterns of behavior. the logical consequence is that it also allows the detection of anomalies and Energy monitoring and energy management provides us with changes in behavior. A clear example is the non-extinguishing of data on the use of various devices and systems. The collected lights, which can serve as scenarios of causes such as insomnia, data give us information on the use of an individual device or dementia or even death. The frequency of lighting in a particular system and the time frames for the use of the devices. Depending room, such as the frequent use of lights in toilets at night, can on the time period, the duration of an individual event and the provide us with information about the disruption of water type of device or system that was used at a given time, we can drainage, ie the detection of the onset of incontinence or prostate specifically build events and content interpretations of events problems in the case of the male population. However, early that trigger event-specific activities. detection and early diagnosis of the disease can have a radical 28 Usability of smart home and home automation data Robotina, August 2020, Kouina, Slovenia impact on further prevention of disease development. In the case reach key data that enable to build additional services in assisted of homes for the elderly, however, caregivers have the option of living and other remote care services. Data API configurations comprehensive insight and prompt action. enable devices to be accessible to third-party clients by calling SMIP Data API Web Services. Content developers may define 3.2 Heating scenarios which things and which variables may be accessible to which third-party. Write access to variables is also defined in Data API Heating and cooling are also the most common solutions in any configurations. home or building. In this case, energy management and HEMS also provide us with information on a person's health through 5. REAL LIFE SOLUTION APPLICATION monitoring the operation of heating and cooling systems. There are, of course, many scenarios. An example of excessive cooling Through energy consumption and the use of various devices, or heating may indicate a physiological change in a person or a HEMS can define various events, which, of course, depend on change in health. the type of building or living environment and the age or health condition of the person. 3.3 Energy consumption scenarios 5.1 Individual home Comprehensive energy management and monitoring through HEMS provides us with rich data to interpret the behavior of the For an individual home as we have already mentioned in the individual through monitoring energy consumption. An obvious paper, the key goal is to monitor the routine through HEMS example of such a scenario is the example of "morning coffee". energy management. Through an appropriate, long enough Hems can clearly identify patterns of behavior and deviations, period to define individual events, the system obtains data to changes, or anomalies in an individual’s routine actions or tasks detect changes in routine tasks, define current health status, through monitoring peak energy consumption. In this case, detect deterioration in health status, or early detection of “morning coffee,” which is routine for most people, can be an potential new disease at an early stage. Notifications to relatives indicator of a person’s condition. Perceived changes in the or caregivers are generated when routines change. This way they performance of routine tasks are systematically interpreted as an have the ability to check the situation and the ability to act event that triggers activities to inform relatives or caregivers. quickly. The benefits of this type of solution are for the elderly and their therapists or doctors, caregivers and, of course, relatives. 4 DATA AVAILABILITY With the growth of the elderly population, solutions such as HEMS and the Ambient Assisted Living service can indirectly help extend the autonomy of the elderly and the independence of The power of big data is in its usability. To assure 3rd party data use it is necessary to assure the data availability in a standardized the elderly, as everything takes place automatically and remotely. This is especially important for seniors living in remote locations. way through standard protocols. This ensures to other provider All involved stakeholders who directly or indirectly care for a to use the collected data and make them a part of an applicable person thus have the opportunity to be informed and to act in a solution, which provides stand-alone services based on this timely manner for the benefit of the quality of life of the elderly information or data. person ore their beloved ones. 6 CONCLUSION We have described the possibilities offered by the energy management solution (HEMS) and the 4S cloud platform in the direction of “Ambient Assisted Living” solutions. We have prepared an environment for the capture, transfer and storage of data and the creation of events that serve to implement other applications of application solutions. We examined the possibilities of using the obtained data to generate events and trigger actions. We have identified scenarios that can provide solutions to the well-being of older people and the early detection of potentially pathological changes and diseases. These solutions can be applied fast and with the least investment in the current infrastructure. Figure 4: HEMS prosumers and consumers configurator. ACKNOWLEDGMENTS 4.1 Data API The paper was supported by the ISE-EMH project funded by the program: Interreg V-A Italy-Slovenia 2014-2020. As mentioned above, Robotinas HEMS (Home Energy Management System) enables to spatialized IT companies to 29 Robotina 2021, August 2021, Kozina, Slovenia D. Palčič et al. REFERENCES DESA | United Nations Department of Economic and Social Affairs (2021) [1] 2021. Growing at a slower pace, world population is expected to reach [2] 2021. Robotina-WikiPage en:hems:hems [] (hiq-universe.com) (2021). 9.7 billion in 2050 and could peak at nearly 11 billion around 2100 | UN 30 Intelligent cognitive assistant technology for (mental) health in the ISE-EMH project Tine Kolenik† Urša Klun Primož Kocuvan Department of Intelligent Department of Intelligent Department of Intelligent Systems Systems Systems Jožef Stefan Institute Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39, 1000 Ljubljana, Jamova cesta 39, 1000 Ljubljana, Jamova cesta 39, 1000 Ljubljana, Slovenia Slovenia Slovenia tine.kolenik@ijs.si ursaklun10@gmail.com primoz.kocuvan@ijs.si Matjaž Gams Department of Intelligent Systems Jožef Stefan Institute Jamova cesta 39, 1000 Ljubljana, Slovenia matjaz.gams@ijs.si ABSTRACT Intelligent cognitive assistant technology (ICAs) has been defined as a technology powered by complex information The paper presents the inclusion of intelligent cognitive assistant processing agents. These can acquire information, put it into technology in an electronic and mobile health system under action and transmit knowledge, bringing together perception, development for the ISE-EMH project. After introducing the intelligence, thinking, calculation, reasoning, imagining and, in project and the intelligent cognitive assistant technology, the the end, conscience [1]. ICAs aspire to: understand context; be paper describes the included assistants in the developing adaptive and flexible; learn and develop; be autonomous; be electronic health system. The main focus of the paper is its communicative, collaborative and social; be interactive and emphasis on the current state of the design of an advanced personalized; be anticipatory and predictive; perceive; act; have adaptive and personalized assistant for mental health. The most internal goals and motivation; interpret; and reason. Such agents in-depth description focuses on the utilization of a generative are usually deployed either as conversational agents – computer pre-trained transformer (based on OpenAI’s GPT-3) to respond systems that converse with humans in (usually written) natural to users’ self-reports about their mood and mental health issues. language – or robots. All these characteristics make them very suitable for various functionalities inside an emHealth platform, KEYWORDS some of which will be presented in this paper, as they also play Attitude and behavior change, digital mental health, electronic a role in the ISE-EMH project. and mobile health system, intelligent cognitive assistant The ISE-EMH project encompasses a platform of emHealth technology and is co-financed by the European Regional Development Fund. As a result of a collaboration of multiple Slovenian and Italian partners, it connects cross-border healthcare systems. The project 1 INTRODUCTION consists of three main components: a mobile application, a web Electronic and mobile health system (emHealth) describes page and a chat system. The mobile application, available for OS healthcare services that are largely enriched by the use of Android, is developed specifically for elderly persons and their information and communication technology (ICT) for its caretakers. The goal is to simplify and improve quality of functionalities. This mostly includes computers and mobile seniors’ lives. This is achieved through a design that provides phones, which make healthcare more flexible and available at all simple usage and implementation of multiple integrations, such times, but can also extend to newer technologies, such as robots. as alarms for taking medicines, fall detection and an SOS call ICT is made useful in a wide range of services, from data keeping option. The web page consists of several relevant information to predictive modeling. One of the existing and still evolving about health concerns, diseases, and fields of medicine, along technologies that benefit emHealth the most is the intelligent with useful links for booking certain services, for waiting queues cognitive assistant technology. for procedures, and of presentations of other helpful platforms. Because all data are gathered in one place and easily available, it allows better access to the healthcare itself, as searching for information is intuitively designed. The chat system is Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed implemented in the web page, and is intended for all users that for profit or commercial advantage and that copies bear this notice and the full have questions concerning health problems or are simply looking citation on the first page. Copyrights for third-party components of this work must for more information about certain services. Its main goal is to be honored. For all other uses, contact the owner/author(s). Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia provide answers, give advice to users, and present information © 2020 Copyright held by the owner/author(s). about waiting queues for certain procedures and services. This is possible through the use of ICAs. 31 Information Society 2021, 4–8 October 2020, Ljubljana, Slovenia T. Kolenik et al. This paper presents the use of the ICA technology in the ISE- analyzing large amounts of data [4]) on trigram words, it EMH emHealth platform, and focuses on an adaptive and allows fast and precise searching. personalized ICA for attitude and behavior change in mental health, which is planned to be implemented into the platform. 2 INTELLIGENT COGNITIVE ASSISTANT TECHNOLOGY IN THE ISE-EMH PROJECT The ICA technology in the ISE-EMH project, similarly as other chat possibilities, is integrated in Rocket.Chat [2], a widely used open source communicating platform. The ICA technology is mostly deployed when a professional is not available to chat with the users. At that moment, the answers are provided by various ICAs. Three assistants are implemented directly in the platform, all of them with different domains they cover in terms of their semantic understanding: the JSI assistant, an assistant for waiting queues, and an assistant for service searching. They all provide their answers, but users can alternatively select only one of them. Others assistants (e.g., an assistant for information on hepatitis) are also included, but are not integrated, and they can be accessed through a link, suggested in the chat window. The three major integrated assistants are described below: Figure 1. Left: An assistant for waiting queues giving information 1. The JSI assistant [3]: An assistant developed in Bottle on oncological procedures. Right: Various possibilities from (Python). Jožef Stefan Institute (JSI) is a partner of the different ICAs when more answers are possible. project, and this ICA’s purpose is to give answers to any question related to the JSI institute and its employees. It can Another planned ICA integration is an intelligent cognitive also provide answers to more general questions. It works by assistant for attitude and behavior change in mental health. This enacting the following pipeline: 1) all stop-words are is going to be an advanced adaptive and personalized ICA, removed in given text to obtain relevant keywords; and 2) planned to be state of the art (SOTA) in the field of ICAs for lemmatization, a procedure that returns the root of a word, is mental health. Its design is described in the next section. used on these keywords to provide more efficient searching through the database. 3 DESIGN FOR AN INTELLIGENT 2. An assistant for waiting queues: An assistant developed COGNITIVE ASSISTANT FOR ATTITUDE in Django (Python), which provides information on waiting AND BEHAVIOR CHANGE IN MENTAL queues in the Slovenian healthcare system of a given health HEALTH service. The data of all the possible procedures are obtained To be able to understand how to design such an ICA, the mental from the website https://cakalnedobe.ezdrav.si/, which makes health issues in the society have to be overviewed. Stress, anxiety them non-obsolete. Based on the selected health service, and depression (SAD) are on the rise in the entire world, with urgency and the region of procedure, information on the figures in certain groups reaching 71% for stress, 12% for medical institutions that provide the service along with first anxiety disorder and 48% for depression [5]. This opens the available time slots are given. For avoiding any doors for technological and scientific interventions to help misunderstandings about what procedures the user wants, mitigate the occurring mental health pandemic. One such stop words are removed and a lemmatizator is used. The user technology is persuasive technology (PT), which tries to change then chooses among all the suggested services. If a procedure attitudes or behaviors without coercion or deception. ICAs can is not available, other solutions are suggested, such as be effective vessels for such goals., as they can communicate in searching in all the available regions. This makes searching natural language. By employing ICAs in mental health, the faster and more precise as well as easier to use, which is benefits can be numerous: they can be free of charge, available enabled through the button-based chat implementation. 24/7, and available in remote locations [6]. Furthermore, people tend to be more comfortable talking to an ICA than to a person 3. An assistant for service searching: The assistant is [7]. implemented as a service search; the services are available on To discover what SOTA in this field was, three major ICAs the ISE-EMH platform website as well. The ICA uses entered for SAD were reviewed, also because only rare review articles keywords to provide descriptions of services, including many on this topic exist [8]. An ICA by Yorita, Egerton, Oakman, Chan useful links. With the stop-words removal and the use of and Kubota [9] is based on the Belief-Desire-Intention Elasticsearch (a search engine that allows faster searching and architecture with three models: “a conversation model for acquiring state information about the individual, measuring their 32 Intelligent cognitive assistant technology in the ISE-EMH Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia project stress level, a Sense of Coherence (SOC) model for evaluating The module architecture represents the following pipeline: the individuals state of stress, and Peer Support model, which uses the SOC to select a suitable peer support type and action it” 1. Information input. Users answer quantitative [Ibid., p. 3762]. The ICA teaches users how to improve their questionnaires on their mental health, and provide a textual mental resilience to stress, which it succeeds in in the reported input on their mood, their experiences while in that mood, and experiment. Another effective ICA is called Woebot [10]. It is similar. If the user has already provided quantitative scores based on a “decision tree with suggested responses that accepts for enough time, only text is necessary. natural language inputs” [Ibid., p. 3]. It intervenes by outputting educational content, personalized messages, and scripted advice 2. The scores and text are automatically preprocessed to by collecting data on users’ emotions and identifying their errors extract the features. in thinking. In one experiment, Woebot was more successful in helping with SAD symptoms than the government-prescribed 3. A pre-trained model is used first to recognize moods, material. The last reviewed ICA is called Tess, which “reduce[s] emotions, sentiment, and other mental health markers in the self-identified symptoms of depression and anxiety” [11]. It uses text, and it uses this on conjunction with the quantitative an extensive emotion ontology to identify the emotions of its scores to forecast mental health trends of the user. This part users from the text. It uses prepared scripts to help the users, and determines various metrics: the severity of mental health collects journal data and user feedback to improve its outputs. In problems, the specificity of mental health problems, and the one experiment, Tess significantly reduced depression and short-term trend of mental health problems. anxiety symptoms as opposed to the government-approved eBook for self-help. 4. The three metrics are sent to the next part of the module. Reviewed ICAs seem to already be at least partly successful, but they do not fully exploit the possibilities PT offers (e.g., 5. A pre-written text is selected according to the metrics, attitude and behavior change theories, user modeling, adaptation, determined in the previous part. The text serves to mitigate personalization). The ICA we are designing takes that into the user’s mental health problems, and, if the forecasted trend account. In this paper, we focus on describing a module in the is negative, to try to break that trend. ICA that utilizes a generative pre-trained transformer (based on GPT-3 [12]) to formulate outputs. 6. The text is sent to be augmented with the next part of the Our ICA surpasses the SOTA by possessing a ‘theory of module. mind’. This is achieved by the ICA a user model with the data on users’ emotions, mental states, and personality, which relies on 7. The text from the previous part is enriched by a generative behavioral and cognitive sciences advances; a reinforcement pre-trained transformer based on GPT-3 with additional text. learning algorithm to learn from historical interactions between This makes responses more varied and alive for the user. the ICA and the user, thus capturing which strategies work and which do not; and ontologies on attitude and behavior change, 8. The enriched text is passed to the GPT humanizer. stress, anxiety, and depression. The focus is however on the following module and its architecture: 9. GPT humanizer return the original text for enrichment if deemed risky for the user. Module architecture: 10. GPT humanizer decides whether the added text by Generative pre-trained transformer based on GPT-3 is acceptable in terms of risk for the user. It consists of a rule- based model that rejects text with certain words, sentiment analyzer that rejects text that fails to reach a certain positive sentiment threshold, and a pre-trained model based on a dataset of risky sentences that rejects text if it is detected as risky. 11. The final text goes through some additional modifications to check that everything is in order. 12. Output of the final text. Example: 1. The user inserts text “Today I have felt very bad. I feel a Figure 2. A module for detecting and forecasting mental health lot of stress because I have a deadline at work coming, and I issues, and responding to the detected and forecasted issues by utilizing fought with my partner yesterday. I am anxious to talk to them a generative pre-trained transformer, based on GPT-3. tonight. The deadline is a bit scary because I did not do a good 33 Information Society 2021, 4–8 October 2020, Ljubljana, Slovenia T. Kolenik et al. job last time, and if it happens again, it might trigger my friends. They probably have some experience with them and depression.” can offer advice as well as another perspective on the situation. Don't ignore or repress the problem if you need 2. Features are extracted. This includes performing sentiment to seek help. You are not a burden and you do have analysis, creating n-grams, vectorizing, calculating basic worth. ” statistical features, and applying psychological and cognitive lexicons on the text. 4 CONCLUSION 3. The features from the second step are used for detecting the The paper presented an emHealth platform being developed in user’s specific mental health problems, their severity and, if the ISE-EMH project. First, the paper focused on the ICA there is historical data of the user, the short-term trend. The technology used in the project, and later it focused on one model returns abnormal levels of stress and depression, and it particular ICA, an advanced adaptive and personalized ICA for forecasts a negative trend. attitude and behavior change in mental health. The field of emHealth is still developing as advancing 4. The information on the mental health of the user is sent to technologies are being integrated into various domains of our the sentence selector. society. The project ISE-EMH represents one such use case of integrating the domain of healthcare with the ICA technology. 5. Due to the abnormal levels of stress and depression as well as a negative trend forecasting, the text, which also takes the ACKNOWLEDGMENTS / ZAHVALA user’s personality, which is more easily persuaded when The authors acknowledge the financial support from the addressing their social circle, into account, selects the text Slovenian Research Agency (research core funding No. P2-0209 “Tackling stress and depression is hard, especially since and the Young researchers’ grant) and the ISE-EMH project many people have problems with them, but it is not funded by the program Interreg V-A Italy-Slovenia impossible. Try to think outside of the current situations and 2014-2020. discuss them with your friends. They probably have some experience with them and can offer advice as well as another REFERENCES perspective on the situation. ”. [1] Oakley, J. 2018. Intelligent Cognitive Assistants (ICA). Workshop Summary. IBM Almaden Research Center. 6. The module takes the selected text and sends it for [2] Rocket.Chat – The Ultimate Communication Platform. https://rocket.chat/ [3] Asistent. http://www.projekt-asistent.si/ijs-en enrichment. [4] Elasticsearch: The Official Distributed Search & Analytics Engine. https://www.elastic.co/elasticsearch/ [5] Twenge, J. M. 2014. Time Period and Birth Cohort Differences in 7. The text is enriched with the following two sentences: Depressive Symptoms in the U.S., 1982–2013. Soc. Indic. Res. 121, 2 “Don't ignore or repress the problem if you need to seek (2014), 437-454. [6] Kolenik, T. and Gams, M. 2021. Persuasive Technology for Mental help. You are not a burden and you do have worth. ” (This Health: One Step Closer to (Mental Health Care) Equality? IEEE was an actual output of GPT-J [13]). Technology and Society Magazine. 40, 1 (2021), 80-86. [7] Lucas, G. M., Gratch, J., King, A., and Morency, L. P. 2014. It’s only a computer: virtual humans increase willingness to disclose. Comp. Human. 8. The enriched text is passed to the GPT humanizer. Behav. 37 (2014), 94-100. [8] Kolenik, T. and Gams, M. 2021. Intelligent Cognitive Assistants for Attitude and Behavior Change Support in Mental Health: State-of-the-Art 9. Does not occur. Technical Review. Electronics. 10, 11 (2021), 1250. [9] Yorita, A., Egerton, S., Oakman, J., Chan, C., and Kubota, N. 2018. A Robot Assisted Stress Management Framework. 2018 IEEE International 10. GPT humanizer extracts features from the added text. It Conference on Systems, Man, and Cybernetics (SMC), Miyazaki, Japan, does not find any risky indicators. 2018. [10] Fitzpatrick, K. K., Darcy, A., and Vierhile, M. 2017. Delivering Cognitive Behavior Therapy to Young Adults With Symptoms of Depression and 11. The final text is passed to the final stage without Anxiety Using a Fully Automated Conversational Agent (Woebot): A modifications. Randomized Controlled Trial. JMIR Ment. Health. 4, 2 (2017), e19. [11] Fulmer, R., Joerin, A., Gentile, B., Lakerink, L., and Rauws, M. 2018. Using Psychological Artificial Intelligence (Tess) to Relieve Symptoms of 12. The user receives the full text output: “Tackling stress and Depression and Anxiety. JMIR Ment. Health. 5, 4 (2018), e64. [12] Brown, Tom B. et al. Language Models are Few-Shot Learners. arXiv. depression is hard, especially since many people have 2005.14165 (2020). problems with them, but it is not impossible. Try to think [13] Wang, B. and Komatsuzaki, A. GPT-J-6B: A 6 Billion Parameter Autoregressive Language Model. https://github.com/kingoflolz/mesh- outside of the current situations and discuss them with your transformer-jax 34 Analysis of a recommendation system used for predicting medical services Gjorgji Noveski Jakob Valič gjorgi.noveski@ijs.si jakob.valic@ijs.si Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia ABSTRACT Collaborative filtering Recommendation systems are widely used in prediction of user Collaborative filtering is based on the idea that users who have preferences given a set of data. More often these preferences similar interest in the past, will be more likely to rate future items are part of the domains of the entertainment or general goods the same way. The recommendation is gathered from data from industry, striving to improve recommendations for items such as various users, but it is tailored just for the specific user that is songs, movies, electronics, etc. In this work, we give an overview doing the query. of using a recommendation system in a electronic and mobile health platform, showcasing the applicability of such a system Item-item collaborative filtering for recommending healthcare services and keywords relating to Another form of collaborative filtering is the item-item collab- user’s initial search query. orative filtering. Instead of looking at users which are similar based on their previous ratings, it looks at the items that are KEYWORDS rated by one user and it suggest a new item which has the high- Electronic and mobile health, recommendation systems, Insieme est similarity between the previously rated ones. This form of filtering performs well when the number of users is higher than 1 INTRODUCTION the numbers of items, which is a good fit for our problem. Recommendations systems have been proven to provide solid recommendations in various tasks, such as movie recommenda- Insieme tions, song recommendations, general goods recommendations The environment where the recommendation system will be ana- on online shopping platforms, etc. With their help, using online lyzed is called ISE-EMH (Insieme). It represents an EMH platform platforms and services has become more enjoyable since they which connects various medical institutions and patients. The tailor suggestions in respect to each individual user. Focusing on platform provides information about services which are obtained the healthcare domain, we can see that recommender systems from medical institutions. An example use case is a patient that are able to offer help in many aspect of a patient’s health [3]. Sim-requires information about certain illnesses, queries information ilarly, they are also used by the healthcare professionals, aiding through keywords on the platform and the platform returns the them in decision making scenarios in order to decrease the risk relevant information. of errors. An interesting approach is proposed by [2], where the The users who are using Insieme are not required to have an entirety of the recommendation system is build as a whole plat- account to use its services. Because of that, a session based recom-form. Their platform takes into consideration the user’s health mendation system is used, meaning the user’s queries (searched status and finds healthcare services which it considers would keywords) are only relevant in only one session. The user may be be of value to the user. Our work is interested in analyzing a able to search for different illnesses in different sessions, but we recommender system in conjunction with an electronic and mo-can not assume their previous searches are related to the current bile health (EMH) platform. We want to recommend relevant one. healthcare services to the users based on their search queries. Having good recommendations allows the user to find relevant 2 EXPERIMENT services in a faster and easier way. In order to carry out the analysis of the recommendation model Problems with recommendation systems for project Insieme, we decided to simulate a small set of input data due to lack of real data. First, we chose a subset of services A recommendation system has the most difficult time achieving and then we simulated some users’ choices. The simulation was good results at the beginning of its work period. This is widely done by generating use cases which simulate a typical user using known as the "cold start" problem. This problem is due to the fact the EMH platform and choosing appropriate services. This was that a system can not infer any significant information about its done because real user interaction data was unobtainable. users or items because previous information is scarce. There are several approaches one can take while designing a recommenda- tion system, such as: Input data Insieme services are organized into medical categories. For the Permission to make digital or hard copies of part or all of this work for personal purpose of the experiment, we chose the following medical cate-or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and gories: dermatology, oncology and infections. From each of those the full citation on the first page. Copyrights for third-party components of this we chose 3 to 4 services, thus obtaining 10 various medical ser-work must be honored. For all other uses, contact the owner/author(s). vices. Each service has keywords describing it. Since the services Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia © 2020 Copyright held by the owner/author(s). refer to the diseases, the keywords refer to the affected part of the body and describe some additional properties of the disease. 35 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Gjorgji Noveski and Jakob Valič Table 1: A subset of services with corresponding keywords Table 2: Users’ choices of services When choosing the subset of services, we carefully selected those filtering model. In addition, an implicit feedback model is used with various keyword intersections. The services and keywords that regards the absence of information in the interaction matrix are shown in Table 1. The ’+’ sign denotes which keywords are as negative feedback. The motive for this is that a user already associated with a service. made a conscious choice about what kind of services he needs Next, we have simulated the interactions between users and information for, so the keywords that the user didn’t search can services. The ’+’ sign denotes that the user has chosen the service. be regarded as negative interactions in the interaction matrix. The users’ choices are in Table 2. When preparing the data, we We built the recommendation model using the LightFM library bore in mind the possible reasons to some users’ choices. E. g., [1]. We compared the outcome of two various models. The first users 1, 7 and 10 might be concerned about some skin problems, model is trained on the users’ choices of the services only. The while user 11 might be concerned about the lung problems and second model is additionally trained on keywords describing the user 2 might be investigating all information about cancer avail- service. Using these two models, we obtained the suggestions for able. However, in order to make data more realistic, the majority each user. of users’ choices don’t have agenda. Results Recommendation model The top recommendations for all users are mostly the same. The Our choice for a recommendation system is LightFM [1]. The differences between the recommended services are minimal, e. g. reason for using this implementation is the ability to create tag for user 2: embeddings by supplying user and item features. In our use case, our item features represent tags that further explain the keywords • pneumonia: 0.13 (items), for e.g. : "Acne" item has the corresponding "Dermatol- • acne: 0.18 ogy" and "Skin" items associated with it. The benefit of using • psoriasis: 0.13 embeddings is that they capture semantic similarities between • dermatitis: 0.15 the keywords, which in turn will result in better inference of • skin cancer: 0.18 the model and provide an option to choose top N most similar • lung cancer: 0.18 keywords. • brain tumor: 0.13 With the help of the learned latent vectors improvement is • Lyme disease: 0.17 achieved on "cold start" scenarios. If the item features were not • tick-borne meningoencephalitis: 0.17 supplied, the model would default back to a pure collaborative • COVID-19: 0.16 36 Recommendation system analysis - project Insieme Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia This data is produced by the model with only interactions. The Digital, which organized the focus group as part of the HoCare model with item features has the same recommendations, with 2.0 project. slightly lower probability. For each user, the top 3 suggested services are acne, skin cancer and lung cancer. These predictions REFERENCES are not satisfactory, since we would like to obtain the suggestions [1] Maciej Kula. 2015. Metadata embeddings for user and item tailored to every user separately. We assume more data would be cold-start recommendations. In Proceedings of the 2nd Work- needed for training of recommendation models. shop on New Trends on Content-Based Recommender Systems co-located with 9th ACM Conference on Recommender Sys- 3 CONCLUSION tems (RecSys 2015), Vienna, Austria, September 16-20, 2015. In this work we analyzed a recommendation system that is used (CEUR Workshop Proceedings). Toine Bogers and Marijn with an EMH platform. The goal was to see if such a system is Koolen, editors. Volume 1448. CEUR-WS.org, 14–21. http: applicable on an EMH platform and offer medical service recom- //ceur-ws.org/Vol-1448/paper4.pdf. mendations to users. Because of limited amount of interaction [2] Choon-oh Lee, Minkyu Lee, Dongsoo Han, Suntae Jung, and data, the recommendation system faces difficulties in learning Jaegeol Cho. 2008. A framework for personalized healthcare meaningful representations. The system requires data which service recommendation. In HealthCom 2008-10th Interna- would be gathered during a longer period of time, in order to tional Conference on e-health Networking, Applications and give more accurate and meaningful suggestions. Services. IEEE, 90–95. [3] Thi Ngoc Trang Tran, Alexander Felfernig, Christoph Trat- ACKNOWLEDGMENTS tner, and Andreas Holzinger. 2021. Recommender systems The paper was supported by the ISE-EMH project funded by the in the healthcare domain: state-of-the-art and research is- program: Interreg V-A Italy-Sovenia 2014-2020. We thank IPM sues. Journal of Intelligent Information Systems, 57, 1, 171– 201. 37 PlatformUptake Methodology for AHA Solution Assessment Žiga Kolar Zdenko Vuk Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia ziga.kolar@ijs.si zdenko.vuk@ijs.si Erik Dovgan Matjaž Gams Jožef Stefan Institute Jožef Stefan Institute Jamova cesta 39 Jamova cesta 39 Ljubljana, Slovenia Ljubljana, Slovenia erik.dovgan@ijs.si matjaz.gams@ijs.si ABSTRACT Course for synergies, knowledge exchange and a common un- The EU PlatformUptake project aims for AHA solution assess- derstanding among all stakeholders in the Active and Healthy ment. It assesses the societal impact of the existing platforms, Ageing market [3]. creates monitoring and evaluation toolkits, collects successful The rest of the paper is organized as follows. Section 2 presents user stories and best practices, promotes interoperability, and the main objectives of the project, project methodology and plat- defines guidelines for a common evolution of such platforms forms for elderly. Clustering and taxonomies are described in within existing policy frameworks and initiatives. In this paper section 3. Finally, section 4 concludes the paper with summary we present (i) PlatformUptake methodology for AHA solution and ideas for future work. assessment and its main objectives, (ii) the results of two ways of clustering, and (iii) the results of taxonomies generation of the 2 METHODOLOGY FOR AHA SOLUTION text descriptions of the EU PlatformUptake platforms for elderly. ASSESSMENT After the data was prepared, we ran the K-means algorithm and hierarchical clustering to get the number of clusters. We also 2.1 Main objectives created a decision tree of platforms for elderly. The main objectives of the EU PlatformUptake methodology for AHA solution assessment are: KEYWORDS • To identify the critical success factors of the development, uptake, clustering, artificial intelligence, health, elder people deployment and spread of open platforms in the Active And Healthy Ageing Domain, through a sophisticated 1 INTRODUCTION tailor-made monitoring methodology. Ageing presents one of the greatest socio-economic challenges of • To develop monitoring and self-evaluation tools to sup- the twenty-first century. According to estimates more than 20% port platform providers and users self-assess their success, of Europeans will be 65 or older by 2025 [5]. Reacting to related uptake, capability gaps and evolution potentials through puzzlements of demographic shifts and ageing in general, and smart assessment and visualization tools. guaranteeing the availability of the required structure to help Eu- • To analyse existing platforms based on the created method- rope utilize the active and healthy ageing sector’s opportunities, ology, by assessing the projects and initiatives hosted by the EU has devoted a high level of resources to ICT projects in the them, their further evolution, uptake, sustainability and field of active and healthy ageing. As such a considerable num- socioeconomic benefits. ber of open source platforms for the development of innovative • To involve end-user communities and related stakehold- solutions in the AHA domain have been created [3]. ers to initiate a knowledge exchange cycle for collecting The EU PlatformUptake methodology for AHA solution as- insights on best practices and challenges of platforms’ sessment assesses the societal impact of these existing platforms, uptake, evolution and costs, etc. create monitoring and evaluation toolkits, collect successful user • To leverage the platform uptake by their user communities stories and best practices, promote interoperability and define as well as their continuous improvement and expansion, guidelines for a common evolution of such platforms within ex- by elaborating and showcasing best-practice models and isting policy frameworks and initiatives. Seeking to support the evaluation guidelines. large-scale uptake of the platforms, the project proposes the cre- • To disseminate the acquired knowledge to end-users for ating of an online information hub which provides descriptive increasing their uptake of existing platforms, and promote and support materials on all existing platforms, the organisation best practice models and identified benefits to foster future of several stakeholder events, as well as Massive Open Online developments. Permission to make digital or hard copies of part or all of this work for personal 2.2 Methodology or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and The EU PlatformUptake methodology for AHA solution assess-the full citation on the first page. Copyrights for third-party components of this ment seeks to deliver an inventory of the state of the art and anal-work must be honored. For all other uses, contact the owner/author(s). yse the use of open service platforms in the Active and Healthy Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia © 2021 Copyright held by the owner/author(s). domain, covering both open platforms – such as UnversAAL, FIWARE and partly-open/proprietary platforms developed by 38 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Žiga Kolar, et al. industry, and address the interactions between these platforms The FIWARE Foundation is the legal independent body pro- [3]. viding shared resources to help achieve the FIWARE mission by To measure the impacts of such platform and enhance their up- promoting, augmenting, protecting, and validating the FIWARE take, the project proposal presents a methodology for monitoring technologies as well as the activities of the FIWARE community, open platform development, adoption and spread across Europe, empowering its members including end users, developers and by listing key factors that determine success or hinderance in rest of stakeholders in the entire ecosystem. their uptake by the end-user groups, and also the evolution of REACH represents a solution that seeks to prevent elderly their ecosystems and stakeholder networks. citizens from loss of function and a decline of being able to per- The proposed methodology shall be employed in the project form Activities of Daily Living (ADLs) independently leading to evaluate the use of open platforms by collecting and process- ultimately to entering Long Term Care (LTC). ing data from past and currently running European projects VAALID (Accessibility and Usability Validation Framework and other initiatives that are built upon such platforms. Follow- for AAL Interaction Design Process) is a STREP project of the ing the knowledge acquisition, the methodology will elaborate 7th Marco Program for Investigation and Development of the evaluation guidelines and best practice models of integrating European Commission, included within the Strategic Objective multiple platforms, taking account of technical, organizational, ‘Accessible and Inclusive ICT’; Thematic Priority ICT-2007.7.2. financial/business and legal aspects. GIRAFF+ is a complex system which can monitor activities Following the assessment of the ecosystem and output of an in the home using a network of sensors, both in and around the extended evidence on the ecosystem, the methodology will create home as well as on the body. support materials for all involved stakeholders to promote the The purpose of the EkoSmart program is to develop a smart large-scale uptake of existing platforms and also their contin- city ecosystem with all the support mechanisms necessary for uous improvement. In concrete terms this action includes the efficient, optimized and gradual integration of individual areas creation of toolkits for Monitoring and Self-Assessment for both into a unified and coherent system of value chains. platform providers and platform users, the creation of an Online PERSONA aims at advancing the paradigm of Ambient Intel- Information Hub which showcases all information through visu- ligence through the harmonisation of Ambient Assisted Living ally appealing smart tools, and the creation and implementation (AAL) technologies and concepts for the development of sus- of a Massive Open Online Course and final project activities to tainable and affordable solutions for the social inclusion and promote synergies and knowledge exchange between the com- independent living of Senior Citizen, integrated in a common munity members [3]. semantic framework. OASIS introduces an innovative, Ontology-driven, Open Refer- 2.3 Platforms ence Architecture and Platform, which will enable and facilitate interoperability, seamless connectivity and sharing of content be- Within the development of the methodology 18 platforms for tween different services and ontologies in all application domains elderly were analysed. These platforms are: relevant to applications for the elderly and beyond. (1) ACTIVAGE The general objective of the AmIVITAL project is the devel- (2) universAAL opment of a new generation of ICT technologies and tools for (3) FIWARE the modelling, design, operation and implementation of Ambient (4) ReAAL Intelligence (AmI) devices and systems to be used for providing (5) VAALID services and personal support for independent living, wellbeing (6) GIRAFF+ and health. (7) EkoSmart REACH2020 represents a solution that seeks to prevent elderly (8) PERSONA citizens from loss of function and a decline of being able to per- (9) OASIS form Activities of Daily Living (ADLs) independently leading (10) AmIVITAL ultimately to entering Long Term Care (LTC). (11) REACH2020 The Amigo project develops open, standardized, interoperable (12) AMIGO middleware and attractive user services for the networked home (13) MPOWER environment. (14) SOPRANO/OPENAAL MPOWER defines and implements an open platform to sim- (15) INTER-IOT plify and speed up the task of developing and deploying services (16) UNCAP for persons with cognitive disabilities and elderly. (17) BeyondSilos SOPRANO designs and develops highly innovative, context- (18) INLIFE aware, smart services with natural and comfortable interfaces for older people at affordable cost, meeting requirements of users, ACTIVAGE consists of a set of Techniques, Tools and Method- family and care providers and significantly extending the time ologies for interoperability between heterogeneous IoT Platforms we can live independently in our homes when older. and an Open Framework for providing Semantic Interoperabil- In the absence of global IoT standards, the INTER-IoT results ity of IoT Platforms for AHA while addressing trustworthiness, will allow any company to design and develop new IoT devices privacy, data protection and security. or services, leveraging on the existing ecosystem, and bring get universAAL enables seamless interoperability of devices, ser- them to market quickly. vices and applications for IoT enabled smart environments. The UNCAP (“Ubiquitous iNteroperable Care for Ageing People “) platform provides the framework for communication, connec- makes use of solutions and technologies developed in previous tivity and compatibility between otherwise disparate products, research projects to develop an open, scalable and privacy-savvy services and devices. 39 PlatformUptake Methodology for AHA Solution Assessment Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia ICT infrastructure designed to help aging people live indepen- dently while maintaining and improving their lifestyle. BeyondSilos aims at further spreading ICT-enabled, joined-up health and social care for older people by developing, piloting and evaluating integrated services based on two generic pathways in a multicentric approach, making extensive use of knowledge and experience gained among early adopters of integrated eCare in Europe. INLIFE aims to prolong and support independent living for elderly with cognitive impairments, through interoperable, open, personalised and seamless ICT services that support home activ- ities, communication, health maintenance, travel, mobility and socialization, with novel, scalable and viable business models, based on feedback from large-scale, multi-country pilots. 3 CLUSTERING AND TAXONOMIES Here we present the results of two ways of clustering and the results of taxonomies generation of the text descriptions of the Figure 1: Result of k-means with four clusters. EU PlatformUptake platforms for elderly. There are 18 platforms, each of which is described by 66 features. First, the text description of the platforms was converted into numeric values, e.g. “yes” gets converted to 10, “no” to 0, “par- tial” to 5. A part of features couldn’t be directly converted into numbers – mainly features with string-type unordered values, e.g. “Any (web)”, “Windows, mobile, Symbian”, “Java”. 3.1 K-means clustering K-means clustering is a method of vector quantization, originally from signal processing, that aims to partition n observations into k clusters in which each observation belongs to the cluster with the nearest mean (cluster centers or cluster centroid), serving as a prototype of the cluster. This results in a partitioning of the data space into Voronoi cells. k-means clustering minimizes within-cluster variances (squared Euclidean distances), but not regular Euclidean distances, which would be the more difficult Weber problem: the mean optimizes squared errors, whereas only the geometric median minimizes Euclidean distances. For instance, Figure 2: Result of k-means with four clusters with labels. better Euclidean solutions can be found using k-medians and k-medoids [2]. The goal of clustering is to determine how many and which cluster groups represent the platforms best, based on the descrip- tions of EU platforms for elderly, created by the EU PlatformUp- take project in Spring 2021. In Figure 1 we can see the results of k-means with four clusters. In Figure 2 we can see the results of k-means with four clusters with labels. The platforms are divided into: • cluster 1: VAALID. • cluster 2: AmIVITAL, EKOSMART, INLIFE, OASIS, sensi- Nact, UNCAP, UNIVERSAL. • cluster 3: ActivAgeR, FIWARE, Giraff+, InterIoTL, REACH, Figure 3: Result of hierarchical clustering. SOFIA2. • cluster 4: AMIGO, BeyondSilos, PERSONA, SOPRANO. It should be noted that there are only 15 dots visible in Figure is distinct from each other cluster, and the objects within each 1 although there are 18 platforms. The reason is that 4 dots are cluster are broadly similar to each other [1]. overlapped one over another, and therefore are not seen in Figure Hierarchical clustering starts by treating each observation as 1 separately. a separate cluster. Then, it repeatedly executes the following two steps: first, identify the two clusters that are closest together, 3.2 Hierarchical clustering and second, merge the two most similar clusters. This iterative Hierarchical clustering also known as hierarchical cluster analy- process continues until all the clusters are merged together. The sis, is an algorithm that groups similar objects into groups called main output of hierarchical clustering is a dendrogram, which clusters. The endpoint is a set of clusters, where each cluster shows the hierarchical relationship between the clusters [1]. 40 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Žiga Kolar, et al. We got the best result using complete linkage (linkage deter- • All the related web servers ensure... <= 1.875 mines which distance to use between sets of observation). • Implemented data analytics analyze environmental . . . <= Figure 3 shows the dendrogram for the hierarchical clustering. 0.5 In general, closer and shorter lines present greater similarity. • Operating systems supported (including . . . <= 0.5 There are three clusters. One is green colored, one is orange In a similar way, all descriptions of the platforms can be ob- colored and the combined one is blue colored. tained from the generated tree, best differentiating between them. The only exception is the second node from the left where it is 3.3 Taxonomies not possible to distinguish between the three platforms. While the previous features lead to all three of them, the algorithm is not able to create further questions to differentiate between them, i.e. using additional features. 4 CONCLUSION AND DISCUSSION We presented the EU PlatformUptake project for AHA solution assessment, its main objectives and its 18 platforms for elderly chosen because they allow elderly to live more healthy and more independently. We studied similarities and differences between the 18 platforms, and presented the results by two ways of clus- Figure 4: A decision tree of platforms for elderly. tering, and the taxonomies generated from the text descriptions of the EU PlatformUptake platforms. We conclude that the plat- forms can be clustered into similar categories and that an effective According to Wikipedia [4], taxonomy is “the practice and decision tree taxonomy can be created for the platforms. Cluster-science of categorization or classification based on discrete sets.” ing and structuring taxonomies for elderly in the proposed way It is a hierarchical classification, in which things are organized enables an integrated understanding of the field of EU platforms into groups or types. Many taxonomies are hierarchies in the for elderly. form of a tree structure, but not all are. Creating taxonomies In future work, deeper analysis of the clustering is required. often corresponds to machine learning (ML) of a decision tree With k-means it is possible to get better and more clear results from input, where each leaf in a decision tree corresponds to a with 6 or more clusters. It is feasible that hierarchical clustering specific object, e.g. a specific plant species, or in our case – a could also yield more clusters which would be better for deeper taxonomy description. The input for the taxonomies generation analysis. in this section is the same as for the all other approaches in this text, e.g. clustering. There are Initially 61 features. Most of these ACKNOWLEDGMENTS features have values that can be easily converted into numeric values, e.g. “yes” gets converted to 10, “no” to 0, “partial” to 5. This study received funding from the European Union’s Hori- A part of features couldn’t be directly converted into numbers zon 2020 research and innovation programme under the grant – mainly features with string-type unordered values, e.g. “Any agreement number 875452. The authors acknowledge the finan- (web)”, “Windows, mobile, Symbian”, “Java”. These features have cial support from the Slovenian Research Agency (research core been broken (hot encoding) into a bigger number of new features, funding No. P2-0209). e.g. “is web”, “is Windows”, “is mobile”, “is Symbian”, “is Java”. After the transformations, there are 66 features. REFERENCES Figure 4 represents the generated taxonomy on the platforms [1] 2021. Hierarchical clustering. https://www.displayr.com/ analyzed in the EU PlatformUptake project. It is a decision tree what-is-hierarchical-clustering/. (2021). where most of the leafs correspond to only one platform. Starting [2] 2021. K-means clustering. https://en.wikipedia.org/wiki/K- from the root (the top of the decision tree), it contains all the means_clustering. (2021). platforms, hence all 1s in the “value” field. The set of all platform [3] 2021. Platform uptake. https://project.platformuptake.eu/ descriptions gets split into the left and the right node based on the about/. (2021). feature/question “All the related web servers ensure maintenance [4] 2021. Taxonomy. https://en.wikipedia.org/wiki/Taxonomy. and corrections against the main known weaknesses <= 1.875?”. (2021). According to the question that was found the most relevant for [5] 2021. The european demographic deficit. https://www. this decision tree / taxonomy, the set of all taxonomies splits into europarl . europa . eu / sides / getDoc . do ? pubRef= - / / EP / two: (1, 0, 0, 1, 1, 0, 0, 0, 0, 0, 1, 1, 0, 0, 1, 0, 1, 0, 0) in the left /TEXT+IM-PRESS+20080414FCS26499+0+DOC+XML+ subtree and (0, 1, 1, 0, 0, 1, 1, 1, 1, 1, 0, 0, 1, 1, 0, 1, 0, 1, 1) in the V0//EN. (2021). right subtree. The procedure repeats until ideally there is just one platform left, i.e. all the other platforms do not correspond to the set of questions except the one. For example, the yellow leaf corresponds/represents to the “AmIVITAL” platform. It’s parent node splits platforms looking at the feature named “Operating systems supported (including mobile) – Java OSGi”. If the value is <= 5, it continues the graph over its left arrow; if the value is > 5, it continues the graph over its right arrow, in this case for a leaf “AmIVITAL”. Therefore, the features (questions) leading to the yellow node, i.e. the AmIVITAL platform, are: 41 What-If Analysis of Countermeasures Against COVID-19 in November 2020 in Slovenia Vito Janko Nina Reščič Tea Tušar Jožef Stefan Institute Jožef Stefan Institute Jožef Stefan Institute Department of Intelligent Systems Department of Intelligent Systems Department of Intelligent Systems Ljubljana, Slovenia Ljubljana, Slovenia Ljubljana, Slovenia vito.janko@ijs.si nina.rescic@ijs.si tea.tusar@ijs.si Mitja Luštrek Matjaž Gams Jožef Stefan Institute Jožef Stefan Institute Department of Intelligent Systems Department of Intelligent Systems Ljubljana, Slovenia Ljubljana, Slovenia mitja.lustrek@ijs.com matjaz.gams@ijs.com ABSTRACT In the study reported here we used the improved version, specialized for Slovenia, to objectively answer a few what-if Choosing best sets of countermeasures against COVID-19 is a questions, such as whether school closure and mask usage were difficult task, and it is often not clear whether the the counter- justified at a particular point in time, or were they an unnecessary measures that were actually chosen were justified. In this paper burden. we studied if the introduction of masks and school opening in These questions were posed to us by the Slovenian Ministry of the times of exponential growth in November 2020 in Slovenia Health. Namely, in August 2021 we sent the our XPRIZE system to were justified or not. all EU Ministries of Health with the motivation to help decision- KEYWORDS makers better select NPIs. No ministry was able or eager to use the system itself so far, but we got some replies and requests for COVID-19, epidemiological models, multi-objective optimization, particular studies, such as the one tackled in this paper. non-pharmaceutical interventions 1 INTRODUCTION 2 DATASET Our system was trained on the data from 235 world regions Coronavirus disease 2019 (COVID-19) is an infectious disease between the dates of March 1, 2020 and April 14, 2021. While that has rapidly spread across the world. Due to its high mor- taking data from Slovenia only might result in a more localized tality rate [4], most countries deemed it too disruptive to let it model, this data does not provide the necessary range of imple-run unchecked and have thus implemented countermeasures mented NPIs and their combinations. against it. The main type of countermeasures, in particular in The main source of data was the "COVID-19 Government re- the times when the vaccines were not yet available, were the sponse tracker" database, collected by the Blavatnik School of non-pharmaceutical interventions (NPI) that include lockdowns, Government at Oxford University [2], that defines which (and closure of schools and workplaces, and required mask usage. in what time interval) NPIs were implemented in each country. Due to the lack of precedent in the recent history, and several The NPIs in this database are listed in Table 1. The database also variables that influence the effect in a particular country, e.g. provides the strictness of the implementations in the form of weather and cultural circumstances, it was and still is hard for numbers, e.g., "Workplace closing – 1" represents that govern-decision-makers and domain experts to determine which NPIs to ment only suggests closure, while "Workplace closing – 3" strictly implement in a given epidemiological situation and what effect demands it. The detailed description for each level of strictness would a particular combination of NPIs have. is provided by the database authors [2]. As we are now in the second year of the pandemic, large Other key data needed for training the system are the numbers databases of data regarding the spread of the virus and imple- of infections and deaths, obtained from the same database. In mented NPIs aimed at stopping it, became available. This in turn addition we used data on weather, mobility, hospitalizations, allows for the use of artificial intelligence (AI) methods to analyze vaccination and 93 features based on country characteristics (e.g., the data, create predictive models, and consequently help the culture, development) from our previous work [3]. decision-makers in their task. It also enables reevaluation of the influence of particular NPIs at a particular time. 3 METHODS In our previous work [5] we built such an AI system, as part of The results in this work were made using an upgraded version the XPRIZE: Pandemic Response Challenge. At that competition, of our XPRIZE system that can predict the number of infections our system achieved second best results, and was significantly given the active NPIs, and propose best NPIs to counter them. upgraded since that time. The whole system is thoroughly described in our previous Permission to make digital or hard copies of part or all of this work for personal work [5]. Here, a quick overview is provided. The system first or classroom use is granted without fee provided that copies are not made or uses historical data of all regions to create a model that predicts distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page. Copyrights for third-party components of this COVID-19 infections given a set of NPIs. A SEIR epidemiological work must be honored. For all other uses, contact the owner /author(s). model is used for this purpose, combined with a machine-learning Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia model that predicts the SEIR models’s parameters as a function © 2021 Copyright held by the owner/author(s). of NPIs. This model is used to predict the infections resulting 42 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Vito Janko, Nina Reščič, Tea Tušar, Mitja Luštrek, and Matjaž Gams Table 1: The NPIs used in our study, and the range of val- ues representing their strictness. NPI Value range C1: School closing [0-3] C2: Workplace closing [0-3] C3: Cancel public events [0-2] C4: Restrictions on gatherings [0-4] C5: Close public transport [0-2] C6: Stay at home requirements [0-3] C7: Restrictions on internal movement [0-2] C8: International travel controls [0-4] H1: Public information campaigns [0-2] H2: Testing policy [0-3] H3: Contact tracing [0-2] H6: Facial Coverings outside the home [0-4] Figure 1: A comparison of actual number of daily infec- from a sequence of NPIs (referred to as "intervention plan") – its tions, with predicted number of infections for the hypo-benefit. thetical case where masks were not used. The system also estimates the cost of each intervention plan. Calculating costs of NPIs is a complex issue that will be dis- cussed in a forthcoming paper. In brief, they consist of economic costs (due to disruption of business and similar), for which some sources are available in the literature [6, 1], and social costs (due to isolation, restriction of freedom and similar). The cost of each plan is traded off against its benefit, as stricter plans results in fewer infections, but are costlier. Finally, the system uses multi-objective optimization to find intervention plans with good trade-off between benefits and costs. The two major improvements of the system for the purpose of this paper are: 1) the added possibility to set a constraint on the maximal number of infections allowed – no plan exceeding this constraint is generated and 2) the added possibility to limit the strictness of any individual NPI. This two changes allowed us to analyze the what-if scenarios of what would happen if a certain NPI were not implemented, and what plans can we implement to have a similar number of infections, but not the undesired NPI. The plans presented in this paper were evaluated using only Figure 2: A comparison of actual number of daily infec- their economic component (estimated GDP loss [in %] for that tions, with predicted number of infections for the hypo- month), but not with the social one. While we repeated all the thetical case where schools were re-opened. experiments using social costs, the results were similar, and the social costs we used are less objective than the economic ones. greatly changes the reproduction rate and consequently leads to 4 RESULTS the exponential growth. All the predictions were made for the time interval 30. 10. 2020 – Such fast growth as was predicted in these to experiments is 30. 11. 2020, for Slovenia. That time interval was chosen because probably too pessimistic, as in reality in the case that the number of the high number of infections observed and strict counter-of infections were starting to grow so alarmingly, the population’s measures imposed. behavior would likely become more cautious – counterbalancing In the first experiment, we tested what would have happened the growth. Nonetheless, the model indicates that the school if masks were not worn in closed spaces, and all other NPIs would closure is a major contributor to regulating COVID-19, even remain the same as the actually implemented. The results are more important than the masks. Please note that scale of the 𝑦 shown in Figure 1. They indicate an increase of infections, which axis differs between Figures 1 and 2. was expected considering that we are simulating lowering the The two described experiments show that removing an NPI countermeasures during the epidemic peak, and no other NPI from the implemented intervention plan will likely result in sub- was simulated instead of masks. stantial growth, which decision-makers would not allow. There- A similar experiment was made for the hypothetical case fore, we attempted to compensate for the missing NPIs with other where schools fully re-opened for a month. The results (Figure NPIs to prevent the exponential growth. 2) show that the number of infections would grow even faster. We used multi-objective optimization to show the best plans This happens due to the exponential nature of the epidemiolo- one can make given the restriction that a certain NPI cannot be gical model, encapsulating the actual nature of the virus infection used, at least not with a strictness exceeding a given threshold. in favorable conditions. Obviously, this reduction in strictness These plans were compared based on the predicted infections 43 Feature library Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Table 2: The weekly strictness of selected intervention plans. The letters identify the plan on the Pareto front approximations in Figures 3 and 4. Strictness 0044 would indicate that the lowest strictness is used for the first two weeks, and highest for the last two ones. NPI a b c d e f g h i C1: School closing 3333 3333 3333 2222 2222 1111 1111 3333 3333 C2: Workplace closing 2222 0000 2000 1000 3000 3002 3301 0000 3000 C3: Cancel public events 2222 2222 2222 2222 2222 2222 2222 2222 2222 C4: Restrictions on gatherings 4444 4444 4442 4444 4444 4444 4444 4444 4444 C5: Close public transport 0022 2222 2222 2222 2222 2222 2222 2222 2222 C6: Stay at home requirements 2222 0010 1100 1111 1110 1111 1111 1100 3110 C7: Restrictions on internal movement 2222 0000 1000 1000 2000 2011 2210 0000 2200 C8: International travel controls 2222 4343 4443 4444 4444 4444 4444 4444 4444 H1: Public information campaigns 2222 2222 2222 2222 2222 2222 2222 2222 2222 H2: Testing policy 2222 3333 3333 3333 3333 3333 3333 3333 3333 H3: Contact tracing 1111 2222 2222 2222 2222 2222 2222 2222 2222 H6: Facial coverings outside the home 4444 4444 4444 4444 4444 4444 4444 0000 0000 Figure 3: Proposed intervention plans using different re- Figure 4: Proposed intervention plans generated using dif- strictions on the value of "School closing". They were eval-ferent restrictions on the value of "Facial covering". They uated based on the predicted number of infections and were evaluated based on the predicted number of infec- estimated GDP loss. The marked plans are explained in tions and estimated GDP loss. The marked plans are ex- Table 2. plained in Table 2. "School closing" NPI, so it is easier to replace by increasing other NPIs. and predicted GDP loss for that month. The resulting Pareto- front approximations for school closure with different levels of 5 DISCUSSION AND CONCLUSIONS strictness are shown in Figure 3. In this study we analyzed what-if scenarios in regards to reduced The blue line in Figure 3 represents the case with no limit-wearing of masks and school re-openings for Slovenia at peak ations when constructing an intervention plan, and obviously infection time in November 2020. The study showed that both of these solutions are substantially better than the intervention plan these changes would worsen the epidemiological situation in the actually implemented, and the plans with limitations. The orange country if no other NPI was introduced instead. Furthermore, for and green lines represent plans that have schools partially or fully school closure the AI model could not find proper replacement in open. These plans are visibly worse in terms of the two desired that situation, suggesting that school closure was justified. The objectives: this happens because the system is compensating for closest viable solution was "Solution e" that proposes only partial the lack of "School closing" NPI with "Workplace closing" NPI, school closing, but compensates it with increased testing and which is more expensive. A sample of the generated plans is international travel control. On the other hand, the model indic- given in Table 2. ated that mask usage could be almost completely compensated This experiment was repeated, this time restricting the strict- with an increase of other NPIs. It cannot judge whether this is ness of the "Facial Coverings" NPI. The results are shown in desirable – that may depend on social costs. Figure 4 and sample plans are given in Table 2. The study has a number of limitations: For the mask analysis, the system found comparable solutions that compensate for the reduced "Facial Coverings" NPI. While (1) The study was done using historic data for Slovenia, while this NPI has a good ratio between benefit and economic cost, the AI system was trained on data from all regions and its benefit in absolute terms is nevertheless smaller than of the was only somewhat tuned to Slovenia. 44 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Vito Janko, Nina Reščič, Tea Tušar, Mitja Luštrek, and Matjaž Gams (2) The data and the resulting model do not contain the in- Acknowledgement formation on vaccination, as it was not available in the First, we would like to acknowledge all other authors who con- tested period. tributed to the development of the presented AI system: Erik (3) The data and the resulting model do not contain the in- Dovgan, David Susič, Carlo De Masi, Tea Tušar, Matej Cigale, formation on the Delta or newer variants, as it was not Matej Marinko, Aljoša Vodopija, Anton Gradišek. available in the tested period. Second, we want to acknowledge XPRIZE, the organizer of (4) The model does not predict what would happen with a the $500K Pandemic Response Challenge competition and their different implementation of the NPI (e.g., stricter testing sponsor Cognizant, who made the development of the system of students/teachers). possible. (5) The study uses costs available from the literature and Finally, we acknowledge the financial support from the Slove- might not fit best Slovenian specifics. nian Research Agency (research core funding No. P2-0209). (6) The study does not use social costs, which are certainly important but difficult to set in a justifiable manner. REFERENCES Because of these limitations, it is not recommended that this [1] 2021. Business closures and partial reopenings due to covid- study be used as a basis for future policies. For such purpose, we 19 could cost the u.s. trillions. https://news.usc.edu/178979/ strongly recommend performing new experiments tailored to business- closures- covid- 19- pandemic- united- states- gdp- the problem we try to address. losses/. Comparing best AI-proposed measures with the actual ones by [2] Thomas Hale, Sam Webster, Anna Petherick, Toby Phil- humans reveals a well-known phenomenon that humans cannot lips and Beatriz Kira. 2020. Oxford covid-19 government on their own consider all possibilities and propose best actions. response tracker, blavatnik school of government. (April Although demonstrated only on a couple of cases here, in our 2020). https : / / www . bsg . ox . ac . uk / research / research - opinion that is a fairly general conclusion valid not only for projects/covid- 19- government- response- tracker. COVID-19 NPIs. In most cases it should still be the human’s role [3] Vito Janko, Gašper Slapničar, Erik Dovgan, Nina Reščič, to make final decisions, but humans should take advantage of AI Tine Kolenik, Martin Gjoreski, Maj Smerkol, Matjaž Gams assistance when possible. and Mitja Luštrek. 2021. Machine learning for analyzing In summary, school closing and masks in general represent non-countermeasure factors affecting early spread of covid- important NPIs, and the decision to use them in peak infection 19. International Journal of Environmental Research and Pub- cases when vaccinations are not available or sufficient, seems lic Health, 18, 13. issn: 1660-4601. doi: 10.3390/ijerph18136750. reasonable. However, unlike the school closing, the masks can be https://www.mdpi.com/1660- 4601/18/13/6750. replaced with other NPIs. Furthermore, vaccinations in particular [4] Gideon Meyerowitz-Katz and Lea Merone. 2020. A system- render NPIs less important – if no new variant of COVID-19 atic review and meta-analysis of published research data appears. on covid-19 infection-fatality rates. International Journal of Infectious Disease, 101, (December 2020), 138–148. An optional note. [5] Reščič Nina, Janko Vito, Susič David, De Masi Carlo, Vodop- ija Aljoša, Marinko Matej, Tušar Tea, Dovgan Erik, Gradišek Anton, Cigale Matej, Gams Matjaž and Luštrek Mitja. 2021. Finding efficient intervention plans against covid-19. In ETAI 2021 (exact name will be given for the camera ready version). (September 2021). [6] Md Z Sadique, Elisabeth J Adams and William J Edmunds. 2008. Estimating the costs of school closure for mitigating an influenza pandemic. BMC public health, 8, 1, 1–7. 45 Effectiveness of non-pharmaceutical interventions in handling the COVID-19 pandemic: review of related studies Janez Tomšič David Susič Matjaž Gams janez2001@gmail.com David.Susic@ijs.si Matjaz.Gams@ijs.si Jožef Stefan Institute Jožef Stefan Institute Jožef Stefan Institute Ljubljana, Slovenia Ljubljana, Slovenia Ljubljana, Slovenia ABSTRACT restrictions on gatherings, public transport closure, stay at home In this paper, we analyse 30 articles on studies that focus on requirements, restrictions on internal movement, international assessing the effectiveness of different non-pharmaceutical in- travel controls, public information campaigns, testing policy, terventions (NPIs) to flatten the pandemic curve. The articles contact tracing and facial coverings. The 12 NPIs listed were reviewed use different methods, data sources, and metrics for chosen because they are independent of each other and cover all NPI effectiveness. They also analyse different regions in different radical government interventions used worldwide. For example, time periods. Here, we rank the interventions from each article curfew is included in the stay at home requirements. using a consistent scoring system. This allows us to rank and The articles included in this review, the corresponding NPI compare the effectiveness of each NPI. data sources, the countries included, and the time frame in which We conclude that school closure, workplace closure and restric- the effectiveness of the interventions was assessed can be seen tions on gatherings are the most effective interventions. Public in Table 1. events cancellation and public information campaigns also ap- pear to have a significant impact. Stay at home requirements, 2.2 Articles not based on OxCGRT data facial coverings, restrictions on internal movement and interna- In articles on studies that obtained NPI data from other sources, tional travel controls have a moderate effect. The least effective intervention policies were checked for compatibility with Ox- NPIs across all studies were found to be public transport closure, CGRT descriptions wherever possible, using documentation from testing policy and contact tracing. the individual NPI databases or descriptions provided by the au- thors (in studies where data collection was conducted by the KEYWORDS authors themselves). Non-pharmaceutical interventions, COVID-19, SARS-CoV-2 In this review, some interventions from the article by Haug et al. [15] were merged into those specified by OxCGRT, with 1 INTRODUCTION the effectiveness of the merged NPI defined as the maximum The COVID-19 pandemic has forced governments around the effectiveness of the nonmerged ones. In this way, both small globe to implement several non-pharmaceutical interventions and mass gathering cancellations were merged into restrictions (NPIs). Researchers have studied the effectiveness of such inter- on gatherings. Border restriction, travel alert and warning were ventions to help governments make more informed decisions in merged into international travel controls. dealing with the crisis. Some interventions have been reassigned to the appropriate The aim of this review is to summarize and compare the find- OxCGRT definitions. Educate and actively communicate with ings and methods of several articles on the impact of NPIs on the public was transformed into public information campaigns, COVID -19 and to determine which interventions are best suited enhance detection system into testing policy, and national lock- to improve the epidemiological situation. down into stay at home requirements. The study by Bo et al. [3] analysed the impacts of four inter-2 METHODOLOGY vention categories, namely traffic restriction, social distancing, mandatory wearing of a face mask in public, and isolation or 2.1 Selection of articles quarantine. Most of these interventions are a combination of For this review, we searched for articles that focused on assessing several OxCGRT interventions, so their effectiveness score was the effectiveness of NPIs in dealing with the ongoing COVID -19 assigned to all the NPIs that comprise them. The score of traffic pandemic. To be included here, articles had to include compar- restriction was assigned to restrictions on internal movement and isons of at least two interventions so that each could be ranked international travel controls, and the social distancing to school from most to least effective. Important works on the spread of closing, public events cancellation, and restriction on gatherings. COVID -19 that do not include a comparison of different NPIs Mandatory face masks in public was assigned to facial coverings (e.g., Chang, S. et al. [5]) are therefore not included in this review. and isolation or quarantine to stay at home orders. Effectiveness of the following NPIs from the Oxford COVID- Banholzer et al. [2] estimated the effectiveness of seven NPIs. 19 Government Response Tracker (OxCGRT) [14] was assessed: They treated bans on small and large gatherings as two seper-school closure, workplace closure, public events cancellation, ate interventions, so they were merged here into restrictions on gatherings. Venue closure and work-from-home order were Permission to make digital or hard copies of part or all of this work for personal merged into workplace closure based on the fact that the authors or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and described venue closure as "closure of some or all non-essential the full citation on the first page. Copyrights for third-party components of this businesses". Their border closure intervention was treated as work must be honored. For all other uses, contact the owner/author(s). international travel controls in this review. Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia © 2021 Copyright held by the owner/author(s). In other articles that included the non-essential business clos- ing intervention it was treated as OxCGRT’s workplace closure. 46 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Janez Tomšič, David Susič, and Matjaž Gams Table 1: Articles included in this review Authors Published in NPI data source Countries included Askitas et al. [1] Nature OxCGRT 175 countries Banholzer et al. [2] Plos One Collected by the authors USA, Canada, Australia and 17 EU countries Bo et al. [3] ScienceDirect Collected by the authors 190 countries Brauner et al. [4] Science Collected by the authors 41 countries (of which 34 are European) Chaudhry et al. [6] The Lancet Collected by the authors 50 countries Chernozhukov et al. [7] ScienceDirect Covid Tracking Project USA Courtemanche et al. [8] Health Affairs John Hopkins University 3138 US counties Deb et al. [9] SSRN OxCGRT 129 countries Dreher et al. [10] ScienceDirect unclear USA Ebrahim et al. [11] JMIR Hikma Health 1320 US counties Esra et al. [12] medRxiv WHO-PHSM 26 countries and 34 US states Haug et al. [15] Nature CCCSL 56 countries, 79 territories Hunter et al. [16] medRxiv IHME 30 European countries Islam et al. [17] BMJ OxCGRT 149 countries Jalali et al. [18] medRxiv Collected by the authors 30 most populous US counties Jüni et al. [19] CMAJ Collected by the authors 144 worldwide geopolitical re- gions Koh et al. [20] ScienceDirect OxCGRT 170 countries Leffler et al. [21] AJTMH OxCGRT 200 countries Li et al. (a) [22] The Lancet OxCGRT 131 countries Li et al. (b) [23] MDPI NSF spatiotemporal center USA Liu et al. [24] BMC Medicine OxCGRT 130 countries and territories Olney et al. [26] American Journal of Epidemi- Collected by the authors USA ology Papadopoulos et al. [27] medRxiv OxCGRT 151 countries Piovani et al. [28] ScienceDirect OxCGRT 37 members of OECF Pozo-Martin et al. [29] Springer Link OxCGRT and WHO-PHSM 37 members of OECD Sharma et al. [30] medRxiv Collected by the authors 114 subnational areas in 7 Eu- ropean countries Stokes et al. [32] medRxiv OxCGRT 130 countries Wibbens et al. [33] Plos One OxCGRT 40 countries and US states Wong et al. [34] Journal of Infection OxCGRT 139 countries Zhang et al. [35] MDPI NY Times and CNN USA National lockdown was mapped to stay at home requirements. graded its effectiveness as 2. International travel controls NPI Some studies measured the effectiveness of bans on small and was less effective. Restrictions on internal movement and public mass gatherings separately. Their results were combined into a transport closures had a negligible impact. single intervention - restrictions on gatherings. In the study by Liu et al. [24], the effectiveness of NPIs was measured in two scenarios: maximum effort (i.e., the NPIs are at 2.3 Ranking effectiveness of NPIs their maximum intensity) and any effort (i.e., the NPIs are active at any intensity). They described interventions as either strong, Different studies estimated the individual impacts of implement- moderate, or weak in each of the scenarios. Their results were ing interventions in different ways. We used a simplified ranking adapted to the simplified ranking system by assigning a value of system by assigning values between 1 and 4 to the NPIs, with 1 1 to NPIs strong in either both scenarios or in any effort scenario. representing the most effective and 4 the least effective interven- NPIs that are strong only at maximum effort were graded 2, tions. Several interventions from individual studies could have moderate NPIs were ranked as 3 regardless of the scenario and been assigned the same value. weak NPIs were ranked as 4. In the articles where the authors have already quantitatively In the article written by Li et al. (a) [22], the impact of NPIs was estimated the impacts of individual NPIs, their results have simply estimated 7 days, 14 days and 28 days after its implementation as been converted to new values, as described above. the ratio between the reproduction number (R) at a given time Askitas et al. [1] graded interventions only descriptively. They point (after 7, 14 or 28 days) and the initial R. We simplified this found that the most effective interventions in reducing the spread by calculating the average between all three ratios for each NPI of COVID-19 were restrictions on gatherings, public events can- and ranking them with values between 1 to 4. cellation, school closure and workplace closure, so they were Wibbens et al. [33] estimated the effectiveness of 11 NPIs at assigned a value of 1. Stay at home requirements were estimated different levels of intensity. They were first ranked at their highest to have a smaller effect after a longer period of time and we 47 NPI effectiveness review Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia intensity and then at an intermediate intensity. Our metric with by COVID-19 and 5 articles [7][9][18][23][27] analysed the im-values between 1 and 4 was then applied based on the average of pact on both cases and deaths. high and intermediate intensity rankings. 3.3 NPI effectiveness estimations 2.4 Comparison with a similar article Some intervention policies were included in more studies than While looking for studies to include in this review we found a others. The ones that were included in most of the studies re- similar article by Mendez-Brito et al. [25]. Their review included viewed are school closure (conclusively assessed in 22 studies), 34 articles on the subject of NPI effectiveness. Here, some of these workplace closure (in 22 studies), restrictions on gatherings (in articles were excluded, most of them because they only found 18 studies), and stay at home requirements (in 20 studies). The conclusive evidence for the effectiveness of a single intervention distribution of estimated effectiveness of these NPIs across in- policy. Furthermore, the study conducted by Flaxman et al. [13] is cluded articles is shown in Figure 1. Contact tracing was only also not included in this review because Soltesz et al. [31] proved analysed in two articles [33][24] and testing policy in four articles that the model used was too sensitive to subtle and realistic [15][24][29][33]. alterations in parameter values. Two additional articles were included in this review [1][30]. However, all other studies included in our paper were already reviewed in the review by Mendez-Brito et al. [25]. Despite that, there are a few differences between our findings and theirs. Slight differences occur in grading effectiveness of NPIs from different studies, most likely because many studies did not present a rank- ing system and so our interpretation of their results might be different than that of Mendez-Brito et al. There are some articles that do not compare impacts of dif- ferent intervention policies, but rather only qualitatively mark them as effective in reducing the COVID-19 incidence. The NPIs assessed in those studies were all graded with the same grade by Mendez-Brito et al. They treated those interventions as the Figure 1: Distribution of effectiveness values for the most least effective according to their scale. Since such NPIs were only frequently represented NPIs (lower values stands for bet-found to be effective but it was not determined how effective ter effectiveness they were, we could not confidently grade them as the most effec- tive interventions, but their impact likewise could not be treated The impacts of NPIs in the included studies are shown in Fig- as negligible. That is why they are marked here as moderately ure 3. Blank cells indicate that the intervention was either not effective, i.e., they are graded with a 2. included in the corresponding study or that the estimates of its effectiveness were inconclusive. Cells are also color coded ac- 3 RESULTS cording to the strength of the interventions in the corresponding articles. A darker shade indicates that the NPI is more effective. 3.1 Assessed regions and time frames The average effectiveness, median effectiveness and the most The reviewed studies assessed the impact of COVID-19 inter- frequent value were calculated for each NPI across all articles vention policies in different time frames. Most studies only anal- and are presented in Figure 2 along with inormation on how ysed the first epidemiological wave. Only Sharma et al. [30] many articles conclusively assessed each NPI and the standard focused on estimating effectiveness of NPIs during the second deviation for each NPI. wave. Wibbens et al. [33] analysed a longer time frame - from March to November 2020. Pozo-Martin et al. [29] analysed both 4 CONCLUSION waves independently, but the estimates of NPI effectiveness dur- Studies estimating the effectiveness of COVID-19 interventions ing the second wave were not as statistically significant. Zhang used different approaches. They analysed data from different et al. [35] also analysed a longer period including both waves. regions and in different time frames. They did not use the same The majority of studies (19 out of 30) analysed only country- models to analyse the data or the same metrics to determine the level data. Two studies [29][28] used data only on countries that effects different interventions had on the pandemic. Nonetheless, are members of the Organisation for Economic Co-operation and their findings were often agreed with each other. Development (OECD). The works [11][26][8][23][18][35][7][10] The most effective interventions across the reviewed studies analysed impacts of NPIs in the United States, either state-level were school closure, workplace closure and restrictions on gather- or county-level. The only more detailed analysis for Europe was ings. School closure was estimated to be among the most effective done by Sharma et al. [30], who analysed 114 subnational areas intervention policies in 9 studies (41% of studies in which it was in 7 European countries. assessed), workplace closure in 5 (23%) studies, and restrictions on gatherings in 8 (44%) studies. Public events cancellation was 3.2 Metrics used in studies also found to be one of the most effective measures in 3 (33%) out The studies in this review used different metrics to evaluate the of 9 studies. Interestingly, public information campaigns appear impacts of the interventions. 10 studies analysed how individual to be as effective as cancellation of public events. NPIs affect the reproduction number [3][4][10][12][15][20][22] Stay at home requirements was also estimated to have a con- [24][26][30]. 8 studies estimated the effectiveness of NPIs based siderable impact, but the interventions mentioned above were on case incidence or case growth rates [1][2][8][11][19][29][33] found to be even more effective. This could be because the stay [35]. 4 articles [6][21][28][32] focused only on mortality caused at home requirements NPI was often implemented as a last resort 48 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Janez Tomšič, David Susič, and Matjaž Gams Figure 2: Averaged effectiveness, median effectiveness and most frequent value (Mode) of each NPI across studies (lower value stands for better effectiveness) along with the number of studies that assessed each NPI (Counts) and standard deviations (Std dev). in addition to many other interventions and its isolated effect analysis measuring the impact of government actions, might only be what it adds on top of those. country preparedness and socioeconomic factors on covid- Facial coverings, restrictions on internal movement and inter- 19 mortality and related health outcomes. EClinicalMedicine, national travel controls were generally associated with having a 25, 100464. moderate effect on flattening the epidemic curve. The least effec- [7] Victor Chernozhukov, Hiroyuki Kasahara, and Paul Schrimpf. tive NPIs were consistently found to be public transport closure, 2021. Causal impact of masks, policies, behavior on early testing policy and contact tracing. covid-19 pandemic in the us. Journal of econometrics, 220, 1, 23–62. REFERENCES [8] Charles Courtemanche, Joseph Garuccio, Anh Le, Joshua [1] Nikolaos Askitas, Konstantinos Tatsiramos, and Bertrand Pinkston, and Aaron Yelowitz. 2020. Strong social distanc- Verheyden. 2021. Estimating worldwide effects of non- ing measures in the united states reduced the covid-19 pharmaceutical interventions on covid-19 incidence and growth rate: study evaluates the impact of social distanc- population mobility patterns using a multiple-event study. ing measures on the growth rate of confirmed covid-19 Scientific reports, 11, 1, 1–13. cases across the united states. Health Affairs, 39, 7, 1237– [2] Nicolas Banholzer, Eva Van Weenen, Adrian Lison, Alberto 1246. Cenedese, Arne Seeliger, Bernhard Kratzwald, Daniel Tsch- [9] Pragyan Deb, Davide Furceri, Jonathan D Ostry, and Nour ernutter, Joan Puig Salles, Pierluigi Bottrighi, Sonja Lehti- Tawk. 2020. The effect of containment measures on the nen, et al. 2021. Estimating the effects of non-pharmaceutical covid-19 pandemic. interventions on the number of new infections with covid- [10] Nickolas Dreher, Zachary Spiera, Fiona M McAuley, Lind- 19 during the first epidemic wave. PLoS one, 16, 6, e0252827. sey Kuohn, John R Durbin, Naoum Fares Marayati, Muham- [3] Yacong Bo, Cui Guo, Changqing Lin, Yiqian Zeng, Hao Bi mad Ali, Adam Y Li, Theodore C Hannah, Alex Gometz, Li, Yumiao Zhang, Md Shakhaoat Hossain, Jimmy WM et al. 2021. Policy interventions, social distancing, and sars- Chan, David W Yeung, Kin On Kwok, et al. 2021. Effec- cov-2 transmission in the united states: a retrospective tiveness of non-pharmaceutical interventions on covid-19 state-level analysis. The American journal of the medical transmission in 190 countries from 23 january to 13 april sciences, 361, 5, 575–584. 2020. International Journal of Infectious Diseases, 102, 247– [11] Senan Ebrahim, Henry Ashworth, Cray Noah, Adesh Kadambi, 253. Asmae Toumi, and Jagpreet Chhatwal. 2020. Reduction of [4] Jan M Brauner, Sören Mindermann, Mrinank Sharma, covid-19 incidence and nonpharmacologic interventions: David Johnston, John Salvatier, Tomáš Gavenčiak, Anna B analysis using a us county–level policy data set. Journal Stephenson, Gavin Leech, George Altman, Vladimir Miku- of medical Internet research, 22, 12, e24614. lik, et al. 2021. Inferring the effectiveness of government [12] Rachel T Esra, Lise Jamesion, Matthew P Fox, Daniel interventions against covid-19. Science, 371, 6531. Letswalo, Nkosinathi Ngcobo, Sithabile Mngadi, Janne [5] Serina Chang, Emma Pierson, Pang Wei Koh, Jaline Ger- Global Estill, Gesine Meyer-Rath, and Olivia Keiser. 2020. ardin, Beth Redbird, David Grusky, and Jure Leskovec. Evaluating the impact of non-pharmaceutical interven- 2021. Mobility network models of covid-19 explain in- tions for sars-cov-2 on a global scale. MedRxiv. equities and inform reopening. Nature, 589, 1, 82–85. [13] Seth Flaxman, Swapnil Mishra, Axel Gandy, H Juliette [6] Rabail Chaudhry, George Dranitsaris, Talha Mubashir, T Unwin, Thomas A Mellan, Helen Coupland, Charles Justyna Bartoszko, and Sheila Riazi. 2020. A country level Whittaker, Harrison Zhu, Tresnia Berah, Jeffrey W Eaton, 49 NPI effectiveness review Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Figure 3: Estimated effectiveness of each NPI in different studies (lower value stands for better effectiveness) 50 Information Society 2021, 4–8 October 2021, Ljubljana, Slovenia Janez Tomšič, David Susič, and Matjaž Gams et al. 2020. Estimating the effects of non-pharmaceutical [25] Alba Mendez-Brito, Charbel El Bcheraoui, and Francisco interventions on covid-19 in europe. Nature, 584, 7820, Pozo-Martin. 2021. Systematic review of empirical stud- 257–261. ies comparing the effectiveness of non-pharmaceutical [14] Thomas Hale, Noam Angrist, Rafael Goldszmidt, Beat- interventions against covid-19. Journal of Infection. riz Kira, Anna Petherick, Toby Phillips, Samuel Webster, [26] Andrew M Olney, Jesse Smith, Saunak Sen, Fridtjof Thomas, Emily Cameron-Blake, Laura Hallas, Saptarshi Majumdar, and H Juliette T Unwin. 2021. Estimating the effect of so- et al. 2021. A global panel database of pandemic policies cial distancing interventions on covid-19 in the united (oxford covid-19 government response tracker). Nature states. American Journal of Epidemiology, 190, 8, 1504– Human Behaviour, 5, 4, 529–538. 1509. [15] Nils Haug, Lukas Geyrhofer, Alessandro Londei, Elma Der- [27] Dimitris I Papadopoulos, Ivo Donkov, Konstantinos Char- vic, Amélie Desvars-Larrive, Vittorio Loreto, Beate Pinior, itopoulos, and Samuel Bishara. 2020. The impact of lock- Stefan Thurner, and Peter Klimek. 2020. Ranking the effec- down measures on covid-19: a worldwide comparison. tiveness of worldwide covid-19 government interventions. MedRxiv. Nature human behaviour, 4, 12, 1303–1312. [28] Daniele Piovani, Maria Nefeli Christodoulou, Andreas [16] Paul Raymond Hunter, Felipe Colon-Gonzalez, Julii Suzanne Hadjidemetriou, Katerina Pantavou, Paraskevi Zaza, Pan- Brainard, and Steve Rushton. 2020. Impact of non-pharmaceutical telis G Bagos, Stefanos Bonovas, and Georgios K Nikolopou- interventions against covid-19 in europe: a quasi-experimental los. 2021. Effect of early application of social distancing in- study. MedRxiv. terventions on covid-19 mortality over the first pandemic [17] Nazrul Islam, Stephen J Sharp, Gerardo Chowell, Sharmin wave: an analysis of longitudinal data from 37 countries. Shabnam, Ichiro Kawachi, Ben Lacey, Joseph M Massaro, Journal of Infection, 82, 1, 133–142. Ralph B D’Agostino, and Martin White. 2020. Physical dis- [29] Francisco Pozo-Martin, Heide Weishaar, Florin Cristea, tancing interventions and incidence of coronavirus disease Johanna Hanefeld, Thurid Bahr, Lars Schaade, and Charbel 2019: natural experiment in 149 countries. bmj, 370. El Bcheraoui. 2021. The impact of non-pharmaceutical [18] Aliea M Jalali, Sumaia G Khoury, JongWon See, Alexis M interventions on covid-19 epidemic growth in the 37 oecd Gulsvig, Brent M Peterson, Richard S Gunasekera, Gen- member states. European journal of epidemiology, 1–12. tian Buzi, Jason Wilson, and Thushara Galbadage. 2020. [30] Mrinank Sharma, Sören Mindermann, Charlie Rogers- Delayed interventions, low compliance, and health dispar- Smith, Gavin Leech, Benedict Snodin, Janvi Ahuja, Jonas B ities amplified the early spread of covid-19. MedRxiv. Sandbrink, Joshua Teperowski Monrad, George Altman, [19] Peter Jüni, Martina Rothenbühler, Pavlos Bobos, Kevin Gurpreet Dhaliwal, et al. 2021. Understanding the effec- E Thorpe, Bruno R Da Costa, David N Fisman, Arthur S tiveness of government interventions in europe’s second Slutsky, and Dionne Gesink. 2020. Impact of climate and wave of covid-19. medRxiv. public health interventions on the covid-19 pandemic: a [31] Kristian Soltesz, Fredrik Gustafsson, Toomas Timpka, Joakim prospective cohort study. Cmaj, 192, 21, E566–E573. Jaldén, Carl Jidling, Albin Heimerson, Thomas B Schön, [20] Wee Chian Koh, Lin Naing, and Justin Wong. 2020. Es- Armin Spreco, Joakim Ekberg, Örjan Dahlström, et al. 2020. timating the impact of physical distancing measures in The effect of interventions on covid-19. Nature, 588, 7839, containing covid-19: an empirical analysis. International E26–E28. Journal of Infectious Diseases, 100, 42–49. [32] Jonathan Stokes, Alex James Turner, Laura Anselmi, Mar- [21] Christopher T Leffler, Edsel Ing, Joseph D Lykins, Matthew cello Morciano, and Thomas Hone. 2020. The relative ef- C Hogan, Craig A McKeown, and Andrzej Grzybowski. fects of non-pharmaceutical interventions on early covid- 2020. Association of country-wide coronavirus mortality 19 mortality: natural experiment in 130 countries. medRxiv. with demographics, testing, lockdowns, and public wear- [33] Phebo D Wibbens, Wesley Wu-Yi Koo, and Anita M Mc- ing of masks. The American journal of tropical medicine Gahan. 2020. Which covid policies are most effective? a and hygiene, 103, 6, 2400. bayesian analysis of covid-19 by jurisdiction. PloS one, 15, [22] You Li, Harry Campbell, Durga Kulkarni, Alice Harpur, 12, e0244177. Madhurima Nundy, Xin Wang, Harish Nair, Usher Net- [34] Martin CS Wong, Junjie Huang, Jeremy Teoh, and Sunny H work for COVID, et al. 2021. The temporal association of Wong. 2020. Evaluation on different non-pharmaceutical introducing and lifting non-pharmaceutical interventions interventions during covid-19 pandemic: an analysis of with the time-varying reproduction number (r) of sars- 139 countries. The Journal of Infection, 81, 3, e70. cov-2: a modelling study across 131 countries. The Lancet [35] Xue Zhang and Mildred E Warner. 2020. Covid-19 policy Infectious Diseases, 21, 2, 193–202. differences across us states: shutdowns, reopening, and [23] Yun Li, Moming Li, Megan Rice, Haoyuan Zhang, Dexuan mask mandates. International Journal of Environmental Sha, Mei Li, Yanfang Su, and Chaowei Yang. 2021. The Research and Public Health, 17, 24, 9520. impact of policy measures on human mobility, covid-19 cases, and mortality in the us: a spatiotemporal perspec- tive. International Journal of Environmental Research and Public Health, 18, 3, 996. [24] Yang Liu, Christian Morgenstern, James Kelly, Rachel Lowe, and Mark Jit. 2021. The impact of non-pharmaceutical interventions on sars-cov-2 transmission across 130 coun- tries and territories. BMC medicine, 19, 1, 1–12. 51 Napovedovanje trendov in optimiziranje ukrepov v boju proti pandemiji COVID-19: Tekmovanje XPRIZE in naslednji koraki Forecasting trends and optimizing the intervention plans against the COVID-19 pandemic: The XPRIZE competition and beyond Mitja Luštrek, Nina Reščič, Matej Cigale, Anton Odsek za inteligentne sisteme Vito Janko, David Susič, Gradišek, Matjaž Gams Institut “Jožef Stefan” Carlo De Masi, Aljoša Ljubljana, Slovenija mitja.lustrek@ijs.si Vodopija, Matej Marinko, nina.rescic@ijs.si, Tea Tušar, Erik Dovgan, vito.janko@ijs.si POVZETEK KEYWORDS V tem preglednem prispevku predstavimo delo, ki smo ga COVID-19, epidemiological models, nonpharmaceutical sodelavci Odseka za inteligentne sisteme opravili v zadnjem letu interventions, multi-objective optimization v povezavi s pandemijo COVID-19. Raziskave modelov, ki predvidevajo prenos okužb v lokalnem okolju in na podlagi tega predlagajo ukrepe za boj proti epidemiji, so najprej potekale v 1 UVOD okviru mednarodnega tekmovanja $500k Pandemic Response Ko se danes ozremo na prve mesece leta 2020, je jasno, da je Challenge, v organizaciji fundacije XPRIZE in podjetja pandemija COVID-19 zahodni svet ujela nepripravljen. Ker je Cognizant. Na tekmovanju se je ekipa JSI vs COVID uvrstila na šlo za nov virus, se najprej ni vedelo, kako kužen je, kako hitro drugo mesto. V nadaljevanju so potekale raziskave v povezavi z se širi, predvsem pa kako se pred njim zaščititi in kako preprečiti Ministrstvom za zdravje RS, da bi ugotovitve in modele lahko v obremenitev bolnišnic in visoka števila težko bolnih in mrtvih. boju proti pandemiji uporabili tudi v praksi. Zelo hitro se je namreč pokazalo, da v primerjavi z nekaterimi drugimi respiratornimi obolenji več obolelih potrebuje KLJUČNE BESEDE bolnišnično oskrbo, nekateri tudi obravnavo na intenzivni negi in COVID-19, epidemiološki modeli, nefarmacevtski ukrepi, pomoč medicinskega ventilatorja. Zdravniki so poleg tega večkriterijska optimizacija potrebovali več mesecev, da so ugotovili, kako najučinkoviteje zdraviti paciente s COVID-19. Pomembno je bilo tudi ABSTRACT podcenjevanje nevarnosti, ker se je pričakovalo, da bo možno s We present an overview of the work that was carried out by the sledenjem okuženih, ki so npr. prileteli z avionom, zajeziti vdor members of the Department of Intelligent Systems in the last year, virusa v državo. Ker je virus sposoben prenašanja tudi preko na related to the COVID-19 pandemic. We were studying the videz zdravih ljudi, ta ukrep ni bil sposoben zajeziti vdora oz. models that forecast the spread of infection in local environment blokirati pri majhnem številu okuženih. and tried to suggest the countermeasures based on the trends. The Med različnimi pristopi v boju proti širjenju okužb se je zelo research first took place within the $500k Pandemic Response hitro uveljavil pristop »lockdowna«, praktično popolnega zaprtja Challenge, organized by the XPRIZE fundation and the company družbe. Ta pristop so najprej uporabili na Kitajskem, v mestu Cognizant. The JSI vs COVID team won the second place in the Wuhan, kjer so najprej zasledili virus, v Evropi so prvi začeli z competition. In the following months, the research focused on zapiranjem v Italiji, februarja, najprej v posameznih občinah na the applicability of the results in practive, in collaboration with severu države, kasneje po celotni državi. V Sloveniji so bili prvi the Ministry of Health. primeri virusa zaznani v začetku marca, sredi marca pa je prišlo do zaprtja države. Časovni potek prvih mesecev pandemije je opisan v [1]. Kmalu se je pokazalo, da popolno zaprtje družbe sicer precej učinkovito omeji širjenje okužb, ni pa učinkovito na dolgi rok, Permission to make digital or hard copies of part or all of this work for personal or classroom use is granted without fee provided that copies are not made or distributed saj je izredno drago za državo in prebivalce tako z ekonomskega for profit or commercial advantage and that copies bear this notice and the full kot tudi z družbenega vidika. Raziskovalci so zato začeli citation on the first page. Copyrights for third-party components of this work must preučevati kombinacije ukrepov, ki bi po eni strani učinkovito be honored. For all other uses, contact the owner/author(s). Information Society 2021, 4-8 October 2021, Ljubljana, Slovenia omejili širjenje okužb, po drugi strani pa bi čim manj prizadeli © 2021 Copyright held by the owner/author(s). 52 Information Society 2021, 4-8 October 2021, Ljubljana, Slovenia M. Luštrek et al. gospodarstvo in državljane. Takih sistemskih rešitev, npr. milijona dolarjev. Še osem ekip je prejelo posebna priznanja in programov, ob pojavu virusa COVID-19 še ni bilo. simbolične nagrade. Iskanje učinkovitih kombinacij ukrepov, ki hkrati ne bi prizadeli družbe, je bilo tudi vodilo tekmovanja $500k Pandemic Response Challenge, v organizaciji fundacije XPRIZE in 3 NEFARMACEVTSKI UKREPI podjetja Cognizant, ki je bilo organizirano konec leta 2020. Na V okviru tekmovanja so ekipe dobile predpisan seznam NPI, tekmovanje se je prijavila tudi ekipa JSI vs COVID, ki prihaja iz ki so ga uporabile za gradnjo preskriptorja, hkrati pa so za vsak Odseka za inteligentne sisteme IJS, in na koncu osvojila drugo ukrep dobile tudi »ceno« uveljavljanja tega ukrepa. Seznam je mesto. V pričujočem prispevku najprej predstavimo tekmovanje, vseboval sledeče NPI: nato skiciramo rešitev, ki jo je razvila ekipa JSI vs COVID, na 1. Zapiranje šol koncu pa orišemo nadaljnje raziskave, ki so potekale v povezavi 2. Omejitev prihoda na delo z Ministrstvom za zdravje Republike Slovenije. Opisana je tudi 3. Preklic javnih dogodkov ponudba evropskim ministrstvom za zdravje za brezplačno 4. Omejitve zbiranja uporabo sistema JSI vs COVID. 5. Omejitev javnega prometa 6. Omejitev izhodov od doma 7. Omejitve gibanja po državi 2 TEKMOVANJE PANDEMIC RESPONSE 8. Omejitve gibanja med državami CHALLENGE 9. Kampanja osveščanja javnosti Tekmovanje $500k Pandemic Response Challenge [2] je 10. Strategija testiranja potekalo med oktobrom 2020 (registracija udeležencev) in 11. Sledenje stikov februarjem 2021 (razglasitev zmagovalcev). Naloga, ki so jo 12. Uporaba zaščitnih mask dobili udeleženci, je bila razviti modele, ki bodo po eni strani čim Ukrepi so se lahko izvajali v strožji ali v milejši obliki. Ukrep natančneje predvideli lokalne izbruhe okužb (»prediktor«), po »Omejitve gibanje po državi« v strožji različici tako prepove drugi strani pa predlagati čim učinkovitejši načrt ukrepov, tako gibanje izven določenega območja, v milejši pa ga samo da se hkrati minimizira število okužb ter ekonomsko škodo odsvetuje, »Omejitve zbiranja« pa z naraščanjem strogosti (»preskriptor«). Beseda ukrepi se v tem primeru nanaša na zmanjšuje število ljudi, ki se lahko zbira. Seveda se je pri ukrepih nefarmacevtske ukrepe (angleško non-pharmaceutical potrebno zavedati, da se jih ljudje lahko držijo ali ne, in od interventions, NPI), ki jih nekoliko podrobneje opišemo v posamezne države je odvisno, kako strogo bo preverjala nadaljevanju. Število sodelujočih ekip je bilo omejeno na 200. spoštovanje ukrepov – pa tudi od tega, kako so ukrepi V prvi fazi tekmovanja so se ekipe osredotočile na analizo predstavljeni javnosti in kako jih ljudje sprejmejo. obstoječih podatkov in strategij boja proti pandemiji v različnih državah. Cilj je bil razviti in ovrednotiti napovedne modele za razvoj pandemije. Pri tem so imele ekipe na razpolago zbirko 4 MODELI podatkov o pandemiji za vrsto držav z Univerze v Oxfordu Modeli prediktorjev in preskriptorjev so oz. bodo podrobneje (Oxford COVID-19 Government Response Tracker (OxCGRT) opisani v konferenčnem prispevku [6] in v prihajajočih [3]), vključno z ukrepi, ki so bili v državah veljavni v različnih publikacijah, zato tu le skiciramo rešitve. obdobjih, ter vzorce temeljnih prediktorjev in preskriptorjev podjetja Cognizant, Evolutionary AI™. 4.1 Prediktor Prediktorji, ki so jih ekipe razvile, so se nato uporabljali za Cilj prediktorja je napovedati število okužb na določenem napovedovanje, napovedi pa so se primerjale z razvojem območju (v državi ali v regiji) za vsak dan, za obdobje več pandemije v realnem svetu. Ta primerjava je bila osnova za mesecev v prihodnost. Pri tem upoštevamo seznam NPI, ki so v uvrstitev v prvi fazi. Pomembno je poudariti, da so vse ekipe državi v danem trenutku v veljavi. dobile na voljo natanko iste podatke in bile ocenjevane po enakih Prediktor uporablja standardni epidemiološki model SEIR, kriterijih. kjer upoštevamo dinamiko med skupinami posameznikov, ki so Prva faza tekmovanja se je zaključila januarja 2021 z Susceptible (dovzetni), Exposed (izpostavljeni), Infected merjenjem dejanskih infekcij. V drugo fazo se je uvrstilo 48 (okuženi) in Removed (ozdraveli ali umrli, se pravi niso več najbolje uvrščenih ekip, ki so prihajale iz 17 držav. V drugi fazi dovzetni za okužbo). Model SEIR [7] je sestavljen iz sklopljenih tekmovanja je bil cilj razvoj preskriptorjev. V tem delu seveda ni diferencialnih enačb (1), kjer uporabljamo parametre, ki določajo bilo mogoče testirati v praksi, zato je testiranje potekalo na verjetnosti za prehode iz ene skupine v drugo: 𝑆 → 𝛽 → 𝐸 → sledeči način: za predikcije se je uporabljal »standardni 𝜎 → 𝐼 → 𝛾 → 𝑅. prediktor«, ki so ga razvili organizatorji [4]. Vsaka ekipa je predlagala do deset strategij intervencije za vsako državo. Pri 𝑑𝑆 𝛽𝑆𝐼 ocenjevanju se je upoštevalo, da je ena strategija od druge boljša, = − 𝑑𝑇 𝑁 če je boljša po enem kriteriju, ne pa hkrati tudi slabša po drugem (kriterija sta tu omejevanje širjenja okužb in cena ukrepov). 𝑑𝐸 𝛽𝑆𝐼 = − 𝜎𝐸 (1) Druga faza se je končala marca 2021, ko so bili razglašeni 𝑑𝑡 𝑁 zmagovalci [5]. Prvo mesto je osvojila španska ekipa 𝑑𝐼 VALENCIA IA4COVID19 iz Valencie, drugo mesto pa = 𝜎𝐸 − 𝛾𝐼 𝑑𝑡 slovenska ekipa JSI vs COVID z zelo podobnim numeričnim rezultatom. Ekipi sta si enakovredno razdelili nagradni sklad pol 53 Napovedovanje trendov in optimiziranje ukrepov v boju proti Information Society 2020, 5–9 October 2020, Ljubljana, Slovenia pandemiji COVID-19: Tekmovanje XPRIZE in naslednji koraki 𝑑𝑅 posamezno oceno potreboval približno 2 s. Z uporabo slednjega = 𝛾𝐼 𝑑𝑡 bi tako lahko ocenili le približno 45 planov, kar ne zadostuje za Pri tem velja zveza 𝑆 + 𝐸 + 𝐼 + 𝑅 = 𝑁 (vsi posamezniki). algoritem tipa NSGA-II. Natančneje, uporabili smo dva Parametre smo določili na sledeč način: 𝛽 (merilo za prenos) je nadomestna modela, enega na osnovi SEIR prediktorja (Sekcija bil določen s prilagajanjem modela na realne podatke o okužbah. 4.1) in enega na osnovi prekalkuliranih planov za različne poteke Z metodami strojnega učenja smo zgradili modele, ki se naučijo pandemije. Slednji deluje tako, da za dano regijo in čas najprej napovedovati ta parameter glede na nabor ukrepov. Pri tem smo ocenimo potek pandemije in nato predpišemo plane, ki se za dani upoštevali dejstvo, da posamezne države/regije uporabljajo potek najbolje odnesejo. Med vsemi kompromisnimi rešitvami, različen nabor NPI in da v posameznih obdobjih število okužb ki jih dobimo tako s prvim kot drugim nadomestnim modelom, narašča ali pada. 𝛽 je bil zato prilagojen za posamezno situacijo. izberemo 10 takih, ki najbolje pokrijejo množico vseh Parametra »optimalnih« kompromisov. Tipičen primer kompromisnih 𝜎 (inkubacijska doba) in 𝛾 (merilo za okrevanje) smo določili na podlagi podatkov iz literature. rešitev v obliki fronte najdemo na Sliki 2. Prediktor, ki smo ga zgradili z naborom različnih vrednosti 𝛽, se tako lahko prilagaja ukrepom, ki jih posamezne države uvajajo v različnih trenutkih. Prediktor deluje s časovno ločljivostjo enega dneva. Slika 1 prikazuje primer napovedi prediktorja v primerjavi z realnimi podatki. Slika 2: Primer desetih kompromisnih rešitev v obliki fronte, dobljenih z metodo iz tekmovanja. 5 SODELOVANJE Z MINISTRSTVOM ZA ZDRAVJE RS Slika 1: Primer dnevnega števila novih okužb za Italijo V okviru delovanja zadnjega avtorja kot državnega svetnika za jeseni 2020. Modra črta predstavlja napoved, črtkana rdeča črta pa število potrjenih primerov. S sivo je označen raziskovalno dejavnost je bil dne 28. 6. 2021 izveden posvet v Državnem svetu na temo uporabe programskih metod za krotenje interval za vhodne podatke. epidemije z naslovom: »Problemi COVID-19 in iskanje optimalnih rešitev za naprej«. Na komisijah DS in na 4.2 Preskriptor plenarnem zasedanju je bila pogosta debata na temo COVID-19, Cilj preskriptorja je izdelati plane intervencij (NPI) za kjer je avtor prispeval s strokovnimi analizami in napovedmi, dal posamezno obdobje za posamezno državo ali regijo. Pri tem je tudi nekaj pobud vladi v smeri najboljšega delovanja. Nekatere želimo predpisati plane, ki predstavljajo dobre kompromise med izmed teh zamisli so bile uresničene, druge ne. omejevanjem širjenja okužb in ekonomsko in družbeno ceno teh Ministrstvo za zdravje RS je poleg tega naročilo nekaj študij. ukrepov. Dodatna zahteva pri tekmovanju je bila časovna Del teh študij je prikazan v Vito Janko itd. »What-If Analysis of omejitev za izdelavo načrta, namreč, v šestih urah je bilo treba Countermeasures Against COVID-19 in November 2020 in izdelati načrte za 235 regij, kar predstavlja v povprečju 90 s na Slovenia« v zborniku konference Informacijska družba 2021. regijo. Tam je pokazano, da je v fazi velike rasti smotrno vpeljati ukrepa Preskripcijo planov lahko naravno predstavimo kot tako zapiranja šol kot nošenja mask. Za november 2020 so večkriterijski optimizacijski problem, kjer želimo minimizirati analize celo pokazale, da se zapiranju šol ne da povsem izogniti, dva konfliktna si kriterija: (1) povprečno število infekcij in (2) pa četudi uporabimo vrsto drugih ukrepov, če želimo ustaviti ekonomska in družbena cena ukrepov. Za potrebe optimizacije rast. Ukrep nošenja mask se glede na naš model izkaže enako smo uporabili dobro poznan genetski algoritem z nedominiranim učinkovit kot kombinacija bolj restriktivnih ukrepov, vendar je sortiranjem II (ang. nondominated sorting genetic algorithm II – predvidoma do posameznikov bolj prijazen. Družbenih cen NSGA-II) [11]. posameznih ukrepov v boju proti pandemiji v tej fazi nismo Zaradi stroge časovne omejitve (90 s na regijo), smo za oceno upoštevali, spada pa to med področja, ki jih bomo obravnavali v kakovosti planov standardni prediktor nadomestili z prihodnje. nadomestnimi modeli. Standardni prediktor je namreč za 54 Information Society 2021, 4-8 October 2021, Ljubljana, Slovenia M. Luštrek et al. 6 SODELOVANJE Z EVROPSKIMI programov, kot na primer razvitih na tekmovanju XPRIZE, MINISTRSTVI omogočila bistveno boljše rezultate. Rezultate smo delili s slovenskim in z evropskimi ministrstvi za zdravje. Julija 2021 smo vsem ministrstvom EU poslali pismo, kjer jim dajemo možnost neposredne uporabe programa XPRIZE JSI, ZAHVALA specializiranega za njihovo državo. Vsaka država torej lahko uporablja z geslom zaščiteno verzijo programa. Poslali smo jim Raziskave so bile financirane s strani ARRS, v sklopu tudi navodila, kako uporabljati programe in na kakšen način jim programske skupine P2-0209. nudimo podporo. Razvite rešitve so bile evropski in svetovni javnosti ponujene brezplačno. VIRI Nimamo podatkov, koliko EU ministrstev je aktivno [1] Derrick Bryson Taylor, A Timeline of the Coronavirus Pandemic, New uporabilo ali uporablja programe, a glede na odzive in glede na York Times, 17. marec 2021, dostopano 10. septembra 2021, https://www.nytimes.com/article/coronavirus-timeline.html to, da imamo kontakte le z nekaj ministrstvi, se zdi, da ne prav [2] Pandemic Response Challenge, dostopano 10. septembra 2021, veliko. https://www.xprize.org/challenge/pandemicresponse [3] T. Hale, N. Angrist, R. Goldszmidt, B. Kira, A. Petherick, T. Phillips, S. Webster, E. Cameron-Blake, L. Hallas, S. Majumdar and H. Tatlow, “A global panel database of pandemic policies (Oxford COVID-19 7 ZAKLJUČEK Government Response Tracker),” Nature Human Behaviour, p. 529–538, 2021. [4] R. Miikkulainen, O. Francon, E. Meyerson, X. Qiu, D. Sargent, E. Canzani V prispevku so pregledno opisane programske rešitve, ki jih je and B. Hodjat, “From Prediction to Prescription: Evolutionary Optimization of Nonpharmaceutical Interventions in the COVID-19 skupina Odseka za inteligentne sisteme razvila v okviru Pandemic,” IEEE Transactions on Evolutionary Computation, vol. 25, no. tekmovanja XPRIZE. Nekatere med njimi so bile v celoti izvirne 2, pp. 386-401, 2021. in so prispevale k drugem mestu na svetovnem prvenstvu na to [5] Announcing the Grand Prize Winners, XPRIZE, 10. marec 2021, temo. dostopano 10. septembra 2021, https://www.xprize.org/challenge/pandemicresponse/articles/pandemic- Po tekmovanju smo razvili vrsto dodatnih rešitev. Eno smo response-challenge-winners uporabili za iskanje ukrepov retroaktivno – ali in v kolikšni meri [6] Nina Reščič et al. Finding efficient intervention plans against Covid-19, je bil kak ukrep upravičen. Ena taka študija je zajemala obdobje presented at ETAI 2021. [7] Model SEIR, dostopano 10. septembra 2021, novembra 2020 v Sloveniji. V večini študij se je pokazalo, da so https://en.wikipedia.org/wiki/Compartmental_models_in_epidemiology# bili človeški ukrepi daleč od optimalnega in bi uporaba The_SEIR_model 55 56 Indeks avtorjev / Author index Bele Klemen ................................................................................................................................................................................. 23 Brandão Lucas ................................................................................................................................................................................ 5 Canzonieri Vincenzo ...................................................................................................................................................................... 9 Cigale Matej ................................................................................................................................................................................. 52 Crovella Sergio ............................................................................................................................................................................... 5 De Masi Carlo .............................................................................................................................................................................. 52 Dovgan Erik ............................................................................................................................................................... 12, 23, 38, 52 Eržen Samo .................................................................................................................................................................................. 20 Gams Matjaž ...................................................................................................................................... 12, 16, 23, 31, 38, 42, 46, 52 Gerlero Virginia ............................................................................................................................................................................. 9 Gradišek Anton ............................................................................................................................................................................ 52 Gratton Rossella ............................................................................................................................................................................. 5 Ilijaš Tomi .................................................................................................................................................................................... 20 Janko Vito .............................................................................................................................................................................. 42, 52 Klun Urša ..................................................................................................................................................................................... 31 Kocuvan Primož ......................................................................................................................................................... 12, 16, 23, 31 Kolar Žiga .................................................................................................................................................................................... 38 Kolenik Tine ................................................................................................................................................................................. 31 Luštrek Mitja .......................................................................................................................................................................... 42, 52 Marinko Matej .............................................................................................................................................................................. 52 Moura Ronald ................................................................................................................................................................................. 5 Noveski Gjorgji ............................................................................................................................................................................ 35 Palčič Devid ................................................................................................................................................................................. 27 Ražman Simon ............................................................................................................................................................................. 27 Reščič Nina ............................................................................................................................................................................ 42, 52 Rizzolio Flavio ............................................................................................................................................................................... 9 Strnad Marjan ............................................................................................................................................................................... 27 Susič David ............................................................................................................................................................................ 46, 52 Tomšič Janez ................................................................................................................................................................................ 46 Tricarico Paola Maura .................................................................................................................................................................... 5 Truccolo Ivana ............................................................................................................................................................................... 9 Tušar Tea ................................................................................................................................................................................ 42, 52 Valič Jakob ............................................................................................................................................................................. 16, 35 Vodopija Aljoša ........................................................................................................................................................................... 52 Vuk Zdenko .................................................................................................................................................................................. 38 57 58 Delavnica projekta Insieme Insieme Project Workshop Matjaž Gams, Primož Kocuvan, Flavio Rizzolio Document Outline 02 - Naslovnica - notranja - D - TEMP 03 - Kolofon - D - TEMP 04 - IS2021 - Predgovor - TEMP 05 - IS2021 - Konferencni odbori 07 - Kazalo - D 08 - Naslovnica - notranja - D - TEMP 09 - Predgovor podkonference - D 10 - Programski odbor podkonference - D 01 - Brandao 02 - Truccolo 03 - Kocuvan_2-final Abstract 1 Introduction 2 An overview of Existing EMH Platforms 2.1 Genoa 2.2 DigiGone 2.3 Doxy.me 2.4 eVisit 2.5 iPath 2.6 MedSymphony 2.7 Bodi Zdrav 2.8 EcoSmart 3 The ISE-EMH platform 3.1 Basic Information 3.2 Detailed Description of the ISE-EMH Platform 4 Analytical Comparison 4.1 Choosing the Features for Comparison 4.2 Results of the Analytical Comparison 5 Empirical Comparison of Simply Accessible Platforms 5.1 Bodi Zdrav User Experience 5.2 EcoSmart User Experience 5.3 ISE-EMH User Experience 6 Conclusion Acknowledgments 04 - Kocuvan Abstract 1 Introduction 1.1 Basic information about the project 1.2 Chosen Android OS and programming language 1.3 Overview of the functions 2 The initial screen 3 The elder's home view and functions 3.1 Sandbox 3.2 Battery 3.3 Mobile phone location and vocal search 3.4 Alarms and reminders 3.5 SOS function and fall detection 3.6 Phone book and pedometer 4 The caretaker view and functions 4.1 Sandbox 4.2 Battery 4.3 Mobile phone location and vocal search 4.4 Alarms and reminders 4.5 SOS function and fall detection 4.6 Phone book and pedometer 5 Conclusion Acknowledgments 05 - Erzen - cakamo 06 - Bele 07 - Palcic - cakamo 08 - Kolenik 09 - Noveski Abstract 1 Introduction 2 Experiment 3 Conclusion Acknowledgments 10 - Kolar Abstract 1 Introduction 2 Methodology for AHA Solution Assessment 2.1 Main objectives 2.2 Methodology 2.3 Platforms 3 Clustering and taxonomies 3.1 K-means clustering 3.2 Hierarchical clustering 3.3 Taxonomies 4 Conclusion and discussion Acknowledgments 11 - Janko Abstract 1 Introduction 2 Dataset 3 Methods 4 Results 5 Discussion and Conclusions 12 - Tomsic Abstract 1 Introduction 2 Methodology 2.1 Selection of articles 2.2 Articles not based on OxCGRT data 2.3 Ranking effectiveness of NPIs 2.4 Comparison with a similar article 3 Results 3.1 Assessed regions and time frames 3.2 Metrics used in studies 3.3 NPI effectiveness estimations 4 Conclusion 13 - Lustrek 12 - Index - D Blank Page Blank Page Blank Page Blank Page Blank Page Blank Page