Informatica 38 (2014) 385-386 385
The Inherent Context Awareness of Natural User Interfaces: a Case Study on Multitouch Displays
Bojan Blažica
XLAB Research, Pot za Brdom 100, 1000 Ljubljana, Slovenia
Jožef Stefan International Postgraduate School, Jamova 39, 1000 Ljubljana, Slovenija
E-mail: bojan.blazica@xlab.si
www.xlab.si
Thesis Summary
Keywords: natural user interfaces, context-awareness, multitouch displays, biometric identification
Received: November 11, 2014
In computer science, context-awareness refers to the capability of a computing device to sense, understand and react to contextual information, i.e. information that is not at the centre of an activity but is still relevant for that activity. A computing device does not necessarily interact with humans at a given moment, but when it does, its Context-Awareness has many implications for human-computer interaction. The thesis that this paper surmises looks at Natural User Interfaces from a Context-Awareness perspective.
Povzetek: V računalništvu se pojem kontekstna ozaveščenost nanaša na sposobnost računalniškega sistema, da zazna, razume in se odzove na informacije, ki izvirajo iz konteksta, v katerem se nahaja in deluje. Imenujemo jih kontekstne informacije in jih definiramo kot tiste informacije, ki sicer niso v centru neke aktivnosti, a so zanjo še vedno pomembne. V primeru, da računalniški sistem interagira s človekom, ima lahko kontekstna ozaveščenost sistema velik vpliv na samo komunikacijo človek-računalnik. Ta članek povzema disertacijo, ki z vidika kontekstne ozaveščenosti obravnava naravne uporabniške vmesnike.
1 Introduction
This paper surmises a PhD thesis [1] that looks at natural user interfaces from a context-awareness perspective (Figure 1). On the one hand, we show that considering natural user interfaces as context-aware systems further increases the expressive power of these interfaces and, on the other hand, we show that natural user interfaces can also represent essential building blocks for context-aware systems and are therefore a viable way towards context-awareness. Research prospects addressed that arise from this perspective are: to what extent are natural user interfaces already inherently context-aware, how to increase the expressiveness of natural user interfaces through context-awareness, do natural user interfaces provide enough information to perform biometric user identification, and how to take advantage of information implicitly conveyed by the user during interaction with natural user interfaces. The specific natural user interfaces used are multitouch displays.
The thesis first reviews the fields of natural user interfaces and context-awareness. Regarding natural user interfaces, as this is an emerging research field, special care is taken to survey all currently available definitions of the term. Similarly, multitouch displays and multitouch interaction are described in more detail as they are considered in the case studies for the thesis. Other related fields such as ubiquitous/pervasive computing, ambient intelligence etc. are also briefly
explained. The presented overview does not merely introduce the topic of the thesis, but also shows how interconnected these fields are and how natural user interfaces are indeed inherently context-aware.
Figure 1 : Context in human-computer interaction.
2 MTi: a Method for user identification on multitouch displays
We have shown how the increased amount and variety of data from natural user interfaces can be exploited to acquire contextual information by developing a biometric user identification method and a clustering algorithm for hand detection, both for multitouch displays. The method
386 Informática 38 (2014) 385-386
B. Blazica
Figure 3 : Illustration of the problem of hand detection on multitouch displays.
for user identification, named MTi, is based on features obtained only from the coordinates of the 5 touchpoints of one of the user's hands (Figure 2). This makes it applicable to (almost) all multitouch displays without requiring additional hardware and regardless of the display's underlying sensing technology. The method was tested on a dataset of 34 users and reported 94.69 % identification accuracy. The method also proved to scale well and has an above-average usability [2].
Figure 2: Illustration of types of features used for user identification on multitouch displays.
3	HDCMD: a clustering algorithm to support hand detection on multitouch displays
Next, we address the problem of hand detection, i.e. detecting how many hands are currently on the surface and associating each touch point to its corresponding hand (Figure 3) [3]. The presented solution - a clustering algorithm with simple heuristics based on the anatomy of the human hand - is software-based and thus again applicable to all multitouch surfaces regardless of their construction. Along with these two, other related methods that increase the expressiveness of multitouch displays are surveyed in [2, 3].
4	A personal perspective on photowork: implicit human computer interaction for photo collection management
Finally, the thesis explores the possibility to use implicit human-computer interaction to aid personal photo collection management. The idea is that the way we interact with natural user interfaces can implicitly disclose additional (contextual) information, which helps a context-aware system to better understand the user. More specifically, we take into account the user's personal relationship with a single photo; whether the photo is of particular importance to the user. We call this personal relationship the user's affinity for a photo.
Experiments revealed that affinity is correlated with the time a user spends viewing a picture. Furthermore, by looking at viewing times, it is also possible to distinguish the task a user is currently performing [4].
5 Conclusion
The positive examples of context acquisition on multitouch displays presented confirm that natural user interfaces are inherently context-aware and show how their expressive power can be further increased by viewing them from a context-aware perspective.
Acknowledgement
The author wants to thank his supervisors Dunja Mladenic and Daniel Vladusic for their advice and support. Funded in part by the European Union, European Social Fund, Operational Program for Human Resources, Development for the Period 2007-2013 and by the Slovenian Research Agency.
References
[1]	Blazica, B. The inherent contex awareness of natural user interfaces: a case study on multitouch displays : doctoral dissertation. Ljubljana, 2013 http ://hci. si/?p=511
[2]	Blazica, B.; Vladusic, D.; Mladenic, D. Mti: A method for user identification for multitouch displays. International Journal of Human-Computer Studies vol. 71, no. 6, p. 691-702 (2013). http://goo.gl/pcyUzl.
[3]	Blazica, B.; Vladusic, D.; Mladenic, D. HDCMD: a Clustering Algorithm to Support Hand Detection on Multitouch Displays. V: First International Conference, SouthCHI 2013, Maribor, Slovenia, July 1-3, 2013. Human factors in computing and informatics : proceedings (Lecture notes in computer science), vol. 7946, p. 803-814 (2013).
[4]	Blazica, B.; Vladusic, D.; Mladenic, D. A personal perspective on photowork: implicit humancomputer interaction for photo collection management. Personal and Ubiquitous Computing, vol. 17, no. 8, p. 1787-1795 (2013). http://goo.gl/sGfvNh.