Visual Impairments and Mobile Touchscreen Interaction: State-of-the-Art,Causes of Visual Impairment,and Design Guidelines

ABSTRACT

This article identifies, catalogues, and discusses factors that are responsible for causing visual impairment of either a pathological or situational nature for touch and gesture input on smart mobile devices. Because the vast majority of interactions with touchscreen devices are highly visual in nature, any factor that prevents a clear, direct view of the mobile device’s screen can have potential negative implications on the effectiveness and efficiency of the interaction. This work presents the first overview of such factors, which are grouped in a catalogue of users, devices, and environments. The elements of the catalogue (e.g., psychological factors that relate to the user, or the social acceptability of mobile device use in public that relates to the social environment) are discussed in the context of current eye pathology classification from medicine and the recent literature in human–computer interaction on mobile touch and gesture input for people with visual impairments, for which a state-of-the-art survey is conducted. The goal of this work is to help systematize research on visual impairments and mobile touchscreen interaction by providing a catalogue-based view of the main causes of visual impairments affecting touch and gesture input on smart mobile devices.     

Point &; click mediated interactions for large home entertainment displays

This work introduces and discusses the implementation details of a novel concept for a home entertainment system together with an affordable controlling interface that uses point & click interactions in order to create, mix and manipulate media screens within the same projection-based display. Scenarios for single and multiple viewers are discussed with users being able to create, reposition, resize, and control their own-defined media screens. The standard and familiar WIMP interaction techniques are transferred from PCs to home entertainment using a motion-sensing remote controller. An optional system feature is described for the automatic configuration of such media screens by analyzing the home environment using computer vision techniques. Observations from initial user studies are reported with regards to the perceived usefulness and acceptability of the proposed system. The main benefit introduced by this work is that of a large entertainment display that becomes shared and personalized while it is being adapted and fit into the home environment.     

Semantic ambient media: From ambient advertising to ambient-assisted living

The term ambient media was in its beginning used only for ambient advertising. Nowadays it denotes the media environment and the communication of information in ubiquitous and pervasive environments. With the addition of intelligence, the new field of semantic ambient media was established. In recent years, the field of semantic ambient media has spread its span from only a few sub-areas, such as ambient advertising, to new ones, such as ambient-assisted living (AAL) and health-monitoring media, significantly supported by intelligence. The study presented in this paper provides an advanced introduction to the field of semantic ambient media including the solutions for threat issues and illustration of success stories of the field. It conducts a survey of the related work and presents a thorough discussion of it. The related work is grouped according to the coverage of the principles of semantic ambient media. Based on the state-of-the-art research, the future possibilities of the field are demonstrated, especially for the ambient-assisted living, audio-visual rendering of media objects, user design principles and the society impact of the field. The paper provides ideas for impacting ambient media and directions and questions for further research. It also discusses the potential of the combination of several research studies.     

Presence bubbles: supporting and enhancing human-human interaction with ambient media

People communicate using speech, gestures and emotions but sometimes these may not be enough in order to express the true intended message. This results in miscommunication and sometimes frustration which are perceived negatively by the interacting participants. We propose an investigation into expressive ambient media in the form of visual surfaces as digital enhancements for Human-Human Interaction. The goal is to allow new modalities for self-expression as well as to enhance communication by displaying personal digital content in the form of Presence Bubbles. Ambient media allow thus communication, presence and socialization opportunities and transform into responsive and expressive media for supporting and enhancing human-human interaction.     

On designing interactivity awareness for ambient displays

In the not-so-distant world in which ambient displays will likely become prevalent, immediate awareness of their interaction affordances to passerbies will be decisive for their usability. However, how to address awareness at user perception level represents a challenge for which little progress has been made so far in contrast with considerable advances in designing interaction techniques for such displays. Even though many interactive ambient displays exist with properly designed interfaces, people may not always be aware of their interactivity. This work addresses the problem of interactivity awareness by focusing on two important questions: How can people tell whether a public display is interactive or not? and, assuming interactivity, How can people tell what the interface is? A study was conducted in order to investigate factors potentially related to perceived interactivity. Results show that people’s evaluations are correct in most cases but they also tend to form incorrect perceptions in many other situations. We found that location, installation, reachability, and displayed content are factors that can influence people’s perceptions. Our findings are complemented with a discussion of techniques that can be used in the practice of designing for interactivity awareness.     

Personalized gesture interactions for cyber-physical smart-home environments

A gesture-based interaction system for smart homes is a part of a complex cyber-physical environment, for which researchers and developers need to address major challenges in providing personalized gesture interactions. However, current research efforts have not tackled the problem of personalized gesture recognition that often involves user identification. To address this problem, we propose in this work a new event-driven service-oriented framework called gesture services for cyber-physical environments (GS-CPE) that extends the architecture of our previous work gesture profile for web services (GPWS). To provide user identification functionality, GS-CPE introduces a two-phase cascading gesture password recognition algorithm for gesture-based user identification using a two-phase cascading classifier with the hidden Markov model and the Golden Section Search, which achieves an accuracy rate of 96.2% with a small training dataset. To support personalized gesture interaction, an enhanced version of the Dynamic Time Warping algorithm with multiple gestural input sources and dynamic template adaptation support is implemented. Our experimental results demonstrate the performance of the algorithm can achieve an average accuracy rate of 98.5% in practical scenarios. Comparison results reveal that GS-CPE has faster response time and higher accuracy rate than other gesture interaction systems designed for smart-home environments.     

Ambient bloom: new business, content, design and models to increase the semantic ambient media experience

Semantic ambient media are the novel trend in the world of media reaching from the pioneering subareas such as ambient advertising to the new and emerging subareas such as ambient assisted living. They will likely shape the upcoming years in terms of modeling smart environments and also media consumption and interaction. This work analyzes semantic ambient media by considering business models, content and media, interaction design and consumer experience, and methods and techniques that are important to create this new form of media. Discussion is led using the state-of-the-art event of the semantic ambient media, which is the annual international workshop on Semantic Ambient Media Experience (SAME). The study also creates a vision for how ambient media will evolve and how they will look like in the future decade.