Architectural patterns for applications with external user interface elements

Coupling mobile devices and other remote interaction technology with software systems surrounding the user enables for building interactive environments under explicit user control. The realization of explicit interaction in ubiquitous or pervasive computing environments introduces a physical distribution of input devices, and technology embedded into the environment of the user. To fulfill the requirements of emerging trends in mobile interaction, common approaches for system design need adaptations and extensions. This paper presents the adaptation and extension of the Model-View-Controller approach to design applications of remote, complementary, duplicated and detached user interface elements.     

Modelling touch-interaction time on smartphones

Accurate performance models are important for interaction technique, application, and hardware design. The limited screen size of mobile devices and use of touch interaction require unique considerations, especially when interacting with large amounts of information. This paper considers the performance impact of target visibility on mobile smartphone applications that provide on- and offscreen content with the commonly used direct-touch interactions and four cursor-based interaction methods for precise selection. Three existing and 12 novel performance models are experimentally validated. Fitts' Law, which was not designed for modelling selection of offscreen targets, did not predict interaction times for mobile interaction methods as accurately as is commonly observed with desktop interaction with onscreen targets. Target visibility was found to greatly impact interaction times (particularly for direct-touch interaction). The presented models that incorporate variables related to target visibility greatly improve predicted interaction times. The use and merits of the top models are discussed, emphasizing the importance and implications of accepted user-interface design guidelines.     

Dynamic Performance of Mobile Haptic Interfaces

The increasing demand for virtual reality applications in several scientific disciplines feeds new research perspectives dealing with robotics, automation, and computer science. In this context, one of the topics is the design of advanced force-feedback devices allowing not only kinesthetic interaction with virtual objects but also locomotion and navigation inside virtual worlds. This has the main advantage to stimulate human vestibular apparatus, thus increasing the overall realism of simulation. Particularly, this paper deals with mobile haptic interfaces (MHIs), built by combining standard force-feedback devices with mobile platforms. We investigated which factors may affect the transparency of this kind of devices, identifying in mobile robot dynamics a possible cause of loss of transparency. Hence, in this paper, we present a method to analyze dynamic performance of an MHI and some basic guidelines to design controller in order to meet desired specifications. Experimental validation of the theoretical results is reported.     

Older adults’ use of mobile device: usability challenges while navigating various interfaces

ABSTRACT

Mobile devices are becoming ubiquitous among older adults, but have also caused unprecedented challenges due to the high demands of interaction techniques and changeable design patterns found across various applications. This paper aims to investigate how older adults navigate with mobile interfaces and identify their potential usability challenges while navigating. To do so, we summarised six state-of-the-art mobile interface design patterns and conducted individual usability test and in-depth interview with 22 older adults. Participants were asked to perform 19 navigation tasks that contain these design patterns under realistic usage scenarios. Follow-up interviews were held to collect their detailed comments on usability issues regarding visual design, ease of understanding, and interaction and navigation of the design patterns, as well as their personal experience. The results found that overall older adults were able to navigate contents more effectively than menus and buttons. Participants experienced great challenges in directing their attention to the menus and buttons, understanding the meaning of icons, and interacting with these menu components. In contrast, the content-oriented navigation design performed better in understanding, navigation, and interaction, which could be a promising direction for elderly-friendly mobile application design. Design implications are further discussed for creating an elderly-friendly mobile interface.     

Using Mobile Communication Devices to Access Virtual Meeting Spaces

This contribution addresses the integration of mobile devices with limited display and interaction capabilities into a multi-user system for tele-conferencing and collaborative work. We first report on a recently conducted case study in which we used a simulation environment to investigate how a small group of geographically dispersed users can jointly solve localisation and route planning tasks when equipped with communication devices that essentially differ with regard to available input and output capabilities. We then present a system architecture that includes gateway components as interfaces between the mobile devices and one or more interactive applications. Finally, we argue that there is a high demand for automated design approaches which are able to generate information presentations that are tailored to the available presentation capabilities of particular target devices.     

The value of handhelds in smart environments

The severe resource restrictions of computer-augmented everyday artifacts imply substantial problems for the design of applications in smart environments. Some of these problems can be overcome by exploiting the resources, I/O interfaces, and computing capabilities of nearby mobile devices in an ad-hoc fashion. We identify the means by which smart objects can make use of handheld devices such as PDAs and mobile phones, and derive the following major roles of handhelds in smart environments: (1) mobile infrastructure access point; (2) user interface; (3) remote sensor; (4) mobile storage medium; (5) remote resource provider; and (6) weak user identifier. We present concrete applications that illustrate these roles, and describe how handhelds can serve as mobile mediators between computer-augmented everyday artifacts, their users, and background infrastructure services. The presented applications include a remote interaction scenario, a smart medicine cabinet, and an inventory monitoring application.     

A unified methodology for the evaluation of accessibility and usability of mobile applications

This article reports a unified methodology developed to evaluate the accessibility and usability of mobile computing applications, which is intended to guarantee universal access as far as possible. As a basis for the methodology, this paper presents an analysis of the accessibility guidelines, conducted to take into account the specificity of mobile systems, as well as a set of usability heuristics, specifically devised for mobile computing. Finally, it presents the results of the application of the proposed methodology to applications that have been semi-automatically developed by the MAIS Designer, a new design tool that provides applications suited to different mobile devices.     

Natural language interface for information management on mobile devices

A natural language interface (NLI) enables the ease-of-use of information systems in performing sophisticated human - computer interaction. To address the challenges of mobile devices to user interaction in information management, we propose an NLI as a promising solution. In this paper, we review state-of-the-art NLI technologies and analyse user requirements for managing notable information on mobile devices. To minimize any technical difficulties arising from developing and improving the usability of NLI systems we develop general principles for NLI design, which fills in a gap in the literature. In order to satisfy user requirements for information management on mobile devices, we innovatively design NLI-enabled information management architecture. It is shown from two usage scenarios that the architecture could lead to reduced effort in user navigation and improved efficiency and effectiveness of managing information on mobile devices. We conclude the article with the implications of this study and suggestions for future direction.     

3D elastic control for mobile devices

To increase the input space of mobile devices, the authors developed a proof-of-concept 3D elastic controller that easily adapts to mobile devices. This embedded device improves the completion of high-level interaction tasks such as visualization of large documents and navigation in 3D environments. It also opens new directions for tomorrow's mobile applications.     

Design and development of a mobile peer-to-peer social networking application

The proliferation of wireless and mobile devices such as personal digital assistants and mobile phones has created a large demand for mobile software applications such as social networking software. In addition, the realization and widespread usage of peer-to-peer (P2P) networking have drastically increased the number of applications utilizing these technologies. The convergence of mobile and P2P networking have generated increasing interest in the mobile peer-to-peer (MP2P) community. In this paper, we describe the design and development of a mobile social software (MoSoSo) based on a P2P network architecture using Juxtapose (JXTA) and Juxtapose for Java MicroEdition (JXME). The MoSoSo application allows users to discover, communicate and share resources with one another. We present three facets of designing the MoSoSo: object-oriented software design, network infrastructure design, and user-interface design. The software has been fully implemented and tested on a variety of mobile devices for use in a campus setting. By studying the design and implementation of the MoSoSo, we hope to benefit the entire mobile application development community by providing common models and insights into developing MP2P software.     

A location-based policy-specification language for mobile devices

The dramatic rise in mobile applications has greatly increased threats to the security and privacy of users. Security mechanisms on mobile devices are currently limited, so users need more expressive ways to ensure that downloaded mobile applications do not act maliciously. Policy-specification languages were created for this purpose; they allow the enforcement of user-defined policies on third-party applications. We have implemented LoPSiL, a location-based policy-specification language for mobile devices. This article describes LoPSiL’s design and implementation, several example policies, and experiments that demonstrate LoPSiL’s viability for enforcing policies on mobile devices.     

Event Perception in Mobile Interaction: Toward Better Navigation History Design on Mobile Devices

This article explores how people perceive interactive activities in order to inform navigation history design on mobile devices. Following event segmentation method, 12 participants were asked to break 6 episodes of mobile interaction into segments, organize the segments, and identify those deemed representative. Three findings emerged. First, when making sense of mobile interaction, users concentrate on the content objects on which actions are performed. This indicates the value of content-centric designs in navigation history and other mobile user interface designs. The content objects are data objects and their collections meaningful to the person dealing with it, for example, photos, messages, or albums. Second, users tend to employ two-level hierarchies in grouping segments, and use the similarity in content objects and applications as a reference. They deem the segments as representative where objects are created or changed, or where sharing or querying acts take place. These findings indicate how a navigation history design should organize and prioritize mobile interaction events. Finally, event perception shows relatively low interparticipant consensus, which indicates that navigation history designs have to accommodate large individual differences.     

MobileAR Browser – A generic architecture for rapid AR-multi-level development

We present our novel generic approach for interfacing web components on mobile devices in order to rapidly develop Augmented Reality (AR) applications using HTML5, JavaScript, X3D and a vision engine. A general concept is presented exposing a generalized abstraction of components that are to be integrated in order to allow the creation of AR capable interfaces on widely available mobile devices.Requirements are given, yielding a set of abstractions, components, and helpful interfaces that allow rapid prototyping, research at application level, as well as commercial applications. A selection of various applications (also commercial) using the developed framework is given, proving the generality of the architecture of our MobileAR Browser.Using this concept a large number of developers can be reached. The system is designed to work with different standards and allows for domain separation of tracking algorithms, render content, interaction and GUI design. This can potentially help groups of developers and researchers with different competences creating their application in parallel, while the declarative content remains exchangeable.     

Trusting mobile user devices and security modules

The market for devices like mobile phones, multifunctional watches, and personal digital assistants is growing rapidly. Most of these mobile user devices need security for their prospective electronic commerce applications. While new technology has simplified many business and personal transactions, it has also opened the door to high-tech crime. We investigate design options for mobile user devices that are used in legally significant applications     

移动环境下神经功能评价多通道交互模型

随着平板电脑、智能手机、智能手表等智能移动设备的普及,利用便携的智能移动设备随时随地进行健康评价受到了国内外学者的广泛关注.人机交互特有的多通道、交互式、人机协同的计算能够有效地提高移动环境下神经功能评价的准确度.然而,目前很少有研究对人机交互在这一应用场景上发挥的作用进行过充分讨论,更没有形成统一的多通道交互模型.为此,首先分析了目前移动设备上主流的神经功能评价方法,归纳总结出了一套适用该应用场景的交互原语和交互任务.然后,在此基础上提出了移动环境下神经功能评价多通道人机交互模型——MINA（multimodal human-computer interaction model for nerve function assessment in mobile environment),并对该模型的移动医学评价和多通道融合特点进行了分析.最后,依据此模型给出神经系统疾病检测的应用实例.实践证明,MINA能够较好地指导交互式神经功能评价应用的设计和开发,多通道融合的方式能够有效地提高医学评价的准确度.     

Evaluating the benefits of multimodal interface design for CoMPASS—a mobile GIS

The context of mobility raises many issues for geospatial applications providing location-based services. Mobile device limitations, such as small user interface footprint and pen input whilst in motion, result in information overload on such devices and interfaces which are difficult to navigate and interact with. This has become a major issue as mobile GIS applications are now being used by a wide group of users, including novice users such as tourists, for whom it is essential to provide easy-to-use applications. Despite this, comparatively little research has been conducted to address the mobility problem. We are particularly concerned with the limited interaction techniques available to users of mobile GIS which play a primary role in contributing to the complexity of using such an application whilst mobile. As such, our research focuses on multimodal interfaces as a means to present users with a wider choice of modalities for interacting with mobile GIS applications. Multimodal interaction is particularly advantageous in a mobile context, enabling users of location-based applications to choose the mode of input that best suits their current task and location. The focus of this article concerns a comprehensive user study which demonstrates the benefits of multimodal interfaces for mobile geospatial applications.     

How mobile phones perform in collaborative augmented reality (CAR) applications

This paper presents the experimental analysis of mobile phones for Augmented Reality marker tracking, a core task that any CAR application must include. The results show that the most time consuming stage is the marker detection stage, followed by the image acquisition stage. Moreover, the rendering stage is decoupled on some devices, depending on the operative system used. This decoupling process allows avoiding low refresh rates, facilitating the collaborative work. However, the use of multicore devices does not significantly improve the performance provided by CAR applications. Finally, the results show that unless a poor network bandwidth makes the network to become the system bottleneck, the performance of CAR applications based on mobile phones will be limited by the detection stage. These results can be used as the basis for an efficient design of CAR systems and applications based on mobile phones.     

Patterns of Mobile Interaction

The design of systems for mobile scenarios covers a wide range of issues, ranging from mobile networking to user interface design for mobile devices. Mobile applications often run distributed on several connected devices, used by many users simultaneously. Considering all issues related to mobile scenarios, a designer might be overwhelmed. As a solution, we propose a specific kind of design patterns which we call mobility patterns, derived from successful mobile applications. They allow a designer to re-use design elements as building blocks in their own designs. After describing the idea of mobility patterns, we give a brief overview of patterns we have identified so far. Two patterns are described in more detail with the help of our research platforms QuickStep and Pocket DreamTeam.     

Multimodal object oriented user interfaces in mobile affective interaction

In this paper, we investigate an object oriented (OO) architecture for multimodal emotion recognition in interactive applications through mobile phones or handheld devices. Mobile phones are different from desktop computers since mobile phones are not performing any processing involving emotion recognition whereas desktop computers can perform such processing. In fact, in our approach, mobile phones have to pass all data collected to a server and then perform emotion recognition. The object oriented architecture that we have created, combines evidence from multiple modalities of interaction, namely the mobile device’s keyboard and the mobile device’s microphone, as well as data from emotion stereotypes. Moreover, the OO method classifies them into well structured objects with their own properties and methods. The resulting emotion detection server is capable of using and handling transmitted information from different mobile sources of multimodal data during human-computer interaction. As a test bed for the affective mobile interaction we have used an educational application that is incorporated into the mobile system.     

移动环境下基于移动代理的应用框架设计

随着网络技术的发展和移动设备的普及，用户希望能随时随地对资源进行访问。但是，由于移动网络连接带宽低且移动单元本身资源贫乏，移动环境为分布应用开发提出了新的挑战。移动代理技术以其节省带宽、异步性等特征，为移动计算提供了良好的解决方案。本文首先讨论了移动代理系统的设计，然后提出一个基于移动代理的移动应用框架
架，最后给出了相应的应用实例。