A user-centric and context-aware solution to interface management and access network selection in heterogeneous wireless environments

This paper investigates issues related to handover management of mobile terminals in heterogeneous wireless environments. The mobile terminals are equipped with multiple wireless radio interfaces and have a limited lifetime battery. Seamless mobility and power utilization efficiency become two important aspects of the handover management. We propose in this work user-centric network selection, power-saving interface management and adaptive handover initiation solutions at the terminal side to support seamless terminal-initiated and terminal-controlled vertical handover. The proposed access network selection is situation-aware and application-aware to suit different communication contexts. It enables terminals to select the most suitable access network according to various access network characteristics. Multiple wireless interfaces of a terminal device are handled in both idle and active communication modes to optimize the power consumption. We also address an adaptive handover initiation scheme to assist the service continuity. We conduct simulations and analyses of the proposed solutions to show their suitability and their efficiency.     

Interaction on-the-go: a fine-grained exploration on wearable PROCAM interfaces and gestures in mobile situations

Wearable projector and camera (PROCAM) interfaces, which provide a natural, intuitive and spatial experience, have been studied for many years. However, existing hand input research into such systems revolved around investigations into stable settings such as sitting or standing, not fully satisfying interaction requirements in sophisticated real life, especially when people are moving. Besides, increasingly more mobile phone users use their phones while walking. As a mobile computing device, the wearable PROCAM system should allow for the fact that mobility could influence usability and user experience. This paper proposes a wearable PROCAM system, with which the user can interact by inputting with finger gestures like the hover gesture and the pinch gesture on projected surfaces. A lab-based evaluation was organized, which mainly compared two gestures (the pinch gesture and the hover gesture) in three situations (sitting, standing and walking) to find out: (1) How and to what degree does mobility influence different gesture inputs? Are there any significant differences between gesture inputs in different settings? (2) What reasons cause these differences? (3) What do people think about the configuration in such systems and to what extent does the manual focus impact such interactions? From qualitative and quantitative points of view, the main findings imply that mobility impacts gesture interactions in varying degrees. The pinch gesture undergoes less influence than the hover gesture in mobile settings. Both gestures were impacted more in walking state than in sitting and standing states by all four negative factors (lack of coordination, jittering hand effect, tired forearms and extra attention paid). Manual focus influenced mobile projection interaction. Based on the findings, implications are discussed for the design of a mobile projection interface with gestures.     

Output-oriented power saving mode for mobile devices

Fast battery discharge is still the most nerve wracking issue for smartphone users. Though many energy saving methods have been studied, still users are not satisfied with their phone’s battery. Power management system provides low power mode when the phone is not in use for a long time. When the user is interacting with the phone, current system assumes the user is interactive and should keep the device in active mode. However, this is not true. After the user’s interaction, the device processes the request and displays the result on the smartphone’s output device. During this period, the user cannot see any meaningful information from the phone. In this paper, we propose a new low power mode where we put smartphone’s output device into low power mode while phone is preparing result for the user. We named this as o-sleep, an output-oriented power saving mode. While a device is processing a user’s request, output from the device may require preparation time. We consider the situation as the device’s output idle time and put the phone’s user interfaces into sleep mode while maintaining other subsystems in active state. To prove our concept, we have applied our technique onto various smartphone applications with varying operation environment. From the experiment, we found that the smartphone entered the o-sleep mode up to 58% of its total usage time in various test scenarios. Usability study supported feasibility of our proposed method.     

Adapting the mobile phone for task efficiency: the case of predicting outgoing calls using frequency and regularity of historical calls

The set of functionalities provided by advanced mobile phones is significantly increasing. However, the small size of mobile phone user interfaces makes it difficult for the user to deal with this large number of functionalities, which could reflect negatively on user performance and the efficiency of mobile phone functionalities. In this paper, we designed and developed an adaptive task-based functionality called ATF on mobile phones, where the task we focused on was to predict the next contact that the user is most likely to call. Furthermore, we conducted comprehensive evaluation of our approach. We show that our approach can successfully predict contacts that a user will most likely call next. Our results uncover the frequency and pattern of regularity in making mobile phone calls and suggest promising avenues for future work for optimising tasks (beyond phone calls) performed with the mobile phone.     

Context Awareness and Mobile Phones

This paper investigates some aspects of how context-awareness can support users of mobile phones, in particular the calling party. The use of mobile and stationary phones is discussed in relation to situational properties of a phone conversation, especially with regards to who might benefit from context-awareness in this context. An initial hypothesis is that mobile phone users communicate context information to each other (verbally) to a much higher degree than do stationary phone users. Mobile phone users could benefit much from context awareness technology, in particular when about to make a call, if they can receive context information regarding the person they are trying to reach prior to establishing the call. We argue that such technology should require low amounts of explicit user interaction, and could lead to less disrupting calls in inappropriate moments, as well as less frustration for the calling party when a call is not answered.     

Evaluating the benefits of multimodal interface design for CoMPASS—a mobile GIS

The context of mobility raises many issues for geospatial applications providing location-based services. Mobile device limitations, such as small user interface footprint and pen input whilst in motion, result in information overload on such devices and interfaces which are difficult to navigate and interact with. This has become a major issue as mobile GIS applications are now being used by a wide group of users, including novice users such as tourists, for whom it is essential to provide easy-to-use applications. Despite this, comparatively little research has been conducted to address the mobility problem. We are particularly concerned with the limited interaction techniques available to users of mobile GIS which play a primary role in contributing to the complexity of using such an application whilst mobile. As such, our research focuses on multimodal interfaces as a means to present users with a wider choice of modalities for interacting with mobile GIS applications. Multimodal interaction is particularly advantageous in a mobile context, enabling users of location-based applications to choose the mode of input that best suits their current task and location. The focus of this article concerns a comprehensive user study which demonstrates the benefits of multimodal interfaces for mobile geospatial applications.     

UX Curve: A method for evaluating long-term user experience

The goal of user experience design in industry is to improve customer satisfaction and loyalty through the utility, ease of use, and pleasure provided in the interaction with a product. So far, user experience studies have mostly focused on short-term evaluations and consequently on aspects relating to the initial adoption of new product designs. Nevertheless, the relationship between the user and the product evolves over long periods of time and the relevance of prolonged use for market success has been recently highlighted. In this paper, we argue for the cost-effective elicitation of longitudinal user experience data. We propose a method called the “UX Curve” which aims at assisting users in retrospectively reporting how and why their experience with a product has changed over time. The usefulness of the UX Curve method was assessed in a qualitative study with 20 mobile phone users. In particular, we investigated how users’ specific memories of their experiences with their mobile phones guide their behavior and their willingness to recommend the product to others. The results suggest that the UX Curve method enables users and researchers to determine the quality of long-term user experience and the influences that improve user experience over time or cause it to deteriorate. The method provided rich qualitative data and we found that an improving trend of perceived attractiveness of mobile phones was related to user satisfaction and willingness to recommend their phone to friends. This highlights that sustaining perceived attractiveness can be a differentiating factor in the user acceptance of personal interactive products such as mobile phones. The study suggests that the proposed method can be used as a straightforward tool for understanding the reasons why user experience improves or worsens in long-term product use and how these reasons relate to customer loyalty.     

A natural language model for managing TV-Anytime information in mobile environments

The TV-Anytime standard describes the structures of categories of digital TV program metadata, as well as user profile metadata for TV programs. We describe a natural language (NL) model for the users to interact with the TV-Anytime metadata and preview TV programs from their mobile devices. The language utilises completely the TV-Anytime metadata specifications (upper ontologies), as well as domain-specific ontologies. The interaction model does not use clarification dialogues, but it uses the user profiles as well as TV-Anytime metadata information and ontologies to rank the possible responses in case of ambiguities. We describe implementations of the model that run on a PDA and on a mobile phone, and manage the metadata on a remote TV-Anytime-compatible TV set. We present user evaluations of the approach. Finally, we propose a generalised implementation framework that can be used to easily provide NL interfaces for mobile devices for different applications and ontologies.     

Worth-centred mobile phone design for older users

The twenty-first century society fights against an inherent tendency to over-classify and label people. In the case of the aged, despite all efforts, the perception of the helpless, feeble older person still prevails. The truth of the matter is that people over sixty often do not fit this profile. The aged are a heterogeneous group with varying different skills and abilities in many different areas. This paper challenges prevalent mobile phone design decisions that appear to have been made based on the erroneous pre-conception of the incapable elder. Designers currently produce “senior” mobile phones that are, at best, inadequate and, at worst, insulting to a sector of society that deserves respect and consideration. Age does indeed influence mobile phone usage, and people over sixty often have specific and special needs, quite apart from age-related limitations, that predict their use of mobile phones. Most mobile phones designed for older users simply reduce the number of features: the so-called simplification approach. Apart from reducing the effectiveness of the phone, this approach often incorporates the fatal design flaw of using numbers or letters, on speed-dial buttons, which requires the user to remember the button–person mappings. In fact, this design rationale reduces the value of the phone to the user. This paper argues that mobile phone design for older users should be worth-centred (Cockton G in Designing worth is worth designing. In: Proceedings of the 4th Nordic conference on human–computer interaction: changing roles. Oslo, Norway, pp 165–174, 2006) rather than simplification-driven. The worth-centred approach maximises worth to the user of the phone. This is achieved by maximising effectiveness while accommodating reduced capabilities. To maximise ease of use, and consequent accessibility, features may have to be reduced in an informed way. To facilitate this, a mapping process is proposed whereby user needs are linked to uses of the phone, and then to the features that facilitate these uses. Needs fall into a number of categories, and each category is characterised by a number of different uses, which form a usage space. Features can be linked to one or more usage spaces, and thus be used to support needs. The first step in the conducted research entailed the identification of the needs of the older mobile phone user. Then, it was determined whether these needs were indeed being met by the uses afforded in existing phones. Having concluded that most users’ needs were not being met, the next step was to capture data on the needs, limitations and expectations of people over the age of sixty. This was achieved by conducting a series of one-to-one interviews with a number of older mobile phone users and also supervising a participatory design experiment. Using the findings of the analysis, a usage space model is proposed, which serves to align feature inclusion with user needs. Based on this usage space model (the theoretical contribution), a prototype mobile phone design is presented as the practical contribution of the paper.     

Pedestrian navigation using the sense of touch

Haptics is a feedback technology that takes advantage of the human sense of touch by applying forces, vibrations, and/or motions to a haptic-enabled user device such as a mobile phone. Historically, human–computer interaction has been visual, data, or images on a screen. Haptic feedback can be an important modality in Mobile Location-Based Services like – knowledge discovery, pedestrian navigation and notification systems. In this paper we describe a methodology for the implementation of haptics in four distinct prototypes for pedestrian navigation. Prototypes are classified based on the user’s navigation guidance requirements, the user type (based on spatial skills), and overall system complexity. Here haptics is used to convey location, orientation, and distance information to users using pedestrian navigation applications. Initial user trials have elicited positive responses from the users who see benefit in being provided with a “heads up” approach to mobile navigation. We also tested the spatial ability of the user to navigate using haptics and landmark images based navigation. This was followed by a test of memory recall about the area. Users were able to successfully navigate from a given origin to a Destination Point without the use of a visual interface like a map. Results show the users of haptic feedback for navigation prepared better maps (better memory recall) of the region as compared to the users of landmark images based navigation.     

A rapid prototyping software infrastructure for user interfaces in ubiquitous augmented reality

Recent user interface concepts, such as multimedia, multimodal, wearable, ubiquitous, tangible, or augmented-reality-based (AR) interfaces, each cover different approaches that are all needed to support complex human–computer interaction. Increasingly, an overarching approach towards building what we call ubiquitous augmented reality (UAR) user interfaces that include all of the just mentioned concepts will be required. To this end, we present a user interface architecture that can form a sound basis for combining several of these concepts into complex systems. We explain in this paper the fundamentals of DWARFs user interface framework (DWARF standing for distributed wearable augmented reality framework) and an implementation of this architecture. Finally, we present several examples that show how the framework can form the basis of prototypical applications.     

A middleware framework for application-aware and user-specific energy optimization in smart mobile devices

Mobile battery-operated devices are becoming an essential instrument for business, communication, and social interaction. In addition to the demand for an acceptable level of performance and a comprehensive set of features, users often desire extended battery lifetime. In fact, limited battery lifetime is one of the biggest obstacles facing the current utility and future growth of increasingly sophisticated “smart” mobile devices. This paper proposes a novel application-aware and user-interaction aware energy optimization middleware framework (AURA) for pervasive mobile devices. AURA optimizes CPU and screen backlight energy consumption while maintaining a minimum acceptable level of performance. The proposed framework employs a novel Bayesian application classifier and management strategies based on Markov Decision Processes and Q-Learning to achieve energy savings. Real-world user evaluation studies on Google Android based HTC Dream and Google Nexus One smartphones running the AURA framework demonstrate promising results, with up to 29% energy savings compared to the baseline device manager, and up to 5×savings over prior work on CPU and backlight energy co-optimization.     

Empowerment through seamfulness: smart phones in everyday life

In this paper, we describe research into use of multifunctional mobile phones by working adults and posit the device as a plausible realization of ubiquitous computing. We investigate how users actively adapt and adopt the different functions in smart phones to suit their needs and lifestyles. Through an interview and diary study, we discover how the smart phone is used in pragmatic and seamful ways, regardless of the interface of the specific phone selected or the particular features available. Users used phones in highly individual manners; mixed and adapted existing functions to meet their own priorities; added some functions and ignored others to create their own portfolio; and blended their use with the specifics of their everyday lives. While these data challenge some assumptions of human–computer interaction and ubiquitous computing, it also presents new research potential in terms of understanding how users take advantage of the multiple features in smart phone devices and how they utilize seamfulness in everyday smart phones practices.     

Ubiquitous Access to Unified Messaging: A Study of Usability and the Use of Pervasive Computing

The advent of mobile devices and the wireless Internet is having a profound impact on the way people communicate, as well as on the user interaction paradigms used to access information that was traditionally accessible only through visual interfaces. Applications for mobile devices entail the integration of various data sources optimized for delivery to limited hardware resources and intermittently connected devices through wireless networks. Although telephone interfaces arise as one of the most prominent pervasive applications, they present interaction challenges such as the augmentation of speech recognition through natural language (NL) understanding and high-quality text-to-speech conversion. This article presents an experience in building an automated assistant that is natural to use and could become an alternative to a human assistant. The Mobile Assistant (MA) can read e-mail messages, book appointments, take phone messages, and provide access to personal-organizer information. Key components are a conversational interface, enterprise integration, and notifications tailored to user preferences. The focus of the research has been on supporting the pressing communication needs of mobile workers and overcoming technological hurdles such as achieving high accuracy speech recognition in noisy environments, NL understanding, and optimal message presentation on a variety of devices and modalities. The article outlines findings from the 2 broad field trials and lessons learned regarding the support of mobile workers with pervasive computing devices and emerging technologies.     

基于信任区和可信计算技术的移动钱包系统

通过移动电子商务，终端用户可以在任何地方、任何时间进行电子交易，给用户提供了方便和舒适的生活，移动电子商务的安全性主要体现在终端的安全性。目前的手机安全措施还不是很完善，针对存在的安全问题，通过对现有的手机的硬件架构基础改进，实现了基于信任区和可信计算技术的移动钱包系统，保证了顾客和商家交易的安全性。     

Studying users’ perception of IoT mobile companion apps

Internet of Things (IoT) products provide over-the-net capabilities such as remote activation, monitoring, and notifications. An associated mobile app is often provided for more convenient usage of these capabilities. The perceived quality of these companion apps can impact the success of the IoT product. We investigate the perceived quality and prominent issues of smart-home IoT mobile companion apps with the aim of deriving insights to: (i) provide guidance to end users interested in adopting IoT products; (ii) inform companion app developers and IoT producers about characteristics frequently criticized by users; (iii) highlight open research directions. We employ a mixed-methods approach, analyzing both quantitative and qualitative data. We assess the perceived quality of companion apps by quantitatively analyzing the star rating and the sentiment of 1,347,799 Android and 48,498 iOS user reviews. We identify the prominent issues that afflict companion apps by performing a qualitative manual analysis of 1,000 sampled reviews. Our analysis shows that users’ judgment has not improved over the years. A variety of functional and non-functional issues persist, such as difficulties in pairing with the device, software flakiness, poor user interfaces, and presence of issues of a socio-technical impact. Our study highlights several aspects of companion apps that require improvement in order to meet user expectations and identifies future directions.     

Interactive faceted search and exploration of open social media data on a touchscreen mobile phone

A central challenge of semantic ambient media applications is designing smart user interfaces that are able to dynamically deal with an a-priori unknown number of data categories and data instances received live from different Linked Open Data sources while at the same time being intuitive and easy to use. In the mobile world, this challenge is even more difficult as the mobile devices have limited interaction possibilities and smaller display size. In this paper, we tackle this challenge and present the user-centered, iterative design of a mobile application for faceted search and exploration of a large, multi-dimensional data set of open social media on a touchscreen mobile phone. The application is called Mobile Facets and provides live retrieval and interactive search and exploration of resources like places, persons, organizations, and events originating from different, integrated social media sources like DBpedia, Eventful, Upcoming, Flickr, and GeoNames. In contrast to existing work, we do not know in advance the number and type of data categories and data instances that will be received as the data is queried live from the sources. While developing Mobile Facets, we have applied a participatory design with a small group of five users. For the final prototype we have conducted a task-based, formative evaluation with 12 additional subjects to investigate the applicability and usability of our Mobile Facets application.     

Puzzle: A mobile application development environment using a jigsaw metaphor

ObjectiveCreate a visual mobile end user development framework, named Puzzle, which allows end users without IT background to create, modify and execute applications, and provides support for interaction with smart things, phone functions and web services.MethodsDesign of an intuitive visual metaphor and associated interaction techniques for supporting end user development in mobile devices with iterative empirical validation.ResultsOur results show that the jigsaw is an intuitive metaphor for development in a mobile environment and our interaction techniques required a limited cognitive effort to use and learn the framework. Integration of different modalities and usage of smart things was relevant for users.ConclusionPuzzle has addressed the main objective. The framework further contributes to the research on mobile end user development in order to create an incentive for users to go beyond consuming content and applications to start creating their own applications.PracticeUsage of a mobile end user development environment has the potential to create a shift from the conventional few-to-many distribution model of software to a many-to-many distribution model. Users will be able to create applications that fit their requirements and share their achievements with peers.ImplicationsThis study has indicated that the Puzzle visual environment has the potential to enable users to easily create applications and stimulate exploration of innovative scenarios through smartphones.     

Comparison of Cognitive Modeling and User Performance Analysis for Touch Screen Mobile Interface Design

The aim of this study is to analyze and comparatively evaluate the usability of touch screen mobile applications through cognitive modeling and end-user usability testing methodologies. The study investigates the accuracy of the estimated results of a cognitive model produced for touch screen mobile phone interfaces. A mobile wallet application was chosen as the mobile software. The CogTool modeling tool was used as the cognitive modeling method. Eight tasks were determined and user tests were conducted in a usability laboratory with 10 participants. The tasks were compared on the basis of step time and total task completion time. This study reveals that CogTool gives approximate estimations with actual user performance on touch screen mobile phone application interfaces. However, if there are special cases in the tasks such that users are very accustomed to the steps or decision-making that is involved in the tasks, the “Think Operation” in CogTool should be modified.     

The influence of user expertise and phone complexity on performance,ease of use and learnability of different mobile phones

The study focuses on usability, ease of use and learnability of three different mobile phones (Nokia 3210, Siemens C35i, Motorola P7389). The first independent variable refers to the complexity of the menu (depth/breadth of the menu tree) and navigation keys (number/functionality). The Nokia phone had the lowest and the Motorola the highest complexity, with the Siemens phone ranging between them. The second independent variable was user expertise: 30 novices and 30 experts solved six telephone tasks. In order to assess effects of learnability, tasks were presented twice. Differences between the mobile phones regarding effectiveness, efficiency and learnability were found: The best performance was shown by Nokia users. The remaining two phones did not differ significantly, although the most complex phone was superior to the phone of medium complexity which had the lowest performance. Moreover, an effect of expertise was confirmed, though suboptimal interfaces were identified as lessening the advantage of expertise. Specific weaknesses of the tested phones are discussed.