Design of Mobile Phones for Older Adults: An Empirical Analysis of Design Guidelines and Checklists for Feature Phones and Smartphones

Design guidelines and checklists are suggested as a useful tool in the development and evaluation of interface design of mobile phones for older adults. Given the intense evolution of mobile phone design, understanding how the design guidelines and checklists have taken into account the advances in mobile phone usability for older adults is important for their correct application and future development. Thus, this study explores the usability dimensions of mobile phone design for older adults and the related changes in terms of time and the type of device (feature phones vs. smartphones) based on an expert coding of the eight mobile phone design guidelines and checklists for older adults published between 2006 and 2014. The results of the expert coding show that design guidelines and checklists most frequently deal with visual and haptic issues (e.g., high contrast, button type, and button size), whilst they hardly ever address various elements of textual interface (e.g., ease of text entry, a button’s feedback, and font type). Over time, the design guidelines and checklists have become more complex in terms of the average number of included usability categories and dimensions. For smartphones, the guidelines, on average, put more emphasis on the screen, touchscreen, text, and exterior related issues, whereas the design guidelines for feature phones stress the usability of the keypad and menus. Besides revealing potential usability dimensions that could be further expanded in the guidelines, this study also highlights the need for research that would empirically validate the design guidelines and checklists in the future.     

The user action framework: a reliable foundation for usability engineering support tools

Although various methods exist for performing usability evaluation, they lack a systematic framework for guiding and structuring the assessment and reporting activities. Consequently, analysis and reporting of usability data are ad hoc and do not live up to their potential in cost effectiveness, and usability engineering support tools are not well integrated. We developed the User Action Framework, a structured knowledge base of usability concepts and issues, as a framework on which to build a broad suite of usability engineering support tools. The User Action Framework helps to guide the development of each tool and to integrate the set of tools in the practitioner's working environment. An important characteristic of the User Action Framework is its own reliability in term of consistent use by practitioners. Consistent understanding and reporting of the underlying causes of usability problems are requirements for cost-effective analysis and redesign. Thus, high reliability in terms of agreement by users on what the User Action Framework means and how it is used is essential for its role as a common foundation for the tools. Here we describe how we achieved high reliability in the User Action Framework, and we support the claim with strongly positive results of a summative reliability study conducted to measure agreement among 10 usability experts in classifying 15 different usability problems. Reliability data from the User Action Framework are also compared to data collected from nine of the same usability experts using a classic heuristic evaluation technique.     

Analytic review of usability evaluation in ISMAR

There has been a rapid increase in research evaluating usability of Augmented Reality (AR) systems in recent years. Although many different styles of evaluation are used, there is no clear consensus on the most relevant approaches. We report a review of papers published in International Symposium of Mixed and Augmented Reality (ISMAR) proceedings in the past decade, building on the previous work of Swan and Gabbard (2005). Firstly, we investigate the evaluation goal, measurement and method of ISMAR papers according to their usability research in four categories: performance, perception and cognition, collaboration and User Experience (UX). Secondly, we consider the balance of evaluation approaches with regard to empirical–analytical, quantitative–qualitative and participant demographics. Finally we identify potential emphases for usability study of AR systems in the future. These analyses provide a reference point for current evaluation techniques, trends and challenges, which benefit researchers intending to design, conduct and interpret usability evaluations for future AR systems.     

A systematic mapping study of API usability evaluation methods

An Application Programming Interface (API) provides a programmatic interface to a software component that is often offered publicly and may be used by programmers who are not the API’s original designers. APIs play a key role in software reuse. By reusing high quality components and services, developers can increase their productivity and avoid costly defects. The usability of an API is a qualitative characteristic that evaluates how easy it is to use an API. Recent years have seen a considerable increase in research efforts aiming at evaluating the usability of APIs. An API usability evaluation can identify problem areas and provide recommendations for improving the API. In this systematic mapping study, we focus on 47 primary studies to identify the aim and the method of the API usability studies. We investigate which API usability factors are evaluated, at which phases of API development is the usability of API evaluated and what are the current limitations and open issues in API usability evaluation. We believe that the results of this literature review would be useful for both researchers and industry practitioners interested in investigating the usability of API and new API usability evaluation methods.     

Evaluating the usability of a mobile guide: the influence of location, participants and resources

When designing a usability evaluation, choices must be made regarding methods and techniques for data collection and analysis. Mobile guides raise new concerns and challenges to established usability evaluation approaches. Not only are they typically closely related to objects and activities in the user's immediate surroundings, they are often used while the user is ambulating. This paper presents results from an extensive, multi-method evaluation of a mobile guide designed to support the use of public transport in Melbourne, Australia. In evaluating the guide, we applied four different techniques; field-evaluation, laboratory evaluation, heuristic walkthrough and rapid reflection. This paper describes these four approaches and their respective outcomes, and discusses their relative strengths and weaknesses for evaluating the usability of mobile guides.     

A Usability Checklist for the Usability Evaluation of Mobile Phone User Interface

In the last decade, the research of the usability of mobile phones has been a newly evolving area with few established methodologies and realistic practices that ensure capturing usability in evaluation. Thus, there exists growing demand to explore appropriate evaluation methodologies that evaluate the usability of mobile phones quickly as well as comprehensively. This study aims to develop a task-based usability checklist based on heuristic evaluations in views of mobile phone user interface (UI) practitioners. A hierarchical structure of UI design elements and usability principles related to mobile phones were developed and then utilized to develop the checklist. To demonstrate the practical effectiveness of the proposed checklist, comparative experiments were conducted on the usability checklist and usability testing. The majority of usability problems found by usability testing and additional problems were discovered by the proposed checklist. It is expected that the usability checklist proposed in this study could be used quickly and efficiently by usability practitioners to evaluate the mobile phone UI in the middle of the mobile phone development process.     

Evaluating the usability of a mobile guide: The influence of location,participants and resources

When designing a usability evaluation, choices must be made regarding methods and techniques for data collection and analysis. Mobile guides raise new concerns and challenges to established usability evaluation approaches. Not only are they typically closely related to objects and activities in the user's immediate surroundings, they are often used while the user is ambulating. This paper presents results from an extensive, multi-method evaluation of a mobile guide designed to support the use of public transport in Melbourne, Australia. In evaluating the guide, we applied four different techniques; field-evaluation, laboratory evaluation, heuristic walkthrough and rapid reflection. This paper describes these four approaches and their respective outcomes, and discusses their relative strengths and weaknesses for evaluating the usability of mobile guides.     

Visualisation improves the usability of voice-operated mobile phone services

Three different versions of a voice-operated mobile city guide service with a hierarchically structured dialogue were evaluated in a mobile setting. One numbered menu style (standard) service, and two services which contained terms derived from underlying real-world referents, were implemented. The real-world referents (metaphors) used were: an office filing system and a computer desktop. It was hypothesized that the use of interface metaphors would allow more participants to visualize the service structure, leading to an improvement in performance relative to the standard service. Forty-two phone users undertook three different tasks with one of the three phone services. User performance and attitudes to the services were recorded, and post-task interviews were conducted. Results showed that significantly more participants using the metaphor-based services visualized the services. Visualizers performed significantly better than non-visualizers, with visualization emerging as a significant predictor of both attitude and performance. We argue that designing speech-based mobile phone services using an appropriate spatial metaphor leads to high levels of visualisation, which allows participants to orient themselves and to navigate more effectively within the hierarchical service architecture. The usability benefits afforded by visualization may become especially important when using phone services in cognitively demanding mobile settings.     

Usability risk level evaluation for physical user interface of mobile phone

The mobile phone has become an indispensable device in everyday life. However, many usability problems were generated by the multi-functionality of mobile phones. There was an important increase on the user interface (UI) design and usability. With the introduction of a variety of forms and keypads in mobile phones, the user's need of controllability, grip-stability, and usability were raised in the UI aspect. Therefore, in this study, we investigated the physical user interface (PUI) elements of mobile phones, and defined the degree of usability risk such as key level value, function level value, and grip level value. Also, this study proposed an evaluation framework to quantitatively measure the usability risk in the mobile phone PUI elements. For this, the mobile phone PUI elements were analyzed and classified by key type, function, and form-factor. Hence, in the case study was investigated and evaluated 133 mobile phones in the market by the proposed framework. As a result, the proposed evaluation framework enabled the measurement of the mobile phone usability risk level in the early stages of the concept design. This made possible the prediction of design problems related with the mobile phone PUI.     

The theoretical foundation and validity of a component-based usability questionnaire

Although software engineers extensively use a component-based software engineering (CBSE) approach, existing usability questionnaires only support a holistic evaluation approach, which focuses on the usability of the system as a whole. Therefore, this paper discusses a component-specific questionnaire for measuring the perceived ease-of-use of individual interaction components. A theoretical framework is presented for this compositional evaluation approach, which builds on Taylor's layered protocol theory. The application and validity of the component-specific measure is evaluated by re-examining the results of four experiments. Here, participants were asked to use the questionnaire to evaluate a total of nine interaction components used in a mobile phone, a room thermostat, a web-enabled TV set and a calculator. The applicability of the questionnaire is discussed in the setting of a new usability study of an MP3 player. The findings suggest that at least part of the perceived usability of a product can be evaluated on a component-based level.     

New techniques for usability evaluation of mobile systems

Usability evaluation of systems for mobile computers and devices is an emerging area of research. This paper presents and evaluates six techniques for evaluating the usability of mobile computer systems in laboratory settings. The purpose of these techniques is to facilitate systematic data collection in a controlled environment and support the identification of usability problems that are experienced in mobile use. The proposed techniques involve various aspects of physical motion combined with either needs for navigation in physical space or division of attention. The six techniques are evaluated through two usability experiments where walking in a pedestrian street was used as a reference. Each of the proposed techniques had some similarities to testing in the pedestrian street, but none of them turned out to be completely comparable to that form of field-evaluation. Seating the test subjects at a table supported identification of significantly more usability problems than any of the other proposed techniques. However a large number of the additional problems identified using this technique were categorized as cosmetic. When increasing the amount of physical activity, the test subjects also experienced a significantly increased subjective workload.     

移动系统可用性综合研究框架*

在研读国内外众多移动系统可用性的研究文献之后,从可用性维度、影响因素、测量方法和工具方面对移动系统可用性研究进行了系统概述,以此为基础提出了一个移动系统可用性的综合研究框架。     

基于云模型的老人手机易用性评估模型研究

目前,老人手机被老年群体广泛使用,但对老人手机易用性评估还尚缺,不利于老人手机研发。针对老人手机易用性评估的问题,提出基于云模型的老人手机易用性评估方法。构建老人手机易用性评估指标体系,并把熵权法和层次分析法相结合来确定各指标间相对权重,利用云模型对指标信息进行评估。实例分析表明该方法可以有效地处理老人手机易用性评估中的不确定度和模糊性,提高易用性评估结果的客观准确性,为老人手机在易用性设计方面提供理论依据。     

Determining the effectiveness of the usability problem inspector: a theory-based model and tool for finding usability problems

Despite the increased focus on usability and on the processes and methods used to increase usability, a substantial amount of software is unusable and poorly designed. Much of this is attributable to the lack of cost-effective usability evaluation tools that provide an interaction-based framework for identifying problems. We developed the user action framework and a corresponding evaluation tool, the usability problem inspector (UPI), to help organize usability concepts and issues into a knowledge base. We conducted a comprehensive comparison study to determine if our theory-based framework and tool could be effectively used to find important usability problems in an interface design, relative to two other established inspection methods (heuristic evaluation and cognitive walkthrough). Results showed that the UPI scored higher than heuristic evaluation in terms of thoroughness, validity, and effectiveness and was consistent with cognitive walkthrough for these same measures. We also discuss other potential advantages of the UPI over heuristic evaluation and cognitive walkthrough when applied in practice. Potential applications of this work include a cost-effective alternative or supplement to lab-based formative usability evaluation during any stage of development.     

Generating educational mobile applications using UIDPs identified by artificial intelligence techniques

Commercial mobile devices vary in brand, size, and functionalities, but they all allow people to interact with educational applications. In software engineering, application development techniques, approaches, methodologies, and processes (e.g., BBD, FDD, RAD, DDD) are often time consuming, costly, or aim at expert developers – which implies that users outside the software development field (e.g., teachers) need great practice to become experienced application developers. This work proposes an artificial-intelligence-based process for generating educational mobile apps from freehand-generated images. The images’ design is based on User Interface Design Pattern (UIDP) representations. As a proof of concept, we introduce EduMatic, an application development tool. To test our process, we assessed Wiki-Comp, an application built with EduMatic, along with three other external educational applications: Khan Academy, Wiki encyclopedia, and Kiwix. According to the evaluation results, Wiki-Comp outstands in functionality, usability, and performance aspects.     

Improving Web usability through visualization

One reason for poor Web site usability is researchers' lack of understanding about how users surf for information. Predictive Web usage visualizations can help analysts uncover traffic patterns and usability problems. The author describes some of the techniques those in the User Interface Research Group at PARC have developed to visualize content changes, linkage structures, site usage, and so on. He explores how these techniques can be used to identify specific usability problems at large Web sites. As part of his work, he have created a predictive visualization model called Information Scent, in which simulated users help uncover patterns and deficiencies in information accessibility. He demonstrates these techniques using a prototype system called ScentViz to illustrate how Web usage analysis can be enhanced using visualization methods     

A model-based framework for classifying and diagnosing usability problems

A great deal of study has been devoted to the problem of how to identify and categorize usability problems; however, there is still a lack of studies dealing with the problem of how to diagnose the causes of usability problems and how to feed them back into design process. The value of classifying usability problems can be enhanced when they are interpreted in connection with design process and activities. Thus, it is necessary to develop a systematic way of diagnosing usability problems in terms of design aspects and applying diagnosis results to improve design process and activities. With this issue in mind, this paper proposes a conceptual framework that supports a systematic classification and diagnosis of usability problems. This paper firstly reviews seven approaches to classifying usability problems. Then, we point out the needs of adopting a model-based approach to classifying and diagnosing usability problems and of developing a comprehensive framework guiding the use of model-based approaches. We then propose a conceptual framework that specifies how a model-based classification and diagnosis of usability problems should be conducted and suggests the combined use of three different types of models, each of which addresses context of use, design knowledge and design activities. Last, we explain how a sound classification scheme of usability problems can be systematically developed, and how the classification of usability problems can be connected to design process and activities on the basis of the framework.     

The influence of design aesthetics in usability testing: Effects on user performance and perceived usability

This article examined the effects of product aesthetics on several outcome variables in usability tests. Employing a computer simulation of a mobile phone, 60 adolescents (14-17 yrs) were asked to complete a number of typical tasks of mobile phone users. Two functionally identical mobile phones were manipulated with regard to their visual appearance (highly appealing vs not appealing) to determine the influence of appearance on perceived usability, performance measures and perceived attractiveness. The results showed that participants using the highly appealing phone rated their appliance as being more usable than participants operating the unappealing model. Furthermore, the visual appearance of the phone had a positive effect on performance, leading to reduced task completion times for the attractive model. The study discusses the implications for the use of adolescents in ergonomic research.     

A comparison of usability evaluation methods for evaluating e-commerce websites

The importance of evaluating the usability of e-commerce websites is well recognised. User testing and heuristic evaluation methods are commonly used to evaluate the usability of such sites, but just how effective are these for identifying specific problems? This article describes an evaluation of these methods by comparing the number, severity and type of usability problems identified by each one. The cost of employing these methods is also considered. The findings highlight the number and severity level of 44 specific usability problem areas which were uniquely identified by either user testing or heuristic evaluation methods, common problems that were identified by both methods, and problems that were missed by each method. The results show that user testing uniquely identified major problems related to four specific areas and minor problems related to one area. Conversely, the heuristic evaluation uniquely identified minor problems in eight specific areas and major problems in three areas.     

An open source usability maturity model (OS-UMM)

User satisfaction has always been a major factor in the success of software, regardless of whether it is closed proprietary or open source software (OSS). In open source projects, usability aspects cannot be improved unless there are ways to test and measure them. Hence, the increasing popularity of open source projects among novice and non-technical users necessitates a usability evaluation methodology. Consequently, this paper presents a usability maturity model specifically aimed at usability-related issues for open source projects. In particular, the model examines the degree of coordination between open source projects and their usability aspects. The measuring instrument of the model contains factors that have been selected from four of our empirical studies, which examine the perspectives of OSS users, developers, contributors and the industry. In addition to presenting the usability maturity model, this paper discusses assessment questionnaires, a rating methodology and two case studies.