A Usability Checklist for the Usability Evaluation of Mobile Phone User Interface

In the last decade, the research of the usability of mobile phones has been a newly evolving area with few established methodologies and realistic practices that ensure capturing usability in evaluation. Thus, there exists growing demand to explore appropriate evaluation methodologies that evaluate the usability of mobile phones quickly as well as comprehensively. This study aims to develop a task-based usability checklist based on heuristic evaluations in views of mobile phone user interface (UI) practitioners. A hierarchical structure of UI design elements and usability principles related to mobile phones were developed and then utilized to develop the checklist. To demonstrate the practical effectiveness of the proposed checklist, comparative experiments were conducted on the usability checklist and usability testing. The majority of usability problems found by usability testing and additional problems were discovered by the proposed checklist. It is expected that the usability checklist proposed in this study could be used quickly and efficiently by usability practitioners to evaluate the mobile phone UI in the middle of the mobile phone development process.     

Evaluation of user interfaces: EVADIS II— a comprehensive evaluation approach

As a result of the importance of the usability approach in system development and the EC's 'Directive concerning the minimum safety and health requirements for VDT workers' (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, 'How can usability requirements comprehensively be considered and evaluated in system development?' Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?' Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.     

Internationalization and localization: evaluating and testing a Website for Asian users

The objective of this study was to combine internationalization and localization of Websites and improvement of Website usability with user-centred design methods. This study designed for internationalization and localization of Websites for Asian users, and implemented usability engineering into every phase of Website usability testing, based on the internationalization and localization perspectives of the honeywell.com/your home Website. The first step was to develop the usage scenarios. Three Asian usability specialists carried out one heuristic evaluation session for the current honeywell.com/your home Website. The usability problems were analysed and possible solutions to these problems were discussed. In the next phase, cluster analysis was utilized to test current information architecture. The results provided options for future information architecture development for this Website. Finally, a performance measurement test was conducted to investigate the performance for Asian users. Based on the results, suggestions for improving the Website usability from the localization perspective were provided. The results demonstrate the user-centred design (UCD) approach and stress international and local issues in Website development to Website designers.     

Internationalization and localization: evaluating and testing a Website for Asian users

The objective of this study was to combine internationalization and localization of Websites and improvement of Website usability with user-centred design methods. This study designed for internationalization and localization of Websites for Asian users, and implemented usability engineering into every phase of Website usability testing, based on the internationalization and localization perspectives of the honeywell.com/your home Website. The first step was to develop the usage scenarios. Three Asian usability specialists carried out one heuristic evaluation session for the current honeywell.com/your home Website. The usability problems were analysed and possible solutions to these problems were discussed. In the next phase, cluster analysis was utilized to test current information architecture. The results provided options for future information architecture development for this Website. Finally, a performance measurement test was conducted to investigate the performance for Asian users. Based on the results, suggestions for improving the Website usability from the localization perspective were provided. The results demonstrate the user-centred design (UCD) approach and stress international and local issues in Website development to Website designers.     

Evaluation of user interfaces: EVADIS II— a comprehensive evaluation approach

Abstract

As a result of the importance of the usability approach in system development and the EC's ‘Directive concerning the minimum safety and health requirements for VDT workers’ (EWG 1990), there is an accepted need for practical evaluation methods for user interfaces. The usability approach and the EC Directive are not restricted to user interface design, as they include the design of appropriate hardware and software, as well as organization, job, and task design. Therefore system designers are faced with many, often conflicting, requirements and need to address the question, ‘How can usability requirements comprehensively be considered and evaluated in system development?’ Customers buying hardware and software and introducing them into their organization ask, (How can I select easy-to-use hardware and software?’ Both designers and customers need an evaluation procedure that covers all the organizational, user, hard- and software requirements. The evaluation method, EVADIS.II, we present in this paper overcomes characteristic deficiencies of previous evaluation methods. In particular, it takes the tasks, the user, and the organizational context into consideration during the evaluation process, and provides computer support for the use of the evaluation procedure.     

A heuristic checklist for an accessible smartphone interface design

Smartphone technology has evolved into a multi-functional device with advanced capabilities, but this mobile technology remains inaccessible to many individuals with visual impairments or upper extremity disabilities. This paper provides a heuristic checklist for accessible smartphone interface design, developed through reviewing existing design standards and guidelines and validating these guidelines with user involvement. Specifically, a set of preliminary user requirements (59 items) was extracted from existing standards, guidelines, and user requirements regarding mobile handheld device accessibility. Subsequently, the requirement set was filtered using a participatory method and then integrated to create an operational version of design guidelines. These guidelines were then used in a heuristic evaluation and usability testing on high-fidelity prototypes produced by a commercial manufacturer. A heuristic checklist for designing accessible smartphones was formed, which may also be applicable to other touchscreen handheld devices (e.g., printer screen) in terms of accessibility features. The initial set of 59 user requirements was re-organized into 44 statements in six general categories: mechanical controls, display, speech and general operation controls, audio feedback controls, touch-operated controls, and others. Using results from both qualitative and quantitative methods provides support, though with some limitations, for this accessibility checklist. This checklist is intended as a practical design support tool for use in early design phases of handheld products. A number of challenges and limitations are discussed as well.     

A methodology for evaluating the usability of audiovisual consumer electronic products

Usability evaluation is now considered an essential procedure in consumer product development. Many studies have been conducted to develop various techniques and methods of usability evaluation hoping to help the evaluators choose appropriate methods. However, planning and conducting usability evaluation requires considerations of a number of factors surrounding the evaluation process including the product, user, activity, and environmental characteristics. In this perspective, this study suggested a new methodology of usability evaluation through a simple, structured framework. The framework was outlined by three major components: the interface features of a product as design variables, the evaluation context consisting of user, product, activity, and environment as context variables, and the usability measures as dependent variables. Based on this framework, this study established methods to specify the product interface features, to define evaluation context, and to measure usability. The effectiveness of this methodology was demonstrated through case studies in which the usability of audiovisual products was evaluated by using the methods developed in this study. This study is expected to help the usability practitioners in consumer electronics industry in various ways. Most directly, it supports the evaluators' plan and conduct usability evaluation sessions in a systematic and structured manner. In addition, it can be applied to other categories of consumer products (such as appliances, automobiles, communication devices, etc.) with minor modifications as necessary.     

IBM computer usability satisfaction questionnaires: Psychometric evaluation and instructions for use

This article describes recent research in subjective usability measurement at IBM, focused on evaluating the psychometric properties of questionnaires designed for use in scenario‐based usability evaluation. The questionnaires address evaluation at both a global overall system level and at a more detailed scenario level. The primary goals of this article are to (a) discuss the psychometric characteristics of IBM questionnaires that measure user satisfaction with computer system usability, and (b) provide the questionnaires, with administration and scoring instructions. For scenario‐level measurement, the 3‐item After‐Scenario Questionnaire (ASQ) has excellent internal consistency, with coefficient alphas across a set of scenarios ranging from .90 to .96. For more global assessment, the Post‐Study System Usability Questionnaire (PSSUQ) also has excellent internal consistency, with an overall coefficient alpha of .97. Preliminary principal factor analysis of 48 PSSUQ questionnaires suggested the presence of three factors named, after varimax rotation, System Usefulness, Information Quality, and Interface Quality, with corresponding coefficient alphas of .96, .91, and .91. Evaluation of 377 PSSUQ questionnaires (modified to allow mailing to respondents in their offices and referred to as the Computer System Usability Questionnaire, or CSUQ) confirmed the structure of the preliminary principal factor analysis. Consequently, usability practitioners can use these questionnaires to help them measure users’ satisfaction with the usability of computer systems in the context of scenario‐based usability studies.     

A systematic framework for evaluating design concepts of a new product

This study aims to suggest a systematic framework to evaluate design concepts for a new product at the concept‐development phase. It focuses especially on evaluating design concepts based on user requirements and implicit tasks, defining trends in technology alternatives, and relating users' perceived value to product functionality. The potential user needs and functional requirements were identified through scenario‐based analysis and hierarchical task analysis. Technology alternatives were also investigated to support users in performing the required tasks effectively and efficiently. For a quantifiable evaluation measure, customer‐perceived value (CPV) attributes were used to evaluate the benefits and costs of the current design concept as compared to perceived alternatives. A case study was conducted to evaluate the design concepts of a new computer‐supported cooperative work (CSCW)‐based system with a tangible user interface, which was designed to support group decision‐making activities, such as business meetings. At the concept‐development phase, engineering specifications are not determined, and cost cannot be precisely estimated. Thus, while we dealt with design‐concept evaluation, we had no choice but to exclude cost attributes, such as monetary expenditure. It is still expected that our framework would be effective in incorporating user‐centered design perspectives early in the process of new product development. © 2010 Wiley Periodicals, Inc.     

An interdisciplinary heuristic evaluation method for universal building design

This study highlights how heuristic evaluation as a usability evaluation method can feed into current building design practice to conform to universal design principles. It provides a definition of universal usability that is applicable to an architectural design context. It takes the seven universal design principles as a set of heuristics and applies an iterative sequence of heuristic evaluation in a shopping mall, aiming to achieve a cost-effective evaluation process. The evaluation was composed of three consecutive sessions. First, five evaluators from different professions were interviewed regarding the construction drawings in terms of universal design principles. Then, each evaluator was asked to perform the predefined task scenarios. In subsequent interviews, the evaluators were asked to re-analyze the construction drawings. The results showed that heuristic evaluation could successfully integrate universal usability into current building design practice in two ways: (i) it promoted an iterative evaluation process combined with multi-sessions rather than relying on one evaluator and on one evaluation session to find the maximum number of usability problems, and (ii) it highlighted the necessity of an interdisciplinary ad hoc committee regarding the heuristic abilities of each profession. A multi-session and interdisciplinary heuristic evaluation method can save both the project budget and the required time, while ensuring a reduced error rate for the universal usage of the built environments.     

基于用户体验的多关节主被动训练仪可用性研究

为提高多关节主被动训练仪的可用性水平,以用户体验理论为基础构建了以用户为触点的多关节主被动训练仪可用性评价指标体系,运用模糊层次分析法(FAHP)和隶属度定量化处理的方法构建多关节主被动训练仪可用性评价模型.以某公司生产的一款多关节主被动训练仪为例,先由专家测评确定评价指标并计算出各指标因素的权重值,再由评分小组操作该...     

A service scenario generation scheme based on association rule mining for elderly surveillance system in a smart home environment

More convenient smart home environments can be constructed by monitoring home appliances, if automated services are supported by their usage information. This paper proposes a scheme for translating association rules among appliances mined from their usage information into service scenarios. A smart home environment is unique in that there exist a limited number of home appliances, some of which operate without interruption like a refrigerator. Furthermore, the number of home appliances is much less than the number of items in itemsets to which existing algorithms for mining association rules have been applied. After showing that the existing algorithms are limited in improving the usefulness of association rule generation by means of adjusting the confidence level due to such unique characteristics, we propose a new service scenario generation scheme which calculates the confidence level based on hypothesis testing. This paper demonstrates that association rule mining based on the dependence between appliances is feasible and its performance is very much comparable with that of an existing association rule mining algorithm. Since home users are allowed to choose their favorable scenarios from meaningful service scenarios generated by the proposed scheme without their intervention, we expect that future smart homes can provide automated services more efficiently to them, especially to surveillance systems for elderly.     

Training software developers and designers to conduct usability evaluations

Many efforts to improve the interplay between usability evaluation and software development rely either on better methods for conducting usability evaluations or on better formats for presenting evaluation results in ways that are useful for software designers and developers. Both of these approaches depend on a complete division of work between developers and evaluators. This article takes a different approach by exploring whether software developers and designers can be trained to conduct their own usability evaluations. The article is based on an empirical study where 36 teams with a total of 234 first-year university students on software development and design educations were trained through an introductory course in user-based website usability testing that was taught in 40 h. They used the techniques from this course for planning, conducting, and interpreting the results of a usability evaluation of an interactive website. They gained good competence in conducting the evaluation, defining user tasks and producing a usability report, while they were less successful in acquiring skills for identifying and describing usability problems.     

Towards a practical method of user interface evaluation

This paper describes a practical method for evaluating the usability of human-computer interfaces. The paper specifies the requirements of such a method, and then outlines our work in developing a method to meet this specification. The method is based on the conduct of realistic tasks with an interactive system and the subsequent systematic elicitation of end-users' and designers' reactions to the interface using a criterion-based evaluation checklist.

Two practical examples are used to illustrate development of the method: (a) evaluation of a prototype production scheduling system, and (b) comparative assessment of the usability of three prototype user interfaces to a public-access database. The paper discusses some issues raised by the method and considers how it can be furthe developed.     

A Usable Security Heuristic Evaluation for the Online Health Social Networking Paradigm

The security, privacy, and usability of health social networking websites remain key concerns for their successful adoption by patients. As far as it is known, no studies have developed a usable security heuristic evaluation to test these websites for the usability of their security and privacy features. This article describes the development of such a usable security heuristic evaluation in the context of the online health social networking paradigm. The tailored method approach was applied to transform themes identified from relevant literature and convert them into high-level heuristics for usable security. Checklist items were created for each high-level heuristic. The result is 13 high-level heuristics with individualized checklist items that help examine usable security. The results can identify which security and privacy features are lacking from a usability perspective and would, therefore, require design improvement. The improvements would enhance the users' experiences, which in turn can assist in the adoption of health social networking websites.     

基于投影的软件易用性评价方法

随着IT行业的不断发展,软件的易用性越来越受到用户和专家的关注与重视。如何科学地量化软件产品的易用性,进而对产品质量进行综合地测评,决定了软件产品的竞争力和用户的满意度。针对软件的易用性,提出了一种基于投影和直觉模糊理论的评价方法。首先,依据用户调查得到的信息,给出评价软件易用性的直觉模糊评价矩阵,并在此基础上建立了评价矩阵的理想决策。其次,借助群决策的框架,给出评价矩阵到理想决策上的投影。然后,基于投影给出测评软件的优劣序。最后,用一个测评实例展示出本方法的有效性和实用性。结果表明,本文提出的模型是一种软件质量综合评价的有效方法。     

A comparison of usability evaluation methods for evaluating e-commerce websites

The importance of evaluating the usability of e-commerce websites is well recognised. User testing and heuristic evaluation methods are commonly used to evaluate the usability of such sites, but just how effective are these for identifying specific problems? This article describes an evaluation of these methods by comparing the number, severity and type of usability problems identified by each one. The cost of employing these methods is also considered. The findings highlight the number and severity level of 44 specific usability problem areas which were uniquely identified by either user testing or heuristic evaluation methods, common problems that were identified by both methods, and problems that were missed by each method. The results show that user testing uniquely identified major problems related to four specific areas and minor problems related to one area. Conversely, the heuristic evaluation uniquely identified minor problems in eight specific areas and major problems in three areas.     

A Heuristic Evaluation for Deaf Web User Experience (HE4DWUX)

Websites that are usable and accessible can have a positive impact on the overall user experience. Usability Inspection Methods (UIMs) can be applied to evaluate and measure the usability. The current research in the fields of Web Accessibility and Human –Computer Interaction (HCI) is in need of additional UIMs that can be applied to also measure the accessibility, in addition to the usability alone. In this article, a novel UIM in the form of a heuristic evaluation is presented. The heuristic evaluation aims to support HCI experts and Web developers in designing and evaluating websites that provide positive user experiences to users who are deaf. This article discusses the development of the Heuristic Evaluation for Deaf Web User Experience (HE4DWUX). Following an iteration cycle, version 2 of the HE4DWUX is presented in Appendix A. An existing three-phase process to develop heuristics for specific application domains was applied to construct the HE4DWUX. The outcome of this research is 12 heuristics, with each containing its own set of checklist items to operationalize its applicability in measuring the Web user experience for users who are deaf. The heuristics and their checklist items can identify important aspects of design that will impact the Web user experience for this particular user group.     

Reviewing the quality of awareness support in collaborative applications

Awareness to users is a valuable feature of a collaborative system. Therefore, the designers of a system of this type may find it useful to receive hints on the awareness support provided by the system when it is under development or evolution. This paper proposes a tool for their use to obtain suggestions on the awareness features provided by the system and those not currently supported by it. The considered kinds of awareness were obtained from a review of a significant number of proposals from the literature. The tool is based on a checklist of design elements related to these awareness types to be applied by the application designer. The construction of this checklist was done as follows. The process started with an analysis of the types of awareness to be provided. This step ended with 54 selected design elements and six awareness types. Experts on the development of collaborative systems used their experience to provide correlations between the design elements and the types of awareness previously identified, thus encapsulating their expertise within the checklist. The proposal was applied to three existing collaborative systems and the results are presented. The obtained results suggest that the checklist is adequate to provide helpful hints that may be used to improve an application's awareness support.     

A vision-oriented approach for innovative product design

Nowadays, business globalization, complex consumer requirements, and high technology development have led to vigorous business competition and market uncertainty. In new product design, it is increasingly important to put more emphasis on innovative concerns than quality and speed of production. Providing unique, innovative and value-added products becomes a crucial strategy for success. This study aims to develop a new product design process that concentrates on radical innovations. Visioning thinking, TRIZ, Image Scale, and Morphological Analysis are integrated for developing designs with a balance of rational/logical analyses and intuitive/creative thinking. The proposed process fulfills the need for radical innovation through enriching heuristic thinking. It is a useful methodology that adds value to products in terms of technological functions, form perceptions, and holistic usability. A proposal of therapy bike design for cerebral palsy children is demonstrated using the new process. Through case demonstration, designers could enhance products with human-centered values while coping with market uncertainty effectively. The applicability of the vision-oriented approach for innovative product design is verified.