Usability testing in 2000 and beyond

Usability testing is a widely used technique to evaluate user performance and acceptance of products and systems. It was introduced in the late 1980s and rose to popularity in the past decade. This paper provides a view of the current status of usability testing as a method and describes how it will be used in the 21st century. Although usability testing may not be the most efficient technique for discovery of usability problems, it is a reliable way to estimate quantitatively users' performance and subjective satisfaction with products. Four major trends in usability testing include: common reporting formats and methods for industry; Internet application and website testing; testing of mobile, handheld devices; and testing in more naturalistic environments such as simulated homes and classrooms. In the 21st century, ‘quick and clean’ usability testing methods are needed to provide valid and reliable data on how well people use products and systems, and how they like using them.     

Usability testing in a competive market: lessons learned

Abstract

Usability testing is a relatively new and rapidly developing field. Newcomers to usability testing typically enter the profession with a knowledge of social science research methods and the belief that usability testing is conducted in a lab using thinking aloud techniques to identify usability defects. Our practice of usability testing at Mead Data Central has shown that these core beliefs represent an approach of limited utility in designing products like the LEX1S®NEXIS® research systems. In this paper, we describe our experiences in conducting traditional usability testing and how we used the results of those efforts to develop more effective methods of testing for Mead Data Central's products and customers.     

Usability Testing and Instruction Librarians

ABSTRACT

This study examines how librarians are experiencing usability testing and how their observations are influencing library instruction. A survey of instruction librarians illustrates how usability testing and library instruction are connected. Survey results prove that instruction librarians are involved in usability testing. Furthermore, their participation in usability studies has led instruction librarians to alter their instructional methods. An overwhelming majority changed one or more instructional tools as a result of usability testing, and many reported creating new instructional resources. The authors add their own insights as both instruction librarians and participants in usability testing.     

The effects of prototype medium on usability testing

Inconsistencies among testing methods and results in previous research prompted this study that builds upon a systematic usability testing research framework to better understand how interface medium influences users' abilities to detect usability flaws in applications. Interface medium was tested to identify its effects on users' perceptions of usability and abilities to detect usability problems and severe usability problems. Results indicated that medium has no effect on users' abilities to detect usability problems or perceptions of usability. However, results did indicate an interaction between the medium and the tested application in which users were able to identify significantly more usability problems on a higher fidelity medium using a particular application. Results also indicated that as users' perceptions of an application's usability increases, the users are less able to detect usability problems in that application. Usability testing should begin early in the design process, even if low fidelity mediums will be used.     

The Impact of Personality on the Subjective Assessment of Usability

The system usability scale (SUS) has been widely employed in both the field and the laboratory as a valid and reliable measure of system usability. Although its psychometric properties are relatively well understood, the impact that differences in users’ personality traits have on their perceived usability of products, services, and systems has not been deeply explored—even though people’s scores on personality traits have been shown to be reliable and predict a staggering array of societally important outcomes in work, school, and life domains. In this study, 268 users assessed the usability of 20 different products retrospectively with the SUS. Five broad personality traits were then measured using the Mini-IPIP scale. Results indicated that measured personality traits do correlate with the rated usability of products, where measures of Openness to Experience and Agreeableness have the strongest positive correlations with subjective usability assessment. Implications for practitioners, designers, and researchers are discussed.     

Game-based Learning and 21st century skills: A review of recent research

Game-based learning and 21st century skills have been gaining an enormous amount of attention from researchers and practitioners. Given numerous studies support the positive effects of games on learning, a growing number of researchers are committed to developing educational games to promote students’ 21st century skill development in schools. However, little is known regarding how games may influence student acquisition of 21st century skills. This paper examines the most recent literature in regard to game-based learning and identified 29 studies which targeted 21st century skills as outcomes. The range of game genres and game design elements as well as learning theories used in these studies are discussed, together with the range of indicators, measures and outcomes for impacts on 21st century skills. The findings suggest that a game-based learning approach might be effective in facilitating students’ 21st century skill development. The paper also provides valuable insights for researchers, game designers, and educators in issues related to educational game design and implementation in general.     

Role of trust in networked production systems

In the 21st century the networked production systems (NPS) are typical in the globalised world. These systems are based on the integration of ICT and new organisational structures. In spite of the abilities of high-tech systems and devices human beings play an important role during the co-operation of the different production units. The cultural differences, the application of computer systems put trust into focus as an important factor of their efficiency and reliable operation. The paper gives an overview how a sufficient level of trust can develop with security mechanism during the different life cycle phases of networked production systems.     

Usability problem identification in culturally diverse settings

There are indications that established methods for evaluating information system usability that have been developed for use in, e.g. Europe or the USA, fail to give reliable results in countries such as India, China or Malaysia. This paper presents the theoretical background, related work and a definition of culture that should be useful for studies of multiple‐country usability testing. This includes a discussion of cultural fit and the consequences of cultural (in)consistencies between stakeholders in system development and use. As an illustrative example of the kind of academic research that needs to be done, a pilot study is described. The pilot study exemplifies themes to explore, who should be participants and where should the study be done, how to find examples of multiple‐country usability testing, how to collect data and how to analyse that data and what kind of results and discussion of results that may be expected. The conclusion summarises the paper, discusses the limitations of the pilot study and gives detailed suggestions to future research.     

Metacognition and system usability: Incorporating metacognitive research paradigm into usability testing

There is an agreement that perceived usability is important beyond actual effectiveness of software systems. Perceived usability is often obtained by self-reports provided after system use. Aiming to improve summative usability testing, we propose a methodology to enhance in-depth testing of users' performance and perceived usability at the task level. The metacognitive research approach allows detailed analysis of cognitive processes. Adapting its methodologies, we propose the Metacognitive Usability Profile (MUP) which includes a comprehensive set of measures based on collecting confidence in the success of each particular task and triangulating it with objective measures. We demonstrate using the MUP by comparing two versions of a project management system. Based on a task analysis we allocated tasks that differ between the versions and let participants (N = 100) use both versions. Although no difference was found between the versions in system-level perceived usability, the detailed task-level analysis exposed many differences. In particular, overconfidence was associated with low performance, which suggests that user interfaces better avoid illusions of knowing. Overall, the study demonstrates how the MUP exposes challenges users face. This, in turn, allows choosing the better task implementation among the examined options and to focus attempts for usability improvement.     

国外海洋油气资源的遥感技术探测

论述了当前国外采用的各种先进的海洋油气资源遥感探测技术及海洋油气遥感影像异常的地质综合分析解译方法,总结了海洋油气藏的基本特征及用遥感技术探测海洋油气的可行性,指出遥感技术用于海洋油气资源探查是今后发展的一个重要方向。     

Banking 2000? Reengineering at the First National Bank of Southern Africa to create a branch of the future

This case study describes how a major bank reorganised, restructured and retrained a branch in order to solve an operational and financial problem. It applied well-trusted information technology in an innovative way to improve client throughput, increase turnover and raise staff productivity. This was undertaken to simultaneously show the way to the rest of the organisation as to how retail banking should be conducted in the 21st century. The branch of the future showed how reengineering could be used to substantially improve a branch's performance while increasing its attractiveness to the bank's clients. This new approach to retail banking is referred to by the authors as Banking 2000? or the branch of the future.     

An alternative structure for next generation regulatory controllers. Part II: Stability analysis,tuning rules and experimental validation

We recently developed, as an alternative to the PID controller, a novel 4-mode control scheme that takes full advantage of modern electronic hardware components, and whose tuning parameters are directly related to controller performance attributes (robustness, set-point tracking, and disturbance rejection); its achievable performance is better than that of the PID controller, and it can be designed and implemented much more directly and transparently. In this companion paper we present robust stability results for the proposed controller, for any given plant/model mismatch; these results are then used to generate simple tuning rules for the controller. The effect of noise on controller performance is explicitly considered and modified tuning rules are proposed for excessively noisy processes. The controller tuning results and procedure are then illustrated via experimental testing and validation. First, water level control in a simple laboratory scale process is used to illustrate the design, tuning, and implementation of the RTDA controller. We use this process primarily to evaluate the controller performance and illustrate how the tuning parameters influence, directly and independently, the controller performance attributes. The controller is then implemented on a pilot-scale physical vapor deposition process to demonstrate its performance on a more complicated process that is prototypical of 21st century manufacturing. We demonstrate that the proposed controller achieves improved performance with minimal tuning effort.     

A decision support system for usability evaluation of web-based information systems

In this study, a decision support system (DSS) for usability assessment and design of web-based information systems (WIS) is proposed. It employs three machine learning methods (support vector machines, neural networks, and decision trees) and a statistical technique (multiple linear regression) to reveal the underlying relationships between the overall WIS usability and its determinative factors. A sensitivity analysis on the predictive models is performed and a new metric, criticality index, is devised to identify the importance ranking of the determinative factors. Checklist items with the highest and the lowest contribution to the usability performance of the WIS are specified by means of the criticality index. The most important usability problems for the WIS are determined with the help of a pseudo-Pareto analysis. A case study through a student information system at Fatih University is carried out to validate the proposed DSS. The proposed DSS can be used to decide which usability problems to focus on so as to improve the usability and quality of WIS.     

一种可行的可用性测试过程

文章综述用户模型法、用户调查法、专家评审法和观察法4类可用性测试方法,根据可用性测试的经验,总结出一套合理实用的可用性测试过程,使得可用性测试的效率高、代价低,尽可能发现更多的可用性问题;提出在程序设计类课程的教学中,应适当向学生介绍可用性测试的方法和过程,以培养学生对自己开发的软件系统进行可用性测试的能力。     

浅析生态设计理念

进入21世纪,人类面临着严峻的生态环境问题,使得生态设计理念成为了研究焦点。产品生态设计是利用生态学的思想,将环境因素融入到产品设计中,优先考虑产品的环境属性,改善产品在整个生命周期内的环境性能,实现从源头上预防污染的目的。其理论基础是生命周期评价理论。所以,也叫做生命周期设计。     

Usability measurement in context

Abstract

Different approaches to the measurement of usability are reviewed and related to definitions of usability in international standards. It is concluded that reliable measures of overall usability can only be obtained by assessing the effectiveness, efficiency and satisfaction with which representative users carry out representative tasks in representative environments.This requires a detailed understanding of the context of use of a product. The ESPRIT MUSiC project has developed tools which can be used to measure usability in the laboratory and the field. An overview is given of the methods and tools for measuring user performance, cognitive workload, and user perceived quality.     

The usability engineering life cycle

A practical usability engineering process that can be incorporated into the software product development process to ensure the usability of interactive computer products is presented. It is shown that the most basic elements in the usability engineering model are empirical user testing and prototyping, combined with iterative design. Usability activities are presented for three main phases of a software project: before, during, and after product design and implementation. Some of the recommended methods are not really single steps but should be used throughout the development process     

关于复杂系统研究的发展情况

复杂系统涉及的范围非常广,20世纪80年代到21世纪初关于复杂系统的研究得到很大程度的发展,成为了当今科学界主要研究方向之一。复杂系统虽然还没有统一的定义,但可以通过几个基本方面,在某种程度上进行描述。研究复杂系统的相关理论和方法主要包括非线性动力学和混沌理论、Agent和基于Agent的建模、网络等。复杂系统的研究目前还依然是探索性的,但随着科技工作者的努力,复杂系统在社会发展中必将起到十分重要的作用。     

Usability Principles for Augmented Reality Applications in a Smartphone Environment

Through the rapid spread of smartphones, users have access to many types of applications similar to those on desktop computer systems. Smartphone applications using augmented reality (AR) technology make use of users' location information. As AR applications will require new evaluation methods, improved usability and user convenience should be developed. The purpose of the current study is to develop usability principles for the development and evaluation of smartphone applications using AR technology. We develop usability principles for smartphone AR applications by analyzing existing research about heuristic evaluation methods, design principles for AR systems, guidelines for handheld mobile device interfaces, and usability principles for the tangible user interface. We conducted a heuristic evaluation for three popularly used smartphone AR applications to identify usability problems. We suggested new design guidelines to solve the identified problems. Then, we developed an improved AR application prototype of an Android-based smartphone, which later was conducted a usability testing to validate the effects of usability principles.