A practical guide to using software usability labs: lessons learned at IBM

Abstract

Usability evaluation is a key component of a user-centred design process. Access to a usability laboratory can greatly facilitate the process of empirically measuring user performance, but the mere presence of a usability laboratory does not assure usable products. Rather, the laboratory must be used within an evaluation process. The process described in this article has five phases: designing the evaluation, preparing to conduct the evaluation, conducting the evaluation, analysing the data, and reporting the results. Lessons learned by the authors while they practised this evaluation process with a variety of products are summarized for possible use by other usability organizations.     

The influence of a knowledge-based system on designers' cognitive activities: a study involving professional web designers

Over the last few decades, several usability knowledge-based systems have been developed to provide user interface designers with usability knowledge (e.g. heuristics, usability guidelines, standards). Such systems are intended to assist designers during the design process, and to improve the usability of the user interface being designed. However, the assumption that such systems actually improve the usability of the resulting user interface remains to be demonstrated: virtually no systems have been empirically tested by designers who create products. In order to confirm this assumption, we conducted an experimental study in which professional web designers had to create webpages, either using a knowledge-based system – MetroWeb – or without it. This study was intended to determine the influence of MetroWeb on the professional web designers' cognitive activity, and to find out whether MetroWeb actually assists them to develop a user-centred design. The results show that the web designers did not very often use MetroWeb while designing webpages; however, rather surprisingly, the designers who did use MetroWeb more often exhibited a user-centred activity than those working without MetroWeb. We conclude this paper by discussing these findings, and suggesting future possible ways of research intended to assist designers to adopt a user-centred approach to their activity.     

A practical guide to using software usability labs: lessons learned at IBM

Usability evaluation is a key component of a user-centred design process. Access to a usability laboratory can greatly facilitate the process of empirically measuring user performance, but the mere presence of a usability laboratory does not assure usable products. Rather, the laboratory must be used within an evaluation process. The process described in this article has five phases: designing the evaluation, preparing to conduct the evaluation, conducting the evaluation, analysing the data, and reporting the results. Lessons learned by the authors while they practised this evaluation process with a variety of products are summarized for possible use by other usability organizations.     

Model-Driven Interactive System Design for Therapy Robots

Physiotherapy using intelligent robots is emerging as a new approach to recovery for many stroke patients. Although therapy robots have a strong potential in dealing with therapeutic and other medical applications, they have not been fully utilized in everyday therapy activities due to concerns over safety and the lack of friendly robot user/patient interaction models. From the viewpoint of software engineering, a user-centred design based on UML (Unified Modelling Language) has been known to be one of the best solutions to satisfy usability since the design process relies heavily on the analysis of users and their tasks to reach their goals. Therefore, a model-driven approach to interactive system design via UML for therapy robots is needed to make them usable in the real world. This paper proposes such approach and introduces a new graphical notation that describes user interface elements and the methods of connection with hardware/software objects. With the proposed abstract interaction models, prototyping interactive systems can be made faster and allows for their evaluation by users and system developers before implementation in order to improve usability from the perspectives of users and system developers.     

ARREST: From work practices to redesign for usability

In this paper we discuss the redesign of a support management system deployed in a small and medium sized enterprise (SME) in the UK. The original system was not fulfilling its needs as it had not captured work practices in a way that was recognizable to the users. The advantages of the redesign included: improved usefulness; improved efficiency and productivity; reduced learning time; improved usability; and increased acceptance among users. The system is used to support complex and distributed cooperative activities taking place in an SME. We evaluated the current system and analysed work practices using a user-centred design and evaluation philosophy. In this paper we discuss how user needs are incorporated into the enhanced design of the support management system. The user-centred design techniques used in this research include interviews, questionnaires, observations and user tests. We present comparative evaluation results that show significant improvement in performance of user tasks using the redesigned support management system. The contribution of this paper is the presentation of a case study to show how a user-centred design and evaluation philosophy can lead to better requirements capture resulting in systems that more accurately capture the users’ conceptual models.     

Bu-Dash: a universal and dynamic graphical password scheme (extended version)

Passwordless authentication is a trending theme in cyber security, while biometrics gradually replace knowledge-based schemes. However, Personal Identification Numbers, passcodes, and graphical passwords are still considered as the primary means for authentication. Passwords must be memorable to be usable; therefore, users tend to choose easy to guess secrets, compromising security. The Android Pattern Unlock is a popular graphical password scheme that can be easily attacked by exploiting human behavioristic traits. Despite its vulnerabilities, the popularity of the scheme has led researchers to propose adjustments and variations that enhance security but maintain its familiar user interface. Nevertheless, prior work demonstrated that improving security while preserving usability remains frequently a hard task. In this paper we propose a novel graphical password scheme built on the foundations of the well-accepted Android Pattern Unlock method, which is usable, inclusive, universal, and robust against shoulder surfing and (basically) smudge attacks. Our scheme, named Bu-Dash, features a dynamic user interface that mutates every time a user swipes the screen. Our pilot studies illustrate that Bu-Dash attracts positive user acceptance rates, it is secure, and maintains high usability levels. We define complexity metrics that can be used to further diversify user input, and we conduct complexity and security assessments.

     

The cobbler's children: usability in the lab

Abstract

Software and product designers regularly wrestle with the usability of their creations in user studies laboratories. The laboratories themselves, however, often languish for lack of the same scrutiny with respect to their own usability. In this article, we trace our efforts toward making a truly usable user studies laboratory. Our lab serves many purposes and many individuals, each requiring a slightly different configuration of equipment. The challenge was to design a user studies environment that would provide ‘instant’ configurations, obviating the need for users to connect components by hand. Toward this end, we purchased equipment that could be controlled directly or indirectly by a computer and designed the wiring plant to all run through a matrix switcher, also under computer control. Upon this technology, we designed a user interface in HyperCard that automatically configures the lab and provides the interface controls required for any one of a set of specific pre-designed tasks that the user may select. The interface also permits the user to customize and save sets of configurations and controls by copying and pasting among cards and also to create novel configurations by manipulating the matrix controller's software switches one by one.     

Mixed reality environment for learning sensing technology applications in Construction: A usability study

Mixed reality is gradually becoming ubiquitous and significant in education owning to the inherent benefits of active participation and tacit knowledge development in a safe and engaging environment. However, limited studies have explored design features that facilitate its use as a pedagogical tool in construction education, particularly in equipping students with experiential skills that are otherwise challenging to obtain due to resource constraints and limited access to construction sites. By evaluating eye tracking, usability questions, and think-aloud protocol data and verbal feedback, this study investigated the usability of a mixed reality environment designed for equipping construction engineering students with competencies for deploying sensing technologies on construction projects. Results revealed features such as accuracy of represented construction activities, quality of animations, and easy access to information and resources as important for designing efficient mixed reality learning environments. While the usability data suggested that the user interface was usable, eye tracking provided profundity on encountered difficulties with the user interface. Through concise outline and sequential design of the user interface, this study revealed that knowledge scaffolding can improve task performance in a mixed reality learning environment. This study adds value to existing literature, in particular by providing insights into the affordances of mixed reality environments that address the technological gap between the construction industry and construction engineering education. The mixed reality learning environment contributes to the cognitive apprenticeship theory through the use of game objects to develop procedural knowledge for addressing construction industry challenges with sensing technologies.     

Distributed Path-Based Inference in Semantic Networks

This paper presents the task model, instruction set, reasoning scheme, software infrastructure, as well as the experimental results, of a new distributed semantic network system. Unlike the synchronous and static marker passing algorithm previously used for parallel semantic network design, our system operates asynchronously, supporting knowledge sharing, dynamic load balancing and duplicate checking. To better the performance in distributed environments, the system has two collaborating components: the slave module, which performs task execution; and the host module, which interacts with the user and processes the information for the slaves. Our current implementation focuses on path-based knowledge inferences, using ANSI C and the MPICH-G2 with flex lexical analyzer and the yacc parser generator. Tests of individual components have been performed on a SUN multiprocessor server. The experiments demonstrate promising speedups.     

mwKAT: A meta-tool-generated knowledge acquisition tool for multimedia workstations development

mwKAT is an interactive knowledge acquisition tool for acquiring domain knowledge about multimedia components. It constructs knowledge bases for a consulting system that produces the design specification for a multimedia workstation according to the user requirements.mwKAT is generated from and executed inGAS, a primitives-based generic knowledge acquisition meta-tool. It contains three acquisition primitives, namely, parameter proposing, constraint proposing, and fix proposing to construct an intermediate knowledge base represented by a dependency model. These primitives identify necessary domain knowledge and guide users to propose significant components, constraints, and fix methods into the dependency model.mwKAT also invokes knowledge verification and validation primitives to verify the completeness, consistency, compilability, and correctness of the intermediate knowledge base.     

Designing and equipping a usability laboratory

Abstract

The goal of usability lab design is to create a space where high quality data capture occurs in an environment that looks and feels like the workplace of the product that is being tested. The lab must be part recording studio and part flexible work environment. Since most work environments are not recording studios, achieving a balance between simulation of the work environment and high quality data capture presents a challenge to usability lab designers. Steps can be taken with a user-centred design process to insure that a usability lab design meets this goal. This paper describes such a user-centred design process and how it. in combination with practical architectural and equipment guidelines derived from the authors' past experience, can be used in the design and redesign of future labs. The authors also discuss what changed in the present lab at NCR as a result of these guidelines.     

The Role of Instructional Design in Persuasion: A Comics Approach for Improving Cybersecurity

Abstract

Although computer security technologies are the first line of defense to secure users, their success is dependent on individuals’ behavior. It is therefore necessary to persuade users to practice good computer security. This interview analysis of users’ conceptualization of security password guessing attacks, antivirus protection, and mobile online privacy shows that poor understanding of security threats influences users’ motivation and ability to practice safe behaviors. An online interactive comic series called Secure Comics was designed and developed based on instructional design principles to address this problem. An eye-tracking experiment suggests that the graphical and interactive components of the comics direct users’ attention and facilitate comprehension of the information. In the evaluations of Secure Comics, results from several user studies show that the comics improve understanding and motivate positive changes in security management behavior. The implication of the findings to better understand the role of instructional design and persuasion in education technology are discussed.     

User-centred design: for users or by users?

Abstract

When ergonomists contribute to the design of products and services they aim to be user-centred. This paper explores two possible meanings of user-centredness; the ergonomist may use theories and findings about human behaviour to act for the user or may help the user to participate in design. Both approaches are well known in ergonomics and they can point in conflicting directions. This paper examines the rationale for the two approaches and presents the results of a survey, which found that the most successful strategy was to mix the two approaches. It offers a classification to support the identification of the appropriate approach to adopt in different situations. The paper proposes, for example, that ‘design by users’ is the appropriate strategy when significant value judgements have to be taken in a local or bespoke design setting. By contrast a ‘design for users’ approach is appropriate for the design of generic products. An additional approach ‘design for users with users’ is introduced for settings that require knowledge about human characteristics and that need users to make value judgements.     

A Multidimensional Product‐Concept Model Enhancing Cross‐Functional Knowledge Creation in the Product Innovation Process: The Case of the Suunto t6 Training Wrist Computer

This article presents a four‐dimensional product‐concept model enhancing cross‐functional knowledge creation in product innovation: the dimensions presented in the new model are: technology, end‐user, brand and business logic. The application of the model is described in the case study of the Suunto t6 training wrist computer. For the Suunto t6 development it was revealed that cross‐functional knowledge creation had happened on an even larger scale than expected. Analysis of the results suggests that certain dimensions of the product concept can be especially indicative of cross‐functional knowledge creation, such as concern with the end‐user’s experience. Furthermore, the new product concept may provide an early warning of innovation‐based diversification. Nicknames for the product concept under development, which requires the verbalisation of tacit subjective associations, can be used as indicators of cross‐functional integration within the organization and as suitable indicators of tacit knowledge. Measurement of all relevant organizational capabilities is discussed.     

A Usable Security Heuristic Evaluation for the Online Health Social Networking Paradigm

The security, privacy, and usability of health social networking websites remain key concerns for their successful adoption by patients. As far as it is known, no studies have developed a usable security heuristic evaluation to test these websites for the usability of their security and privacy features. This article describes the development of such a usable security heuristic evaluation in the context of the online health social networking paradigm. The tailored method approach was applied to transform themes identified from relevant literature and convert them into high-level heuristics for usable security. Checklist items were created for each high-level heuristic. The result is 13 high-level heuristics with individualized checklist items that help examine usable security. The results can identify which security and privacy features are lacking from a usability perspective and would, therefore, require design improvement. The improvements would enhance the users' experiences, which in turn can assist in the adoption of health social networking websites.     

Evaluation of integrated software development environments: Challenges and results from three empirical studies

Evidence shows that integrated development environments (IDEs) are too often functionality-oriented and difficult to use, learn, and master. This article describes challenges in the design of usable IDEs and in the evaluation of the usability of such tools. It also presents the results of three different empirical studies of IDE usability. Different methods are sequentially applied across the empirical studies in order to identify increasingly specific kinds of usability problems that developers face in their use of IDEs. The results of these studies suggest several problems in IDE user interfaces with the representation of functionalities and artifacts, such as reusable program components. We conclude by making recommendations for the design of IDE user interfaces with better affordances, which may ameliorate some of most serious usability problems and help to create more human-centric software development environments.     

A part affordance-based approach for capturing detailed design knowledge

Design knowledge reuse is widely accepted as an effective strategy for designers to develop robust artifacts with less time and lower cost. However, there has been very little research on how to help designers capture detailed design knowledge for reuse. As a result, most detailed design knowledge still has to remain in designers’ memories as tacit knowledge, which can easily get lost due to oblivion or the mobility of designers. Therefore, this paper attempts to develop a part affordance-based approach for externalizing and capturing detailed design knowledge for effective reuse. It first introduces a part model for representing the detailed design-related information. Based on the relational theory for design, the concept, part affordance, is then employed to help designers externalize and capture various lifecycle factors that are implicit in a detailed design. Based on the affordance constraint axiom, a systematic approach is then proposed for deriving tacit design knowledge from captured part affordances through the analysis of extreme working situations. The proposed approach has been implemented as the Design Knowledge-Capturing System (DKCS). A fixture design case has been employed to illustrate the proposed approach.     

Usability testing — on a budget: a NASA usability test case study

Abstract

Okay, so you've purchased a graphical user interface (GUI) builder tool to help you quickly build a sophisticated user interface, and your developers promise to follow a particular style guide (e.g., OSF/Motif, Apple/Macintosh) when creating the GUI. This is definitely a step in the right direction, but it is no guarantee that the application's user interface will be usable; that is, where the user interface helps, rather than hinders, the end-users from doing their jobs. Numerous techniques for testing the usability and user satisfaction of an application's GUI are available, such as design walk-throughs, field testing with beta releases, demonstrations of prototypes to future end-users, and user questionnaires. One of the most effective techniques is usability testing with defined tasks and metrics, and yet, it is not commonly used in project development life cycles at the National Aeronautics and Space Agency (NASA). This paper discusses a low-budget, but effective, approach we used at NASA's Goddard Space Flight Center (GSFC) to perform structured usability testing. It did not require any additional staff or a usability laboratory, but did successfully identify problems with the application's user interface. The purpose of the usability testing was two-fold: (1) to test the process used in the usability test; and (2) to apply the results of the test to improving the subject software's user interface. This paper will discuss the results from the test and the lessons learned. It will conclude with a discussion of future plans to conduct cost benefit analysis and integrate usability testing as a required step in a project's development life cycle.     

Finding a needle in a haystack: an interactive video archive explorer for professional video searchers

Professional video searchers typically have to search for particular video fragments in a vast video archive that contains many hours of video data. Without having the right video archive exploration tools, this is a difficult and time consuming task that induces hours of video skimming. We propose the video archive explorer, a video exploration tool that provides visual representations of automatically detected concepts to facilitate individual and collaborative video search tasks. This video archive explorer is developed by employing a user-centred methodology, which ensures that the tool is more likely to fit to the end user needs. A qualitative evaluation with professional video searchers shows that the combination of automatic video indexing, interactive visualisations and user-centred design can result in an increased usability, user satisfaction and productivity.