A Collaborative Usability Evaluation (CUE) Model for Health IT Design and Implementation

ABSTRACT

In this article, the authors describe a collaborative usability evaluation (CUE) model based on a participatory ergonomics framework. The model was applied to an organization’s health IT design and implementation processes. Individuals received education on usability evaluation principles and methods and had an opportunity for hands-on experience with usability testing. Actual usability evaluations increased participant confidence in their ability to independently design and lead such evaluations. The CUE model has since been applied to the organization’s iterative process of health IT design, has been institutionalized by the health IT department, and is integrated in its health IT implementation timelines.     

Design of Mobile Phones for Older Adults: An Empirical Analysis of Design Guidelines and Checklists for Feature Phones and Smartphones

Design guidelines and checklists are suggested as a useful tool in the development and evaluation of interface design of mobile phones for older adults. Given the intense evolution of mobile phone design, understanding how the design guidelines and checklists have taken into account the advances in mobile phone usability for older adults is important for their correct application and future development. Thus, this study explores the usability dimensions of mobile phone design for older adults and the related changes in terms of time and the type of device (feature phones vs. smartphones) based on an expert coding of the eight mobile phone design guidelines and checklists for older adults published between 2006 and 2014. The results of the expert coding show that design guidelines and checklists most frequently deal with visual and haptic issues (e.g., high contrast, button type, and button size), whilst they hardly ever address various elements of textual interface (e.g., ease of text entry, a button’s feedback, and font type). Over time, the design guidelines and checklists have become more complex in terms of the average number of included usability categories and dimensions. For smartphones, the guidelines, on average, put more emphasis on the screen, touchscreen, text, and exterior related issues, whereas the design guidelines for feature phones stress the usability of the keypad and menus. Besides revealing potential usability dimensions that could be further expanded in the guidelines, this study also highlights the need for research that would empirically validate the design guidelines and checklists in the future.     

Older adults’ use of mobile device: usability challenges while navigating various interfaces

ABSTRACT

Mobile devices are becoming ubiquitous among older adults, but have also caused unprecedented challenges due to the high demands of interaction techniques and changeable design patterns found across various applications. This paper aims to investigate how older adults navigate with mobile interfaces and identify their potential usability challenges while navigating. To do so, we summarised six state-of-the-art mobile interface design patterns and conducted individual usability test and in-depth interview with 22 older adults. Participants were asked to perform 19 navigation tasks that contain these design patterns under realistic usage scenarios. Follow-up interviews were held to collect their detailed comments on usability issues regarding visual design, ease of understanding, and interaction and navigation of the design patterns, as well as their personal experience. The results found that overall older adults were able to navigate contents more effectively than menus and buttons. Participants experienced great challenges in directing their attention to the menus and buttons, understanding the meaning of icons, and interacting with these menu components. In contrast, the content-oriented navigation design performed better in understanding, navigation, and interaction, which could be a promising direction for elderly-friendly mobile application design. Design implications are further discussed for creating an elderly-friendly mobile interface.     

Age-related differences in the initial usability of mobile device icons

Mobile devices offer much potential to support older adults (age 65+). However, older adults have been relatively slow to adopt mobile devices. Although much ongoing HCI research has examined usability problems to address this issue, little work has looked at whether existing graphical icons are harder to use for this population compared with younger adults. We conducted a qualitative exploratory study and a follow-up experimental study to determine which icon characteristics help initial icon usability for older adults. We found that our older participants did have more problems using existing mobile device icons, but that particular icon characteristics – semantically close meaning (i.e. natural, close link between depicted objects and associated function), familiar, labelled and concrete (i.e. those depicting real-world objects) – improved icon usability for them. We discuss how these findings can help icon designers to create mobile device icons that are more suited to the abilities and technology experience of older adults.     

Evidence for an elders' advantage in the naive product usability judgments of older and younger adults

OBJECTIVE: To determine whose naive judgments of consumer product usability are more accurate--those of younger or older adults. Accuracy is here defined as judgments compatible with results from performance-based usability tests. BACKGROUND: Older adults may be better able to predict usability problems than younger adults, making them particularly good participants in studies contributing to the user-centered design of products. This advantage, if present, may stem from older adults' motivation for more usable products or from their experience adapting their own environments to meet their changing physical, cognitive, and sensory needs. METHOD: Sixty older participants (ages 65-75 years) and 60 younger ones (ages 18-22 years) evaluated illustrations of consumer products on specific criteria (e.g., readability, learnability, or error rates). They either rated a single design for each product or ranked six alternative designs. They also explained their choices, indicated which features were most critical for usability, and selected usability-enhancing modifications. RESULTS: Although there was no reliable age difference in the amount of usability information provided in the open-ended explanations, older adults were more accurate at ranking alternative designs, selecting the most usability-critical features, and selecting usability-enhancing modifications (all ps < .05). CONCLUSION: The usability judgments of older adults are more accurate than those of younger adults when these judgments are solicited in a fixed-alternative, but not open-ended, format. APPLICATION: Because older adults are more discerning about potential product usability problems, they may be particularly valuable as research participants in early-stage design research (prior to the availability of working prototypes).     

Evaluating websites for older adults: adherence to ‘senior-friendly’ guidelines and end-user performance

Older adults in the US are the fastest-growing demographic, and also the largest-growing group of internet users. The aim of this research was to evaluate websites designed for older adults in terms of (i) how well they adhere to ‘senior-friendly’ guidelines and (ii) overall ease of use and satisfaction. In Experiment I, 40 websites designed for older adults were heuristically evaluated based on their adherence to usability guidelines derived by the National Institute on Aging and the National Library of Medicine. In Experiment II, three websites with varying levels of guideline compliance were evaluated by older adults in a usability test. Results from this study indicate that the website most compliant with the ‘senior-friendly’ guidelines resulted in higher task success, but did not result in significantly better efficiency, satisfaction, or preference. These findings demonstrate the importance of using both guidelines and usability testing when designing websites for older adults.     

Generation,Adoption, and Tuning of Usability Evaluation Multimethods

The selection and customization of usability evaluation methods, given the peculiarities of their application domains, still remains a critical issue; this especially when dealing with complex products and/or nonexpert usability evaluators. Moreover, as time goes by, the quality of the evaluation results has a heavier impact on the product design process. Starting from classic usability evaluation methods, the research described in this article generates multimethods semiautomatically. It allows quantitative characterization of these multimethods before their application in the field and exploits the comparison between this prior assessment and a final estimate, made after adoption, to update the information used by the method selection process. The most critical issue related to usability, subjectivity, is considered and dealt with throughout the entire research. A case study, done at the end of the development phase, helps validate the proposed approach to usability evaluation.     

Consumer Health Informatics Interventions Must Support User Workflows,Be Easy-To-Use,and Improve Cognition: Applying the SEIPS 2.0 Model to Evaluate Patients’ and Clinicians’ Experiences with the CONDUIT-HID Intervention

ABSTRACT

The aim of this research was to gain a holistic understanding of patients’ and clinicians’ experiences with the CONDUIT-HID (CONtrolling Disease Using Inexpensive Technology—Hypertension In Diabetes) intervention, intended to be a technology-enabled consumer health informatics (CHI) approach to control hypertension. We examined patients’ experiences utilizing the technologies to share patient blood pressure data with their care team via a qualitative analysis of patient (n = 21) and clinician (n = 5) interviews. Using the SEIPS 2.0 sociotechnical systems model, our evaluation revealed that minimizing usability issues and supporting participant workflow were important—but not sufficient—for CHI intervention success. The ability of the CHI intervention to support the cognitive development of patients’ self-management skills and to facilitate strategic collaboration among care team members was also important. These insights can provide CHI and the human–computer interaction (HCI) communities with a framework of generalizable findings to better design future CHI interventions.     

Website design attributes for retrieving health information by older adults: an application of architectural criteria

The objectives of this research were to identify design attributes to develop easy-to-use websites for older adults. Forty-one males and 58 females (age range 58–90) were asked to retrieve information on a health-related topic from the NHS Direct and Medicdirect websites, and were asked to fill in a website evaluation questionnaire. An exploratory factor analysis of data identified navigation/search usability, link usability, usefulness and colour as important dimensions of a senior-friendly website. A two-stage, three-component regression model with these dimensions as predictor variables and the satisfaction level in using a website as the dependent variable has been proposed.     

Redesigning websites for older adults: a case study

Website redesign is a complex task that requires the organised use of design methods and guidelines as well as meaningful assessments. An important reason for website redesign is to enhance the usability and accessibility for ‘non-traditional’ user groups like the older population. The paper presents a case study of the redesign of a touristic web portal in order for it to be senior-friendly. The redesign process involved: (a) identification of a set of 7 principles and 45 guidelines (7p/45g) for web design for older adults; (b) heuristic evaluation of the original website on the basis of the identified 7p/45g set; (c) redesign of the website in an interactive online prototype; and (d) comparative summative usability evaluation, which involved 12 older users and post hoc interviews and questionnaires. The results showed that the redesigned website was significantly more usable and acceptable than the original. This study makes two contributions in the field: first, it outlines a web redesign process for older adults, which is effective in terms of time and cost, making use of various assessments of experts and users; second, it presents the application of this process in a manner that can be adopted and repeated in other redesign cases.     

Exploring influencing factors of technology use for active and healthy ageing support in older adults

ABSTRACT

Aim of this study is to investigate the influence of technological and social cognitive factors for the use of sensor-based technologies for active and healthy ageing (AHA) support by older adults. In a mixed methods approach, data was initially obtained from an online questionnaire completed by older health technology users and used in a regression analysis, where factors from the Technology Acceptance Model (TAM) and the Social Cognitive Theory (SCT) served as predictors for health technology use (HTU). Further, in-depth interviews were conducted with older adults to gain insights into technology use and physical activity behaviour of older adults. The regression analysis showed that the TAM and SCT factors accounted for a significant proportion of variance (39.5%) in HTU. Significant predictors of HTU were physical activity (.399**), social support (.287*), and expectations regarding individual health (.440*) and physical appearance (?.470**), indicating physical activity as mediator for HTU. The qualitative analysis indicated the conflation of technology support with social environments as key for physical activity behaviour in older adults. The findings indicate physical activity as a mediator in HTU by older adults and suggest that the consideration of social factors in health technology design may facilitate the uptake of AHA technologies.     

Usability professionals—current practices and future development

The usability concept has now received such a wide recognition in information technology (IT) development that working with usability can be regarded as a profession in its own right. In recent research projects, we have surveyed and studied usability work on an individual level in a number of Swedish development organisations, including success factors and obstacles. What we have seen relates to the individual usability professional and her background and experiences, the organisation in which she operates, the development process, communication and communication means, and finally the attitudes and basic values held by the people involved.In this paper, we compile and reflect on selected findings from different studies on usability work in practical systems¹ development in a number of Swedish organisations. We discuss our findings from a practical point of view and relate them to the research of others within the international HCI community. Finally, we discuss some issues we consider important for the future development of the practice of usability that we believe is of interest to the international community of usability professionals.     

Comparison of four different mobile devices for measuring heart rate and ECG with respect to aspects of usability and acceptance by older people

In the area of product design and usability, most products are developed for the mass-market by technically oriented designers and developers for use by persons who themselves are also technically adept by today's standards. The demands of older people are commonly not given sufficient consideration within the early developmental process. In the present study, the usability and acceptability of four different devices meant to be worn for the measurement of heart rate or ECG were analyzed on the basis of qualitative subjective user ratings and structured interviews of twelve older participants. The data suggest that there was a relatively high acceptance concerning these belts by older adults but none of the four harnesses was completely usable. Especially problematic to the point of limiting satisfaction among older subjects were problems encountered while adjusting the length of the belt and/or closing the locking mechanism. The two devices intended for dedicated heart rate recording yielded the highest user ratings for design, and were clearly preferred for extended wearing time. Yet for all the devices participants identified several important deficiencies in their design, as well as suggestions for improvement. We conclude that the creation of an acceptable monitoring device for older persons requires designers and developers to consider the special demands and abilities of the target group.     

Using simulation to aid decision making in managing the usability evaluation process

ContextThis paper is developed in the context of Usability Engineering. More specifically, it focuses on the use of modelling and simulation to help decision-making in the scope of usability evaluation.ObjectiveThe main goal of this paper is to present UESim: a System Dynamics simulation model to help decision-making in the make-up of the usability evaluation team during the process of usability evaluation.MethodTo develop this research we followed four main research phases: (a) study identification, (b) study development, (c) running and observation and finally, (d) reflexion. In relation with these phases the paper describes the literature revision, the model building and validation, the model simulation and its results and finally the reflexion on it.ResultsWe developed and validated a model to simulate the usability evaluation process. Through three different simulations we analysed the effects of different compositions of the evaluation team on the outcome of the evaluation. The simulation results show the utility of the model in the decision making of the usability evaluation process by changing the number and expertise of evaluators employed.ConclusionOne of the main advantages of using such a simulation model is that it allows developers to observe the evolution of the key indicators of the evaluation process over time. UESim represents a customisable tool to help decision-making in the management of the usability evaluation process, since it makes it possible to analyse how the key process indicators are affected by the main management options of the Usability Evaluation Process.     

The contribution of apparent and inherent usability to a user's satisfaction in a searching and browsing task on the Web

Previous research found that apparent and inherent usability had big influences on user's acceptance of computer systems. This research extended these findings to the area of user's satisfaction. The study investigated the impact of inherent and apparent usability on user's satisfaction of Web page designs while the two main tasks on the World Wide Web, searching and browsing, were performed. The results indicated that inherent usability was the main factor contributing to user's satisfaction in both searching and browsing. General principles and specific recommendations for the design and evaluation of Web page designs are derived from these findings.     

A Systematic Mapping Study of HCI Practice Research

ABSTRACT

Human–computer interaction (HCI) practice has emerged as a research domain in the HCI field and is growing. The need to transfer HCI practices to the industry began significantly with the works of Nielsen on usability engineering. To date, methods and techniques for designing, evaluating, and implementing interactive systems for human use have continued to emerge. It is, therefore, justified to conduct a systematic mapping study to determine the landscape of HCI practice research. A Systematic Mapping Study method was used to map 142 studies according to research type, topic, and contribution. These were then analyzed to determine an overview of HCI practice research. The objective was to analyze studies on HCI practice and present prominent issues that characterize the HCI practice research landscape. Second, to identify pressing challenges regarding HCI practices in software/systems development companies. The results show that HCI practice research has steadily increased since 2012. The majority of the studies explored focused on evaluation research that largely contributed to the evaluation methods or processes. Most of the studies were on design tools and techniques, design methods and contexts, design work and organizational culture, and collaboration and team communication. Interviews, case studies, and survey methods have been prominently used as research methods. HCI techniques are mostly used during the initial phase of development and during evaluation. HCI practice challenges in companies are mostly process-related and on performance of usability and user experience activities. The major challenge seems to be to find a way to collect and incorporate user feedback in a timely manner, especially in agile processes. There are areas identified in this study as needing more research.     

Age-Related Differences in Eye Tracking and Usability Performance: Website Usability for Older Adults

Cognitive decline is inherent with age. Despite known cognitive limitations, older adults are generally not taken into account during website design. Understanding age-related differences in website navigation is instructive for website design, especially considering the growing number of older adults who use the Internet. This article presents usability and eye-tracking data from five independent website usability studies that included younger and older participants. Overall results revealed age-dependent differences in eye movement and performance during website navigation on some of the sites. In particular, older participants had lower accuracy in one study and took longer to complete tasks in two studies compared to younger participants, they looked at the central part of the screen more frequently than younger participants in two studies, and they looked at the peripheral left part of the screen less frequently and took longer to first look at the peripheral top part of the screen than younger participants in one study. These data highlight the potential for age-related differences in performance while navigating websites and provide motivation for further exploration. Implications for website design and for usability practitioners are discussed.     

The impact of age on website usability

As the general and working populations age in most developed nations, the study of website usability for older adults is becoming increasingly relevant. Website usability is concerned with both utilitarian (i.e. functional) and hedonic (i.e. pleasure-related) aspects. A new website usability model is proposed that considers the effects of age on website usability through cognitive antecedents that are most relevant to age-related effects. Specifically, spatial ability is the declining cognitive skill of particular interest in this research. A laboratory experiment was conducted where younger and older participants interacted with an experimental website. The results suggest that age has a pronounced impact on performance as a mediated effect through declining levels spatial ability and mental model accuracy as well as through a direct effect suggesting the presence of other objective and subjective changes associated with aging that could impact performance. Perceived disorientation was also examined within the proposed website usability model, revealing both expected and surprising findings.     

No time to waste: the role of timing and complementarity of alignment practices in creating business value in IT projects

Despite significant research progress, alignment-related issues are among the top concerns of executives. Previous studies mostly focus on a company-wide strategic level of alignment; while this ‘top-down’ view has benefits, it largely ignores the operational practices that help achieve alignment in IT projects as well as the impact that timing and complementarity of practices have on achieving alignment. In our research we apply four alignment practices – communication, shared understanding, management commitment and IT investment evaluation – to individual IT projects rather than at a company level; specifically, we look at the role of timing and complementarity for these alignment practices throughout different project phases. A detailed analysis of six IT projects carried out at three companies in the telecommunications industry reveals that IT projects creating higher business value employ all four alignment practices immediately from the start. No project was able to recover from failing to establish these four alignment practices in the first phase. While supporting the importance of complementarity of alignment practices, our findings also add the importance of the earliness of this complementarity.     

Age differences in search of web pages: the effects of link size, link number, and clutter

Reaction time, eye movements, and errors were measured during visual search of Web pages to determine age-related differences in performance as a function of link size, link number, link location, and clutter. Participants (15 young adults, M = 23 years; 14 older adults, M = 57 years) searched Web pages for target links that varied from trial to trial. During one half of the trials, links were enlarged from 10-point to 12-point font. Target location was distributed among the left, center, and bottom portions of the screen. Clutter was manipulated according to the percentage of used space, including graphics and text, and the number of potentially distracting nontarget links was varied. Increased link size improved performance, whereas increased clutter and links hampered search, especially for older adults. Results also showed that links located in the left region of the page were found most easily. Actual or potential applications of this research include Web site design to increase usability, particularly for older adults.