Too good to be bad: Favorable product expectations boost subjective usability ratings

In an experiment conducted to study the effects of product expectations on subjective usability ratings, participants (N = 36) read a positive or a negative product review for a novel mobile device before a usability test, while the control group read nothing. In the test, half of the users performed easy tasks, and the other half hard ones, with the device. A standard usability test procedure was utilized in which objective task performance measurements as well as subjective post-task and post-experiment usability questionnaires were deployed. The study revealed a surprisingly strong effect of positive expectations on subjective post-experiment ratings: the participants who had read the positive review gave the device significantly better post-experiment ratings than did the negative-prime and no-prime groups. This boosting effect of the positive prime held even in the hard task condition where the users failed in most of the tasks. This finding highlights the importance of understanding: (1) what kinds of product expectations participants bring with them to the test, (2) how well these expectations represent those of the intended user population, and (3) how the test situation itself influences and may bias these expectations.     

Cross-user analysis: Benefits of skill level comparison in usability testing

This study presents a cross-user usability test approach and analysis technique that extends beyond merely identifying the existence of a usability problem to introducing an empirical basis for identifying the type of usability problem that exists. For experimental purposes, 60 users were tested with three levels of user-competency determined by experience in using: (1) computers, and (2) the tested application. Applying the Tukey honestly significant difference (HSD) test to each test element provided statistical comparison between different experience levels. Analysis results between experience levels suggested which levels encountered usability problems. The authors demonstrate that statistical calculations of cross-user data can render empirical support for categorizing usability problems.     

The System Usability Scale: Past,Present, and Future

The System Usability Scale (SUS) is the most widely used standardized questionnaire for the assessment of perceived usability. This review of the SUS covers its early history from inception in the 1980s through recent research and its future prospects. From relatively inauspicious beginnings, when its originator described it as a “quick and dirty usability scale,” it has proven to be quick but not “dirty.” It is likely that the SUS will continue to be a popular measurement of perceived usability for the foreseeable future. When researchers and practitioners need a measure of perceived usability, they should strongly consider using the SUS.     

Interpretation of a cross-cultural usability evaluation: A case study based on a hypermedia system for rare species management in Namibia

We present the results of a usability evaluation of a locally developed hypermedia information system aiming at conservation biologists and wildlife managers in Namibia. Developer and end user come from different ethnic backgrounds, as is common to software development in Namibia and many developing countries. To overcome both the cultural and the authoritarian gap between usability evaluator and user, the evaluation was held as a workshop with usability evaluators who shared the target users’ ethnic and social backgrounds. Different data collection methods were used and results as well as specific incidences recorded. Results suggest that it is difficult for Namibian computer users to evaluate functionality independently from content. Users displayed evidence of a passive search strategy and an expectation that structure is provided rather than self generated. The comparison of data collection methods suggests that questionnaires are inappropriate in Namibia because they do not elicit a truthful response from participants who tend to provide answers they think are “expected”. The paper concludes that usability goals and methods have to be determined and defined within the target users’ cultural context.     

The effect of testing location on usability testing performance, participant stress levels, and subjective testing experience

The effect of testing location on usability test elements such as stress levels and user experience is not clear. A comparison between traditional lab testing and synchronous remote testing was conducted. The present study investigated two groups of users in remote and traditional settings. Within each group participants completed two tasks, a simple task and a complex task. The dependent measures were task time taken, number of critical incidents reported, and user-reported anxiety score. Task times differed significantly between the physical location condition; this difference was not meaningful for real world application, and likely introduced by overhead regarding synchronous remote testing methods. Critical incident reporting counts did not differ in any condition. No significant differences were found in user reported stress levels. Subjective assessments of the study and interface also did not differ significantly. Study findings suggest a similar user testing experience exists for remote and traditional laboratory usability testing.     

A survey of usability capability maturity models: implications for practice and research

Improving the position and effectiveness of user-centred design (UCD) in software and product development is a challenge in many companies. One step towards improvements is to carry out a usability capability maturity (UCM) assessment to evaluate the strengths and weaknesses of a development organisation in UCD. While several diverse UCM models have been introduced, little research has been published in the public domain on these models. The paper aims to fill this gap by analysing the main features of the models. The results show that most models represent different approaches – although some of them have the same roots – meaning that understanding one model is not enough to understand the basics of another model. All models can be used for examining the status of UCD in individual development projects. In addition, models provide various means for assessment of the status of UCD in other organisational areas. The level of documentation of models varies a lot, and very few empirical research results exist. Based on the results, implications for practice and research are suggested.     

Metacognition in computation: A selected research review

Various disciplines have examined the many phenomena of metacognition and have produced numerous results, both positive and negative. I discuss some of these aspects of cognition about cognition and the results concerning them from the point of view of the psychologist and the computer scientist, and I attempt to place them in the context of computational theories. I examine metacognition with respect to both problem solving (e.g., planning) and to comprehension (e.g., story understanding) processes of cognition.     

Reliability and usability of an internet-based computerized cognitive testing battery in community-dwelling older people

Cognitive decline is an early feature of neurodegenerative conditions. CogState has developed a game-like computerized test battery with demonstrated acceptability, validity, reliability, stability, efficiency and sensitivity to detecting cognitive decline in older people under supervised conditions. This study aimed to evaluate an internet-based version of this test when used remotely and self-administered in a cohort of healthy, community-dwelling older adults aged 55 and above over a 12 month period at 1–3 monthly intervals. Test usability and reliability was examined in terms of acceptability, stability and reliability. Of 150 participants (age: 63.6 ± 5.6, range 55–83 years), 143 (95%) successfully completed a valid baseline test. Of these, 67% completed 3 month and 43% 12 months of testing. Technical difficulties were reported by 9% of participants. For those participants who completed 12 months tests, all tasks showed moderate to high stability and test–retest reliability.This brief computerized test battery was shown to have high acceptability for baseline self-administered testing and moderate to high stability for repeated assessments over 12 months. Attrition was high between baseline and 3 months. These data suggest that this tool may be useful for high frequency monitoring of cognitive function over 6–12 months, and deserves further evaluation.     

The influence of design aesthetics in usability testing: Effects on user performance and perceived usability

This article examined the effects of product aesthetics on several outcome variables in usability tests. Employing a computer simulation of a mobile phone, 60 adolescents (14-17 yrs) were asked to complete a number of typical tasks of mobile phone users. Two functionally identical mobile phones were manipulated with regard to their visual appearance (highly appealing vs not appealing) to determine the influence of appearance on perceived usability, performance measures and perceived attractiveness. The results showed that participants using the highly appealing phone rated their appliance as being more usable than participants operating the unappealing model. Furthermore, the visual appearance of the phone had a positive effect on performance, leading to reduced task completion times for the attractive model. The study discusses the implications for the use of adolescents in ergonomic research.     

Interface design and emotions experienced on B2C Web sites: Empirical testing of a research model

This paper examines the impact of four Web site interface features on the cognitive process that trigger online shoppers’ emotions, operationalized as mental states of readiness that arise from appraisal of events and considered as direct antecedents to approach or avoidance behaviors. A research model was tested with data collected from 215 Web shopping episodes for low-touch merchandise. Results show that shoppers experienced all six emotions posited in the model. The emotions of liking and joy were experienced intensively by a substantial number of shoppers. The results also demonstrate that interface features – key components of the usability of a Web site – influenced the three cognitive appraisals illustrated in the research model. Moreover, the cognitive appraisals of situational state and control potential impacted the six emotions examined. This paper also highlights several theoretical contributions and managerial implications that should help managers and Web site managers improve the interface design of their Web sites in order to facilitate information gathering and better support online shopping processes.     

Strange Bedfellows: Human-Computer Interaction, Interface Design, and Composition Pedagogy

As digital interfaces increasingly mediate our access to information, the design of these interfaces becomes increasingly important. Designing digital interfaces requires writers to make rhetorical choices that are sometimes technical in nature and often correspond with principles taught in the computer science subfield of human-computer interaction. We propose that an HCI-informed writing pedagogy can complicate for both writing and computer science students the important role audience should play when designing traditional and digital interfaces. Although it is a subtle shift in many ways, this pedagogy seemed to complicate student understanding of the relationship between audience and the texts/interfaces they created: it was not just the “human” (beliefs, attitudes, values, demographics) or the “computer” (the software or hardware or other types of mediation) that mattered but rather the “interaction” between the two. First we explore some of the ways in which writing code and writing prose have merged and paved the way for an HCI-informed writing pedagogy. Next we examine some parallels between human-computer interaction principles and composition principles. Finally, we refer to assignments, student responses, and anecdotal evidence from our classes where an HCI-informed writing pedagogy drew—or could have drawn—student attention more acutely to various audience-related technical and rhetorical interface design choices.     

On retrieving information from external knowledge stores: Feeling-of-findability,feeling-of-knowing and Internet search

The ease with which individuals can access information has changed drastically with the advent of the Internet. Understanding how this change in our information landscape influences thinking represents an important question for psychological science. Research has demonstrated that we have a fairly accurate sense of the relative availability of internal information – a feeling-of-knowing. Here we examine the extent to which individuals have developed a sense of the relative availability of information stored on the Internet (i.e., externally) – a feeling-of-findability. Results demonstrate that when individuals do not know the answer to a question their feeling-of-findability accurately predicts the amount of time it takes them to locate the answer on the Internet. Furthermore, this feeling-of-findability, when individuals do not know the answer to a question, is unrelated to individuals’ feeling-of-knowing, despite the fact that the latter is also demonstrated to predict search times. Instead, feeling-of-findability appears to be predicted by intuitions about how difficult it will be to generate a successful search query and the popularity of the type of information sought.     

Employing think-aloud protocols and constructive interaction to test the usability of online library catalogues: a methodological comparison

This paper describes a comparative study of three usability test approaches: concurrent think-aloud protocols, retrospective think-aloud protocols, and constructive interaction. These three methods were compared by means of an evaluation of an online library catalogue, which involved four points of comparison: number and type of usability problems detected; relevance of the problems detected; overall task performance; and participant experiences. The results of the study showed that there were only few significant differences between the usability test approaches, mainly with respect to manner of problem detecting, task performance and participant experience. For the most part, the usability methods proved very much comparable, revealing similar numbers and types of problems that were equally relevant. Taking some practical aspects into account, a case can be made for preferring the concurrent think-aloud protocols over the other two methods.     

A Cognitive-Affective Model of Perceived User Satisfaction (CAMPUS): The complementary effects and interdependence of usability and aesthetics in IS design

Affective dimensions of human–computer interaction design have the potential to elicit emotions and behaviours. However, there is little research into which affective treatments are systematically tested, let alone assessed in light of additional cognitive dimensions. In this study, we formulate and empirically test a Cognitive-Affective Model of Perceived User Satisfaction (CAMPUS) that displays high explanatory power (R² = .69). CAMPUS offers a comprehensive framework for assessing both direct effects of perceptions of cognitive and affective dimensions on satisfaction and the complex interplay between these two in terms of system design and use. Implications for theory and practice are discussed.     

Incorporating customization and personalization into game-based learning: A cognitive style perspective

In recent years, customization and personalization were widely applied to accommodate the needs of different cognitive style groups. Such two approaches have different advantages and disadvantages but there is a lack of studies to compare these two approaches from the perspective of game-based learning, which is currently popular in educational settings. To this end, we developed a customized game-based learning system and a personalized game-based learning system and conducted two empirical studies to examine how cognitive styles affected learner's reactions to these two game-based learning systems. The results from the customized game-based learning system showed that Holists might not always favor to listen to music because they frequently switched on/off music. On the other hand, Serialists did not prefer to use hints. In addition, learners with the customized game-based learning system had more positive perceptions while learners with the personalized game-based learning system had more negative perceptions though both systems were useful to enhance learners' learning performance, regardless of their cognitive styles.     

Design and usability testing of a learning and plagiarism avoidance tutorial system for paraphrasing and citing in English: A case study

The prevalence of information and communication technology (ICT) has considerably converted the means of/for publication and circulation, as well as transforming academia and English pedagogy. However, with the availability and convenience of online resources, one of the critical issues emerged is that non-native English speakers are constantly accused of committing textual plagiarism: either intentionally or unintentionally. While many writing tools and plagiarism detectors are available to help solve the problem, none of them were customized for the great population of Chinese learners of English. Accordingly, DWright—a Chinese-interfaced online writing tutorial for paraphrasing and citing English—was developed in the hope of ensuring academic integrity through the avoidance of textual plagiarism.The purpose of this study was to evaluate the design and efficacy of DWright regarding plagiarism prevention and improvement of English writing. A questionnaire and semi-structured interview were administrated to participants to test DWright and its technology acceptance model (TAM) to identify the effect of perceived usefulness, usability and user attitudes of DWright. Results show that users of DWright were in high agreement regarding the content effectiveness of all DWright-based tasks, indicating the reading activities, multiple-choice exercises and paraphrasing practices were effective to help users enhance writing knowledge and skills to avoid plagiarism. Furthermore, perceived usefulness and system usability affected DWright user attitudes significantly and positively, which mirrors their attitudes toward continued use of DWright. In this study, DWright met its users' needs by extending their knowledge to avoid plagiarism while simultaneously enhancing their paraphrasing and writing skills. The conclusion suggests that tutorial designers, content experts and subject teachers should support effective communication to improve content usefulness, so as to help users achieve their writing goals with a research proven learning and plagiarism avoidance tutorial system.     

Advancing the assessment of automated deception detection systems: Incorporating base rate and cost into system evaluation

In the last two decades, there has been an increased interest in automated deception detection systems (ADDs) for use in screening, although little attention has been paid to the usefulness of these systems. ADDs use various means, both invasive and non‐invasive, to ascertain individual intent to deceive or engage in malicious behaviour. Many papers introducing ADDs use signal detection theory to compare a technique's ability to detect malicious intent with other techniques, but in doing so, they do not include contextual information such as base rate and cost. In this paper, we aim to improve future research by showing how the inclusion of contextual information provides a more realistic picture of the research. Through both theoretical arguments and a real‐data example, we show that especially for those contexts where malicious intent is infrequent (ie, with low base rates of deception) that not factoring in the base rate overestimates the accuracy and therefore usefulness. We conclude with recommendations for how future research should provide a fuller picture of the accuracy and usefulness of ADDs.