Usability Ratings for Everyday Products Measured With the System Usability Scale

This article characterizes the usability of 14 common, everyday products using the System Usability Scale (SUS). More than 1,000 users were queried about the usability of these products using an online survey methodology. The study employed two novel applications of the SUS. First, participants were not asked to perform specific tasks on these products before rating their usability but were rather asked to assess usability based on their overall integrated experience with a given product. Second, some of the evaluated products were assessed as a class of products (e.g., “microwaves”) rather than a specific make and model, as is typically done. The results show clear distinctions among different products and will provide practitioners and researchers with important known benchmarks as they seek to characterize and describe results from their own usability studies.     

Variations in the Usability of Independent Web-Based Personal Health Records

ABSTRACT

Previous research on personal health records (PHRs) has focused on applications that are “tethered” to a specific electronic health record (EHR). However, there is a gap in research on the usability of unaffiliated, independent PHRs, as well as research on college-aged PHR users. Therefore, we performed a comparative usability study with 18 college-aged participants on three popular, freely available, independent PHRs. Participants completed the same six tasks in three different PHRs. Dependent variables included task time, mouse movement, mouse clicks, keystrokes, errors, and user satisfaction ratings based on the Computer System Usability Questionnaire (CSUQ). Analysis of variance (ANOVA) was used to determine the significance in the difference of the means for each dependent variable. Results showed statistically significant differences in CSUQ survey categories, errors, and keystrokes. Results supported one of the three PHRs (“System A”) as having better usability than its tested counterparts; participants scored best in System A of all categories of the CSUQ, and used System A with less mouse movement, commission errors, and keystrokes than the other PHRs.     

Sensemaking in Technology-Use Mediation: Adapting Groupware Technology in Organizations

Understanding how people in organizations appropriate and adapt groupware technologies to local contexts of use is a key issue for CSCW research, since it is critical to the success of these technologies. In this paper, we argue that the appropriation and adaptation of groupware and other types of advanced CSCW technologies is basically a problem of sensemaking. We analyze how a group of “technology-use mediators” (Orlikowski et al. Org. Sci. (1995) 6(4), 423) in a large, multinational company adapted a groupware technology (a “virtual workspace”) to the local organizational context (and vice versa) by modifying features of the technology, providing ongoing support for users, and promoting appropriate conventions of use. Our findings corroborate earlier research on technology-use mediation, which suggests that such mediators can exert considerable influence on how a particular technology will be established and used in an organization. However, we also find that the process of technology-use mediation is much more complex and indeterminate than prior research suggests. The reason being, we argue, that new, advanced CSCW technologies, such as “virtual workspaces” and other groupware applications, challenge the mediators’ and users’ sensemaking, because the technologies are equivocal and, therefore, open to many possible and plausible interpretations.     

Commentary: Usability in Theory and Practice

This commentary discusses Noam Tractinsky’s article “The Usability Construct: A Dead End?” (Human–Computer Interaction, 2017). That article debates the conceptual qualities of “usability” as a key concept in human–computer interaction. It provides two main conclusions: (a) the usefulness of the concept of “usability” to human–computer interaction theories has been limited and (b) a notion of “construct” is useful for identifying weaknesses of the concept of “usability” and thereby provides reasons for its limited usefulness. The article is interesting and provides an in-depth analysis of the concept of usability. Unfortunately, the use of this analysis is not convincing. Instead, redefinition of “usability” should be based on a constructive approach.     

The influence of user expertise and prototype fidelity in usability tests

An empirical study examined the impact of user expertise and prototype fidelity on the outcomes of a usability test. User expertise (expert vs. novice) and prototype fidelity (paper prototype, 3D mock-up, and fully operational appliance) were manipulated as independent variables in a 2 × 3 between-subjects design. Employing a floor scrubber as a model product, 48 users carried out several cleaning tasks. Usability problems identified by participants were recorded. Furthermore, performance, system management strategies and perceived usability were measured. The results showed that experts reported more usability problems than novices but these were considered to be less severe than those reported by novices. Reduced fidelity prototypes were generally suitable to predict product usability of the real appliance. The implications for the running of usability tests are specific to the fidelity of the prototype.     

Development and Evaluation of a Groupware System to Support Chemical Design Processes

This article presents the concept of a groupware system adapted to the early stages of chemical design processes. To ensure best usability for chemical engineers during the chemical plant design, a field study in an international chemical group was carried out. Several theoretical concepts of cooperation and workspace design models were analyzed to meet the requirements of cooperative and distributed work during the design process. Based on this, a new groupware concept was developed that consists of 4 main design elements. This concept was evaluated empirically for the relevance of the design elements and efficiency of use.     

Enhancing usability testing through datamining techniques: A novel approach to detecting usability problem patterns for a context of use

Usability is a software attribute usually associated with the “ease of use and to learn” of a given interactive system. Nowadays usability evaluation is becoming an important part of software development, providing results based on quantitative and qualitative estimations. In this context, qualitative results are usually obtained through a Qualitative Usability Testing process which includes a number of different methods focused on analyzing the interface of a particular interactive system. These methods become complex when a large number of interactive systems belonging to the same context of use have to be jointly considered to provide a general diagnosis, as a considerable amount of information must be visualized and treated simultaneously. However, diagnosing the most general usability problems of a context of use as a whole from a qualitative viewpoint is a challenge for UE nowadays. Identifying such problems can help to evaluate a new interface belonging to this context, and to prevent usability errors when a novel interactive system is being developed. From a quantitative viewpoint, condensing results in singles scores, metrics or statistical functions is an acceptable solution for processing huge amounts of usability related information. Nevertheless, QUT processes need to keep their richness by prioritizing the “what” over the “how much/how many” questions related to the detection of usability problems.To cope with the above situation, this paper presents a new approach in which two datamining techniques (association rules and decision trees) are used to extend the existing Qualitative Usability Testing process in order to provide a general usability diagnosis of a given context of use from a qualitative viewpoint. In order to validate our proposal, usability problems patterns belonging to academic webpages in Spanish-speaking countries are assessed by processing 3450 records which store qualitative information collected by means of a Heuristic Evaluation.     

The System Usability Scale: Past,Present, and Future

The System Usability Scale (SUS) is the most widely used standardized questionnaire for the assessment of perceived usability. This review of the SUS covers its early history from inception in the 1980s through recent research and its future prospects. From relatively inauspicious beginnings, when its originator described it as a “quick and dirty usability scale,” it has proven to be quick but not “dirty.” It is likely that the SUS will continue to be a popular measurement of perceived usability for the foreseeable future. When researchers and practitioners need a measure of perceived usability, they should strongly consider using the SUS.     

SWOF - an open framework for shared workspaces to support different cooperation tasks

Computer supported collaborative work (CSCW) allows people to cooperate by computers from different places and at different times. To enable an easier integration of such collaborative components into web-based communities and portals, a Shared Workspace Open Framework (SWOF) was developed. This framework provides the basic features of shared workspaces and can be customized to different cooperation cases. High usability is an important aspect of the implementation. To achieve these aims SWOF focuses on an information space with more task-suited item-types that can help to pre-structure the information. Thus, on the one hand the system can help the users to write down the needed information in a consistent way and, on the other, could reduce the arguments between the group members on how to structure their workspace. As a use case for SWOF the development of a Web-based portal for the community of man-machine interaction was chosen. In the project MMI-Interaktiv, a portal is built with a SWOF-based shared workspace component. An evaluation for this use case is presented.     

SWOF - an open framework for shared workspaces to support different cooperation tasks

Computer supported collaborative work (CSCW) allows people to cooperate by computers from different places and at different times. To enable an easier integration of such collaborative components into web-based communities and portals, a Shared Workspace Open Framework (SWOF) was developed. This framework provides the basic features of shared workspaces and can be customized to different cooperation cases. High usability is an important aspect of the implementation. To achieve these aims SWOF focuses on an information space with more task-suited item-types that can help to pre-structure the information. Thus, on the one hand the system can help the users to write down the needed information in a consistent way and, on the other, could reduce the arguments between the group members on how to structure their workspace. As a use case for SWOF the development of a Web-based portal for the community of man-machine interaction was chosen. In the project MMI-Interaktiv, a portal is built with a SWOF-based shared workspace component. An evaluation for this use case is presented.     

一种基于黑板的共享对象协同操纵系统

文中提出一个基于黑板的共享对象协同操纵模型，对其中的关键技术黑板结构、共享对象的分布、并发控制、协作感知进行了详细的讨论。     

融合社交信息的局部潜在空间推荐方法

随着社交网络的发展,越来越多的研究利用社交信息来改进传统推荐算法的性能,然而现有的推荐算法大多忽略了用户兴趣的多样化,未考虑用户在不同社交维度中关心的层面不同,导致推荐质量较差.为了解决这个问题,提出了一种同时考虑全局潜在因子和不同子集特定潜在因子的推荐方法LSFS,使得推荐过程既考虑了用户共享偏好又考虑了用户在不同子集中的特定偏好.考虑到参与到不同社交维度的用户对不同的项目感兴趣,首先根据用户的社交关系将用户划分到不同的子集中;其次通过截断奇异值分解技术建模用户对项目的评分,其中全局潜在因子捕获用户共享的层面,而不同用户子集的特定潜在因子捕获用户关心的特定层面;最后,结合全局与局部潜在因子预测用户对未评分项目的评分.实验结果表明该方法可行且有效.     

Investigating users’ perspectives on e-learning: An integration of TAM and IS success model

The purpose of this paper is to examine an integrated model of TAM and D&M to explore the effects of quality features, perceived ease of use, perceived usefulness on users’ intentions and satisfaction, alongside the mediating effect of usability towards use of e-learning in Iran. Based on the e-learning user data collected through a survey, structural equations modeling (SEM) and path analysis were employed to test the research model. The results revealed that “intention” and “user satisfaction” both had positive effects on actual use of e-learning. “System quality” and “information quality” were found to be the primary factors driving users’ intentions and satisfaction towards use of e-learning. At last, “perceived usefulness” mediated the relationship between ease of use and users’ intentions. The sample consisted of e-learning users of four public universities in Iran. Past studies have seldom examined an integrated model in the context of e-learning in developing countries. Moreover, this paper tries to provide a literature review of recent published studies in the field of e-learning.     

Behavioral data mining to produce novel and serendipitous friend recommendations in a social bookmarking system

In the last few years, social media systems have experienced a fast growth. The amount of content shared in these systems increases fast, leading users to face the well known “interaction overload” problem, i.e., they are overwhelmed by content, so it becomes difficult to come across interesting items. To overcome this problem, social recommender systems have been recently designed and developed in order to filter content and recommend to users only interesting items. This type of filtering is usually affected by the “over-specialization” problem, which is related to recommendations that are too similar to the items already considered by the users. This paper proposes a friend recommender system that operates in the social bookmarking application domain and is based on behavioral data mining, i.e., on the exploitation of the users activity in a social bookmarking system. Experimental results show how this type of mining is able to produce accurate friend recommendations, allowing users to get to know bookmarked resources that are both novel and serendipitous. Using this approach, the impact of the “interaction overload” and the “over-specialization” problems is strongly reduced.     

Evaluation of earcons for portable digital electronic products considering perceptual user characteristics

Earcons, nonverbal sound feedback, have been used for electronic products to give appropriate feedback information for the selected user functions. This study evaluated earcon usability of a portable digital electronic product based on cognition time, error rate, and subjective feelings using 20 male and female subjects. For subjective evaluation, the study assessed various earcons by subjective impression of sounds using 7‐point rating scales. For earcon usability performance, major user functions were used for the product with currently available earcons and for the product with the new earcons (suggested by this study), which considered perceptual characteristics, such as loudness and melody. Statistical results from the study indicated that the new earcons significantly reduced user error rates and therefore generally improved user performance on major functions, such as “PLAY,” “OFF,” “STOP,” “FF” (fast forward), and “REW” (rewind). Subjective data results also showed that users were more satisfied with the new, melody‐based sound feedback. Practical guidelines for sound feedback design of a small digital product are suggested. © 2011 Wiley Periodicals, Inc.     

Work-domain knowledge in usability evaluation: Experiences with Cooperative Usability Testing

Usability evaluation helps to determine whether interactive systems support users in their work tasks. However, knowledge about those tasks and, more generally, about the work-domain is difficult to bring to bear on the processes and outcome of usability evaluation. One way to include such work-domain knowledge might be Cooperative Usability Testing, an evaluation method that consists of (a) interaction phases, similar to classic usability testing, and (b) interpretation phases, where the test participant and the moderator discuss incidents and experiences from the interaction phases. We have studied whether such interpretation phases improve the relevance of usability evaluations in the development of work-domain specific systems. The study included two development cases. We conclude that the interpretation phases generate additional insight and redesign suggestions related to observed usability problems. Also, the interpretation phases generate a substantial proportion of new usability issues, thereby providing a richer evaluation output. Feedback from the developers of the evaluated systems indicates that the usability issues that are generated in the interpretation phases have substantial impact on the software development process. The benefits of the interpretation phases may be explained by the access these provide both to the test participants’ work-domain knowledge and to their experiences as users.     

Do handheld devices facilitate face‐to‐face collaboration? Handheld devices with large shared display groupware to facilitate group interactions

Abstract   One-to-one computing environments change and improve classroom dynamics as individual students can bring handheld devices fitted with wireless communication capabilities into the classrooms. However, the screens of handheld devices, being designed for individual-user mobile application, limit promotion of interaction among groups of learners. This study proposes a design of classrooms that incorporates personal workspace and public workspace. Students use handheld devices as private workspace and work with peers on public workspace with shared displays through their handheld devices. Experiments confirmed that students with only handheld devices did not demonstrate expected participation ratios and actively interact with group members. The proposed shared display groupware promoted shared understanding of the workspace and increased awareness of partner actions. Collaboration was enhanced by creating the opportunity for students to use handheld devices to perform ideal communication patterns and avoiding ineffective communication patterns.     

Templates for Cross-Cultural and Culturally Specific Usability Testing: Results From Field Studies and Ethnographic Interviewing in Three Countries

The cultural diversity of users of technology challenges our methods for usability testing. This article suggests templates for cross-culturally and culturally specific usability testing, based on studies of usability testing in companies in Mumbai, Beijing, and Copenhagen. Study 1 was a cross-cultural field study of think-aloud testing done by usability vendor companies in the three countries. The result was a grounded theory of cultural variations in the production of a usability problem list. Study 2 was a follow-up, ethnographic interview study of how the companies typically perform usability tests. The result was the construction of templates for usability testing. The culturally specific templates were in Mumbai “user-centered evaluation,” Copenhagen “client-centered evaluation,” and Beijing “evaluator-centered evaluation.” The findings are compared with related research, and the implications are pointed out. The templates can be seen as a simple and practical way to plan, compare, and improve the way usability testing is carried out in multiple, different cultures and countries.     

Using a shared workspace and wireless laptops to improve collaborative project learning in an engineering design class

Two different technologies, groupware (a shared workspace) and shared wireless laptop computers, were implemented in a project design class in a civil engineering course. The research interest was in the way these technologies supported resource sharing within and across project groups and in the forms of group collaboration that resulted. The initiative was evaluated using both qualitative (e.g. pyramid discussion) and quantitative methods (e.g. survey, logs of usage). The results showed that these technologies helped improve group sharing of resources and supported different kinds of group collaboration. The shared workspace provided a location-independent central repository of resources around which group activities were coordinated whereas the laptops provided a focal point for the face-to-face discussion of these resources. The paper discusses the importance of embedding supportive technologies and the different forms of learner collaboration mediated by each technology.