Usability Principles for Augmented Reality Applications in a Smartphone Environment

Through the rapid spread of smartphones, users have access to many types of applications similar to those on desktop computer systems. Smartphone applications using augmented reality (AR) technology make use of users' location information. As AR applications will require new evaluation methods, improved usability and user convenience should be developed. The purpose of the current study is to develop usability principles for the development and evaluation of smartphone applications using AR technology. We develop usability principles for smartphone AR applications by analyzing existing research about heuristic evaluation methods, design principles for AR systems, guidelines for handheld mobile device interfaces, and usability principles for the tangible user interface. We conducted a heuristic evaluation for three popularly used smartphone AR applications to identify usability problems. We suggested new design guidelines to solve the identified problems. Then, we developed an improved AR application prototype of an Android-based smartphone, which later was conducted a usability testing to validate the effects of usability principles.     

一种可行的可用性测试过程

文章综述用户模型法、用户调查法、专家评审法和观察法4类可用性测试方法,根据可用性测试的经验,总结出一套合理实用的可用性测试过程,使得可用性测试的效率高、代价低,尽可能发现更多的可用性问题;提出在程序设计类课程的教学中,应适当向学生介绍可用性测试的方法和过程,以培养学生对自己开发的软件系统进行可用性测试的能力。     

A comparison of usability evaluation methods for evaluating e-commerce websites

The importance of evaluating the usability of e-commerce websites is well recognised. User testing and heuristic evaluation methods are commonly used to evaluate the usability of such sites, but just how effective are these for identifying specific problems? This article describes an evaluation of these methods by comparing the number, severity and type of usability problems identified by each one. The cost of employing these methods is also considered. The findings highlight the number and severity level of 44 specific usability problem areas which were uniquely identified by either user testing or heuristic evaluation methods, common problems that were identified by both methods, and problems that were missed by each method. The results show that user testing uniquely identified major problems related to four specific areas and minor problems related to one area. Conversely, the heuristic evaluation uniquely identified minor problems in eight specific areas and major problems in three areas.     

A comparison of methods currently used in inclusive design

Inclusive design has unique challenges because it aims to improve usability for a wide range of users. This typically includes people with lower levels of ability, as well as mainstream users. This paper examines the effectiveness of two methods that are used in inclusive design: user trials and exclusion calculations (an inclusive design inspection method). A study examined three autoinjectors using both methods (n = 30 for the user trials). The usability issues identified by each method are compared and the effectiveness of the methods is discussed. The study found that each method identified different kinds of issues, all of which are important for inclusive design. We therefore conclude that a combination of methods should be used in inclusive design rather than relying on a single method. Recommendations are also given for how the individual methods can be used more effectively in this context.     

Usability testing in 2000 and beyond

Usability testing is a widely used technique to evaluate user performance and acceptance of products and systems. It was introduced in the late 1980s and rose to popularity in the past decade. This paper provides a view of the current status of usability testing as a method and describes how it will be used in the 21st century. Although usability testing may not be the most efficient technique for discovery of usability problems, it is a reliable way to estimate quantitatively users' performance and subjective satisfaction with products. Four major trends in usability testing include: common reporting formats and methods for industry; Internet application and website testing; testing of mobile, handheld devices; and testing in more naturalistic environments such as simulated homes and classrooms. In the 21st century, ‘quick and clean’ usability testing methods are needed to provide valid and reliable data on how well people use products and systems, and how they like using them.     

Determining the effectiveness of the usability problem inspector: a theory-based model and tool for finding usability problems

Despite the increased focus on usability and on the processes and methods used to increase usability, a substantial amount of software is unusable and poorly designed. Much of this is attributable to the lack of cost-effective usability evaluation tools that provide an interaction-based framework for identifying problems. We developed the user action framework and a corresponding evaluation tool, the usability problem inspector (UPI), to help organize usability concepts and issues into a knowledge base. We conducted a comprehensive comparison study to determine if our theory-based framework and tool could be effectively used to find important usability problems in an interface design, relative to two other established inspection methods (heuristic evaluation and cognitive walkthrough). Results showed that the UPI scored higher than heuristic evaluation in terms of thoroughness, validity, and effectiveness and was consistent with cognitive walkthrough for these same measures. We also discuss other potential advantages of the UPI over heuristic evaluation and cognitive walkthrough when applied in practice. Potential applications of this work include a cost-effective alternative or supplement to lab-based formative usability evaluation during any stage of development.     

Must evaluation methods be about usability? Devising and assessing the utility inspection method

Whereas research in usability evaluation abounds, few evaluation approaches focus on utility. We present the utility inspection method (UIM), which prompts evaluators about the utility of the system they evaluate. The UIM asks whether a system uses global platforms, provides support infrastructure, is robust, gives access to rich content, allows customisation, offers symbolic value and supports companionship among users and between users and developers. We compare 47 participants’ use of UIM and heuristic evaluation (HE). The UIM helps identify more than three times as many problems as HE about the context of activities; HE helps identify 2.5 times as many problems as UIM about the interface. Usability experts consider the problems found with UIM more severe and more complex to solve compared to those found with HE. We argue that UIM complements existing usability evaluation methods and discuss future research on utility inspection.     

Analyzing the Uses of a Software Modeling Tool

While a lot of progress has been made in improving analyses and tools that aid software development, less effort has been spent on studying how such tools are commonly used in practice. A study into a tool's usage is important not only because it can help improve the tool's usability but also because it can help improve the tool's underlying analysis technology in a common usage scenario. This paper presents a study that explores how (beginner) users work with the Alloy Analyzer, a tool for automatic analysis of software models written in Alloy, a first-order, declarative language. Alloy has been successfully used in research and teaching for several years, but there has been no study of how users interact with the analyzer. We have modified the analyzer to log (some of) its interactions with the user. Using this modified analyzer, 11 students in two graduate classes formulated their Alloy models to solve a problem set (involving two problems, each with one model). Our analysis of the resulting logs (total of 68 analyzer sessions) shows several interesting observations; based on them, we propose how to improve the analyzer, both the performance of analyses and the user interaction. Specifically, we show that: (i) users often perform consecutive analyses with slightly different models, and thus incremental analysis can speed up the interaction; (ii) users' interaction with the analyzer is sometimes predictable, and akin to continuous compilation, the analyzer can precompute the result of a future action while the user is editing the model; and (iii) (beginner) users can naturally develop semantically equivalent models that have significantly different analysis time, so it is useful to study manual and automatic model transformations that can improve performance.     

Barriers common to mobile and disabled web users

World Wide Web accessibility and best practice audits and evaluations are becoming increasingly complicated, time consuming, and costly because of the increasing number of conformance criteria which need to be tested. In the case of web access by disabled users and mobile users, a number of commonalities have been identified in usage, which have been termed situationally-induced impairments; in effect the barriers experienced by mobile web users have been likened to those of visually disabled and motor impaired users. In this case, we became interested in understanding if it was possible to evaluate the problems of mobile web users in terms of the aggregation of barriers-to-access experienced by disabled users; and in this way attempt to reduce the need for the evaluation of the additional conformance criteria associated with mobile web best practice guidelines. We used the Barrier Walkthrough (BW) method as our analytical framework. Capable of being used to evaluate accessibility in both the disabled and mobile contexts, the BW method would also enable testing and aggregation of barriers across our target user groups.We tested 61 barriers across four user groups each over four pages with 19 experts and 57 non-experts focusing on the validity and reliability of our results. We found that 58% of the barrier types that were correctly found were identified as common between mobile and disabled users. Further, if our aggregated barriers alone were used to test for mobile conformance only four barrier types would be missed. Our results also showed that mobile users and low vision users have the most common barrier types, while low vision and motor impaired users experiencing similar rates of severity in the barriers they experienced. We conclude that the aggregated evaluation results for blind, low vision and motor impaired users can be used to approximate the evaluation results for mobile web users.     

Tracking the effectiveness of usability evaluation methods

Abstract

We present a case study that tracks usability problems predicted with six usability evaluation methods (claims analysis, cognitive walkthrough, GOMS, heuristic evaluation, user action notation, and simply reading the specification) through a development process. We assess the method's predictive power by comparing the predictions to the results of user tests. We assess the method's persuasive power by seeing how many problems led to changes in the implemented code. We assess design-change effectiveness by user testing the resulting new versions of the system. We concludethatpredictivemethodsarenot as effective as the HCI field would like and discuss directions for future research.     

Examining modality usage in a conversational multimodal application for mobile e-mail access

As Third Generation (3G) networks emerge they provide not only higher data transmission rates but also the ability to transmit both voice and low latency data within the same session. This paper describes the architecture and implementation of a multimodal application (voice and text) that uses natural language understanding combined with a WAP browser to access email messages on a cell phone. We present results from the use of the system by users as part of a laboratory trial that evaluated usage. The user trial also compared the multimodal system with a text-only system that is representative of current products in the market today. We discuss the observed modality issues and highlight implementation problems and usability concerns that were encountered in the trial. Findings indicate that speech was used the majority of the time by participants for both input and navigation even though most of the participants had little or no prior experience with speech systems (yet did have prior experience with text-only access to applications on their phones). To our knowledge this represents the first implementation and evaluation of its kind using this combination of technologies on an unmodified cell phone. Design implications resulting from the study findings and usability issues encountered are presented to inform the design of future conversational multimodal mobile applications.     

Tracking the effectiveness of usability evaluation methods

. We present a case study that tracks usability problems predicted with six usability evaluation methods (claims analysis, cognitive walkthrough, GOMS, heuristic evaluation, user action notation, and simply reading the specification) through a development process. We assess the method's predictive power by comparing the predictions to the results of user tests. We assess the method's persuasive power by seeing how many problems led to changes in the implemented code. We assess design-change effectiveness by user testing the resulting new versions of the system. We concludethatpredictivemethodsarenot as effective as the HCI field would like and discuss directions for future research.     

In situ evaluation of recommender systems: Framework and instrumentation

This paper deals with the evaluation of the recommendation functionality inside a connected consumer electronics product in prototype stage. This evaluation is supported by a framework to access and analyze data about product usage and user experience. The strengths of this framework lie in the collection of both objective data (i.e., “What is the user doing with the product?”) and subjective data (i.e., “How is the user experiencing the product?”), which are linked together and analyzed in a combined way. The analysis of objective data provides insights into how the system is actually used in the field. Combined with the subjective data, personal opinions and evaluative judgments on the product quality can be then related to actual user behavior. In order to collect these data in a most natural context, remote data collection allows for extensive user testing within habitual environments. We have applied our framework to the case of an interactive TV recommender system application to illustrate that the user experience of recommender systems can be evaluated in real-life usage scenarios.     

A method to elicit architecturally sensitive usability requirements: its integration into a software development process

In the Human-Computer Interaction (HCI) community, software usability has primarily been concerned with the presentation of information, more precisely with the user interface. However, some usability problems can prove costly to fix if the changes require modifications that reach beyond the presentation layer, namely those that cannot be easily accommodated by the software architecture. Taking into account some usability requirements earlier in the software development cycle, specifically prior to the architectural design phase, can reduce the cost of these modifications. There is a scarcity of methods and guidelines with the scope to direct users in eliciting the usability requirements that can impact the software architecture. This paper proposes a usability-driven adaptation of the quality attribute workshop (QAW) to assist software development organizations in discovering and documenting usability requirements. It shows how this method can be integrated into a software development process, by discussing how the existing software framework workflows can be adjusted to take this new activity into consideration. A preliminary exercise was conducted to help discern the utility and the limits of the proposed method. Participants with different levels of knowledge of usability and comprehension of the system being developed found the method constructive, as it guided them in identifying the architecturally relevant usability requirements. It also helped determine the usability aspects that would not necessarily have been defined if this technique had not been employed.     

Leveraging app usage contexts for app recommendation: a neural approach

The large volume and variety of apps pose a great challenge for people to choose appropriate apps. As a consequence, app recommendation is becoming increasingly important. Recently, app usage data which record the sequence of apps being used by a user have become increasingly available. Such data record the usage context of each instance of app use, i.e., the app instances being used together with this app (within a short time window). Our empirical data analysis shows that a user has a pattern of app usage contexts. More importantly, the similarity in the two users’ preferences over mobile apps is correlated with the similarity in their app usage context patterns. Inspired by these important observations, this paper tries to leverage the predictive power of app usage context patterns for effective app recommendation. To this end, we propose a novel neural approach which learns the embeddings of both users and apps and then predicts a user’s preference for a given app. Our neural network structure models both a user’s preference over apps and the user’s app usage context pattern in a unified way. To address the issue of unbalanced training data, we introduce several sampling methods to sample user-app interactions and app usage contexts effectively. We conduct extensive experiments using a large real app usage data. Comparative results demonstrate that our approach achieves higher precision and recall, compared with the state-of-the-art recommendation methods.

     

A prediction and auto-execution system of smartphone application services based on user context-awareness

Smartphones have emerged as suitable environments for user context-awareness and intelligent service provision due to the high penetration rate, the high usability, various embedded sensors, and so on. In particular, its most unique characteristic is the usage of various applications. However, the most of existing studies through the three steps process (log collection, context inference, and service provision) did not consider smartphone applications (Apps) as the target service. Smartphone users still have to use Apps with manual controls by own decision. Therefore, in this paper, we propose a system to predict smartphone applications based on inferring user context. We define a mobile context model with a new level of context (Situation) and its inference method to perceive a user’s intention or purpose related to the App usage. Based on the Situation context, the system predicts Apps which can be useful and helpful for a user and automatically executes it on his/her smartphone. With the proposed system, it will be possible to autonomously provide and manage smartphone application services without users’ perception or intervention.     

Usability testing in 2000 and beyond

Usability testing is a widely used technique to evaluate user performance and acceptance of products and systems. It was introduced in the late 1980s and rose to popularity in the past decade. This paper provides a view of the current status of usability testing as a method and describes how it will be used in the 21st century. Although usability testing may not be the most efficient technique for discovery of usability problems, it is a reliable way to estimate quantitatively users' performance and subjective satisfaction with products. Four major trends in usability testing include: common reporting formats and methods for industry; Internet application and website testing; testing of mobile, handheld devices; and testing in more naturalistic environments such as simulated homes and classrooms. In the 21st century, 'quick and clean' usability testing methods are needed to provide valid and reliable data on how well people use products and systems, and how they like using them.     

Designing personal attentive user interfaces in the mobile public safety domain

In the mobile computing environment, there is a need to adapt the information and service provision to the momentary attentive state of the user, operational requirements and usage context. This paper proposes to design personal attentive user interfaces (PAUI) for which the content and style of information presentation is based on models of relevant cognitive, task, context and user aspects. Using the police work environment as the application domain, relevant attributes of these aspects are identified based on literature and domain analyses. We present a user-centered design (UCD) method for the iterative development and validation of the proposed PAUI. Application of this approach provided requirements for (1) adaptation to users’ attentive state, (2) notification, (3) information processing and task switching support and (4) user modeling. We aim at refining and validating the models and requirements through continuing empirical evaluation.     

Effectiveness of user testing and heuristic evaluation as a function of performance classification

For different levels of user performance, different types of information are processed and users will make different types of errors. Based on the error's immediate cause and the information being processed, usability problems can be classified into three categories. They are usability problems associated with skill-based, rule-based and knowledge-based levels of performance. In this paper, a user interface for a Web-based software program was evaluated with two usability evaluation methods, user testing and heuristic evaluation. The experiment discovered that the heuristic evaluation with human factor experts is more effective than user testing in identifying usability problems associated with skill-based and rule-based levels of performance. User testing is more effective than heuristic evaluation in finding usability problems associated with the knowledge-based level of performance. The practical application of this research is also discussed in the paper.