Evaluating the usability of virtual reality user interfaces

A walkthrough method for evaluating virtual reality (VR) user interfaces is described and illustrated with a usability assessment of a virtual business park application. The method is based on a theory of interaction that extends Norman's model of action. A walkthrough analysis method uses three models derived from the theory. The first model describes goal-oriented task action, the second exploration and navigation in virtual worlds, while the third covers interaction in response to system initiative. Each stage of the model is associated with generic design properties that specify the necessary support from the system for successful interaction. The evaluation method consists of a checklist of questions using the properties and following the model cycle. Use of the method uncovered several usability problems. Approaches to evaluation of VR applications and future work are discussed.     

Investigating the usability of assistive user interfaces

A prototype e-mail system was developed for cognitively disabled users, with four different interfaces (free format, idea prompt, form fill and menu driven). The interfaces differed in the level of support provided for the user and complexity of facilities for composing e-mail messages. Usability evaluation demonstrated that no one interface was superior because of individual differences in usability problems, although the majority of users preferred interfaces which did not restrict their freedom of expression (free format). In contrast to traditional evaluation studies, no common pattern of usability errors emerged, demonstrating the need for customisation of interfaces for individual cognitively disabled users. A framework for customising user interfaces to individual users is proposed, and usability principles derived from the study are expressed as claims following the task artefact cycle.     

Assessing the usability of icons in user interfaces

Abstract

This paper presents a methodology and results of an experiment to assess the usability of menu items constructed of text, icons, and text-and-icons. Attributes of menu items are used to form a matrix which can be used to classify menu items for use in certain applications, tasks, or with users of particular experience levels. An experiment was conducted to validate a portion of the attribute matrix. Performance measures were accuracy of selection and time to make a selection. Results suggest that menus constructed of a mixed format (text and icons) result in the fewest number of incorrect selections by users. No significant differences in the time to make a selection were found.     

Evaluating and improving the usability of a user manual

This research deals with user-testing as a means to improve the usability of a user manual. In Study 1 we analysed the effects of three different methods for user-testing a commercial user manual to a patient administrative system. Thirty nurses read the manual and noted difficulties found by either (1) making underlinings, (2) writing questions, or (3) verbalizing their comments aloud. Underlinings were more associated with difficult words or concepts, and question writing with broader content issues. Comments verbalized aloud produced the greatest variety of information and the greatest number of comments. In Study 2 we analysed the effects of rewriting the user manual on the basis of the results from the user- tests performed in Study 1 and some general writing principles. The modified manual group spent significantly less time (21% less) on the tasks, issued significantly fewer ineffective commands and made significantly fewer obstructive errors, compared to the original manual group. However, no differences were found with respect to error recovery. These results suggest that user-testing by means of the investigated methods can be an important means to enhance the usability of user instruction manuals.     

The influence of design decisions on the usability of direct manipulation user interfaces

It is assumed that the usability of direct manipulation user interfaces is influenced by a number of design aspects. In this experimental study, the order of command specification and the type of function activation were manipulated in a 2 × 2 factorial design, in order to test hypotheses H1, that object-function specification contributes more to usability than function-object specification; and H2. that the type of function activation (clicking vs. dragging) will influence the usability of direct manipulation user interfaces. Sixty-four subjects, balanced by sex, without computer experience, were assigned randomly to the four experimental conditions. The dependent variables include performance data such as time, efficiency and error rates (logfile-recording), and subjective user rating of the user interface (questionnaire). Whereas HI had to be rejected in this general form, a more elaborated analysis showed significant differences between the factor levels in terms of performance time and syntactically correct actions. Furthermore, the results of the study demonstrated evidence for H2.     

Information at a glance [augmented reality user interfaces]

What if we could visualize and interact with information directly in the context of our surroundings? Our research group is exploring how augmented reality (AR) could someday make this possible. AR integrates a complementary virtual world with the physical world-for example, by using head-tracked see-through head-worn displays to overlay graphics on what we see. Instead of looking back and forth between the real world and a PDA, we look directly at the real world and the virtual information overlaid on it. At the heart of this approach is context-aware computing, computing systems that are sensitive to the context in which they operate, ranging from human relationships to physical location. For example, information might be tied to specific locations within a global, Earth-centered, coordinate system. How can we design effective mobile AR user interfaces? We've been trying to answer this question in part by developing experimental AR research prototypes. In AR, as in work on information visualization using desktop technologies, the amount of information available can far exceed what a system can legibly display at a given time, necessitating information filtering. Julier et al. (2000) have developed information filtering techniques for AR that depend on the user's goals, object importance, and proximity. We assume that a system can accomplish information filtering of this sort and that our system is displaying everything it should.     

Presence and simulator sickness predict the usability of a virtual reality attention task

Virtual Reality - Attention is the ability to actively process specific information within one’s environment over longer periods of time while disregarding other details. Attention is an...     

Evaluating user interfaces in context: the ecological value of time-and-motion studies

Findings are summarized from a study which set out to identify cognitive stumbling blocks in the user interface of a large computer system used by telephone operators in Australia. Intermittent observations were made of operators' actions in the workplace over a period of eight months before, during and after system implementation. Numerous weaknesses were identified in the user interface, but the most interesting aspect of the study turned out to be the analysis of the overall jobs operators are required to do, as a by-product of the intended study. The insight into operator job demands led to changes in job selection criteria and training, which were found to match actual job demands quite poorly at the time the study was conducted. The paper describes the process by which the usability assessment of the user-system interface led to a comprehensive job analysis. Supporting quantitative data are presented, together with anecdotal examples which demonstrate the importance of conducting systems analysis in the working context, thereby maximizing the ecological value of the resulting research data.     

Hand-adaptive user interface: improved gestural interaction in virtual reality

Most interactive user interfaces (UIs) for virtual reality (VR) applications are based on the traditional eye-centred UI design principle, which primarily considers the user’s visual searching efficiency and comfort, but the hand operation performance and ergonomics are relatively less considered. As a result, the hand interaction in VR is often criticized as being less efficient and precise. In this paper, the user’s arm movement features, such as the choice of the hand being used and hand interaction position, are hypothesized to influence the interaction results derived from a VR study. To verify this, we conducted a free hand target selection experiment with 24 participants. The results showed that (a) the hand choice had a significant effect on the target selection results: for a left hand interaction, the targets located in spaces to the left were selected more efficiently and accurately than those in spaces to the right; however, in a right hand interaction, the result was reversed, and (b) the free hand interactions at lower positions were more efficient and accurate than those at higher positions. Based on the above findings, this paper proposes a hand-adaptive UI technique to improve free hand interaction performance in VR. A comprehensive comparison between the hand-adaptive UI and traditional eye-centred UI was also conducted. It was shown that the hand-adaptive UI resulted in a higher interaction efficiency and a lower physical exertion and perceived task difficulty than the traditional UI.

     

A rapid prototyping software infrastructure for user interfaces in ubiquitous augmented reality

Recent user interface concepts, such as multimedia, multimodal, wearable, ubiquitous, tangible, or augmented-reality-based (AR) interfaces, each cover different approaches that are all needed to support complex human–computer interaction. Increasingly, an overarching approach towards building what we call ubiquitous augmented reality (UAR) user interfaces that include all of the just mentioned concepts will be required. To this end, we present a user interface architecture that can form a sound basis for combining several of these concepts into complex systems. We explain in this paper the fundamentals of DWARFs user interface framework (DWARF standing for distributed wearable augmented reality framework) and an implementation of this architecture. Finally, we present several examples that show how the framework can form the basis of prototypical applications.     

Evaluating the usability of natural language query languages and interfaces to Semantic Web knowledge bases

The need to make the contents of the Semantic Web accessible to end-users becomes increasingly pressing as the amount of information stored in ontology-based knowledge bases steadily increases. Natural language interfaces (NLIs) provide a familiar and convenient means of query access to Semantic Web data for casual end-users. While several studies have shown that NLIs can achieve high retrieval performance as well as domain independence, this paper focuses on usability and investigates if NLIs and natural language query languages are useful from an end-user's point of view. To that end, we introduce four interfaces each allowing a different query language and present a usability study benchmarking these interfaces. The results of the study reveal a clear preference for full natural language query sentences with a limited set of sentence beginnings over keywords or formal query languages. NLIs to ontology-based knowledge bases can, therefore, be considered to be useful for casual or occasional end-users. As such, the overarching contribution is one step towards the theoretical vision of the Semantic Web becoming reality.     

On the usability and likeability of virtual reality games for education: The case of VR-ENGAGE

Educational software games aim at increasing the students’ motivation and engagement while they learn. However, if software games are targeted to school classrooms they have to be usable and likeable by all students. Usability of virtual reality games may be a problem because these games tend to have complex user interfaces so that they are more attractive. Moreover, if the games acquire an educational content they may lose the attractiveness and appeal that they have on users who are familiar with commercial games. Consequently, likeability may also be questioned. In this paper, we address the issue of usability and likeability of a virtual reality game that is meant to teach students geography. We describe the evaluation experiments conducted, which involved three categories of students in terms of their level of game-playing expertise: novice, intermediate and expert game players. The evaluation results showed that the game was indeed usable and likeable but there was scope for usability and likeability improvement so that the educational benefits may be maximised for all categories of students. The evaluation studies reported in this paper, revealed important issues about further research on virtual reality educational games.     

Reengineering user interfaces

Most developers would like to avoid redesigning a system around a new interface. But turning a character-based interface into a graphical one requires significant time and resources. The authors describe how this process can be partially automated, giving the results of their own reverse-engineering effort     

Permissive user interfaces

User interfaces often only support one way to do a task when the physical interface or the requirements of the task would permit other ways. In contrast, a user interface that supports multiple approaches is permissive. This paper argues that permissive user interfaces are easier to use—and even when they are not applicable for particular applications, considering permissiveness is a productive design heuristic.Many user interfaces are difficult to use yet very easily demonstrated or explained by experts—with the results that users become frustrated because hindsight makes usability problems look like the user's own fault. The lack of permissiveness in such user interfaces explains this paradox.     

用户认知驱动的VR自然交互认知负荷研究

针对目前虚拟现实自然交互系统中用户认知负荷难以量化问题,以量化认知负荷为目的,提出一种基于概率神经网络的多通道信息融合模型。首先根据认知理论建立“认知—行为—环境”框架,并在此基础上研究用户认知的数学表征（方法,过程,机理）,构建多通道认知模型;然后将多通道信息融合方法引入虚拟现实自然交互系统,建立一种面向虚拟现实自然交互系统的认知负荷模型;通过VR隧道应急救援系统实验对系统中用户认知进行验证。验证结果表明,具体提出的认知负荷模型具有一定的可行性。     

A virtual reality approach for usability assessment: case study on a wheelchair-mounted robot manipulator

The aim of the paper is to address an innovative methodology for assessing the usability of a product. This methodology is particularly suitable for designing products that provide their main functions through their control interfaces. In particular, this case study relates to the usability assessment of two control devices for a wheelchair-mounted robot manipulator to assist physically disabled people. The study focuses on defining a synthetic usability index on the basis of two currently used methods: the multi criteria decision analysis and the Saaty’s analytic hierarchy process. Several virtual reality (VR)-based experiments have been conducted, set up in accordance with a cross-array experimental plan, that adequately caters for both control and noise factors. Quantitative measures and subjective user evaluations have been collected to maximize the effectiveness, the efficiency and the satisfaction perceived by users while using the product. Compared to the literature on the subject, the proposed approach provides both more flexibility in defining quantitative indexes and more adequate results, even when involving only a small sample of users in the participatory design session. The use of VR technologies for the collection of the experimental data has been essential in terms of safety, costs and repeatability of the tests, as well as of the robustness with respect to noise factors.     

Three views of virtual reality: nonimmersive virtual reality

Nonimmersive virtual reality (VR), which places the user in a 3D environment that can be directly manipulated with a conventional graphics workstation using a monitor, a keyboard; and a mouse, is discussed. The scene is displayed with the same 3D depth cues used in immersive VR: perspective view, hidden-surface elimination, color, texture, lighting, shading and shadows. As in immersive VR, animation and simulation are interactively controlled in response to the user's direct manipulation. Much of the technology used to support immersive and nonimmersive VR is the same. They use the same 3D modeling and rendering and many of the same interaction techniques. The advantages and applications of nonimmersive VR systems are discussed. Immersive and nonimmersive VR systems are compared and hybrid possibilities are reviewed     

Evaluating display fidelity and interaction fidelity in a virtual reality game

In recent years, consumers have witnessed a technological revolution that has delivered more-realistic experiences in their own homes through high-definition, stereoscopic televisions and natural, gesture-based video game consoles. Although these experiences are more realistic, offering higher levels of fidelity, it is not clear how the increased display and interaction aspects of fidelity impact the user experience. Since immersive virtual reality (VR) allows us to achieve very high levels of fidelity, we designed and conducted a study that used a six-sided CAVE to evaluate display fidelity and interaction fidelity independently, at extremely high and low levels, for a VR first-person shooter (FPS) game. Our goal was to gain a better understanding of the effects of fidelity on the user in a complex, performance-intensive context. The results of our study indicate that both display and interaction fidelity significantly affect strategy and performance, as well as subjective judgments of presence, engagement, and usability. In particular, performance results were strongly in favor of two conditions: low-display, low-interaction fidelity (representative of traditional FPS games) and high-display, high-interaction fidelity (similar to the real world).