Human performance in a multiple-task environment: Effects of automation reliability on visual attention allocation

Diagnostic automation has been posited to alleviate the high demands of multiple-task environments; however, mixed effects have been found pertaining to performance aid success. To better understand these effects, attention allocation must be studied directly. We developed a multiple-task environment to study the effects of automation on visual attention. Participants interacted with a system providing varying levels of automation and automation reliability and then were transferred to a system with no support. Attention allocation was measured by tracking the number of times each task was viewed. We found that participants receiving automation allocated their time according to the task frequency and that tasks that benefited most from automation were most harmed when it was removed. The results suggest that the degree to which automation affects multiple-task performance is dependent on the relative attributes of the tasks involved. Moreover, there is an inverse relationship between support and cost when automation fails.     

Model-based automation of the design of user interfaces to digitalcontrol systems

Digital control systems, like those controlling the functions of buildings or industrial processes, pose a number of special problems for good user interface design. The general problems of providing usability, common to all systems, include difficulty in accessing and applying principles of good design. In addition, digital control systems can have multiple users, with multiple roles, and each installation has different configurations of systems, controls, and user interface devices. Providing interactions for the users of building control systems is often achieved by manually implementing each required display. This is an expensive solution, which often produces less than optimal results. We address these problems through the automation of user interface design. Our solution, called DIGBE (dynamic interface generation for building environments), separates the domain knowledge, interaction design, and presentation heuristics into multiple collaborating models. Each model contains knowledge about a particular aspect of interface design, and uses this knowledge to dynamically create each user interface that is needed to support the users of a control system. DIGBE demonstrates that it is possible to automatically and dynamically create consistent and individualized user interfaces from model-based design knowledge     

Mobile information access in the real world: A story of three wireless devices

The importance of the user perspective to the wireless information access experience cannot be understated: simply put, users will not indulge in devices that are perceived to be difficult to use and in technologies that do not offer quality infotainment – combined information and entertainment – content. In this paper, we investigate the impact that mobile devices have on the user wireless infotainment access experience in practice. To this end, we have undertaken an empirical study placed in a ‘real-world’ setting, in which participants undertook typical infotainment access tasks on three different wireless-enabled mobile devices: a laptop, a personal digital assistant and a head mounted display device. Results show that, with the exception of participants’ level of self-consciousness when using such devices in public environments, the user wireless information access experience is generally unaffected by device type. Location was shown, though, to be a significant factor when users engage in tasks such as listening to online music or navigation. Whilst the interaction between device and environment was found to influence entertainment-related tasks in our experiments, the informational ones were not affected. However, the interaction effects between device and user type was found to affect both types of tasks. Lastly, a user’s particular computing experience was shown to influence the perceived ease of wireless information access only in the case of online searching, irrespective of whether this is done for primarily informational purposes or entertainment ones.     

Using job-shop scheduling tasks for evaluating collocated collaboration

Researchers have begun to explore tools that allow multiple users to collaborate across multiple devices in collocated environments. These tools often allow users to simultaneously place and interact with information on shared displays. Unfortunately, there is a lack of experimental tasks to evaluate the effectiveness of these tools for information coordination in such scenarios. In this article, we introduce job-shop scheduling as a task that could be used to evaluate systems and interactions within computer-supported collaboration environments. We describe properties that make the task useful, as well as evaluation measures that may be used. We also present two experiments as case studies to illustrate the breadth of scenarios in which this task may be applied. The first experiment shows the differences when users interact with different communicative gesturing schemes, while the second demonstrates the benefits of shared visual information on large displays. We close by discussing the general applicability of the tasks.     

An examination of interactions in a three-dimensional virtual world

Three-dimensional (3D) virtual worlds hold the users' attention by providing rich interaction in an environment similar to the real world. User engagement duration is known to increase in environments with intense interaction. However, information in the literature about whether gender, experience, or spatial ability affects interaction in these environments is limited. In this study, these three factors are compared to users' depth of interaction in a 3D virtual world. In addition, the relationships between engagement duration, spatial ability, and depth of interaction are examined to investigate whether the first two factors can predict the third. Findings showed that users' depth of interaction was not influenced by gender, but experience and spatial ability did affect interaction. A strong relationship was determined between depth of interaction and engagement duration, and a moderate relationship was found between depth of interaction and spatial ability. Findings indicated that when designing 3D environments, it is important to consider which kinds of tasks provide more interaction and to what extent spatial abilities affect interaction, as well as to prepare activities that will increase engagement duration and to devise strategies to enhance depth of interaction.     

A Spoken Dialogue System for Navigation in Non-Immersive Virtual Environments

Navigation is the process by which people control their movement in virtual environments and is a corefunctional requirement for all virtual environment (VE) applications. Users require the ability to move, controllingorientation, direction of movement and speed, in order to achieve a particular goal within a VE. Navigation israrely the end point in itself (which is typically interaction with the visual representations of data) but applicationsoften place a high demand on navigation skills, which in turn means that a high level of support for navigationis required from the application. On desktop systems navigation in non‐immersive systems is usually supportedthrough the usual hardware devices of mouse and keyboard. Previous work by the authors shows that many usersexperience frustration when trying to perform even simple navigation tasks — users complain about getting lost,becoming disorientated and finding the interface `difficult to use'. In this paper we report on work in progressin exploiting natural language processing (NLP) technology to support navigation in non‐immersive virtualenvironments. A multi‐modal system has been developed which supports a range of high‐level (spoken) navigationcommands and indications are that spoken dialogue interaction is an effective alternative to mouse and keyboardinteraction for many tasks. We conclude that multi‐modal interaction, combining technologies such as NLP withmouse and keyboard may offer the most effective interaction with VEs and identify a number of areas where furtherwork is necessary. ACM CSS: I.3.6 Computer Graphics Methodology and Techniques—Interaction and Techniques, I.3.7 Three‐DimensionalGraphics and Realism—Virtual Reality, I.2.7 Natural Language Processing—Speech Recognitionand Synthesis     

Comparison of a brain-based adaptive system and a manual adaptable system for invoking automation

OBJECTIVE: Two experiments are presented examining adaptive and adaptable methods for invoking automation. BACKGROUND: Empirical investigations of adaptive automation have focused on methods used to invoke automation or on automation-related performance implications. However, no research has addressed whether performance benefits associated with brain-based systems exceed those in which users have control over task allocations. METHOD: Participants performed monitoring and resource management tasks as well as a tracking task that shifted between automatic and manual modes. In the first experiment, participants worked with an adaptive system that used their electroencephalographic signals to switch the tracking task between automatic and manual modes. Participants were also divided between high- and low-reliability conditions for the system-monitoring task as well as high- and low-complacency potential. For the second experiment, participants operated an adaptable system that gave them manual control over task allocations. RESULTS: Results indicated increased situation awareness (SA) of gauge instrument settings for individuals high in complacency potential using the adaptive system. In addition, participants who had control over automation performed more poorly on the resource management task and reported higher levels of workload. A comparison between systems also revealed enhanced SA of gauge instrument settings and decreased workload in the adaptive condition. CONCLUSION: The present results suggest that brain-based adaptive automation systems may enhance perceptual level SA while reducing mental workload relative to systems requiring user-initiated control. APPLICATION: Potential applications include automated systems for which operator monitoring performance and high-workload conditions are of concern.     

Bio-signal-based geometric modeling application for physically disabled users

This paper discusses the challenges in achieving bio-signal-based design environments. While the main motivation of this paper was to provide a user interface for physically disabled people to express their artistic natures, a special emphasis is given on graphical user interface design where bio-signals are the single input source. Among three bio-signal sources investigated—electromyography, electrooculography and electroencephalography (EEG)—stimulus-based human–computer interaction design (EEG feature extraction method) is found to be the most promising for achieving design environments to perform complex tasks. In the proposed stimulus-based brain–computer-interaction application, the user communication with a computer is achieved by coupling intended functionalities with stimuli signals on the computer screen. Constant focus on the intended command stimulates the brain. In return, the brain releases a response signals (steady state visual evoked potential). In theory, brain’s response signals and the stimulus signals are identical. Once successfully identified, the presence of a signal pattern that is identical to the one of the alternative stimulus signals (paired with a command in a user interface) indicates the intention of a user. Since each option is associated with a unique signal pattern, multiple options can simultaneously be offered to users. The main challenge of working with stimulus signals is that the response signals are weak and they are buried inside of highly polluted EEG signals that include brain’s natural activities. In this paper, we introduce a signal processing algorithm based on Lorenz systems of differential equations for identifying the source of stimulus signals. Our experiments strongly suggest that bio-signal-based design environments to perform complex tasks, including geometric modeling can be achieved by utilizing stimulus-based signal processing methodology.     

Factors affecting performance on a target monitoring task employing an automatic tracker

The experiments in this paper examined the extent to which performance on a task employing an automatic tracker was similar to performance on tasks employing other types of automation that have been studied more extensively. Automated target tracking is being used in many sensor and navigation systems to improve performance and help the operator cope with increased data loads. With many automated systems these goals are not met. In particular, the operator often misses errors made by the automated system and may report no decrease in workload. Several hypotheses have been offered for the operator's failure to monitor an automated system adequately. These include lack of experience with the manual task, a vigilance decrement, complacency, and inappropriate level of automation. The relevance of each of these hypotheses to failure to monitor an automatic tracker adequately was examined. Performance and perceived workload on a target tracking task employing an automatic tracker, in which participants had to detect and then update the position of several targets (e.g. ships) at regular intervals, were measured as a function of number of targets, training with the manual task, experience, and time on task. The results suggested that failure to detect errors made by the automated system was due largely to the lack of visibility of the automation errors relative to other errors. However, complacency could not be ruled out entirely. Unlike some other tasks, the availability of a reliable automatic tracker did lead to a substantial reduction in perceived workload.     

Factors affecting performance on a target monitoring task employing an automatic tracker

The experiments in this paper examined the extent to which performance on a task employing an automatic tracker was similar to performance on tasks employing other types of automation that have been studied more extensively. Automated target tracking is being used in many sensor and navigation systems to improve performance and help the operator cope with increased data loads. With many automated systems these goals are not met. In particular, the operator often misses errors made by the automated system and may report no decrease in workload. Several hypotheses have been offered for the operator's failure to monitor an automated system adequately. These include lack of experience with the manual task, a vigilance decrement, complacency, and inappropriate level of automation. The relevance of each of these hypotheses to failure to monitor an automatic tracker adequately was examined. Performance and perceived workload on a target tracking task employing an automatic tracker, in which participants had to detect and then update the position of several targets (e.g. ships) at regular intervals, were measured as a function of number of targets, training with the manual task, experience, and time on task. The results suggested that failure to detect errors made by the automated system was due largely to the lack of visibility of the automation errors relative to other errors. However, complacency could not be ruled out entirely. Unlike some other tasks, the availability of a reliable automatic tracker did lead to a substantial reduction in perceived workload.     

Applied virtual environments to support learning of social interaction skills in users with Asperger's Syndrome

Asperger's Syndrome (AS) is an autistic spectrum disorder characterised by normal to high IQ but with marked impairment in social skills. Successful social skills training appears to be best achieved either in situ or in role-play situations where users can explore different outcomes resulting from their social behaviour. Single user virtual environments (SVEs) provide an opportunity for users with AS to learn social interaction skills in a safe environment which they can visit as many times as they like.The use of game-like tasks can provide an incentive and can also be used to guide the user through progressive learning stages. Collaborative virtual environments (CVEs) allow several users to interact simultaneously within the virtual environment, each taking different perspectives or role-play characters. Within the AS interactive project a series of SVEs and CVEs have been developed in collaboration with users and professional groups with an overall aim of supporting social skills learning. Initial evaluation studies have been carried out which have been used to both inform and refine the design of these virtual environments (VEs) as well as giving an insight into the understanding and interpretation of these technologies by users with AS.     

Quality of experience in distributed databases

This paper proposes an approach to improve the level of Quality of Experience (QoE) that distributed database systems provide. Quality of Experience is a measure of users?? satisfaction when using a certain service or application. Therefore, the main objective of this paper is to provide mechanisms to increase users?? satisfaction when accessing distributed database systems. In traditional database systems, users cannot specify execution-related constraints. Then, the database system cannot evaluate if user expectations are satisfied and neither the system can take corrective actions when necessary. In this work, we present the QoE-oriented distributed database system (QoE-DDB). It allow users to specify Data Access Requirements (DARs) and aims to please users by satisfying the DARs they define. We define a set of types of Data Access Requirements and propose some SQL extensions that enable users to specify execution-related requirements. Proposed types of DARs include execution deadline and priority, execution start and finish times, data availability and freshness degrees, and disconnected execution mode. In our QoE-DDB, each user??s command is transformed into one or more tasks that are executed by data services. Community modules and local data services negotiate Service Level Objectives (SLOs) for each task, which improves the system??s dependability. We propose both QoE-oriented scheduling and dynamic data placement strategies. Proposed architecture and scheduling strategies enable the system to be used in a wide range of distributed environments, from tightly-coupled homogeneous environments (e.g. composed by off-the-shelf computers connected by a LAN) to highly heterogeneous and geographically distributed systems, where data services have some degree of autonomy. Traditional performance indicators (e.g. throughput and response time) are not adequate to measure the QoE a system provides. We also propose some specialized Key Performance Indicators (KPIs) to estimate the QoE level a database system provides. Finally, we present experimental results obtained through the use of benchmark data and queries together with a prototype that implements proposed strategies. In our experiments, we consider realistic scenarios and compare proposed scheduling strategies with their best-effort oriented counterparts. Obtained results prove the importance of our QoE-oriented approach.     

Analyzing the Adequacy of Interaction Paradigms in Artificial Reality Experiences

In Artificial Reality experiences, that is, interactive, unencumbered, full-body, 2D vision-based virtual reality (VR) experiences (heirs of the seminal Videoplace by Myron Krueger), there are two possible interaction paradigms, namely, first-person and third-person—which differ significantly from the classic VR first- and third-person notions. Up until now, these two paradigms had not been compared or objectively analyzed in such systems. Moreover, most systems are based on the third-person paradigm without a specific justification, most probably due to the influence of the original Videoplace system and because it is the only paradigm available in commercial development tools and leisure systems. For example, many rehabilitation projects have chosen to use these VR systems because of their many advantages. However, most of these projects and research have blindly adopted the third-person paradigm. Hence, the field of virtual rehabilitation has analyzed the beneficial properties of these systems without considering the first-person paradigm that could potentially present better adequacy. To find and understand potential differences between the two paradigms, we have defined an application categorization from which we developed two full-body interactive games and set up an experiment to analyze each game in both paradigms. We studied how 39 participants played these games and we quantitatively and qualitatively analyzed how each paradigm influenced the experience, the activity and the behavior of the users and the efficiency in accomplishing the required goals. We present the results of these experiments and their general implications, and especially for virtual rehabilitation due to the potential impact these systems may have in the well-being of many people.     

Establishing a baseline for text entry for a multi-touch virtual keyboard

Multi-touch, which has been heralded as a revolution in human–computer interaction, provides features such as gestural interaction, tangible interfaces, pen-based computing, and interface customization—features embraced by an increasingly tech-savvy public. However, multi-touch platforms have not been adopted as “everyday” computer interaction devices that support important text entry intensive applications such as word processing and spreadsheets. In this paper, we present two studies that begin to explore user performance and experience with entering text using a multi-touch input. The first study establishes a benchmark for text entry performance on a multi-touch platform across input modes that compare uppercase-only to mixed-case, single-touch to multi-touch and copy to memorization tasks. The second study includes mouse style interaction for formatting rich text to simulate a word processing task using multi-touch input. As expected, our results show that users do not perform as well in terms of text entry efficiency and speed using a multi-touch interface as with a traditional keyboard. Not as expected was the result that degradation in performance was significantly less for memorization versus copy tasks, and consequently willingness to use multi-touch was substantially higher (50% versus 26%) in the former case. Our results, which include preferred input styles of participants, also provide a baseline for further research to explore techniques for improving text entry performance on multi-touch systems.     

Generating recommendations for consensus negotiation in group personalization services

There are increasingly many personalization services in ubiquitous computing environments that involve a group of users rather than individuals. Ubiquitous commerce is one example of these environments. Ubiquitous commerce research is highly related to recommender systems that have the ability to provide even the most tentative shoppers with compelling and timely item suggestions. When the recommendations are made for a group of users, new challenges and issues arise to provide compelling item suggestions. One of the challenges a group recommender system must cope with is the potentially conflicting preferences of multiple users when selecting items for recommendation. In this paper, we focus on how individual user models can be aggregated to reach a consensus on recommendations. We describe and evaluate nine different consensus strategies and analyze them to highlight the benefits of group recommendation using live-user preference data. Moreover, we show that the performance is significantly different among strategies.     

Simulating collaborative systems by means of awareness of interaction among intelligent agents

New specific collaboration models for distributive systems are needed for enabling effective collaboration processes between users of these dynamic environments. However, collaboration in distributive systems depends on several elements such as a negotiation mechanism, a load-balancing strategy, a mutual exclusion mechanism, and a learning strategy. Each of these elements should be analyzed separately in terms of efficiency and effectiveness in order to improve these two features in the global collaborative system. This paper presents a multi-agent based simulation tool and a simulation methodology used to evaluate the efficiency and effectiveness of a collaborative system by analyzing both the global system of each of the strategies or mechanisms involved in the system. Intelligent agents used in the simulation make use of the concept of awareness of interaction for quantifying the degree, nature and quality of the interaction between the agents.     

Supporting cognitive processing with spatial information presentations in virtual environments

While it has been suggested that immersive virtual environments could provide benefits for educational applications, few studies have formally evaluated how the enhanced perceptual displays of such systems might improve learning. Using simplified memorization and problem-solving tasks as representative approximations of more advanced types of learning, we are investigating the effects of providing supplemental spatial information on the performance of learning-based activities within virtual environments. We performed two experiments to investigate whether users can take advantage of a spatial information presentation to improve performance on cognitive processing activities. In both experiments, information was presented either directly in front of the participant, at a single location, or wrapped around the participant along the walls of a surround display. In our first experiment, we measured memory scores and analyzed participant strategies for a memorization and recall task. In addition to comparing spatial and non-spatial presentations, we also varied field of view and background imagery. The results showed that the spatial presentation caused significantly better memory scores. Additionally, a significant interaction between background landmarks and presentation style showed that participants used more visualization strategies during the memorization task when background landmarks were shown with spatial presentations. To investigate whether the advantages of spatial information presentation extend beyond memorization to higher level cognitive activities, our second experiment employed a puzzle-like task that required critical thinking using the presented information. Focusing only on the effects of spatial presentations, this experiment measured task performance and mental workload. The results indicate that no performance improvements or mental workload reductions were gained from the spatial presentation method compared with a non-spatial layout for our problem-solving task. The results of these two experiments suggest that supplemental spatial information can affect mental strategies and support performance improvements for cognitive processing and learning-based activities. However, the effectiveness of spatial presentations is dependent on the nature of the task and a meaningful use of space and may require practice with spatial strategies.     

Evaluation of virtual fixtures for a robot programming by demonstration interface

We investigate the effectiveness of several types of virtual fixtures in a robot programming by demonstration interface. We show that while all types of virtual fixtures examined yield a significant reduction in the number of errors in tight tolerance peg-in-hole tasks, color and sound fixtures generally outperform a tactile fixture in terms of both execution time of successful trials and error rate. We have found also that when users perceive that the task is very difficult but the system is providing some help by means of a virtual fixture, they tend to spend more time trying to achieve a successful task execution. Thus, for difficult tasks the benefits of virtual fixturing are better reflected in a reduction of the error rate than in a decreased execution time. We conjecture that these trends are related to the limitations of currently available interfaces for human-robot interaction through virtual environments and to the different strategies adopted by the users to cope with such limitations in high-accuracy tasks.     

Navigational tools for desktop virtual environment interfaces

Desktop systems typically rely on a two-dimensional (2D) software interface and general purpose hardware (mouse, keyboard) for interaction with a three-dimensional (3D) virtual environment. These interfaces must provide all the functionality required to navigate through and interact with the virtual environment, yet research into the usability aspects of the tools presented on these software interfaces indicates that the majority of users experience some degree of frustration when using them to perform even relatively simple tasks. This paper begins with a study of usability issues for interfaces to virtual environments on desktop systems, and details a series of experiments performed to evaluate the usability of a number of navigational tools. Participants were tested on the time taken to complete a number of navigational tasks with a series of interfaces presenting different navigational tools. The tools presented were a speed control function, a you-are-here (YAH) map, a function enabling the user to mark and teleport to any location within the presented environment, and an undo function. Results indicate that the visual presentation of navigational aids improves navigation performance, in terms of the time taken to complete tasks, and also improves user satisfaction with the desktop system.     

Distraction from multiple in-vehicle secondary tasks: vehicle performance and mental workload implications

This study investigates the impact of multiple in-vehicle information systems on the driver. It was undertaken using a high fidelity driving simulator. The participants experienced, paced and unpaced single tasks, multiple secondary tasks and an equal period of 'normal' driving. Results indicate that the interaction with secondary tasks led to significant compensatory speed reductions. Multiple secondary tasks were shown to have a detrimental affect on vehicle performance with significantly reduced headways and increased brake pressure being found. The drivers reported interaction with the multiple in-vehicle systems to significantly impose more subjective mental workload than either a single secondary task or 'normal driving'. The implications of these findings and the need to integrate and manage complex in-vehicle information systems are discussed.