Peripheral visual feedback: a powerful means of supporting effective attention allocation in event-driven, data-rich environments

Breakdowns in human-automation coordination in data-rich, event-driven domains such as aviation can be explained in part by a mismatch between the high degree of autonomy yet low observability of modern technology. To some extent, the latter is the result of an increasing reliance in feedback design on foveal vision--an approach that fails to support pilots in tracking system-induced changes and events in parallel with performing concurrent flight-related tasks. One possible solution to the problem is the distribution of tasks and information across sensory modalities and processing channels. A simulator study is presented that compared the effectiveness of current foveal feedback and two implementations of peripheral visual feedback for keeping pilots informed about uncommanded changes in the status of an automated cockpit system. Both peripheral visual displays resulted in higher detection rates and faster response times, without interfering with the performance of concurrent visual tasks any more than does currently available automation feedback. Potential applications include improved display designs that support effective attention allocation in a variety of complex dynamic environments, such as aviation, process control, and medicine.     

Force and tactile feedback in preloaded cantilever snap-fits under manual assembly

The purpose of this study was to investigate the tactile feedback of snap-fit fasteners when used in manual assembly. An important aspect of this assembly process is the assembler’s ability to perceive the snap-fit’s engagement. This sensing of engagement yields a high level of confidence that assembly is both complete and secure. Force and tactile feedback are critical elements in this process. Many snap-fits are used in conjunction with sealing elements that produce a constant force that remains in effect after snap-fit assembly is completed. The effect of sealing element preload magnitude and stiffness was studied using a test station, force deflection measurements, and jury pool data. A low value of preload with low stiffness was determined to be most favorable in terms of force and tactile feedback, with no preload only slightly less favorable. In order to sense the engagement signal of the catch, some resistance to assembly was found to be beneficial. A dimensionless term called “engagement signal-to-hold-force ratio” is proposed as an additional way of rating the effect of assembly forces for snap-fits. It was found that higher signal-to-hold-force ratio, as well as higher values of engagement signal, corresponded to higher confidence of assembly among experimental subjects.

Relevance to Industry

In the automotive industry, force and tactile feedback are essential for sensing the full engagement of snap-fit parts during the assembly of critical components, such as electrical and fuel system interconnects. Both the design of the snap-fit and the presence or absence of preload will affect force and tactile feedback. Many of these applications require the compression of an elastic gasket with a preload needed for air, fuel, or electrical isolation. Preload and preload stiffness factors are examined in this paper with the aim of achieving a better understanding of these effects so that snap-fit confidence of assembly and assembly robustness can be enhanced in industrial settings.     

Effective attention allocation behavior and its measurement: a preliminary study

In general, evaluation of human–machine interface design remains a challenging task. Specifically, there remains a lack of method for tracking effective human operator's attention. This paper presents a study aimed at devising such a method. This method is based on a combination of operators' eye movement and hand movement behaviors. The eye movement reflects the operators' cognitive process and attention allocation, while the hand movement reflects the operators' physical action, which is the result of a cognitive process. Effectiveness of that piece of cognition (eye movement) can therefore be evaluated based on the result of an action (hand movement). The said measure, which may be called the hand–eye measure, is examined for its sensitivity to a good or poor operation behavior and patterns that are further correlated to the operator's behavior and performance. At present, the patterns across the whole operation period are explored. A reference system is employed to validate the hand–eye measure.     

Control of large scale discrete event systems: Task allocation and coordination

To control a large scale discrete event system, decentralized control and hierarchical control can be used, where several local supervisors are used to control events in local sites and a coordinator is used to coordinate the local supervisors. Two important problems that need to be solved in such a control architecture are task allocation and coordination. That is, how to allocate tasks to the supervisors, and how to coordinate those tasks. We propose and solve a task allocation problem of assigning tasks to the local supervisors and a minimal intervention problem of coordinating tasks so that the intervention by the coordinator is minimal.     

Automation transparency: implications of uncertainty communication for human-automation interaction and interfaces

Operators of highly automated driving systems may exhibit behaviour characteristic for overtrust issues due to an insufficient awareness of automation fallibility. Consequently, situation awareness in critical situations is reduced and safe driving performance following emergency takeovers is impeded. A driving simulator study was used to assess the impact of dynamically communicating system uncertainties on monitoring, trust, workload, takeovers, and physiological responses. The uncertainty information was conveyed visually using a stylised heart beat combined with a numerical display and users were engaged in a visual search task. Multilevel analysis results suggest that uncertainty communication helps operators calibrate their trust and gain situation awareness prior to critical situations, resulting in safer takeovers. In addition, eye tracking data indicate that operators can adjust their gaze behaviour in correspondence with the level of uncertainty. However, conveying uncertainties using a visual display significantly increases operator workload and impedes users in the execution of non-driving related tasks.

Practitioner Summary: This article illustrates how the communication of system uncertainty information helps operators calibrate their trust in automation and, consequently, gain situation awareness. Multilevel analysis results of a driving simulator study affirm the benefits for trust calibration and highlight that operators adjust their behaviour according to multiple uncertainty levels.     

Acquisition and transfer of attention allocation strategies in a multiple-task work environment

OBJECTIVE: Payoff effects on strategy development and change were assessed in a synthetic work environment, SYNWORK1. BACKGROUND: Many work settings require several tasks to be performed concurrently. It is important to know how the strategies used in performing the respective tasks vary with payoffs. METHOD: Sixty students performed four tasks in SYNWORK1, for which points are received for correct responses and lost for incorrect responses. Individual-task payoffs were varied between participants and were changed after 8 and 12 sessions to examine the effects of a previous strategy on development of a new strategy. RESULTS: Participants were sensitive to initial payoffs and modified their strategies when payoffs changed. However, residual effects of prior payoffs were evident. CONCLUSION: Payoffs for multiple-task environments need to be explicit, and practice should be provided for strategy development. When payoffs change, strategies adopted reflect current and previous payoffs. APPLICATION: The findings can be applied to the design of payoff schedules for multiple-task environments.     

Task allocation and coordination process in distributed agile software development: an ontology based approach

Information Technology and Management - Distributed agile software development (DASD) has gained much popularity over the past years. It relates to Agile Software Development (ASD) being executed...     

Not all trust is created equal: dispositional and history-based trust in human-automation interactions

OBJECTIVE: We provide an empirical demonstration of the importance of attending to human user individual differences in examinations of trust and automation use. BACKGROUND: Past research has generally supported the notions that machine reliability predicts trust in automation, and trust in turn predicts automation use. However, links between user personality and perceptions of the machine with trust in automation have not been empirically established. METHOD: On our X-ray screening task, 255 students rated trust and made automation use decisions while visually searching for weapons in X-ray images of luggage. RESULTS: We demonstrate that individual differences affect perceptions of machine characteristics when actual machine characteristics are constant, that perceptions account for 52% of trust variance above the effects of actual characteristics, and that perceptions mediate the effects of actual characteristics on trust. Importantly, we also demonstrate that when administered at different times, the same six trust items reflect two types of trust (dispositional trust and history-based trust) and that these two trust constructs are differentially related to other variables. Interactions were found among user characteristics, machine characteristics, and automation use. CONCLUSION: Our results suggest that increased specificity in the conceptualization and measurement of trust is required, future researchers should assess user perceptions of machine characteristics in addition to actual machine characteristics, and incorporation of user extraversion and propensity to trust machines can increase prediction of automation use decisions. APPLICATION: Potential applications include the design of flexible automation training programs tailored to individuals who differ in systematic ways.     

Nonlinear output‐feedback control of torsional vibrations in drilling systems

This paper considers the design of a nonlinear observer‐based output‐feedback controller for oil‐field drill‐string systems aiming to eliminate (torsional) stick–slip oscillations. Such vibrations decrease the performance and reliability of drilling systems and can ultimately lead to system failure. Current industrial controllers regularly fail to eliminate stick–slip vibrations under increasingly challenging operating conditions caused by the tendency towards drilling deeper and inclined wells, where multiple vibrational modes play a role in the occurrence of stick–slip vibrations. As a basis for controller synthesis, a multi‐modal model of the torsional drill‐string dynamics for a real rig is employed, and a bit–rock interaction model with severe velocity‐weakening effect is used. The proposed model‐based controller design methodology consists of a state‐feedback controller and a (nonlinear) observer. Conditions, guaranteeing asymptotic stability of the desired equilibrium, corresponding to nominal drilling operation, are presented. The proposed control strategy has a significant advantage over existing vibration control systems as it can effectively cope with multiple modes of torsional vibration. Case study results using the proposed control strategy show that stick–slip oscillations can indeed be eliminated in realistic drilling scenarios in which industrial controllers fail to do so. Moreover, key robustness aspects of the control system involving the robustness against uncertainties in the bit–rock interaction and changing operational conditions are evidenced. Copyright © 2017 John Wiley & Sons, Ltd.     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.     

Overlay networks for task allocation and coordination in large-scale networks of cooperative agents

This paper proposes a novel method for scheduling and allocating atomic and complex tasks in large-scale networks of homogeneous or heterogeneous cooperative agents. Our method encapsulates the concepts of searching, task allocation and scheduling seamlessly in a decentralized process where no accumulated or centralized knowledge or coordination is necessary. Efficient searching for agent groups that can facilitate the scheduling of tasks is accomplished through the use of a dynamic overlay structure of gateway agents and the exploitation of routing indices. The task allocation and the scheduling of complex tasks are accomplished by combining dynamic reorganization of agent groups and distributed constraint optimization methods. Experimental results display the efficiency of the proposed method.     

Situation awareness and attention allocation measures for quantifying telepresence experiences in teleoperation

This study assessed the utility of measures of situation awareness (SA) and attention allocation for quantifying telepresence, the sense of being present at a remote site, in a teleoperation task scenario. Attention and SA have been identified as cognitive constructs potentially underlying telepresence. The motivation for this research was to establish an objective measure of telepresence and investigate the relationship between telepresence and teleoperation performance. Twenty‐four research participants performed a virtual ordnance disposal task at varying levels of difficulty (LODs). The task involved locating, identifying, and disposing of virtual land mines in an outdoor environment using a simulated remote‐control rover with a robotic arm. Performance, SA, and attention allocation were recorded along with subjective assessments of telepresence. Results demonstrated LOD effects on performance and telepresence. Regression analysis revealed LOD and attention to explain significant portions of the variance in telepresence. Results of the study provide further evidence that telepresence may share a relationship with performance, and that cognitive constructs, such as attention and SA, may serve as alternative, objective measures of telepresence. © 2004 Wiley Periodicals, Inc. Hum Factors Man 14: 51–67, 2004.     

Human performance in a multiple-task environment: Effects of automation reliability on visual attention allocation

Diagnostic automation has been posited to alleviate the high demands of multiple-task environments; however, mixed effects have been found pertaining to performance aid success. To better understand these effects, attention allocation must be studied directly. We developed a multiple-task environment to study the effects of automation on visual attention. Participants interacted with a system providing varying levels of automation and automation reliability and then were transferred to a system with no support. Attention allocation was measured by tracking the number of times each task was viewed. We found that participants receiving automation allocated their time according to the task frequency and that tasks that benefited most from automation were most harmed when it was removed. The results suggest that the degree to which automation affects multiple-task performance is dependent on the relative attributes of the tasks involved. Moreover, there is an inverse relationship between support and cost when automation fails.     

Design of a discrete Kalman filter for a tactile sensor in a feedback control system

This paper presents a design methodology for the optimum linear filter and predictor applied to robot sensor signals, as well as a sensitivity analysis of the kalman algorithms for uncertainties in the estimation of signal and noise parameters. Simulation of the filtration and prediction processes was made assuming a first-order spectrum of a pure signal and white measurement noise. Calculations of the algorithm errors, dependent on the accuracy of the signal and noise parameter estimation, were done for various spectra of the signal and for various signal-to-noise ratios. Furthermore, the sensitivity curves of the Kalman filter and predictor are presented. The outlined considerations might be helpful for designers when synthesizing optimum linear digital filters applied to sensor signals. Although that particular procedure has been designed for a robot tactile sensor application, the conclusions are more general, and applications may be found elsewhere.     

Decision-making support in weakly structured subject domains: Analysis of situations and evaluation of alternatives

A model of decision support in weakly structured dynamic situations is considered. This model is based on integration of a fuzzy cognitive situation model and a hierarchical model of evaluation. This model encompasses all stages of the decision-making process from structuring of the situation to generation of alternative decisions and the choice of the best alternative and is directed to simulation of expert knowledge about the situation, as well as his ideas and preferences concerning the control goal.     

Multimodal focus attention and stress detection and feedback in an augmented driver simulator

This paper presents a driver simulator, which takes into account the information about the user’s state of mind (level of attention, fatigue state, stress state). The user’s state of mind analysis is based on video data and biological signals. Facial movements such as eyes blinking, yawning, head rotations, etc., are detected on video data: they are used in order to evaluate the fatigue and the attention level of the driver. The user’s electrocardiogram and galvanic skin response are recorded and analyzed in order to evaluate the stress level of the driver. A driver simulator software is modified so that the system is able to appropriately react to these critical situations of fatigue and stress: some audio and visual messages are sent to the driver, wheel vibrations are generated and the driver is supposed to react to the alert messages. A multi-threaded system is proposed to support multi-messages sent by the different modalities. Strategies for data fusion and fission are also provided. Some of these components are integrated within the first prototype of OpenInterface: the multimodal similar platform.     

Optimization network flow control and price coordination with feedback: proposal of a new distributed algorithm

The usage of price instruments was found to offer an interesting opportunity for reactive congestion flow control in communication or computer networks where the objective is to maximize the total utility of all traffic sources over their transmission rates. The proposed control mechanisms were, amongst others, based on the Price Method—known also as the Interaction Balance Method—and both synchronous and asynchronous versions were developed. It was, however, so far assumed that the transmission network model was known exactly and that no traffic was lost and the traffic routes were fixed and known to the transmitting sources. In this paper, we formulate the flow control problem in a modified form, allowing in particular for the routing decisions to be hidden from the sources, and recall the concept of the Price Coordination with Feedback (known also under the name of the Interaction Balance Method with Feedback—IBMF). It is then shown that the use of this approach allows for proposing the new distributed algorithm for pricing of network services. The existence of an optimal solution in steady state is proved under reasonable assumptions and the convergence issues are discussed. Two examples are provided to illustrate the operation of the proposed algorithm and to compare its performance with the usage of classical price coordination.     

Effect of tactile feedback on movement speed and precision during work-related tasks using a computer mouse

Effects of tactile feedback on movement accuracy and speed were studied. Younger and older participants performed three tasks (1, select and drag word; 2, menu navigation; 3, select and drag cell) using commercial software and a mouse with or without tactile feedback. Task time and error number were recorded. Tasks were divided according to presence or absence of tactile feedback, and participants were divided into subgroups (high, average, low) based on Task 1 performance. Overall, older participants took longer (p < .0001) and made more errors (p < .001) than younger participants. There was an effect of feedback by task in younger participants for all six outcomes (p < 0.02). At the task level, with feedback, younger participants reduced performance time (13%) and errors (24%) on Task 1. Low- and average-performance younger participants benefited most from feedback for Task 1. Older low-performance participants also benefited from feedback for Task 1. For Task 3, older participants tended to take more time and make more errors with feedback. Tactile feedback may enhance performance when feedback is event related. Older people may not integrate sensation as well as younger individuals to enhance performance. Potential applications of this research include the development of tactile feedback interfaces to facilitate computer use.     

Display signaling in augmented reality: effects of cue reliability and image realism on attention allocation and trust calibration

This experiment seeks to examine the relationships among three advanced technology features (presentation of target cuing, reliability of target cuing, and level of image reality and the attention and trust given to that information). The participants were 16 military personnel who piloted an unmanned air vehicle and searched for targets camouflaged in terrain, which was presented at two levels of image realism. Cuing was available for some targets, and the reliability of this information was manipulated at two levels (100% and 75%). The results showed that the presence of cuing aided target detection for expected targets but drew attention away from the presence of unexpected targets. Cuing benefits and attentional tunneling were both reduced when cuing became less reliable. Increasing image realism was compelling but increased reliance on the cuing information when those data were reliable. Potential applications include a cost-benefit analysis of how trust modulates attention in the use of automated target recognition systems and the extent to which increased realism may influence this trust.