Predicting real-world ergonomic measurements by simulation in a virtual environment

Virtual reality techniques have been increasingly used for ergonomic applications. However, it is always important to know whether the results obtained in a virtual environment (VE) are representative of a real environment (RE) ones. This paper presents our preliminary experimental results on the relationship between ergonomic measurements in VE and RE for some typical “drilling” tasks. The same tasks were carried out by thirty male manufacturing factory workers in both VE and RE. Five evaluation indices - three objective (elbow angle, maximum force capacity reduction, and task completion time) and two subjective (BPD - Body Part Discomfort and RPE - Rated Perceived Exertion) - were used to evaluate the similarities between VE and RE for the selected “drilling” tasks. Four of these indices (all except elbow angle) were significantly higher (p < 0.05) in VE than in RE, which indicates that subjects experienced more discomfort and grew fatigued more quickly in VE. However, linear correlations (Pearson’s rho: 0.635-0.807) between VE and RE were found for two of the five indices (BPD and maximum force capacity reduction).

Relevance to industry

Using digital mock-ups and virtual reality simulations, industrial work activities can be evaluated to identify potential ergonomic problems during an early design stage, which reduces design time and costs, increases quality and improves customer satisfaction. A validated linear relationship can provide a reference for work design in virtual reality.     

Incorporating Tactile Cues into Human‐Centered Virtual Product Design

This study introduces a high‐fidelity tactile feedback mechanism to capture ergonomics attributes of users inside virtual product development cycle. The research question posed regards whether the tactile feedback mechanism in virtual experiment proposes high fidelity of ergonomics results when compared to physical experiment outcomes. This question was evaluated through an objective and a subjective study. Objective study was composed of an ergonomics product assessment experiment, where two different cart designs (a commercial cart and a prototype model) were evaluated for ergonomic adequacy (L4/L5 compression forces). Subjective evaluation was consisted of a questionnaire to assess subjects' preferences regarding which cart model they preferred in three different design categories; maneuverability, accessibility, and ergonomics. Results showed that tactile feedback mechanism was able to replicate the physical test conditions in virtual environment with high fidelity, and prototype cart model received higher mean ratings when compared to commercial cart model in each design category.     

The effects of static and adaptive performance feedback in game-based training

Training in virtual environments (VEs) has the potential to establish mental models and task mastery while providing a safe environment in which to practice. Performance feedback is known to contribute to this learning; however, the most effective ways to provide feedback in VEs have not been established. The present study examined the effects of differing feedback content, focusing on adaptive feedback. Participants learned search procedures during multiple missions in a VE. A control group received only a performance score after each mission. Two groups additionally received either detailed or general feedback after each mission, while two other groups received feedback that adapted based on their performance (either detailed-to-general, or general-to-detailed). Groups that received detailed feedback from the start of training had faster performance improvement than all other groups; however, all feedback groups showed improved performance and by the fourth mission performed at levels above the control group. Results suggest that detailed feedback early in the training cycle is the most beneficial for the fastest learning of new task skills in VEs.     

Enhancing virtual environment spatial awareness training and transfer through tactile and vestibular cues

Haptic interaction has been successfully incorporated into a variety of virtual environment (VE) systems, yet designing multimodal VE training systems remains challenging as each cue incorporated during training should maximise learning and training transfer. This study examined the impact of incorporating two independent, spatialised tactile cues and vestibular cues into a military VE training environment with the goal of empirically examining whether such cues could enhance performance within the training environment and also that knowledge and skills gained during training could transfer to another environment. The results showed that tactile cues enhanced spatial awareness and performance during both repeated training and within a transfer environment, yet there were costs associated when two independent tactile cues were presented during training. In addition, results suggest that spatial awareness benefits from a tactile point indicator may be impacted by vestibular cues, as performance benefits were seen when tactile cues were paired with head tracking. To fully realise training potential, it is essential to determine how best to leverage multimodal capacity of VE training systems by identifying how multimodal training cues may advance knowledge, skills and attitudes of trainees. Results from this study provide design guidelines for incorporating tactile cues in VE training environments to enhance spatial awareness.     

Creation and evaluation of a multi-sensory virtual assembly environment

The multi-modal information presentation, integrated into the virtual environment （VE）, has potential for stimulating different senses, improving the user＇s impression of immersion, and increasing the amount of information that is accepted and processed by the user＇s perception system. The increase of the useful feedback information may reduce the user＇s cognitive load, thus enhancing the user＇s efficiency and performance while interacting with VEs. This paper presents our creation of a multi-sensory virtual assembly environment （VAE） and the evaluation of the effects of multi-sensory feedback on the usability. The VAE brings together complex technologies such as constraint-based assembly simulation, optical motion tracking technology, and real-time 3D sound generation technology around a virtual reality workbench and a common software platform. The usability evaluation is in terms of its three attributes： efficiency of use, user satisfaction, and reliability. These are addressed by using task completion times （TCTs）, questionnaires, and human performance error rates （HPERs）, respectively. Two assembly tasks have been used to perform the experiments, using sixteen participants. The outcomes showed that the multi-sensory feedback could improve the usability. They also indicated that the integrated feedback offered better usability than either feedback used in isolation. Most participants preferred the integrated feedback to either feedback （visual or auditory） or no feedback. The participants＇ comments demonstrated that nonrealistic or inappropriate feedback had negative effects on the usability, and easily made them feel frustrated. The possible reasons behind the outcomes are also analysed by using a unifying human computer interaction framework. The implications, concluded from the outcomes of this work, can serve as useful guidelines for improving VE system design and implementation.     

Spatialized Vibrotactile Feedback Improves Goal-Directed Movements in Cluttered Virtual Environments

Spatial awareness in virtual reality (VR) is a dominant research topic. It plays an essential role in the assessment of human operators’ behavior in simulated tasks, notably for the evaluation of the feasibility of manual maintenance tasks in cluttered industrial settings. In such contexts, it is decisive to evaluate the spatial and temporal correspondence between the operator’s movement kinematics and that of his/her virtual avatar in the virtual environment (VE). Often, in a cluttered VE, direct kinesthetic (force) feedback is limited or absent. We tested whether vibrotactile (cutaneous) feedback would increase visuo-proprioceptive consistency, spatial awareness, and thus the validity of VR studies, by augmenting the perception of the operator’s contact(s) with virtual objects. We present experimental results obtained using a head-mounted display (HMD) during a goal-directed task in a cluttered VE. Results suggest the contribution of spatialized vibrotactile feedback to visuo-proprioceptive consistency.     

Effects of four types of non-obtrusive feedback on computer behaviour, task performance and comfort

This study investigated the effects of non-obtrusive feedback on continuous lifted hand/finger behaviour, task performance and comfort. In an experiment with 24 participants the effects of two visual and two tactile feedback signals were compared to a no-feedback condition in a computer task. Results from the objective measures showed that all types of feedback were equally effective to reduce lifted hand/finger behaviour (effectiveness) compared to absence of feedback, while task performance was not affected (efficiency). In contrast to objective measures, subjective user experience was significantly different for the four types of feedback signals. Continuous tactile feedback appeared to be the best signal; not only the effectiveness and efficiency were rated reasonable, it also scored best on perceived match between signal and required action. This study shows the importance of including user experiences when investigating usability of feedback signals. Non-obtrusive feedback embedded in products and environments may successfully be used to support office workers to adopt healthy, productive and comfortable working behaviour.     

A sequenced multimodal learning approach to support students' development of conceptual learning

Virtual learning environments can now be enriched not only with visual and auditory information, but also with tactile and kinesthetic feedback. However, the way to successfully integrate haptic feedback on a multimodal learning environment is still unclear. This study aims to provide guidelines on how visuohaptic simulations can be implemented effectively, thus the research question is: Under what conditions do visual and tactile information support students' development of conceptual learning of force‐related concepts? Participants comprised 170 undergraduate students of a Midwestern University enrolled in a physics for elementary education class. Four experiments were conducted using four different configurations of multimodal learning environments: Visual feedback only, haptic force feedback only, visual and haptic force feedback at the same time, and sequenced modality of haptic feedback first and visual feedback second. Our results suggest that haptic force feedback has the potential to enrich learning when compared with visual only environments. Also, haptic and visual modalities interact better when sequenced one after another rather than presented simultaneously. Finally, exposure to virtual learning environments enhanced by haptic forced feedback was a positive experience, but the ease of use and ease of interpretation was not so evident.     

Beyond the visuals: tactile augmentation and sensory enhancement in an arthroscopy simulator

This paper considers tactile augmentation, the addition of a physical object within a virtual environment (VE) to provide haptic feedback. The resulting mixed reality environment is limited in terms of the ease with which changes can be made to the haptic properties of objects within it. Therefore sensory enhancements or illusions that make use of visual cues to alter the perceived hardness of a physical object allowing variation in haptic properties are considered. Experimental work demonstrates that a single physical surface can be made to ‘feel’ both softer and harder than it is in reality by the accompanying visual information presented. The strong impact visual cues have on the overall perception of object hardness, indicates haptic accuracy may not be essential for a realistic virtual experience. The experimental results are related specifically to the development of a VE for surgical training; however, the conclusions drawn are broadly applicable to the simulation of touch and the understanding of haptic perception within VEs.     

The use of force feedback and auditory cues for performance of an assembly task in an immersive virtual environment

Using an immersive virtual environment, this study investigated whether the inclusion of force feedback or auditory cues improved manipulation performance and subjective reports of usability for an assembly task. Twenty-four volunteers (12 males and 12 females) were required to assemble and then disassemble five interconnecting virtual parts with either auditory, force, or no feedback cues provided. Performance for the assembly task was measured using completion time and number of collisions between parts, while the users preferences across conditions were evaluated using subjective reports of usability. The results indicated that the addition of force feedback slowed completion time and led to more collisions between parts for males. In contrast, females exhibited no change in the mean completion time for the assembly task but did show an increase in collision counts. Despite these negative performance findings when adding force feedback, users did report perceived increases in realism, helpfulness and utility towards the assembly task when force feedback was provided. Unlike force feedback, the results showed that auditory feedback, indicating that parts had collided during the assembly task, had no negative performance effects on the objective measures while still increasing perceived realism and overall user satisfaction. When auditory cues and force feedback were presented together, performance times, number of collisions, and usability were not improved compared to conditions containing just auditory cues or force feedback alone. Based on these results, and given the task and display devices used in the present study, the less costly option of excluding auditory and force feedback cues would produce the best performance when measured by the number of collisions and completion time. However, if increased ratings of usability for an assembly task are desired while maintaining objective performance levels and reduced cost, then the inclusion of auditory feedback cues is best.     

虚拟实验热触觉再现系统的设计与实现

针对目前虚拟实验热触觉装置缺乏的现状,设计了一种基于PTC（Positive Temperature Coefficient）加热片的热触觉感知复现系统。该系统利用HTC VIVE的红外定位,将虚拟环境下热源的空间位置映射为现实环境下制热模块的空间位置。根据虚拟实验中不同位置和不同温度的热源,使用两个步进电机控制热源的位置和方向,并提供4个自由度的热源定位方法;使用PID控制算法控制PTC加热片产生热风的温度,获得不同的热触觉感知。该系统以一种非接触式的触觉感知来呈现虚拟实验的热触觉,保证了热触觉产生的真实性。通过铝热反应虚拟实验,验证该系统的实用性,结果表明该系统的热触觉再现能够提高虚拟实验的真实感受,补偿触觉上缺失的热反馈。     

Ergonomic aspects of aircraft keyboard design: the effects of gloves and sensory feedback on keying performance

Increasing requirements for aircraft keyboards have indicated inadequacies in ergonomic standards for airborne environments, in particular for tasks involving unskilled, discontinuous data entry, with emphasis on accuracy rather than speed. Four experiments are reported from a programme of research aimed at the development and application of methodologies for assessing factors relevant to keyboard design. Experiment 1 demonstrated that the effects of various aircrew gloves on a representative aircraft data entry task are highly situation specific. Experiment 2 showed that gloves impaired performance on a high compatibility continuous keying task, with relatively high keying rates, and that this impairment was not reduced by additional auditory feedback. Experiment 3 demonstrated that the high compatibility keying task provided a successful methodology for investigating the effects of key displacement and resistance on speed and accuracy. Experiment 4 indicated the relative contributions of reduced tactility and mobility to the effects on keying and manipulative performance caused by aircrew gloves. Together, these experiments suggest that kinaesthetic and tactile feedback associated with keying are relatively unaffected by aircrew gloves, and that restrictions on mobility caused by gloves may be more important for continuous data entry involving relatively high keying rates     

Comparing and combining real and virtual experimentation: an effort to enhance students' conceptual understanding of electric circuits

Abstract  The purpose of this study was to investigate value of combining Real Experimentation (RE) with Virtual Experimentation (VE) with respect to changes in students' conceptual understanding of electric circuits. To achieve this, a pre–post comparison study design was used that involved 88 undergraduate students. The participants were randomly assigned to an experimental (45 students) and a control group (43 students). Each group attended a one semester course in physics for preservice elementary school teachers. Both groups used the same inquiry-based curriculum materials. Participants in the control group used RE to conduct the study's experiments, whereas, participants in the experimental group used RE in the first part of the curriculum and VE in another part. Conceptual tests were administered to assess students' understanding of electric circuits before, during and after the teaching intervention. Results indicated that the combination of RE and VE enhanced students' conceptual understanding more than the use of RE alone. A further analysis showed that differences between groups on that part of the curriculum in which the experimental group used VE and the control group RE, in favour of VE.     

Assessing the effects of positive feedback and reinforcement in the introduction phase of an ergonomic intervention

Resistance to change is common in ergonomic interventions, often resulting in negative consequences when the intervention's effectiveness is studied. A lab-based study assessed the effects of positive reinforcement during the intervention process. On Day 1 all participants performed a simple screw-driving task that placed stress on the cervicobrachial region through static loading. On Day 2 a control group received basic information about ergonomics and then performed the task using an ergonomic intervention that has been shown to reduce loading on these muscle groups. The experimental group received the same basic information but also received positive reinforcement while performing the task with the ergonomic intervention. Subjective task assessment surveys and body-part discomfort surveys were administered, and these, along with speed of performance, were assessed in both groups. The results showed a significantly (p < .05) more positive subjective impression of the intervention for the feedback group than for the control group (29%-57% improvement) with no real changes in either the performance or discomfort levels. Applications of this research include improving workers' acceptance of ergonomic interventions in industrial and other settings. The reinforcement technique evaluated in this paper has yielded consistently positive effects in our ongoing ergonomic intervention research.     

The use of auditory feedback in the design of touch-input devices

In workspaces with complex displays and controls, and limited space in which to mount them, the use of touchscreen displays as input devices offers significant advantages. To overcome the lack of tactile feedback on touchscreen input devices, auditory feedback has been suggested. This paper examines the results of an experiment looking at the effect of auditory feedback on typing performance. While error rates were not affected by the feedback, the addition of auditory feedback to a typing task did improve typing speeds under all tested conditions. This indicates that the addition of auditory feedback to touchscreen input devices provides a measurable benefit and should be considered, where possible.     

Training the officer of the deck

The need to train people for increasingly complex tasks given diminishing budgets makes simulation-based training attractive in both the commercial and military sectors. Training systems that use virtual environment (VE) technology are potentially both reconfigurable and portable. The same equipment could provide a wide range of simulations at or near the trainees' worksite, letting them train close to home. To explore the potential of VE technology for naval training, the US Navy is sponsoring the Virtual Environment Technology for Training (VETT) program at three Boston sites. The VETT program experiments with VE systems employing visual and audio feedback and haptic interaction (i.e. tactile and kinesthetic/force interactions). The experiments extend from basic research into human psychophysics to the development of human/machine interfaces and computational systems, and applied training. The goal of integrating basic and applied research in this way is to determine the advantages and limitations of VE technology for training. The initial project focuses on naval officers, specifically the officer of the deck on a submarine     

A comparison of tactile,auditory, and visual feedback in a pointing task using a mouse-type device

A mouse was modified to add tactile feedback via a solenoid-driven pin projecting through a hole in the left mouse button. An experiment is described using a target selection task under five different sensory feedback conditions (‘normal’, auditory, colour, tactile, and combined). No differences were found in overall response times, error rates, or bandwidths; however, significant differences were found in the final positioning times (from the cursor entering the target to selecting the target). For the latter, tactile feedback was the quickest, normal feedback was the slowest. An examination of the spatial distributions in responses showed a peaked, narrow distribution for the normal condition, and a flat, wide distribution for the tactile (and combined) conditions. It is argued that tactile feedback allows subjects to use a wider area of the target and to select targets more quickly once the cursor is inside the target. Design considerations for human-computer interfaces are discussed.     

Fatigue evaluation in maintenance and assembly operations by digital human simulation in virtual environment

Virtual human techniques have been used a lot in industrial design in order to consider human factors and ergonomics as early as possible, and it has been integrated into VR applications to complete ergonomic evaluation tasks. In order to generalize the evaluation task in VE, especially for physical fatigue evaluation, we integrated a new fatigue model into a virtual environment platform. Virtual Human Status is proposed in this paper in order to assess the difficulty of manual handling operations, especially from the physical perspective. The decrease of the physical capacity before and after an operation is used as an index to indicate the difficulty level. The reduction of physical strength is simulated in a theoretical approach on the basis of a fatigue model in which fatigue resistances of different muscle groups were regressed from 24 existing maximum endurance time models. A framework based on digital human modeling technique is established to realize the comparison of physical status. An assembly case in airplane assembly is simulated and analyzed under the framework in VRHIT experiment platform. The endurance time and the decrease of the joint moment strengths are simulated. The experimental result in simulated operations under laboratory conditions confirms the feasibility of the theoretical approach: integration of virtual human simulation into virtual reality for physical fatigue evaluation.     

Virtual hell: a trip through the flames

In immersive virtual environments, our eyes and ears are usually the most important channels of perception. However, you might expect that their creators would include other senses in VEs so as to represent the largest range of stimuli. After all, humans are creatures of many senses and rely upon all of them in day-to-day living. In crafting virtual worlds for exploration, VE developers usually aim for as much realism as possible given the constraints of equipment and cost. Unfortunately, force and tactile feedback still have a long way to go, although tactile feedback models can simulate local geometry, texture reproduction, and heat flux and temperature for replicating collision detection. To investigate the latter options, I developed a complete thermal feedback system, the VR Thermal Kit, which includes different components to model, control, and physically generate stimuli on the operator's skin (usually the hand and exposed parts of the body). The results of my study concentrate on interaction paradigms and haptic rendering. They also enhance the level of immersion     

Navigation in 3D virtual environments: Effects of user experience and location-pointing navigation aids

In this paper, we describe the results of an experimental study whose objective was twofold: (1) comparing three navigation aids that help users perform wayfinding tasks in desktop virtual environments (VEs) by pointing out the location of objects or places; (2) evaluating the effects of user experience with 3D desktop VEs on their effectiveness with the considered navigation aids. In particular, we compared navigation performance (in terms of total time to complete an informed search task) of 48 users divided into two groups: subjects in one group had experience in navigating 3D VEs while subjects in the other group did not. The experiment comprised four conditions that differed for the navigation aid that was employed. The first and the second condition, respectively, exploited 3D and 2D arrows to point towards objects that users had to reach; in the third condition, a radar metaphor was employed to show the location of objects in the VE; the fourth condition was a control condition with no location-pointing navigation aid available. The search task was performed both in a VE representing an outdoor geographic area and in an abstract VE that did not resemble any familiar environment. For each VE, users were also asked to order the four conditions according to their preference. Results show that the navigation aid based on 3D arrows outperformed (both in terms of user performance and user preference) the others, except in the case when it was used by experienced users in the geographic VE. In that case, it was as effective as the others. Finally, in the geographic VE, experienced users took significantly less time than inexperienced users to perform the informed search, while in the abstract VE the difference was significant only in the control and the radar conditions. From a more general perspective, our study highlights the need to take into specific consideration user experience in navigating VEs when designing navigation aids and evaluating their effectiveness.