The affect of contact force sensations on user performance in virtual assembly tasks

The development of a realistic virtual assembly environment is challenging because of the complexity of the physical processes and the limitation of available VR technology. Many research activities in this domain primarily focused on particular aspects of the assembly task such as the feasibility of assembly operations in terms of interference between the manipulated parts. The virtual assembly environment reported in this research is focused on mechanical part assembly. The approach presented addresses the problem of part-to-part contacts during the mating phase of assembly tasks. The system described calculates contact force sensations by making their intensity dependent on the depth of penetration. However the penetration is not visible to the user who sees a separate model, which does not intersect the mating part model. The two 3D models of the part, the off-screen rendered model and the on-screen rendered model are connected by a spring-dumper arrangement. The force calculated is felt by the operator through the haptic interface when parts come in contact during the mating phase of the assembly task. An evaluation study investigating the effect of contact force sensation on user performance during part-to-part interface was conducted. The results showed statistically significant effect of contact force sensation on user performance in terms of task completion time. The subjective evaluation based on feedback from users confirmed that contact force sensation is a useful cue for the operator to find the relative positions of components in the final assembly state.     

Neuro-cognitively inspired haptic user interfaces

Haptic systems and devices are a recent addition to multimodal systems. These devices have widespread applications such as surgical simulations, medical and procedural training, scientific visualizations, assistive and rehabilitative devices for individuals who have physical or neurological impediments and assistive devices for individuals who are blind. While the potential of haptics in natural human machine interaction is undisputable, the realization of such means is still a long way ahead. There are considerable research challenges to development of natural haptic interfaces. The study of human tactile abilities is a recent endeavor and many of the available systems still do not incorporate the domain knowledge of psychophysics, biomechanics and neurological elements of haptic perception. Development of smart and effective haptic interfaces and devices requires extensive studies that link perceptual phenomena with measurable parameters and incorporation of such domain knowledge in the engineering of haptic interfaces. This paper presents design, development and usability testing of a neuro-cognitively inspired haptic user interface for individuals who are blind. The proposed system design is inspired by neuro-cognitive basis of haptic perception and incorporates the computational aspects and requirements of multimodal information processing system. Usability testing of the system suggests that a biologically inspired haptic user interfaces may form a powerful paradigm for haptic user interface design.

Desktop haptic virtual assembly using physically based modelling

This research investigates the feasibility of using a desktop haptic virtual environment as a design tool for evaluating assembly operations. Bringing virtual reality characteristics to the desktop, such as stereo vision, further promotes the use of this technology into the every day engineering design process. In creating such a system, the affordability and availability of hardware/software tools is taken into consideration. The resulting application combines several software packages including VR Juggler, open dynamics engine (ODE)/open physics abstraction layer (OPAL), OpenHaptics, and OpenGL/GLM/GLUT libraries to explore the benefits and limitations of combining haptics with physically based modelling. The equipment used to display stereo graphics includes a Stereographics emitter, Crystal Eyes shutter glasses, and a high refresh rate CRT Monitor. One or two-handed force feedback is obtained from various PHANTOM haptic devices from SensAble Technologies Inc. The application’s ability to handle complex part interactions is tested using two different computer systems, which approximate the higher and lower end of a typical engineer’s workstation. Different test scenarios are analyzed and results presented.     

Augmenting graphical user interfaces with haptic assistance for motion-impaired operators

Haptic assistance is an emerging field of research that is designed to improve human–computer interaction (HCI) by reducing error rates and targeting times through the use of force feedback. Haptic feedback has previously been investigated to assist motion-impaired computer users, however, limitations such as target distracters have hampered its integration with graphical user interfaces (GUIs). In this paper two new haptic assistive techniques are presented that utilise the 3DOF capabilities of the Phantom Omni. These are referred to as deformable haptic cones and deformable virtual switches. The assistance is designed specifically to enable motion-impaired operators to use existing GUIs more effectively. Experiment 1 investigates the performance benefits of the new haptic techniques when used in conjunction with the densely populated Windows on-screen keyboard (OSK). Experiment 2 utilises the ISO 9241-9 point-and-click task to investigate the effects of target size and shape. The results of the study prove that the newly proposed techniques improve interaction rates and can be integrated with existing software without many of the drawbacks of traditional haptic assistance. Deformable haptic cones and deformable virtual switches were shown to reduce the mean number of missed-clicks by at least 75% and reduce targeting times by at least 25%.     

Interaction model between elastic objects for haptic feedback considering collisions of soft tissue

The simulation of organ–organ interaction is indispensable for practical and advanced medical VR simulator such as open surgery and indirect palpation. This paper describes a method to represent real-time interaction between elastic objects for accurate force feedback in medical VR simulation. The proposed model defines boundary deformation of colliding elements based on temporary surface forces calculated by temporary deformation. The model produces accurate deformation and force feedback considering collisions of objects as well as prevents unrealistic overlap of objects. A prototype simulator of rectal palpation is constructed on general desktop PC with a haptic device, PHANToM. The system allows users to feel different stiffness of a rear elastic object located behind another elastic object. The results of experiments confirmed the method expresses organ–organ interaction in real-time and produces realistic and perceivable force feedback.     

A user study of command strategies for mobile robot teleoperation

This article presents a user study of mobile robot teleoperation. Performance of speed, position and combined command strategies in combination with text, visual and haptic feedback information were evaluated by experiments. Two experimental tasks were designed as follows: positioning of mobile robot and navigation in complex environment. Time for task completion and motion accuracy were measured and compared for different command strategies and types of feedback. Role of haptic, text and visual feedback information in combination with described command strategies is outlined.     

A hybrid reality environment and its application to the study of earthquake engineering

Visualization can provide the much needed computer-assisted design and analysis environment to foster problem-based learning, while Virtual Reality (VR) can provide the environment for hands-on manipulation, stimulating interactive learning in engineering and the sciences. In this paper, an interactive 2D and 3D (hybrid) environment is described, which facilitates collaborative learning and research and utilizes techniques in visualization and VR, therefore enhancing the interpretation of physical problems within these fields. The environment described, termed VizClass, incorporates a specially designed lecture room and laboratory integrating both 2-D and 3-D spatial activities by coupling a series of interactive projection display boards (touch-sensitive whiteboards) and a semi-immersive 3D wall display. The environment is particularly appealing for studying critical, complex engineering problems, for example, where time-varying feature modifications and coupling between multiple modes of movement are occurring. This paper describes the hardware architecture designed for this new hybrid environment as well as an initial application within the environment to the study of a real case history building subjected to a variety of earthquakes. The example simulation uses field measured seismic data sources, and illustrations of simple visual paradigms to provide an enhanced understanding of the physical model, the damage accumulated by the model, and the association between the measured and observed data. A detailed evaluation survey was also conducted to determine the merits of the presented environment and the techniques implemented. Results substantiate the plausibility of using these techniques for more general, everyday users. Over 70% of the survey participants believed that the techniques implemented were valuable for engineers.

Broadband Cellular Radio Telecommunication Technologies in Distance Learning: A Human Factors Field Study

This paper presents a virtual classroom field experiment utilising broadband cellular radio telecommunication technologies and involving real users. The aim of the investigation was to experiment in the field cellular broadband systems in realising virtual classroom situations. In particular, we investigated learners’ performance and usability aspects of the multimedia tele-education system including ease of use, usefulness, telepresence and users’ satisfaction about the system. Results confirmed the effectiveness of the multimedia system in terms of both technical and psychological features. Some operational results and practical solutions were obtained as well. They included the basic features of multimedia systems used in virtual classroom and the correct procedures for training teachers and learners in the equipment use.     

Experiments in haptic-based authentication of humans

With the rapid advancement of the technological revolution, computer technology such as faster processors, advanced graphic cards, and multi-media systems are becoming more affordable. Haptics technology is a force/tactile feedback technology growing in disciplines linked to human–computer interaction. Similar to the increasing complexity of silicon-based components, haptics technology is becoming more advanced. On the other hand, currently available commercial haptics interfaces are expensive, and their application is mostly dedicated to enormous research projects or systems. However, the trend of the market is forcing haptic developers to release products for use in conjunction with current keyboards and mice technologies. Haptics allows a user to touch, fell, manipulate, create, and/or alter simulated three-dimensional objects in a virtual environment. Most of the existing applications of haptics are dedicated to hone human physical skills such as sensitive hardware repair, medical procedures, handling hazardous substances, etc. These skills can be trained in a realistic virtual world, and describe human behavioural patterns in human–computer interaction environments. The measurement of such psychomotor patterns can be used to verify a person’s identity by assessing unique-to-the-individual behavioural attributes. This paper explores the unique behaviour exhibited by different users interacting with haptic systems. Through several haptic-based applications, users’ physical attributes output data from the haptic interface for use in the construction of a biometric system.

An augmented reality interface for visualizing and interacting with virtual content

In this paper, a novel AR interface is proposed that provides generic solutions to the tasks involved in augmenting simultaneously different types of virtual information and processing of tracking data for natural interaction. Participants within the system can experience a real-time mixture of 3D objects, static video, images, textual information and 3D sound with the real environment. The user-friendly AR interface can achieve maximum interaction using simple but effective forms of collaboration based on the combinations of human–computer interaction techniques. To prove the feasibility of the interface, the use of indoor AR techniques are employed to construct innovative applications and demonstrate examples from heritage to learning systems. Finally, an initial evaluation of the AR interface including some initial results is presented.     

Human Factors in the Design of Supervisory Control Systems and Human–Machine Interfaces for Highly Automated Complex Systems

The design of control systems and human–machine interfaces in the field of complex and safety-critical environments remains today an open issue, in spite of the high technological evolution of the last decades. The increasing use of automation has improved efficiency, safety and ease of operations but, at the same time, it has complicated operators’ situation awareness and has changed the nature of their possible errors. The research activity described in this paper is an attempt to develop a methodological framework to support designers of control systems and human–machine interfaces. In particular, it focuses on the need for a deeply recursive approach related to the implementation of the systemic and human aspects of the design process of a human–machine system, intended as a Joint Cognitive System. A validating case study has been performed, based on the full application of the framework on the control of the turbine/alternator system of a thermoelectric power plant in northern Italy. Correspondence and offprint requests to: M. Piccini, Politecnico di Torino Dipartimento di Energetica, C.so Duca degli Abruzzi 24, 10129 Turin, Italy. Tel.: +39 011 564 4413; Fax: +39 011 564 4499; Email: mipiccin@polito.it     

Exploring user experiences as predictors of MMORPG addiction

The overuse of Massively Multiplayer Online Role Playing Games (MMORPGs) is becoming a significant problem worldwide, especially among college students. Similar to Internet addiction, the pathological use of MMORPG is a kind of modern addiction that can affect students’ lives on both a physical and a psychological level. The purpose of this study is to understand MMORPG addiction from a user experience design approach. We first developed a complete model that includes eleven factors (challenge, fantasy, curiosity, control, reward, cooperation, competition, recognition, belonging, obligation and role-playing) to represent users’ experience in MMORPGs. After that, we design a questionnaire to measure student’ gaming experience and level of addiction. Students’ demography information, including gender and game playing habits, was also collected. Four hundred and eighteen Taiwanese college students aged 18–25 years old took part in this online survey. Regression analysis was then conducted to evaluate the relative explanatory power of each variable, with addiction score as the dependent variable and the eleven user experience factors as the independent variables. The results of regression analysis reveal five critical factors (curiosity, role-playing, belonging, obligation and reward) that can be used to predict MMORPG addiction. In addition, this study also infers possible casual mechanisms for increasing college students’ level of addiction. The implications of our findings for both design and educational practitioners were also discussed.     

Head tracking using stereo

Head tracking is an important primitive for smart environments and perceptual user interfaces where the poses and movements of body parts need to be determined. Most previous solutions to this problem are based on intensity images and, as a result, suffer from a host of problems including sensitivity to background clutter and lighting variations. Our approach avoids these pitfalls by using stereo depth data together with a simple human-torso model to create a head-tracking system that is both fast and robust. We use stereo data (Commercial equipment and materials are identified in order to adequately specify certain procedures. In no case does such identification imply recommendation or endorsement by the National Institute of Standards and Technology, nor does it imply that the materials or equipment identified are necessarily the best available for the purpose.) to derive a depth model of the background that is then employed to provide accurate foreground segmentation. We then use directed local edge detectors on the foreground to find occluding edges that are used as features to fit to a torso model. Once we have the model parameters, the location and orientation of the head can be easily estimated. A useful side effect from using stereo data is the ability to track head movement through a room in three dimensions. Experimental results on real image sequences are given. Accepted: 13 August 2001     

Two-handed assembly with immersive task planning in virtual reality

Assembly modelling is the process of capturing entities and activity information related to assembling and assembly. Currently, most CAD systems have been developed to ease the design of individual components, but are limited in their support for assembly designs and planning capability, which are crucial for reducing the cost and processing time in complex design, constraint analysis and assembly task planning. This paper presents a framework of a two-handed virtual assembly (VA) planner for assembly tasks, which coordinates two hands jointly for feature-based manipulation, assembly analysis and constraint-based task planning. Feature-based manipulation highlights the important assembling features (e.g. dynamic reference frames, moving arrow, mating features) to guide users for the ease of assembly and in an efficient and fluid manner. The users can freely navigate and move the mating pair along the collision-free path. The free motion of two-handed input in assembly is further restricted to the allowable motion guided by the constraints recognised on-line. The allowable motion in assembly is planned by the logic steps derived from the analysis of constraints and their translation in the progress of assembly. No preprocessing or predefined assembly sequence is necessary since the planning is produced in real-time upon the two-handed interactions. Mating features and constraints in databases are automatically updated after each assembly to simplify the planning process. The two-handed task planner has been developed and experimented for several assembly examples including a drill (12-parts) and a robot (17-parts). The system can be generally applied for the interactive task planning of assembly-type applications.     

The Display of Electronic Commerce within Virtual Environments

In today’s competitive business environment, the majority of companies are expected to be represented on the Internet in the form of an electronic commerce site. In an effort to keep up with current business trends, certain aspects of interface design such as those related to navigation and perception may be overlooked. For instance, the manner in which a visitor to the site might perceive the information displayed or the ease with which they navigate through the site may not be taken into consideration. This paper reports on the evaluation of the electronic commerce sites of three different companies, focusing specifically on the human factors issues such as perception and navigation. Heuristic evaluation, the most popular method for investigating user interface design, is the technique employed to assess each of these sites. In light of the results from the analysis of the evaluation data, virtual environments are suggested as a way of improving the navigation and perception display constraints.     

Assembly simulations in virtual environments with optimized haptic path and sequence

Virtual assembly and disassembly simulations can be accomplished in intuitive and effective ways using haptic information in virtual environments (VEs). Potential problems in a given assembly scheme can be predicted by a user who may be able to suggest an alternative scheme in the VE. This paper describes an intelligent virtual assembly system in which an optimal assembly algorithm is used to allow haptic interactions during virtual assembly operations. This algorithm provides optimal paths for haptic guidance as well as an assembly sequence of the parts to be assembled. The performance of the given assembly schemes was simulated using a virtual assembly system. Experimental results showed that the haptic-path sequence-guidance (HSG) mode gave the best performance improvement in terms of accumulated assembly time (28.33%) and travel distance (15.05%) compared to the unguided mode, while the sequence-guidance (SG) mode alone increased performance by 15.33% for assembly time and 11.36% for travel distance. The experimental results were analyzed by the sub-tasks of gripper selection, inter-part movement, and part assembly. For the HSG mode, the greatest contributor to the time and distance reductions was the optimized haptic path, while for the SG mode, the reduced numbers of gripper exchanges and orientation change made the greatest contributions to reducing the assembly time and the travel distance. As a result, the optimized haptic path, as well as sequence guidance, enhanced the working performance of virtual assembly tasks.     

一个面向维修工程的虚拟人素分析系统软件框架

在工业产品、武器装备的维修性设计中，用虚拟现实技术进行人素分析，可以有效地提高产品的可维修性。该文提出了一个虚拟人素分析软件框架，并在此基础上，基于虚拟人平台—Jack，设计实现了一个虚拟人素分析系统。该系统根据给定的虚拟人体模型和虚拟样机进行人素分析并产生人素分析报告。该软件实现了维修仿真过程建模，以及基于维修仿真过程的人素分析功能，给用户提供了一个高效的分析环境。软件系统的应用实例研究表明，该文提出的软件框架是合理有效的。     

虚拟现实技术在专门用途英语教学中的应用研究综述

阐述了虚拟现实技术与专门用途英语教学之间的联系,介绍了近年来国内外虚拟现实技术应用在专门英语教学中的相关案例、理论探讨及应用模式,对比了现阶段国内外研究的优势和不足,分析了虚拟现实及其相关技术对计算机辅助专门用途英语教学的意义和应用价值,对今后虚拟现实技术如何更好地服务于计算机辅助专门英语教学进行了探讨,并对未来相关研究工作进行了展望。     

Representations in Human–Computer Systems Development

When system developers design a computer system (or other information artefact), they must inevitably make judgements as to how to abstract the domain and how to represent this abstraction in their designs. Over the years human–computer interaction, or more generally information systems design, has had a history of developing competing methods and models for both the process and products of its development. Various paradigms have been suggested, often trying to keep pace with the changing nature of the design problem; from batch processing to interactive systems to work situations and most recently to designing for household environments. It appears timely, then, to review the nature of the design problem that faces the developers of human–computer systems and to consider some of the impact that different representations and different conceptualisations may have on their activities. Green (1998) has suggested that a single model of developing human–computer systems is not desirable, instead arguing for a number of limited theories each of which provides a useful perspective. The aim of this paper is to place competing methods side by side in order to see their strengths and weaknesses more clearly. The central tenet of the paper is that different views of both the human–computer system design process and the different abstractions, or models, that are produced during the design process have varying degrees of utility for designers. It is unlikely that any single method or modelling approach will be optimal in all circumstances. Designers need to be aware of the range of views that exist and of the impact that taking a particular approach may have on the design solution.