Two-handed assembly with immersive task planning in virtual reality

Assembly modelling is the process of capturing entities and activity information related to assembling and assembly. Currently, most CAD systems have been developed to ease the design of individual components, but are limited in their support for assembly designs and planning capability, which are crucial for reducing the cost and processing time in complex design, constraint analysis and assembly task planning. This paper presents a framework of a two-handed virtual assembly (VA) planner for assembly tasks, which coordinates two hands jointly for feature-based manipulation, assembly analysis and constraint-based task planning. Feature-based manipulation highlights the important assembling features (e.g. dynamic reference frames, moving arrow, mating features) to guide users for the ease of assembly and in an efficient and fluid manner. The users can freely navigate and move the mating pair along the collision-free path. The free motion of two-handed input in assembly is further restricted to the allowable motion guided by the constraints recognised on-line. The allowable motion in assembly is planned by the logic steps derived from the analysis of constraints and their translation in the progress of assembly. No preprocessing or predefined assembly sequence is necessary since the planning is produced in real-time upon the two-handed interactions. Mating features and constraints in databases are automatically updated after each assembly to simplify the planning process. The two-handed task planner has been developed and experimented for several assembly examples including a drill (12-parts) and a robot (17-parts). The system can be generally applied for the interactive task planning of assembly-type applications.     

A virtual reality-based experimentation environment for the verification of human-related factors in assembly processes

This paper investigates the use of virtual reality (VR)-based methods for the verification of performance factors related to manual assembly processes. An immersive and interactive virtual environment has been created to provide functionality for realistic process experimentation. Ergonomic models and functions have been embedded into the VR environment to support verification and constrain experimentation to ergonomically acceptable conditions. A specific assembly test case is presented, for which a semi-empirical time model is developed employing statistical design experimentation in the virtual environment. The virtual experimentation results enable the quantification and prediction of the influence of a number of process parameters and their combination at the process cycle time.     

Knowledge-based process planning for electronic assembly

The conventional method of electronic assembly planning is subjective and depends greatly on the skills, memory, knowledge and experience of the planners. This gives rise to inconsistencies which in turn create problems in the manufacturing environment. Since the computer logics are systematic and consistent, it is conceivable that an automatic electronic assembly planning system would have advances over manual systems. Due to the frequent advances in technology the electronic industry is a very dynamic one. This affects the effectiveness of the current process plans and the applicability of the system that generates them. In this paper important issues which address the automatic generation of process plans will be presented. The framework of a system which takes into account the dynamic nature of the environment will be introduced as an example.The original version of this paper was presented at the 2nd International Symposium on Robotics and Manufacturing (ISRAM), Albuquerque, New Mexico, 16–18 November 1988. The published proceedings of this meeting may be ordered from: CAD Laboratory for Systems/Robotics, EECE Dept., UNM, Albuquerque, NM 87131, U.S.A.     

Operations planning for collect-and-place machines in PCB assembly

Collect-and-place machines represent one of the most popular types of placement machines in automated printed circuit board (PCB) assembly. For scheduling the operations of this type of machinery, a three-stage heuristic solution approach is presented. In the first stage, the feeders (component types) are assigned to locations in the magazine of the placement machine. In the second stage, based on the assignment of component feeders to magazine positions, the component placement sequence is determined. Apparently, for a collect-and-place machine, this problem is similar to the well-known vehicle-routing problem. Therefore, we adapt standard methods for vehicle-routing problems, namely savings heuristics introduced by Clark and Wright [Clark, G., & Wright, J. W. (1964). Scheduling vehicles from a central delivery depot to a number of delivery points. Operations Research Quarterly, 12, 568–581]. Finally, local search principles are applied in order to improve the feeder assignment and the component placement sequence obtained. Numerical experiments are performed in order to compare the performance of the various savings-based heuristics under different experimental settings.     

An examination of interactions in a three-dimensional virtual world

Three-dimensional (3D) virtual worlds hold the users' attention by providing rich interaction in an environment similar to the real world. User engagement duration is known to increase in environments with intense interaction. However, information in the literature about whether gender, experience, or spatial ability affects interaction in these environments is limited. In this study, these three factors are compared to users' depth of interaction in a 3D virtual world. In addition, the relationships between engagement duration, spatial ability, and depth of interaction are examined to investigate whether the first two factors can predict the third. Findings showed that users' depth of interaction was not influenced by gender, but experience and spatial ability did affect interaction. A strong relationship was determined between depth of interaction and engagement duration, and a moderate relationship was found between depth of interaction and spatial ability. Findings indicated that when designing 3D environments, it is important to consider which kinds of tasks provide more interaction and to what extent spatial abilities affect interaction, as well as to prepare activities that will increase engagement duration and to devise strategies to enhance depth of interaction.     

Semiotics of virtual reality as a communication process

ABSTRACT

The experience of immersive virtual reality (VR) can be considered as a communication process between human beings, mediated by computer systems, which uses visualisation and other sensory stimulation. In this paper, we analyse how VR characteristics can be explored using semiotic theory and, with methods of generative semiotics, we explore aspects of narrative and interaction in VR. We propose a semiotic analysis of VR communication focusing on syntax, semantics and pragmatics and considering also some principles of generative semiotics. The syntactic level is analysed as determined by the characteristics of the visual communication adopted. The semantic of VR is related to the functional model chosen to realise the virtual system. The pragmatic of VR is based on the human–computer interaction that changes the user's role. We explore how these aspects can be characterised in the context of VR communication design and what principles can be adopted for a VR application, and we present an analysis and a classification of the iconic signs that are being used in VR. Moreover, we present a framework that can be used to classify and describe different kinds of virtual reality systems and to better understand communication in VR, and we use it to classify eight popular systems for e-learning and collaboration.     

A repository for design, process planning and assembly

This paper provides an introduction to the Design, Planning and Assembly Repository available through the National Institute of Standards and Technology (NIST). The goal of the Repository is to provide a publically accessible collection of 2D and 3D CAD and solid models from industry problems. In this way, research and development efforts can obtain and share examples, focus on benchmarks, and identify areas of research need. The Repository is available through the World Wide Web at URL http://www.parts.nist.gov/parts.     

Real-virtual components interaction for assembly simulation and planning

Assembly motion simulation is crucial during conceptual product design to identify potential assembly issues and enhance assembly efficiency. This paper presents a novel assembly simulation system incorporating real-virtual components interaction in an augmented-reality based environment. With this system, users can manipulate and assemble virtual components to real components during assembly simulation using natural gestures through an enhanced bare-hand interface. Accurate and realistic movements of virtual components can be obtained based on constraints analysis, determination of resultant forces from contacts with real components and manipulation from the user's hands (forces and torques applied, etc.) during assembly simulation. A prototype system has been developed, and a case study of an automobile clutch was conducted to validate the effectiveness and intuitiveness of the system.     

Student perceptions of a virtual field trip to replace a real field trip

Abstract  This study examines student perceptions on the use of virtual field trips (VFT) as part of their university experience and in particular the extent to which they could replace real field trips. While students were extremely positive about the potential of VFT to provide valuable learning experiences (and in particular a VFT constructed by the authors of this paper) nearly all of the students were insistent that it could not, and should not, replace real field trips. Furthermore when the same students were re-approached after having been on a real field trip, these perceptions were strengthened and they thought VFT could be most effective in preparing for, or revising after, a real field trip.     

Predicting real-world ergonomic measurements by simulation in a virtual environment

Virtual reality techniques have been increasingly used for ergonomic applications. However, it is always important to know whether the results obtained in a virtual environment (VE) are representative of a real environment (RE) ones. This paper presents our preliminary experimental results on the relationship between ergonomic measurements in VE and RE for some typical “drilling” tasks. The same tasks were carried out by thirty male manufacturing factory workers in both VE and RE. Five evaluation indices - three objective (elbow angle, maximum force capacity reduction, and task completion time) and two subjective (BPD - Body Part Discomfort and RPE - Rated Perceived Exertion) - were used to evaluate the similarities between VE and RE for the selected “drilling” tasks. Four of these indices (all except elbow angle) were significantly higher (p < 0.05) in VE than in RE, which indicates that subjects experienced more discomfort and grew fatigued more quickly in VE. However, linear correlations (Pearson’s rho: 0.635-0.807) between VE and RE were found for two of the five indices (BPD and maximum force capacity reduction).

Relevance to industry

Using digital mock-ups and virtual reality simulations, industrial work activities can be evaluated to identify potential ergonomic problems during an early design stage, which reduces design time and costs, increases quality and improves customer satisfaction. A validated linear relationship can provide a reference for work design in virtual reality.     

Using requirement-functional-logical-physical models to support early assembly process planning for complex aircraft systems integration

The assembly line process planning connects product design and manufacturing through translating design information to assembly integration sequence. The assembly integration sequence defines the aircraft system components installation and test precedence of an assembly process. This activity is part of the complex systems integration and verification process from a systems engineering view. In this paper, the complexity of modern aircraft is defined by classifying aircraft system interactions in terms of energy flow, information data, control signals and physical connections. At the early conceptual design phase of assembly line planning, the priority task is to understand these product complexities, and generate the installation and test sequence that satisfies the designed system function and meet design requirements. This research proposes a novel method for initial assembly process planning that accounts for both physical and functional integrations. The method defines aircraft system interactions by using systems engineering concepts based on traceable RFLP (Requirement, Functional, Logical and Physical) models and generate the assembly integration sequence through a structured approach. The proposed method is implemented in an industrial software environment, and tested in a case study. The result shows the feasibility and potential benefits of the proposed method.     

The evaluation of user experience of a human walking and a driving simulation in the virtual reality

Virtual reality (VR) has been implemented in various applications such as gaming, e-learning, rehabilitation, etc. Although many studies have been conducted to evaluate the user experiences (UX) of various applications in VR, there are few studies evaluating the UX of VR in a holistic view. Since user experience means holistic feelings caused by attributes like efficiency, effectiveness, pleasure, and attractiveness, it is necessary to evaluate user experience in many respects, including presence, usability, and workload. Therefore, this study aims to evaluate the UX of VR in four various metrics: presence, workload, usability, and flow. In this study, the user experience factor was confirmed through the previous studies comparing visual display terminals (VDT) and head-mounted displays (HMD). In addition, the UX difference between VDT and HMD in two different situations, walking and driving, is analyzed. The data of 26 participants were analyzed and found that there were significant differences in the four studied metrics related to UX. In the driving situation, the usability value was higher in VDT, whereas they were higher in HMD for walking. Comparing the two contexts in HMD, walking was significantly higher in three metrics: presence, usability, and flow. The results show that HMD is better in terms of user experience than VDT, and HMD is more suitable for walking situations than for driving situations. Through this study, in terms of user experience, it is suitable to replace walking tasks with VR when the experiment in a real-world environment is impossible.     

Assembly simulations in virtual environments with optimized haptic path and sequence

Virtual assembly and disassembly simulations can be accomplished in intuitive and effective ways using haptic information in virtual environments (VEs). Potential problems in a given assembly scheme can be predicted by a user who may be able to suggest an alternative scheme in the VE. This paper describes an intelligent virtual assembly system in which an optimal assembly algorithm is used to allow haptic interactions during virtual assembly operations. This algorithm provides optimal paths for haptic guidance as well as an assembly sequence of the parts to be assembled. The performance of the given assembly schemes was simulated using a virtual assembly system. Experimental results showed that the haptic-path sequence-guidance (HSG) mode gave the best performance improvement in terms of accumulated assembly time (28.33%) and travel distance (15.05%) compared to the unguided mode, while the sequence-guidance (SG) mode alone increased performance by 15.33% for assembly time and 11.36% for travel distance. The experimental results were analyzed by the sub-tasks of gripper selection, inter-part movement, and part assembly. For the HSG mode, the greatest contributor to the time and distance reductions was the optimized haptic path, while for the SG mode, the reduced numbers of gripper exchanges and orientation change made the greatest contributions to reducing the assembly time and the travel distance. As a result, the optimized haptic path, as well as sequence guidance, enhanced the working performance of virtual assembly tasks.     

Construction of a computer-simulated mixed reality environment for virtual factory layout planning

To survive the cut-throat competition in the manufacturing industry, many companies have introduced digital manufacturing technology. Digital manufacturing technology not only shortens the product development cycle times but also improves the precision of engineering simulation. However, building the virtual objects needed for a digital manufacturing environment requires skilled human resources; it is also costly and time-consuming. A high precision environment with the similar resources is also needed for a high precision simulation. In this paper, we propose a method of constructing a mixed reality-based digital manufacturing environment. The method integrates real objects, such as real images, with the virtual objects of a virtual manufacturing system. This type of integration minimizes the cost of implementing virtual objects and enhances the user's sense of reality. We studied several methods and derived a general framework for the system. Finally, we developed our idea into a virtual factory layout planning system. To assign the pose and position of real objects in virtual space, we applied a circle-based tracking method which uses a safety sign instead of the planar-square-shaped marker generally used for registration. Furthermore, we developed the framework to encapsulate simulation data from legacy data and process data for visualization based on mixed reality.     

A practical approach to a process planning expert system for manufacturing processes

In this paper the methodical techniques applied by the human process planning expertise is simulated. It considers the process plans design, or process selection. Software modules are designed to generate a process plan or several plans for a new part according to the input data from its engineering drawing. A specific module for each surface type, to match the surface parametric data and the required quantities with respect to the capability matrices, in order to locate the most eligible process plan is identified and used.     

Comprehensive modelling and simulation of cylindrical nanoparticles manipulation by using a virtual reality environment

With the expansion of nanotechnology, robots based on atomic force microscope (AFM) have been widely used as effective tools for displacing nanoparticles and constructing nanostructures. One of the most limiting factors in AFM-based manipulation procedures is the inability of simultaneously observing the controlled pushing and displacing of nanoparticles while performing the operation. To deal with this limitation, a virtual reality environment has been used in this paper for observing the manipulation operation. In the simulations performed in this paper, first, the images acquired by the atomic force microscope have been processed and the positions and dimensions of nanoparticles have been determined. Then, by dynamically modelling the transfer of nanoparticles and simulating the critical force-time diagrams, a controlled displacement of nanoparticles has been accomplished. The simulations have been further developed for the use of rectangular, V-shape and dagger-shape cantilevers. The established virtual reality environment has made it possible to simulate the manipulation of biological particles in a liquid medium.     

Learning in serious virtual worlds: Evaluation of learning effectiveness and appeal to students in the E-Junior project

The objective of this study is to present and to evaluate the E-Junior application: a serious virtual world (SVW) for teaching children natural science and ecology. E-Junior was designed according to pedagogical theories and curricular objectives to help children learn about the Mediterranean Sea and its environmental issues while playing. In this study, we present data about the E-Junior evaluation. A class in a serious virtual world (virtual group) was compared with a traditional type of class (traditional group) that contained identical learning objectives and contents but lacked a gaming aspect. Data collection consisted of quantitative and qualitative measures on a sample of 48 children. With regards to learning effectiveness, the results showed that the serious virtual world does not present statistically significant differences with the traditional type of class. However, students from the virtual group reported enjoying the class more, being more engaged, and having greater intentions to participate than students from the traditional group. The plausible explanation for this can be found in the qualitative data. The implications of these results and improvement proposals are also discussed in this work.     

A computer simulation in surgery for a human hip joint

For patients who have early signs of hip joint disease resulting from a structural abnormality, various surgeries for correcting abnormal stress distribution can be useful to prevent the progression of the disease. However, it is difficult to confirm the optimal procedure for surgery. To deal with this problem, we devised a computer program to support preoperative planning. Hip images obtained by computed tomography were loaded into our program, and a three-dimensional voxel model was created. Then the pressure distribution on the hip joint was analyzed with a rigid-body spring analysis (computational nonlinear mechanical analysis). This system has a module for performing virtual surgery. This program allows the hip joint mechanics to be evaluated easily, so that the advantages and disadvantages of various surgical methods can be examined biomechanically prior to surgery. However, this system has several problems that should be solved in the near future. This work was presented in part at the 12th International Symposium on Artifi cial Life and Robotics, Oita, Japan, January 25–27, 2007     

Second Life as a support element for learning electronic related subjects: A real case

Looking for more active and motivating methodological alternatives from the students’ perspective, which promote analysis and investigation abilities that make the student a more participative agent and some learning processes are facilitated, a practical study was conducted in the University of La Sabana (Chía, Colombia), in Computing Engineering (INF) and Agroindustrial Production Engineering (IPA) academic programs under the guiding question: Is Second Life an attractive tool for students, does it increment motivation, promote participation and facilitate learning processes in electronic related subjects in Engineering degrees?Second Life (SL) platform was then used as a traditional on-site class complement in electronic related subjects. A follow up was conducted with several groups during a year and a half, some students followed the subject with the traditional on-site methodology and others did it following a mixed methodology that combined traditional on-site lessons with virtual sessions trough SL, making a comparison and applying a quantitative as well as a qualitative methodology for the information analysis and recollection.After the specified study period it was found that using SL as part of the methodology, increments motivation toward electronic related subjects, and promotes participation and investigation, due to the fact that this tool makes learning possible in a funnier, more interactive and deeper way than if it had been taught in a master class. Having real-time access from a virtual classroom to all types of audiovisual information, helps more senses get involved in the learning process, making it more effective; nevertheless a weakness was detected related to the attention and concentration: students tend to get distracted, because their classroom is the computer and from there they have access not only to the many worlds of SL but to all the social networks and the Internet in general.     

Investigating learners’ attitudes toward virtual reality learning environments: Based on a constructivist approach

The use of animation and multimedia for learning is now further extended by the provision of entire Virtual Reality Learning Environments (VRLE). This highlights a shift in Web-based learning from a conventional multimedia to a more immersive, interactive, intuitive and exciting VR learning environment. VRLEs simulate the real world through the application of 3D models that initiates interaction, immersion and trigger the imagination of the learner. The question of good pedagogy and use of technology innovations comes into focus once again. Educators attempt to find theoretical guidelines or instructional principles that could assist them in developing and applying a novel VR learning environment intelligently. This paper introduces the educational use of Web-based 3D technologies and highlights in particular VR features. It then identifies constructivist learning as the pedagogical engine driving the construction of VRLE and discusses five constructivist learning approaches. Furthermore, the authors provide two case studies to investigate VRLEs for learning purposes. The authors conclude with formulating some guidelines for the effective use of VRLEs, including discussion of the limitations and implications for the future study of VRLEs.