Perception of safe robot idle time in virtual reality and real industrial environments

The main objective of this study was to investigate human perception of safe idle time of an industrial robot in a virtual reality environment. Studying operators' perception of robot operational characteristics such as safe robot idle time can help develop hazard prevention strategies, and ultimately improve robot safety. Results of data collected from 32 participants showed that robot size and speed had significant effects on the perception of safe robot idle time. This study also examined operators' perceived acceptability level for the robot speed, perceived level of hazard of robot motion, perceived chance of error, and self-reported mental workload. Results of this study were compared to the findings of Rahimi, M., Karwowski, W. [1990. Human perception of robot safe speed and idle time. Behaviour & Information Technology 9(5), 381–389], in which their experiment was conducted in a real industrial environment. This study demonstrated the feasibility of testing human perception of dynamic moving objects in a virtual reality environment. The virtual reality technology is believed to be capable of modeling a complex machinery system such as a robotic system.Relevance to industry: Human perception of the operational characteristics of industrial robots is an important concern for robot safety since misperception can cause robot operators to err, which in turn can cause injuries and fatalities. Through this study we (1) understand human perception, safety behavior, and decision making in a robotic system and (2) demonstrate the capability of modeling a complex machinery system using virtual reality technology. Our experiments designed to study human perception of safe robot idle time could lead to safety interventions and guidelines or hazard prevention strategy development.     

Using virtual reality for industrial design learning: a methodological proposal

ABSTRACT

Nowadays, the different computer tools available enable designers to create complex industrial prototypes. The use of these tools is constrained by the limitations imposed by common devices, such as screens and displays. Recently emerged virtual and augmented reality techniques have started being used as supports in many learning and industrial environments. Beyond the new possibilities that these tools offer for designing industrial objects, the underlying question is whether these new technologies could improve the creativity of designers to enable them to get a better understanding of the designing process itself. This paper presents a methodological proposal for the deployment of an industrial design engineering course aimed not only at learning different design techniques, but also at assessing the creativity skills of students. The practical contents include the use of virtual reality devices, to help the designer overcome the limitations of prototype visualisation and make better designing decisions. Moreover, a creativity test will be performed at the beginning and at the end of the course to assess the changes in creativity skills taking place within the experiment group versus the changes in a traditional learning (control) group.     

基于虚拟技术的MACS实训室的构建

针对过程控制专业实验实训及工厂培训设备M A C S控制系统价格特别昂贵,培训效果差的状况,提出建立基于虚拟技术的MACS实训室的构想。虚拟MACS实训室采用虚拟仪器技术、虚拟现实技术、计算机网络技术、多媒体技术,将昂贵的工业控制系统扩展到网络化的多媒体教学中,实现了由多人共用一套实训设备到每人单独使用此套实训设备的转变,充分利用了硬件资源,极大地节省了实训设备的投资,提高了教学质量与教学效果。     

A neurophysiological approach to assess training outcome under stress: A virtual reality experiment of industrial shutdown maintenance using Functional Near-Infrared Spectroscopy (fNIRS)

Shutdown maintenance, i.e., turning off a facility for a short period for renewal or replacement operations is a highly stressful task. With the limited time and complex operation procedures, human stress is a leading risk. Especially shutdown maintenance workers often need to go through excessive and stressful on-site trainings to digest complex operation information in limited time. The challenge is that workers’ stress status and task performance are hard to predict, as most trainings are only assessed after the shutdown maintenance operation is finished. A proactive assessment or intervention is needed to evaluate workers’ stress status and task performance during the training to enable early warning and interventions. This study proposes a neurophysiological approach to assess workers’ stress status and task performance under different virtual training scenarios. A Virtual Reality (VR) system integrated with the eye-tracking function was developed to simulate the power plant shutdown maintenance operations of replacing a heat exchanger in both normal and stressful scenarios. Meanwhile, a portable neuroimaging device – Functional Near-Infrared Spectroscopy (fNIRS) was also utilized to collect user’s brain activities by measuring hemodynamic responses associated with neuron behavior. A human–subject experiment (n = 16) was conducted to evaluate participants’ neural activity patterns and physiological metrics (gaze movement) related to their stress status and final task performance. Each participant was required to review the operational instructions for a pipe maintenance task for a short period and then perform the task based on their memory in both normal and stressful scenarios. Our experiment results indicated that stressful training had a strong impact on participants’ neural connectivity patterns and final performance, suggesting the use of stressors during training to be an important and useful control factors. We further found significant correlations between gaze movement patterns in review phase and final task performance, and between the neural features and final task performance. In summary, we proposed a variety of supervised machine learning classification models that use the fNIRS data in the review session to estimate individual’s task performance. The classification models were validated with the k-fold (k = 10) cross-validation method. The Random Forest classification model achieved the best average classification accuracy (80.38%) in classifying participants’ task performance compared to other classification models. The contribution of our study is to help establish the knowledge and methodological basis for an early warning and estimating system of the final task performance based on the neurophysiological measures during the training for industrial operations. These findings are expected to provide more evidence about an early performance warning and prediction system based on a hybrid neurophysiological measure method, inspiring the design of a cognition-driven personalized training system for industrial workers.     

Systematic literature review on augmented reality in smart manufacturing: Collaboration between human and computational intelligence

Smart manufacturing offers a high level of adaptability and autonomy to meet the ever-increasing demands of product mass customization. Although digitalization has been used on the shop floor of modern factory for decades, some manufacturing operations remain manual and humans can perform these better than machines. Under such circumstances, a feasible solution is to have human operators collaborate with computational intelligence (CI) in real time through augmented reality (AR). This study conducts a systematic review of the recent literature on AR applications developed for smart manufacturing. A classification framework consisting of four facets, namely interaction device, manufacturing operation, functional approach, and intelligence source, is proposed to analyze the related studies. The analysis shows how AR has been used to facilitate various manufacturing operations with intelligence. Important findings are derived from a viewpoint different from that of the previous reviews on this subject. The perspective here is on how AR can work as a collaboration interface between human and CI. The outcome of this work is expected to provide guidelines for implementing AR assisted functions with practical applications in smart manufacturing in the near future.     

Integrated manufacture: A role for virtual reality?

Virtual Reality (VR) is fast becoming an affordable technology with potentially wide-ranging applications in many professions including education, medicine and industry. Its advantages over existing technology are primarily that users can visualise, feel involvement and interact with virtual representations of real world activities in real time.

A recently completed study, funded by the EPSRC, examined the feasibility of VR as a tool for UK manufacturing industry. Of primary interest was whether manufacturers perceive a use for VR in their inudstry and, if so, what impact they envisage it will have within their company. A national survey was distributed to over 2,000 UK manufacturing companies randomly selected and interviews carried out with existing users of VR technology. A brief summary of these results is presented.

A demonstration application was developed in desktop VR representing the manufacture of a consumer product in which various stages of the manufacturing process were featured including initial design, manufacture, and testing. The demonstration application is described in detail and user assessments are presented. On the basis of these findings, the potential future role of VR in integrated manufacture is discussed. Relevance to industry

In only a short time, virtual environments have become a focus of serious consideration as a tool for manufacturing and other industry. In order that VR has greatest industrial utility we need to examine and develop its potential as a specialism-free integrating medium within a simultaneous engineering approach.     

虚拟现实技术在数字化校园中的应用

继多媒体、计算机网络之后,虚拟现实技术成为在教育领域内最具有应用前景的技术。随着该技术的不断发展,它在数字化校园的建设与应用中将发挥其特有的优势。本文将从虚拟现实技术的概念及特点出发,进一步探讨虚拟现实技术在数字化校园中的具体应用。     

The reality of virtual schools: A review of the literature

Virtual schooling was first employed in the mid-1990s and has become a common method of distance education used in K-12 jurisdictions. The most accepted definition of a virtual school is an entity approved by a state or governing body that offers courses through distance delivery – most commonly using the Internet. While virtual schools can be classified in different ways, the three common methods of delivery are by independent, asynchronous or synchronous means. Presently, the vast majority of virtual school students tended to be a select group of academically capable, motivated, independent learners. The benefits associated with virtual schooling are expanding educational access, providing high-quality learning opportunities, improving student outcomes and skills, allowing for educational choice, and achieving administrative efficiency. However, the research to support these conjectures is limited at best. The challenges associated with virtual schooling include the conclusion that the only students typically successful in online learning environments are those who have independent orientations towards learning, highly motivated by intrinsic sources, and have strong time management, literacy, and technology skills. These characteristics are typically associated with adult learners. This stems from the fact that research into and practice of distance education has typically been targeted to adult learners. The problem with this focus is that adults learn differently than younger learners. Researchers are calling for more research into the factors that account for K-12 student success in distance education and virtual school environments and more design research approaches than traditional comparisons of student achievement in traditional and virtual schools.     

Tangible authoring of 3D virtual scenes in dynamic augmented reality environment

This paper proposes tangible interfaces and interactions for authoring 3D virtual and immersive scenes easily and intuitively in tangible augmented reality (AR) environment. It provides tangible interfaces for manipulating virtual objects in a natural and intuitive manner and supports adaptive and accurate vision-based tracking in AR environments. In particular, RFID is used to directly integrate physical objects with virtual objects and to systematically support the tangible query of the relation between physical objects and virtual ones, which can provide more intuitive tangibility and a new way of virtual object manipulation. Moreover, the proposed approach offers an easy and intuitive switching mechanism between tangible environment and virtual environment. This paper also proposes a context-adaptive marker tracking method which can remove an inconsistent problem while embedding virtual objects into physical ones in tangible AR environments. The context-adaptive tracking method not only adjusts the locations of invisible markers by interpolating the locations of existing reference markers and those of previous ones, but also removes a jumping effect of movable virtual objects when their references are changed from one marker to another. Several case studies for generating tangible virtual scenes and comparison with previous work are given to show the effectiveness and novelty of the proposed approach.     

Immersive virtual reality in the age of the Metaverse: A hybrid-narrative review based on the technology affordance perspective

Immersive virtual reality (VR) that utilizes head-mounted displays (HMD) is one of the key emerging technologies of the 21st century and has drawn keen attention from consumers, practitioners, and scholars in various disciplines. Nevertheless, the information systems (IS) discipline has neglected immersive VR, given that only a handful of studies have been published in mainstream IS journals. However, the recent advancements in immersive VR technology provide new opportunities for organizations and IS researchers. In light of these points, we reviewed the immersive VR literature to provide a holistic view of opportunities and challenges for organizations and future research directions for the IS field. By examining the technical capabilities of immersive VR and the previous literature, we identified five affordances: embodiment, interactivity, navigability, sense-ability, and create-ability. Our review of the 151 studies from the IS and related fields synthesized how these affordances were utilized in various research domains. Guided by the affordance-actualization theory, we also identified the strategic opportunities and challenges that come with implementing VR. The actualization of immersive VR affordances in organizations is indeed a fruitful area for IS scholars as there are various venues to move the IS field as well as the VR research in the organizational context forward.     

A methodology for solid modelling in a virtual reality environment

This paper presents a methodology for solid modelling in a virtual reality (VR) environment. Solid modelling in the VR environment is precisely performed in an intuitive manner through constraint-based manipulations. A hierarchically structured and constraint-based data model is developed to support solid modelling in the VR environment. Constraint-based manipulations are realized by allowable motions for precise 3D interactions in the VR environment. A mathematical matrix is presented for representing allowable motions. A procedure-based degree-of-freedom combination method for 3D constraint solving is presented for deriving the allowable motions from constraints. A rule-based constraint recognition engine is developed for both constraint-based manipulations and implicitly incorporating constraints into the VR environment. A prototype system has been implemented to testify the feasibility of the presented methodology.     

A research agenda for augmented and virtual reality in architecture,engineering and construction

This paper presents a study on the usage landscape of augmented reality (AR) and virtual reality (VR) in the architecture, engineering and construction sectors, and proposes a research agenda to address the existing gaps in required capabilities. A series of exploratory workshops and questionnaires were conducted with the participation of 54 experts from 36 organisations from industry and academia. Based on the data collected from the workshops, six AR and VR use-cases were defined: stakeholder engagement, design support, design review, construction support, operations and management support, and training. Three main research categories for a future research agenda have been proposed, i.e.: (i) engineering-grade devices, which encompasses research that enables robust devices that can be used in practice, e.g. the rough and complex conditions of construction sites; (ii) workflow and data management; to effectively manage data and processes required by AR and VR technologies; and (iii) new capabilities; which includes new research required that will add new features that are necessary for the specific construction industry demands. This study provides essential information for practitioners to inform adoption decisions. To researchers, it provides a research road map to inform their future research efforts. This is a foundational study that formalises and categorises the existing usage of AR and VR in the construction industry and provides a roadmap to guide future research efforts.     

Industrial maintenance decision-making: A systematic literature review

The increasing competition among industries has leveraged the emergence of various tools and methods for maintenance decision-making support. This paper identifies in literature the application areas of industrial maintenance decision-making, the relationships between these areas and the ways in which authors integrate tools and methods. This information makes it possible to identify trends and deficiencies in this context, helping to centralize the efforts required for future work. This work follows a series of structured steps for a systematic literature review of papers related to the main topic available in online databases. The selected papers are subject to a content assessment and grouped according to the application areas. The direct comparison between these areas and the construction of a relational matrix provide a quantitative interpretation of the results and well-structured information. Additionally, this paper proposes a framework based on information from the literature, which summarizes the origin and flow of information used in the development of models, showing the relationship among application areas of decision making. The research undertaken identifies trends focused on joint production systems optimization and increasing the deployment of methods for autonomous equipment predictions.     

A quantitative analysis of the effectiveness of laparascopy and endoscopy virtual reality simulators

The increasing use of virtual reality (VR) simulators in surgical training makes it imperative that definitive studies be performed to assess their training effectiveness. Indeed in this paper we report the meta-analysis of the efficacy of virtual reality simulators in (1) the transference of skills from the simulator training environment to the operating room and (2) their ability to discriminate between the experience levels of its users. The task completion time and the error score were the two study outcomes collated and analyzed in this meta-analysis. Sixteen studies were identified from a computer-based literature search (1996-2004). The meta-analysis of the random-effects model (because of the heterogeneity of the data) revealed that training on virtual reality simulators did lessen the time taken to complete a given surgical task as also clearly differentiate between the experienced and the novice trainees. Meta-analytic studies such as the one reported here would be very helpful in the planning and setting up of surgical training programs and for the establishment of reference ‘learning curves’ for a specific simulator and surgical task. If any such programs already exist they can then indicate the improvements to be made in the simulator used such as providing for more variety in their case scenarios based on the state and/or rate of learning of the trainee.     

Application of virtual reality in hospital facilities design

The airborne particles present in certain hospital environments, such as the tuberculosis isolation or operating rooms, can be extremely harmful for patients and/or hospital personnel. An important issue during the design of hospital facilities is an efficient airborne particle removal system. A near-optimal setup of the parameters that affect the airflow, and consequently the airborne particle trajectories within the room is desirable. Computational Fluid Dynamics (CFD) is an alternative to tedious and time-consuming experimental investigations during the design phase, when a large number of alternatives need to be evaluated. The main limitations of CFD application in building design are the high level of skill required, the complexity of the setup phase, and the difficulty of output data interpretation using common 2D (two-dimensional) display devices. A virtual reality (VR) environment can help in overcoming some of these limitations. A CFD/VR procedure for design of contaminant-free hospital facilities is presented in this paper. By means of a VR preprocessing step, inferior solutions can be discharged to drastically reduce the number of configurations to investigate. Then, a CFD/VR tool is used to explore the restricted set of room layouts. The 3D (three-dimensional), immersive visualisation of an indoor space and of the particle motion inside it allows the user to really see the particle flows and consequently understand the effects of room parameters on particle motion throughout the room. In this way a close-to-optimal configuration of the room layout and of the ventilation system can be achieved more speedily and more conveniently compared to traditional CFD investigations.     

ClinicaVR: Classroom-CPT: A virtual reality tool for assessing attention and inhibition in children and adolescents

Having garnered interest both in clinic and research areas, the Virtual Classroom (Rizzo et al., 2000) assesses children's attention in a virtual context. The Digital MediaWorks team (www.dmw.ca) has evolved the original basic classroom concept over a number of iterations to form the ClinicaVR Suite containing the Classroom-CPT as one of its components. The present study has three aims: investigate certain validity and reliability aspects of the tool; examine the relationship between performance in the virtual test and the attendant sense of presence and cybersickness experienced by participants; assess potential effects of gender and age on performance in the test. The study was conducted with 102 children and adolescents from Grade 2 to Grade 10. All participants were enrolled in a regular school program. Results support both concurrent and construct validity as well as temporal stability of ClinicaVR: Classroom-Continuous Performance Test (CPT). Gender exerted no effect on performance, while age did. The test did not cause much cybersickness. We recommend ClinicaVR: Classroom-CPT as an assessment tool for selective and sustained attention, and inhibition, in clinic and research domains.     

A method to measure the uniformity of the virtual image distance in an augmented reality or virtual reality device

The virtual image distance in an augmented reality (AR) or virtual reality (VR) device is an important factor in determining its performance. In this paper, a method for measuring the virtual image distance and its uniformity is proposed. The setup for measurement consists of a two-dimensional spatial sensor array on the translational stage and a pinhole array plate placed in front of the AR or VR device. As the distance between the pinhole plate and the two-dimensional spatial sensor array were changed, the positions of the rays through each pinhole were measured by a two-dimensional spatial sensor array. The ray trajectories through each pinhole were obtained by fitting these positions to straight lines. The relative distances between these ray trajectories were calculated with respect to the distance from the pinhole plate. This method is effective for measuring the uniformity, such as the azimuthal dependence of the virtual image distance or the dependence of the virtual image distance on the distance from the optical axis of the VR lens.     

A virtual try-on system in augmented reality using RGB-D cameras for footwear personalization

This paper presents a system for design evaluation of footwear using commercial depth-sensing technologies. In a mixed reality environment, the system allows users to virtually try on 3D shoe models in a live video stream. A two-stage object tracking algorithm was developed to correctly align shoe models to moving feet during the try-on process. Color markers on the user's foot enabled markerless tracking. Tracking was driven by an iterative closest point (ICP) algorithm that superimposed the captured depth data and predefined reference foot models. Test data showed that the two-stage approach resulted in increased positional accuracy compared with tracking using only surface registration. Trimming the reference model using the instant view angle increased the computational efficiency of the ICP algorithm. The proposed virtual try-on function is an effective tool for realizing human-centered design. This study also demonstrated a new application of RGB-D cameras to product design.     

Assessing mental workload in virtual reality based EOT crane operations: A multi-measure approach

BackgroundEye-movement metrics and subjective workload measures are extensively used to determine mental workload of participants. The aim of this study was to assess Electric overhead travelling (EOT) crane operators’ mental workload variability based on eye movement metrics such as fixation frequency, fixation duration, saccade duration, saccade amplitude, and fixation/saccade ratio during EOT crane operations in virtual reality (VR) based EOT crane simulator.MethodsA 2^k (k = 3) factorial experiment with factors namely, hazardous scenario, activity level, and trial was designed and conducted to demonstrate the proposed assessment approach. Throughout the experiment, we recorded the eye movements of 12 EOT crane operators of a steel industry of authors’ country. Post experiment, the National Aeronautics and Space Administration task load index (NASA-TLX) was adopted as a subjective workload measure and run time of task completion was recorded. Eye-movement metrics, subjective workload measure, run time were tested with multivariate analysis of variance (MANOVA), and three way repeated measure analysis of variance (ANOVA).ResultsAt the level of α = 0.05, the experimental factors significantly influence the means of eye movement metrics, subjective ratings and run time. There was also significant influence among their interactions. A positive correlation was also found for eye movements metrics with NASA-TLX and run time.ConclusionsEye movement metrics help in understanding the mental workload of participants unobtrusively and continuously. Analysis of subjective workload measure and run time along with eye-gaze analysis provide a deeper understanding on the pattern of mental workload.