Evaluation of Internal and External Validity of a Virtual Environment for Learning a Long Procedure

Virtual reality is a frequently used tool in vocational training. Nevertheless, its efficiency has not been systematically tested. The main goal of this study is to assess the effectiveness of a virtual environment (VE) for learning a complex procedure in the biomedical domain. Two experiments were performed. The first one assessed internal validity of the VE, which is the effectiveness of using a VE in the process of learning a new procedure. The second one tested external validity of the VE, which is the participants’ ability to reproduce the acquired skills in a real context. We find that internal and external validity must be evaluated before using a virtual environment in long-term procedure learning. The results of such evaluations are a basis for future experiments aiming to optimize learning conditions in a VE and transferring the acquired skills in a real context.     

Perceived usefulness of,engagement with,and effectiveness of virtual reality environments in learning industrial operations: the moderating role of openness to experience

The development of virtual reality (VR) in enhancing the effectiveness of the learning process, with its interactive, immersive, and intuitive pedagogical environment, has become a necessity for corporations with increasingly complex operations. However, VR users’ perceptions, openness and learning effectiveness are seldom comprehensively evaluated, particularly in learning complex industrial operations. In this study, grounded in the technology acceptance model, a moderated mediation model of perceived usefulness, ease of use, openness to experience, and engagement in VR-based learning was developed. The model was empirically validated using responses collected from 321 users who were trained on aircraft and cargo terminal operations powered by a novel VR-based learning platform. A survey to measure openness to experience and a pre-training performance test were carried out, followed by a post-training survey of learners’ intrinsic factors, including the influence of perceived usefulness, openness to experience, and attitude towards learning. The study revealed that learners with an open attitude towards experiencing new technology tend to perceive VR technology as a useful platform for training. In addition, the learners with more positive views of VR technology-supported training were more engaged in learning.

     

How do varied populations interact with embodied conversational agents? Findings from inner-city adolescents and prisoners

Two studies were conducted to identify individual characteristics that predict behavioral responses to violence prevention interventions. These studies used embodied conversational agents (ECAs) to create hypothetical social situations (called virtual vignettes) to assess interpersonal competency skills. One study was of male inner-city African–American adolescents, and the second was of male prisoners in a state correctional system. In pre- and post-intervention sessions, participants interacted with an ECA that tried to entice them into making risky decisions. The virtual vignette sessions tested participants’ negotiation and conflict resolution skills. Results showed differing tendencies for participants to be engaged by the virtual vignettes. The vignettes were sufficiently realistic to elicit differences in behavior among the adolescents, but generally not for the prisoners. Prior acceptance, accessibility, and usability data suggest that most users readily accept ECAs as valid conversational partners. The evidence presented here suggests that the technology – or the setting in which the technology is used – is not by itself sufficient to actively engage users. The usefulness of virtual vignettes to adequately predict future behavior may be at least partially influenced by participant characteristics.     

Empirical evidence,evaluation criteria and challenges for the effectiveness of virtual and mixed reality tools for training operators of car service maintenance

The debate on effectiveness of virtual and mixed reality (VR/MR) tools for training professionals and operators is long-running with prominent contributions arguing that there are several shortfalls of experimental approaches and assessment criteria reported within the literature. In the automotive context, although car-makers were pioneers in the use of VR/MR tools for supporting designers, researchers started only recently to explore the effectiveness of VR/MR systems as mean for driving external operators of service centres to acquire the procedural skills necessary for car maintenance processes. In fact, from 463 journal articles on VR/MR tools for training published in the last thirty years, we identified only eight articles in which researchers experimentally tested the effectiveness of VR/MR tools for training service operators’ skills. To survey the current findings and the deficiencies of these eight studies, we use two main drivers: (i) a well-known framework of organizational training programmes, and (ii) a list of eleven evaluation criteria widely applied by researchers of different fields for assessing the effectiveness of training carried out with VR/MR systems. The analysis that we present allows us to: (i) identify a trend among automotive researchers of focusing their analysis only on car service operators’ performance in terms of time and errors, by leaving unexplored important pre- and post-training aspects that could affect the effectiveness of VR/MR tools to deliver training contents – e.g., people skills, previous experience, cibersickness, presence and engagement, usability and satisfaction and (ii) outline the future challenges for designing and assessing VR/MR tools for training car service operators.     

Development and validation of a confined space rescue training prototype based on an immersive virtual reality serious game

Due to the inherently hazardous nature of the confined space environment and a lack of effective rescue training on on-site personnel attempting the rescue of fellow workers in a confined space, accidents or even multiple fatalities may happen. For this reason, improving the skill set of rescuers is of vital importance. Immersive virtual reality serious games (IVR SG) have been shown to be effective and are now widely used in safety training. However, there is little research on its application in confined space rescue training. To bridge this gap, we have developed a confined space rescue training system (called Rescue Success) based on IVR SG, and then tested its effectiveness in a real environment. We carried out a single-blind, randomized comparative experiment to study and compare the learning effectiveness of using IVR SG compared to an E-lecture in terms of behavioral skills, knowledge acquisition, self-efficacy, and simulation-induced sickness. Our results show that both training methods significantly improve the participant’s rescue behavioral skills, knowledge, and self-efficacy. Overall, IVR SG performed better than an E-lecture in all aspects except simulation-induced sickness, where there is no difference between the two methods.     

Virtual reality training system for maintenance and operation of high-voltage overhead power lines

The maintenance of high-voltage overhead power lines involves high-risk procedures; the accidents involving live lines maintenance can be lethal. This paper presents the architecture and main features of a novel non-immersive virtual reality training system for maintenance of high-voltage overhead power lines. The general aim of this work was to provide electric utilities a suitable workforce training system to train and to certify operators working in complex and unsafe environments. The developed system has three components: the virtual warehouse, interactive 3D environments, and a learning management system. The workforce training system consists of thirty-one maintenance maneuvers, including the application of different techniques and equipment designed for various structures. Additionally, the system, using 3D animations, illustrates the safety conditions required before starting the maintenance procedures. To fit the worker’s different skill levels, the system has three operation modes: learning, practice, and evaluation, which can be accessed according to the trainee’s level of knowledge. The system is currently used to train thousands of overhead power lines operators of an electric utility in Mexico. The system has demonstrated to be a cost-effective tool for transferring skills and knowledge to new workers while reducing the time and money invested in their training.     

Assistive robot application based on an RFID control architecture and a wireless EOG interface

This paper describes an assistive robot application that combines a portable wireless interface based on electrooculography (EOG) and Radiofrequency Identification (RFID) technology. This assistive application is aimed at handicapped users who suffer from a severe motor disability. To that end, a realistic application has been designed. It consists of an environment in which users can bring a glass and a water bottle closer with only the help of their eye movement using a real robot arm. RFID will be used as a support to the EOG interface in a shared control architecture by storing information of the objects in tags placed on the scene. Five volunteers tested the assistive robot application. The results obtained show that all of them were able to finish the tests in a suitable time and the results improved with practice and training. This proves that the assistive robot application can be a feasible way to help handicapped users.     

The Impact of Serial Solutions’ Summon™ on Information Literacy Instruction: Librarian Perceptions

Web-scale discovery systems, such as Serial Solutions’ Summon? Service, are replacing older federated search technologies as the tool for users to access library resources quickly and easily. The impact of Web-scale discovery systems on instruction is not known. This study of instruction librarian perceptions of Summon's impact on instruction and student information literacy skills is based on a survey. The survey reveals librarians’ ambivalence toward Summon. While some librarians agree that Summon has the potential to change the way librarians teach information literacy skills, it has not been fully integrated into the classroom.     

Software requirements and acceptance testing

In this paper we 1) review industry acceptance testing practices and 2) present a systematic approach to scenario analysis and its application to acceptance testing with the aim to improve the current practice. It summarizes the existing practice into categories and identifies the serious weakness. Then, a new approach based on the formal scenario analysis is presented. It is systematic, and easily applicable to any software or system. A simple, yet realistic example is used to illustrate its effectiveness. Finally, its benefits and its applicability are summarized.     

Product interface design: A participatory approach based on virtual reality

The usability of the user interface is a key aspect for the success of several industrial products. This assumption has led to the introduction of numerous design methodologies addressed to evaluate the user-friendliness of industrial products. Most of these methodologies follow the participatory design approach to involve the user in the design process. Virtual Reality is a valid tool to support Participatory Design, because it facilitates the collaboration among designers and users.The present study aims to evaluate the feasibility and the efficacy of an innovative Participatory Design approach where Virtual Reality plays a ‘double role’: a tool to evaluate the usability of the virtual product interface, and a communication channel that allows users to be directly involved in the design process as co-designers.In order to achieve these goals, we conducted three experiments: the purpose of the first experiment is to determine the influence of the virtual interface on the usability evaluation by comparing “user–real product” interaction and “user–virtual product” interaction. Subsequently, we tested the effectiveness of our approach with two experiments involving users (directly or through their participation in a focus group) in the redesign of a product user interface. The experiments were conducted with two typologies of consumer appliances: a microwave oven and a washing machine.     

The changing nature of user attitudes toward virtual world technology: A longitudinal study

Virtual world technologies have been utilized in gaming for a number of years but only recently have they been applied as a serious tool for business. Many business applications have been identified, including the use of virtual worlds for team collaboration, training, and education, but a question remains about whether users will accept the premise that virtual worlds represent useful environments for engaging in business functions. We address this question by examining user reactions to virtual worlds. The first study looks at attitudes of users of the virtual world Second Life during three time periods (i.e., before exposure to the environment, after an information session and discussion of Second Life, and after use of the environment). Two variables, user acceptance of virtual world technologies and user self-efficacy, were examined as the primary dependent measures. Results show that while self-efficacy increases over time, user acceptance decreases in a highly correlated pattern. A second study investigates the underlying causes of the observed pattern of user acceptance using a content analysis of written reflections of user experiences. Both studies paint a detailed picture of user intentions and some of the reasons these intentions developed after use. The paper concludes with a discussion of the implications of these results for business managers and researchers.     

Establishing the Usability of a Virtual Training System for Assembly Operations within the Automotive Industry

Virtual training systems deliver training within a virtual environment (VE) using virtual reality (VR) or augmented reality (AR) technologies. However, to be fully accepted as a valid tool for training within the automotive industry, evidence is required on the ability of these systems to deliver effective and efficient training to the relevant users. This paper aims to investigate the effectiveness and efficiency of the first prototype of the virtual training system (VTS) developed within the VISTRA (Virtual Simulation and Training of Assembly and Service Processes in Digital Factories) project (FP7‐ICT‐285176), using real end users from the OPEL automotive plant in Rüsselsheim, Germany. Two separate and independent studies were employed that used objective and subjective methods of investigation to establish performance and usability measures. The objective results show that virtual training was effective in reducing error during task performance when compared to traditional training. The subjective results concluded that the opinions of the participants were mainly positive concerning the overall use of the VTS for assembly operation training; however, a number of issues were highlighted and reported to the developers for further advancement of the system.     

The wise cursor: assisted selection in 3D serious games

In recent years, the evolution of 3D graphics hardware and software has lead to a growing interest for serious games in three-dimensional virtual environments for learning, training, and rehabilitation. Many of these games are based on a first-person-shooter paradigm in which users navigate through the environment, select, and manipulate virtual objects. The target users of these applications are not necessarily usual gamers, and they often have difficulties in navigating and interacting in the 3D environment. This paper proposes the wise cursor, a new method for selection that improves the usability and accessibility of mouse-driven serious games in 3D environments. At each user click, the proposed method computes a list of objects candidates to be selected and their probability of being the desired one. Depending on the uncertainty of the probability distribution, either one object is selected or a mechanism to clarify the selection is proposed. In the former case, if the selected object is within the user avatar’s scope, the action associated to it is realized, otherwise the application automatically navigates toward it. In this way, selection and navigation are eased. The empirical results of the usability tests show that this technique is fast, practical, and that it requires little user’s skills. Thus, it can make serious games usable for a wider range of users who can concentrate on the training objectives without technological barriers.     

How virtual reality affects perceived learning effectiveness: a task–technology fit perspective

The application of virtual reality (VR) in improving users’ learning outcomes, especially in perceived learning effectiveness, is a new area. VR provides visualisation and interaction within a virtual world that closely resembles a real world, bringing an immersive study experience. It also has two special features: representational fidelity and immediacy of control. However, only when the technology fits the tasks that users are performing will it be adopted. In addition, technology itself cannot improve learning outcomes; certain learning behaviours, such as reflective thinking, should be prompted first so that learning outcomes can be improved. The research hypotheses derived from this model have empirically been validated using the responses to a survey among 180 users. These responses have been examined through SmartPLS 2.0. Surprisingly, task–technology fit does not moderate the relationship between VR and technology quality and the relationship between VR and technology accessibility. From this study, we can conclude that VR will influence reflective thinking and further indirectly improve perceived learning effectiveness.     

虚拟现实在变电站仿真中的应用

虚拟环境就是采用以计算机技术为核心的现代高科技生成逼真的三维环境,是对现场环境的最真实仿真.该系统基于OSG1.0开发库,利用3DMax建模工具构建模型场景,VC 和OSG编程来实现场景的显示、场景的漫游、碰撞检测和设备的状态变化,构筑一个三维、动态、实时、可交互的数据平台, 展现包含各种数据的动态的实体效果.用户可以自然地和虚拟环境中的客体进行交互,相互影响,从而产生亲临现场的感受和体验.最终该系统满足了变电站仿真培训的要求.     

Extended technology acceptance model of Internet utilization behavior

Based on the belief–attitude–performance chain, this study combines Davis’s technology acceptance model (TAM) and the information behavior model to develop an extended TAM for Internet use. The theoretical model was tested via a questionnaire survey of 203 Taiwanese office workers. The empirical results not only confirmed TAM, but also showed that the relevance of information needs strongly determines perceived usefulness, perceived ease of use, and user attitudes toward Internet use for information seeking, as well as strongly influencing individual performance during the information use stage. More importantly, relevance has greater positive effect on perceived performance and perceived usefulness for enterprise Intranet users than for simple/interactive Intraweb users. Enterprise Intranet users have more positive attitudes toward the Internet and more positive perceptions of system effectiveness for supporting office tasks than simple/interactive Intraweb users. Furthermore, perceived ease of use is the strongest determinant of user attitude toward Internet use in both enterprise Intranet and simple/interactive Intraweb applications. Overall, the extended TAM explains the behavior of enterprise Intranet users better than that of simple/interactive Intraweb users. Limitations of this study are discussed.     

Augmented reality using personal projection and retroreflection

The support of realistic and flexible training simulations for military, law enforcement, emergency response, and other domains has been an important motivator for the development of augmented reality technology. An important vision for achieving this goal has been the creation of a versatile “stage” for physical, emotional, and cognitive training that combines virtual characters and environments with real world elements, such as furniture and props. This paper presents REFLCT, a mixed reality projection framework that couples a near-axis personal projector design with tracking and novel retroreflective props and surfaces. REFLCT provides multiple users with personalized, perspective-correct imagery that is uniquely composited for each user directly into and onto a surrounding environment, without any optics positioned in front of the user’s eyes or face. These characteristics facilitate team training experiences which allow users to easily interact with their teammates while wearing their standard issue gear. REFLCT can present virtual humans who can make deictic gestures and establish eye contact without the geometric ambiguity of a typical projection display. It can also display perspective-correct scenes that require a realistic approach for detecting and communicating potential threats between multiple users in disparate locations. In addition to training applications, this display system appears to be well matched with other user interface and application domains, such as asymmetric collaborative workspaces and personal information guides.     

The design and delivery of crew resource management training: exploiting available resources

Despite widespread acceptance throughout commercial and military settings, crew resource management (CRM) training programs have not escaped doubts about their effectiveness. The current state of CRM training is an example of how an entire body of pertinent research and development has not had the impact on practice that it could. In this paper we outline additional resources (i.e., principles, information, findings, and guidelines) from the team training and training effectiveness research literatures that can be used to improve the design and delivery of CRM training. Some of the resources discussed include knowledge about training effectiveness, training teamwork-related skills, scenario design, and performance measurement. We conclude with a discussion of emerging resources as well as those that need to be developed. The purpose of this paper is to provide the CRM training developer with better access to resources that can be applied to the design and delivery of CRM training programs.     

Acceptance of e-commerce services: the case of electronicbrokerages

This paper examines human motivations underlying individual acceptance of business-to-consumer (B2C) electronic commerce services. Such acceptance is the key to the survival of firms in this intensely competitive industry. A modified theory of planned behavior (TPB) is used to hypothesize a model of e-commerce service acceptance, which is then tested using a field survey of 172 e-brokerage users. We found TPB was useful in explaining e-commerce service acceptance, however, acceptance motivations were significantly different from that of typical IS products. Based on a broader conceptualization of TPB's subjective norm to include both external (mass-media) and interpersonal influences, we report that subjective norm is an important predictor of e-commerce acceptance, behavioral control has minimal impact on e-commerce acceptance, and external influence is a significant determinant of subjective norm. Implications of these findings in light of e-commerce research and practice are discussed     

A crowdsourcing method to develop virtual human conversational agents

Educators in medicine, psychology, and the military want to provide their students with interpersonal skills practice. Virtual humans offer structured learning of interview skills, can facilitate learning about unusual conditions, and are always available. However, the creation of virtual humans with the ability to understand and respond to natural language requires costly engineering by conversation knowledge engineers (generally computer scientists), and incurs logistical cost for acquiring domain knowledge from domain experts (educators). We address these problems using a novel crowdsourcing method entitled Human-centered Distributed Conversational Modeling. This method facilitates collaborative development of virtual humans by two groups of end-users: domain experts (educators) and domain novices (students). We implemented this method in a web-based authoring tool called Virtual People Factory. Using Virtual People Factory, medical and pharmacy educators are now creating natural language virtual patient interactions on their own. This article presents the theoretical background for Human-centered Distributed Conversational Modeling, the implementation of the Virtual People Factory authoring tool, and five case studies showing that Human-centered Distributed Conversational Modeling has addressed the logistical cost for acquiring knowledge.