Human Interaction with Motion Planning Algorithm

This paper presents an interactive motion planning system to compute free collision motion in a numerical model. The system is based on interaction between a user and a motion planning algorithm. On one hand the user moves the object with an interactive device and on the other hand a motion planning algorithm searches a solution in the configuration space. The interaction aims at improving the guidance of an operator during a robot motion task in a virtual environment with the help of an automatic path planning algorithm. Existing works use a two-step decomposition which limits the interaction between the user and the ongoing process. We propose a modification of a classic motion planning method, the Rapidly-exploring Random Tree to build an Interactive-RRT. This method is based on exchanging pseudo-forces between the algorithm and the user, and on data gathering (labels) from the virtual scene. Examples are shown to illustrate the Interactive motion planning system with different interactive devices (space mouse and haptic arm). We analyze the influence of the user’s dexterity to find a solution depending on various parameters of the algorithm and we show how we can adapt these parameters to a user.     

Intelligent Cruise-Control Navigation: A new navigation/travel method for use in virtual environments

An important feature of virtual reality is the facility for the user to move around a virtual environment in a natural and easily controlled manner. Navigation, also called locomotion, travel or motion, involves changing the perspective of the user in the virtual environment (VE). It allows the user to move in the VE as well as reorient themselves to look at the world differently. Natural locomotion methods are able to contribute to a sense of presence and reality. The illusion of presence can be lost through unnatural experiences during travel in the VE. This can be caused by poor interactive metaphors or by experiences which do not agree with the user's everyday understanding of the real world. This paper focuses on the navigation method in the VE, one of the major interfaces for the interactivity between human and VE in virtual reality circumstances and worlds. It proposes a new navigation method. Intelligent Cruise-Control Navigation (ICCN), which provides a natural and user-centred approach to navigation in the VE and can improve the user's sense of reality and presence. ICCN is composed of three major phases: Constant Velocity Navigation, Collision Detection and Avoidance, and Path Adjustment. The ICCN can reduce the user's fatigue and improve the user's presence in the VE. The small experimental study reported in this paper suggests that the ICCN will be a natural, straightforward, and useful navigation interface in VE.     

基于虚拟引擎的交互系统研究

针对网络环境下虚拟交互的应用问题,提出虚拟交互引擎模型结构。借助多Agent协同以及3D交互互感的接口标准,为虚拟现实系统提供Agent库、交互规则库等4个核心模块结构库。通过交互库构建用户设计输入操作感知等虚拟交互子系统,并给出虚拟交互引擎解决方案。     

虚拟环境的用户意图捕获

目的虚拟制造环境中需要复杂精确的3D人机交互。目前的虚拟环境(VE)的主要问题是人在交互过程中的认知和操作负荷太重,交互效率亟需提高。解决此问题的重要途径是提高机器的认知能力。方法本文研究了用户意图的分析和抽取,并建立多通道用户意图理解的算法,以此来提高交互效率。结果结合虚拟装配应用给出了典型意图的实验结果并给予分析。通过实验对多通道意图的可用性和可靠性,以及基于意图系统的实时性进行了评估。实验是虚拟装配空间中用户拾取对象意图的实验。当3维鼠标和对象距离为5 000 mm时,传统系统中操作平均耗时5.344 7 s,而基于意图的系统中平均耗时2.326 6 s。基于意图的系统极大地降低了操作的时间和复杂度。结论采用意图驱动的系统情景转换能在虚拟环境工作中有效地降低人的认知负荷,并能很好地帮助系统开发者进行混成系统的建模和分析,降低开发的复杂度。实践结果表明用户意图理解的多通道模型和算法能极大地提高交互式系统的交互自然性和交互效率。该方法不仅适用于本文所用的虚拟装配系统,对于所有的虚拟环境应用场景都有同样的有效性。     

沉浸式虚拟训练中的虚拟手操控技术研究

将人手的自然动作加入沉浸式虚拟现实系统中,可有效提高系统的沉浸感和交互性,简便快捷、成本低廉的虚拟手建模及操控技术,有助于此项内容的推广应用;在对虚拟手骨骼结构模型分析的基础上,提出了基于Leap Motion的虚拟手控制方法和流程;以维修训练为任务背景,设计了四种用于虚拟手操作的手势规则,并对实现流程和关键技术进行了分析;构建基于Unity3D引擎的原型系统对虚拟手进行仿真验证,结果表明,基于Leap Motion的虚拟手操控简单、交互自然,可以满足虚拟现实系统交互需要。     

虚拟教学环境中基于草图的自适应用户界面研究

将草图技术、自适应技术与传统的虚拟现实技术相结合应用于虚拟教学过程中,通过增强用户和系统之间个性化需求的交互处理来提高系统的智能性和友好性,进而有效提高虚拟教学环境的应用效果.旨在虚拟环境中构建基于草图的自适应用户界面并应用于虚拟教学来满足特定教学需求,着重结合实例分析了基于自适应草图用户界面的虚拟教学环境中的草图上下文处理机制.在上述研究基础上,设计开发了一个虚拟教学原型系统,实验证明该系统在用户体验上有明显的改善.     

An immersive multi-agent system for interactive applications

This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the problem of dynamic navigation of individual and groups of characters in the multi-agent system. A natural interface is designed for the interaction between the user and the virtual characters, as well as the virtual environment, based on gesture recognition. To evaluate the efficiency of the dynamic navigation method, performance results are provided. The presented system has the potential to be used in the training and evaluation of emergency evacuation and other real-time applications of crowd simulation with interaction.     

基于手势的交互式虚拟环境

交互式虚拟现实系统能够为用户提供直观自然的用户接口。使用手势,用户可以更加高效地与虚拟环境进行交互。但随着系统中识别手势的增多,正确手势的识别率和系统的性能会跟着下降。文章提出用交互状态解决手势识别率和系统性能下降问题,并给出了实现方案。     

基于MFC和OpenGL的虚拟人体经络穴位模型实现方法

现有人体经络穴位模型较少具有触摸交互功能,本文基于OpenGL(Open Graphics Library)和MFC(Microsoft Foundation Classes)技术框架,仿真实现交互式虚拟人体经络穴位模型系统,并介绍其实现方法和过程。首先建立人体模型以及穴位点、经络线段。然后利用OpenGL和MFC对模型、穴位点、经络线加载和渲染。详细阐述系统实现的框架和流程,以及在开发过程中涉及的关键技术,如使用动态方向调整来显示穴位点名称,使用全局函数传递消息来提高模型加载响应的效率,给出触摸点坐标快速获取的方法。最后将模型成功部署到触摸式一体机上。模型系统具有较好的交互性,无需鼠标,仅用手指触摸即能实现模型的放大、缩小、旋转、定穴、经络循环等操作。模型能够帮助使用者学习经络知识,其穴位记忆功能能够测试使用者对穴位掌握情况,也为穴位爱好者提供较真实的经络穴位学习环境。     

Design evaluation and modification of mechanical systems in virtual environments

Design is one of the most important stages in the manufacturing cycle and influences all the subsequent stages of product development. In the context of today’s iterative design methodology, the modification of any design is a process involving many evaluations and improvements to the solutions chosen in earlier stages. For this purpose, in the most recent decade, 3D computer simulations have become common tools used within industry. Whilst virtual reality (VR) technology is seen as the interaction technology of the future, much of the current research in this area is carried out to explore the potential benefits and added value brought by the integration of this into standard software technologies currently used at various stages in manufacturing life cycle. A lot of attention has been given to exploring the usability and benefits of interactive VR for assembly planning, knowledge elicitation and design and simulation. However, little research has focussed on the analytical aspects of the design process, for example in the use of VR as an interface for simulation software in finite element methods and multi-body systems. This paper introduces research focussing on applications of virtual environments (VEs) for interactive design evaluation and modification adapted and used with standard simulation software. The use of such interactive visualisation offers the engineer more realistic real-time representations of the design and advanced facilities to interact with the model during the design process. While design evaluation is based mainly on visualisation; design modification requires interactive changes of the model during the simulation, and the interfaces described here highlights such applications. In this work, two software prototypes have been developed using VR technology. First, a software tool called design evaluation in virtual environments is presented together with an application in civil engineering to illustrate the mode of operation and added value of the use of an interactive visualisation environment. Linking the simulation software with the VE provides real time bi-directional communication of graphical information which can be successfully achieved even within the limits of current computer technology. The tool includes a suite of software modules and a user interface to facilitate the link between the simulation results and the VE. The second tool facilitates the design modification in virtual environments system by providing real time dynamic simulation. Two dynamic approaches are investigated in order to study the real time simulation issues in the context of design modification and system performance: the classic approach based on rigid interconnected bodies, and a new and novel approach developed by the authors based on particles dynamics. Both implementations have been tested and compared on a mechanism application under the same computing conditions. The research applications presented demonstrate the practicality, flexibility and versatility of the visualisation in virtual environment in design evaluation and modification. However, the computer efficiency whilst carrying out real time dynamic simulation is limiting the range of applications to models of moderate size; however, this is an improvement on previous similar applications.     

Physics-based virtual reality for task learning and intelligent disassembly planning

Physics-based simulation is increasingly important in virtual manufacturing for product assembly and disassembly operations. This work explores potential benefits of physics-based modeling for automatic learning of assembly tasks and for intelligent disassembly planning in desktop virtual reality. The paper shows how realistic physical animation of manipulation tasks can be exploited for learning sequential constraints from user demonstrations. In particular, a method is proposed where information about physical interaction is used to discover task precedences and to reason about task similarities. A second contribution of the paper is the application of physics-based modeling to the problem of disassembly sequence planning. A novel approach is described to find all physically admissible subassemblies in which a set of rigid objects can be disassembled. Moreover, efficient strategies are presented aimed at reducing the computational time required for automatic disassembly planning. The proposed strategies take into account precedence relations arising from user assembly demonstrations as well as geometrical clustering. A motion planning technique has also been developed to generate non-destructive disassembly paths in a query-based approach. Experiments have been performed in an interactive virtual environment including a dataglove and motion tracker that allows realistic object manipulation and grasping.     

用户认知驱动的VR自然交互认知负荷研究

针对目前虚拟现实自然交互系统中用户认知负荷难以量化问题,以量化认知负荷为目的,提出一种基于概率神经网络的多通道信息融合模型。首先根据认知理论建立“认知—行为—环境”框架,并在此基础上研究用户认知的数学表征（方法,过程,机理）,构建多通道认知模型;然后将多通道信息融合方法引入虚拟现实自然交互系统,建立一种面向虚拟现实自然交互系统的认知负荷模型;通过VR隧道应急救援系统实验对系统中用户认知进行验证。验证结果表明,具体提出的认知负荷模型具有一定的可行性。     

Construction of feature-matching perception in virtual assembly

An important characteristic of virtual assembly is interaction.Traditional direct manipulation in virtual assembly relies on dynamic collision detection,which is very time-consuming and even impossible in desktop virtual assembly environment.Feature-matching is a critical process in harmonious virtual assembly,and is the premise of assembly constraint sensing.This paper puts forward an active object-based feature-matching perception mechanism and a feature-matching interactive computing process,both of which make the direct manipulation in virtual assembly break away from collision detection.They also help to enhance virtual environment understandability of user intention and promote interaction performance.Experimentl results show that this perception mechanism can ensure that users achieve real-time direct manipulation in desktop virtual environment.     

虚拟交互角色的应用探讨

虚拟交互角色源自于儿童的"幻想玩伴"理论,其形成与发展的过程伴随着儿童社会技能的建立而发挥着促进的作用,并为整个人生奠定积极基础。本文对儿童幻想玩伴形式进行深入的诠释,并将虚拟交互角色概念的应用案例进行分析探讨,构建应用性虚拟角色交互界面所应具备的特点和需要注意的问题。     

Guided Exploration with Dynamic Potential Fields: the Cubical Path System

Exploring unknown models or scenes is a highly interactive and dynamic process. Systems for automatic presentation of models or scenes either require cinematographic rules, direct human interaction, framesets, or pre-calculation of paths to a known goal. In this paper we present a system which can deal with rapidly changing user interest in objects of a scene or model as well as with dynamic models and changes of the camera position introduced interactively by the user or through cuts. We describe CubicalPath, a new potential field-based camera control system that helps with the exploration of virtual environments.     

具有协同交流功能的虚拟物理实验室

为解决虚拟实验中师生不能有效交流的问题,采用VRML-JS-Java通信机制实现了一种具有交流功能的虚拟物理实验室.介绍了实验室总体设计和场景层次结构,阐述了三维场景的建模过程和流程,重点对交互功能的服务器端、客户端、用户间的交互与感知,以及VRML虚拟场景与Java接口等关键技术进行实现.经过发布与测试,表明虚拟物理实验室具有良好的层次性、可重用性和互操作性,在大学实验教学改革中具有较高的实用价值.     

基于zSpace的岩心钻机虚拟拆装系统

提高虚拟拆装系统的交互性和立体显示效果, 对虚拟仿真项目的实际应用具有积极意义. 以地质工程中的关键设备岩心钻机为对象, 利用Unity3D和zSpace软硬件平台, 开发一套岩心钻机虚拟拆装系统. 系统拆装模块主要利用六自由度触控笔, 对岩心钻机整机和分动箱、回转器等重要部件, 实现抓取、缩放、爆炸、模型重置等交互操作. 通过zView插件包, 实现拆装过程的增强现实显示效果. 添加配套的钻机简介、工艺流程、安全规范等理论知识模块, 完善虚拟实训体系. 实践表明, 该系统具有交互性强, 显示效果佳的特点, 在实践教学中取得了较好的效果.     

Vision‐tangible interactive display method for mixed and virtual reality: Toward the human‐centered editable reality

Building a human‐centered editable world can be fully realized in a virtual environment. Both mixed reality (MR) and virtual reality (VR) are feasible solutions to support the attribute of edition. Based on the current development of MR and VR, we present the vision‐tangible interactive display method and its implementation in both MR and VR. We address the issue of MR and VR together because they are similar regarding the proposed method. The editable mixed and virtual reality system is useful for studies, which exploit it as a platform. In this paper, we construct a virtual reality environment based on the Oculus Rift, and an MR system based on a binocular optical see‐through head‐mounted display. In the MR system about manipulating the Rubik's cube, and the VR system about deforming the virtual objects, the proposed vision‐tangible interactive display method is utilized to provide users with a more immersive environment. Experimental results indicate that the vision‐tangible interactive display method can improve the user experience and can be a promising way to make the virtual environment better.     

A physically based approach with human–machine cooperation concept to generate assembly sequences

Assembly plan is considered one of the important stages to minimize the cost of manufacturer and to ensure the safety of assembly operation, the main problem of assembly sequence planning approach is how to reduce the deviation from the real manufacture conditions. In this paper, we have extensively investigated a novel approach to automatically generate the assembly sequences for industrial field, which is especially applied to other large-scale structures. A physically based assembly representation model includes not only the pre-determined basic assembly information, such as precedence relations between parts or subassemblies, geometric constraints, different assembly types, and also the dynamic real-time physical properties, such as the center position of gravity, the force strength of the part, et al. This representation model considered the influences on optimum sequences by assembly operations will be modified by the feedback from interactive virtual environment. Then, we select the safety, efficiency and complexity as the optimization objectives. A hybrid search approach may be used to find the optimum assembly sequence, which will be integrated into an interactive assembly virtual environment (IAVE). It means that the results of assembly interaction can be provided to update the assembly planning model as a feedback, by which the approach will take advantages of the immune memory for local optimum search. The user can adjust the assembly sequences with obvious good objective by interaction with IAVE to improve the performance of the search algorithm. We describe human–machine cooperation (HMC) method for ASP in this work, by which human also can play a pivotal role instead of pure soft-computing. A series of numerical experiments are done to validate the performance of the physically based approach (PBA) to generate assembly sequence, which shows the efficiency and the operability to guide the assembly work.     

Reactive virtual agent learning for NUI-based HRI applications

The natural user interface (NUI) has been investigated in a variety of fields in application software. This paper proposes an approach to generate virtual agents that can support users for NUI-based applications through human–robot interaction (HRI) learning in a virtual environment. Conventional human–robot interaction (HRI) learning is carried out by repeating processes that are time-consuming, complicated and dangerous because of certain features of robots. Therefore, a method is needed to train virtual agents that interact with virtual humans imitating human movements in a virtual environment. Then the result of this virtual agent can be applied to NUI-based interactive applications after the interaction learning is completed. The proposed method was applied to a model of a typical house in virtual environment with virtual human performing daily-life activities such as washing, eating, and watching TV. The results show that the virtual agent can predict a human’s intent, identify actions that are helpful to the human, and can provide services 16 % faster than a virtual agent trained using traditional Q-learning.