An immersive multi-agent system for interactive applications

This paper presents an interactive multi-agent system based on a fully immersive virtual environment. A user can interact with the virtual characters in real time via an avatar by changing their moving behavior. Moreover, the user is allowed to select any character as the avatar to be controlled. A path planning algorithm is proposed to address the problem of dynamic navigation of individual and groups of characters in the multi-agent system. A natural interface is designed for the interaction between the user and the virtual characters, as well as the virtual environment, based on gesture recognition. To evaluate the efficiency of the dynamic navigation method, performance results are provided. The presented system has the potential to be used in the training and evaluation of emergency evacuation and other real-time applications of crowd simulation with interaction.     

Interactive teaching of task-oriented robot grasps

This paper focuses on the problem of grasp stability and grasp quality analysis. An elegant way to evaluate the stability of a grasp is to model its wrench space. However, classical grasp quality measures suffer from several disadvantages, the main drawback being that they are not task related. Indeed, constructive approaches for approximating the wrench space including also task information have been rarely considered. This work presents an effective method for task-oriented grasp quality evaluation based on a novel grasp quality measure. We address the general case of multifingered grasps with point contacts with friction.The proposed approach is based on the concept of programming by demonstration and interactive teaching, wherein an expert user provides in a teaching phase a set of exemplar grasps appropriate for the task. Following this phase, a representation of task-related grasps is built. During task planning and execution, a grasp could be either submitted interactively for evaluation by a non-expert user or synthesized by an automatic planning system. Grasp quality is then assessed based on the proposed measure, which takes into account grasp stability along with its suitability for the task. To enable real-time evaluation of grasps, a fast algorithm for computing an approximation of the quality measure is also proposed. Finally, a local grasp optimization technique is described which can amend uncertainties arising in supplied grasps by non-expert users or assist in planning more valuable grasps in the neighborhood of candidate ones.The paper reports experiments performed in virtual reality with both an anthropomorphic virtual hand and a three-fingered robot hand. These experiments suggest the effectiveness and task relevance of the proposed grasp quality measure.     

Optimizing layout using spatial quality metrics and user preferences

Computational design is one of the most common tasks of immersive computer graphics projects, such as games, virtual reality and special effects. Layout planning is a challenging phase of architectural design, which requires optimization across several conflicting criteria. We present an interactive layout solver that assists designers in layout planning by recommending personalized space arrangements based on architectural guidelines and user preferences. Initialized by the designer’s high-level requirements, an interactive evolutionary algorithm is used to converge on an ideal layout by exploring the space of potential solutions. The major contributions of our proposed approach are addressing subjective aspects of the design to generate personalized layouts; and the development of a genetic algorithm with a multi-parental recombination method that improves the chance of generating higher quality offspring. We demonstrate the ability of our method to generate feasible floor plans which are satisfactory, based on spatial quality metrics and designer’s taste. The results show that the presented framework can measurably decrease planning complexity by producing layouts which exhibit characteristics of human-made design.     

Human knowledge acquisition from 3D interaction in virtual environments

Recently,virtual environment (VE) based design and planning,which is a kind of interaction intensive computing,has shown great potential.Most users of such systems are specialists or technicians and their 3D interactions with the system embed a great deal of knowledge and skills.Thus,how to integrate human knowledge and skills into VE is a great challenge to researchers in the human computer interaction field.This paper proposes a method for acquiring user knowledge from VE.The main ideas and work include,abstracting the interactive process and formalizing the interactive semantics,as well as providing a semantic model of an interactive information repository,from which user interactive processes can be retrieved and all kinds of application logics can be established.A virtual assembly planning system is introduced as an example of a practical application of this method.Experiments show that the related models can well capture user knowledge and retrieve the interaction process.     

Head-Tracked Stereo Viewing with Two-Handed 3 D Interaction for Animated Character Construction

In this paper, we demonstrate how a new interactive 3 D desktop metaphor based on two-handed 3 D direct manipulation registered with head-tracked stereo viewing can be applied to the task of constructing animated characters. In our configuration, a six degree-of-freedom head-tracker and CrystalEyes shutter glasses are used to produce stereo images that dynamically follow the user head motion. 3 D virtual objects can be made to appear at a fixed location in physical space which the user may view from different angles by moving his head. To construct 3 D animated characters, the user interacts with the simulated environment using both hands simultaneously: the left hand, controlling a Spaceball, is used for 3 D navigation and object movement, while the right hand, holding a 3 D mouse, is used to manipulate through a virtual tool metaphor the objects appearing in front of the screen. In this way, both incremental and absolute interactive input techniques are provided by the system. Hand-eye coordination is made possible by registering virtual space exactly to physical space, allowing a variety of complex 3 D tasks necessary for constructing 3 D animated characters to be performed more easily and more rapidly than is possible using traditional interactive techniques. The system has been tested using both Polhemus Fastrak and Logitech ultrasonic input devices for tracking the head and 3 D mouse.     

沉浸式虚拟装配中物体交互特征建模方法研究

沉浸式虚拟装配是一种“人在回路”的虚拟现实系统,可用于大型设备的设计及装配人员的培训等领域.作为其关键技术之一,如何实现“真实”人对“虚拟”物体的操作,仍是一项具有挑战性的工作.针对这一问题,提出了一种基于交互区和状态机的物体交互特征建模方法.通过交互区,虚拟物体可理解用户的操作意图,并据此发生相应的状态转换,进而正确监测用户的交互行为.该方法可实现包括使用工具在内的复杂装配操作以及对装配流程的管理控制.通过在实际项目中的应用验证了该方法的有效性.     

一种视觉信息融合数据手套设计研究

基于数据手套的手势交互是人机交互技术的重要实现方式,手势识别和空间定位效率对虚拟交互任务中的沉浸感和体验感有重要影响.目前主要的数据手套产品的复杂结构导致相应建模及手势识别算法设计复杂度高,难以兼顾成本和效率.面向应用需求,完成了一种融合视觉信息的数据手套应用系统方案设计.首先,在手势识别上,通过将3指弯曲度数据和视觉采集模型相结合的方法,有效简化了手势的建模和匹配过程,实现对特定手势的快速识别.其次,在空间位置定位上,提出了一种高效的基于特征校验的霍夫变换算法：通过摄像头捕获发光球的圆形轮廓以计算其三维空间位置;使用基于直径方程的改进型霍夫梯度算法降低圆形检测识别复杂度;对于连续帧图像,采用了基于可信度的特征校验法则进行快速处理,进一步提高算法整体效率.最后,该方案在沉浸式中学虚拟实验系统中应用,系统运行稳定,具有良好的交互体验.     

Task‐driven latent active correction for physics‐inspired input method in near‐field mixed reality applications

Calibration accuracy is one of the most important factors to affect the user experience in mixed reality applications. For a typical mixed reality system built with the optical see‐through head‐mounted display, a key problem is how to guarantee the accuracy of hand–eye coordination by decreasing the instability of the eye and the head‐mounted display in long‐term use. In this paper, we propose a real‐time latent active correction algorithm to decrease hand–eye calibration errors accumulated over time. Experimental results show that we can guarantee an effective calibration result and improve the user experience with the proposed latent active correction algorithm. Based on the proposed system, experiments about virtual buttons are also designed, and the interactive performance regarding different scales of virtual buttons is presented. Finally, a direct physics‐inspired input method is constructed, which shares a similar performance with the gesture‐based input method but provides a lower learning cost due to its naturalness.     

基于LeapMotion的个体化手部建模与实时交互

虚拟人手的实时建模是以人机交互为核心的VR/AR系统中的重要技术难点,传统的基于LeapMotion的手部跟踪视觉反馈仅展示骨架动画信息,缺少应用演示的真实感。由此,提出了一个快速高效的个体化手部建模与实时交互系统,重点阐述了该系统的的工作原理以及算法。以含有骨架信息的静止手模型为模板,使用LeapMotion跟踪用户手的骨架数据作为输入,提出了基于骨骼尺寸的手部模型自动个体化建模算法。提供了一种基于不完整骨骼运动数据的线性蒙皮技术的改进算法,可快速计算不同手部姿势下骨架之间的变换矩阵,动态更新虚拟手的姿势。实验结果表明,此系统在实验的硬件平台上运行速度可以达到60 f/s,误差稳定在4 mm以内,能为人机交互任务实时提供真实可靠的视觉反馈。     

Two-handed tangible interaction techniques for composing augmented blocks

Modeling tools typically have their own interaction methods for combining virtual objects. For realistic composition in 3D space, many researchers from the fields of virtual and augmented reality have been trying to develop intuitive interactive techniques using novel interfaces. However, many modeling applications require a long learning time for novice users because of unmanageable interfaces. In this paper, we propose two-handed tangible augmented reality interaction techniques that provide an easy-to-learn and natural combination method using simple augmented blocks. We have designed a novel interface called the cubical user interface, which has two tangible cubes that are tracked by marker tracking. Using the interface, we suggest two types of interactions based on familiar metaphors from real object assembly. The first, the screw-driving method, recognizes the user??s rotation gestures and allows them to screw virtual objects together. The second, the block-assembly method, adds objects based on their direction and position relative to predefined structures. We evaluate the proposed methods in detail with a user experiment that compares the different methods.     

基于启发式Bi-RRT算法的虚拟手臂运动规划

为提高虚拟人手臂在复杂多障碍物空间进行维修时的运动路径规划效率,提出一种基于启发式Bi-RRT的路径规划算法。在运用Bi-RRT算法的基础上,结合启发式搜索思想对腕关节进行路径规划,通过比较多个随机搜索点与目标点的距离,保留距离目标点最近的随机搜索点,使随机树有方向地进行扩展,减少无效的搜索。仿真实验结果表明,启发式Bi-RRT算法较传统的RRT算法和Bi-RRT算法有更好的搜索效果,搜索效率更高,对于7自由度虚拟手臂的运动路径规划也有较好的效果。     

基于LAN的计算机视觉与虚拟现实集成模型研究

进入80年代后期,虚拟现实技术取得了迅速的发展,目前作为一个全新的技术生长点受到了国内外学者的广泛关注。文中提出了一个基于LAN计算机视觉与虚拟现实集成模型。论述了系统实现及实验结果。研究结果表明,把计算机视觉技术与虚拟现实技术“集成”在一起,从现实世界精炼出关键参数供虚拟现实技术所用,可构成简便灵活的虚拟现实系统,文中在研究大空间跟踪中采用了该模型,证明了该模型的可行性及优点。     

Live path: adaptive agent navigation in the interactive virtual world

We present a novel approach to adaptive navigation in the interactive virtual world by using data from the user. Our method constructs automatically a navigation mesh that provides new paths for agents by referencing the user movements. To acquire accurate data samples from all the user data in the interactive world, we use the following techniques: an agent of interest (AOI), a region of interest (ROI) map, and a discretized path graph (DPG). Our method enables adaptive changes to the virtual world over time and provides user-preferred path weights for smart-agent path planning. We have tested the usefulness of our algorithm with several example scenarios from interactive worlds such as video games. In practice, our framework can be applied easily to any type of navigation in an interactive world. In addition, it may prove useful for solving previous pathfinding problems in static navigation planning.     

Facial animation retargeting and control based on a human appearance space

Expressive facial animations are essential to enhance the realism and the credibility of virtual characters. Parameter‐based animation methods offer a precise control over facial configurations while performance‐based animation benefits from the naturalness of captured human motion. In this paper, we propose an animation system that gathers the advantages of both approaches. By analyzing a database of facial motion, we create the human appearance space. The appearance space provides a coherent and continuous parameterization of human facial movements, while encapsulating the coherence of real facial deformations. We present a method to optimally construct an analogous appearance face for a synthetic character. The link between both appearance spaces makes it possible to retarget facial animation on a synthetic face from a video source. Moreover, the topological characteristics of the appearance space allow us to detect the principal variation patterns of a face and automatically reorganize them on a low‐dimensional control space. The control space acts as an interactive user‐interface to manipulate the facial expressions of any synthetic face. This interface makes it simple and intuitive to generate still facial configurations for keyframe animation, as well as complete temporal sequences of facial movements. The resulting animations combine the flexibility of a parameter‐based system and the realism of real human motion. Copyright © 2010 John Wiley & Sons, Ltd.     

Deformable Motion: Squeezing into Cluttered Environments

We present an interactive method that allows animated characters to navigate through cluttered environments. Our characters are equipped with a variety of motion skills to clear obstacles, narrow passages, and highly constrained environment features. Our control method incorporates a behavior model into well‐known, standard path planning algorithms. Our behavior model, called deformable motion, consists of a graph of motion capture fragments. The key idea of our approach is to add flexibility on motion fragments such that we can situate them into a cluttered environment via constraint‐based formulation. We demonstrate our deformable motion for realtime interactive navigation and global path planning in highly constrained virtual environments.     

A simple footskate removal method for virtual reality applications

Footskate is a common problem encountered in interactive applications dealing with virtual character animations. It has proven difficult to fix without the use of complex numerical methods, which require expert skills for their implementations, along with a fair amount of user interaction to correct a motion. On the other hand, deformable bodies are being increasingly used in virtual reality (VR) applications, allowing users to customize their avatar as they wish. This introduces the need of adapting motions without any help from a designer, as a random user seldom has the skills required to drive the existing algorithms towards the right solution. In this paper, we present a simple method to remove footskate artifacts in VR applications. Unlike previous algorithms, our approach does not rely on the skeletal animation to perform the correction but rather on the skin. This ensures that the final foot planting really matches the virtual character’s motion. The changes are applied to the root joint of the skeleton only so that the resulting animation is as close as possible to the original one. Eventually, thanks to the simplicity of its formulation, it can be quickly and easily added to existing frameworks.     

Development of a virtual aquarium system interacting with a smart device

New applications of smart devices interacting with other computing devices are recently providing interesting and feasible solutions in ubiquitous computing environments. In this study, we propose an interactive virtual aquarium system that interacts with a smart device as a user interface. We developed a virtual aquarium graphic system and a remote interaction application of a smart device for building an interactive virtual aquarium system. We performed an experiment that demonstrates the feasibility and the effectiveness of the proposed system as an example of a new type of interactive application of a smart display, where a smart device serves as a remote user interface.     

虚拟现实手部康复训练系统的设计与实现

将虚拟现实技术应用到康复医学领域,可有效克服传统康复训练方法的局限性,实现安全、舒适和主动的康复训 练。本文设计并实现了一套虚拟现实手部康复训练系统,系统由交互设备、人机交互软件和虚拟环境三部分组成。交互 设备采用 5DT 公司生产的 5DT Data Glove 14 Ultra 数据手套,而人机交互软件运用 Visual Studio 2012 作为开发工具,基于 MFC 编写,实现了用户管理、数据采集、手势信号分类、实时手势识别测试等功能。构建的虚拟场景使用 Flash 游戏, 通过 MFC 和 Flash 游戏之间通讯使用者能使用手势信号实现游戏操控。本文的实验结果表明：虚拟现实手部康复训练系 统能够指导使用者进行有效的手部康复训练,Flash 康复训练游戏能有效提高使用者进行康复训练的积极性和主动性。     

Interactive 3D content insertion in images for multimedia applications

This article addresses the problem of creating interactive mixed reality applications where virtual objects interact in images of real world scenarios. This is relevant to create games and architectural or space planning applications that interact with visual elements in the images such as walls, floors and empty spaces. These scenarios are intended to be captured by the users with regular cameras or using previously taken photographs. Introducing virtual objects in photographs presents several challenges, such as pose estimation and the creation of a visually correct interaction between virtual objects and the boundaries of the scene. The two main research questions addressed in this article include, the study of the feasibility of creating interactive augmented reality (AR) applications where virtual objects interact in a real world scenario using the image detected high-level features and, also, verifying if untrained users are capable and motivated enough to perform AR initialization steps. The proposed system detects the scene automatically from an image with additional features obtained using basic annotations from the user. This operation is significantly simple to accommodate the needs of non-expert users. The system analyzes one or more photos captured by the user and detects high-level features such as vanishing points, floor and scene orientation. Using these features it will be possible to create mixed and augmented reality applications where the user interactively introduces virtual objects that blend with the picture in real time and respond to the physical environment. To validate the solution several system tests are described and compared using available external image datasets.     

绳驱超冗余空间柔性机械臂遥操作系统设计与实验研究

针对绳驱超冗余空间柔性机械臂的运动特点和在大范围工作空间快速运动、避障及狭小空间作业和触碰情况下精细操作等在轨遥操作任务需求,设计并研制了适应多种工况的基于手控器和虚拟现实（VR）手柄组合的遥操作人机交互系统。对狭小空间作业和避障等工况进行分析,提出了基于自由度动态组合和末端―臂形同步规划的遥操作方法。最后,完成了穿越卫星太阳帆板狭缝的典型实验,实验验证了遥操作系统的工程实用性,以及柔性机械臂遥操作运动规划方法的可行性。因此,配合使用多种人机交互方式组合的遥操作系统和相应的运动规划方法,可使操作者以更加直观自然的方式参与到遥操作系统中,有效提高操作员完成复杂遥操作任务的安全性和操作性能。