Study on interaction-induced symptoms with respect to virtual grasping and manipulation

Owing to the popularity of various hand tracking interfaces, there have been numerous applications developed to provide intuitive hand interaction with the virtual world. As users start with great anticipation, they end up with dissatisfaction due to difficulties of manipulation or physical tiredness coming very short. Although the task itself is rather trivial in a real life situation, it requires much effort in the virtual environment. We address this awkwardness as ‘VR interaction-induced fatigue symptom’ and hypothesize its causes based on our observations. We argue that the source of the fatigue comes from the restricted sensory information of the VR interfaces, and that users try to accommodate the missing sensory feedback by excessive motion leading to wrong posture or bad timing. We demonstrate our hypothesis by conducting experiments of two types of virtual interaction scenarios: object transport and 3D selection. Furthermore, by analyzing the behaviors of users' action collected from our experiment, we derive essential factors to be considered in designing VR applications, and propose a conceptual interaction model for orchestrating virtual grasping.     

一种基于非线性弹簧模型的虚拟手交互新方法

基于虚拟手的交互技术在人机交互和人机工程学测试等应用中发挥着重要的作用。为了实现直观自然、实时准确、接近真实世界中的虚拟手与虚拟物体的交互,并计算出反馈作用力,首先提出了用非线性弹簧模型计算抓取作用力,使虚拟手和虚拟环境之间实现了基于物理的交互;然后将计算结果以视觉渲染的形式反馈给用户,并对仿真的速率做了定量分析,以便使仿真速率可以达到屏幕刷新频率和力反馈刷新频率的要求。实验结果表明,虚拟手不仅可以直观自然地抓取3维虚拟物体,而且和3维物体之间能够进行实时交互,同时可计算出反馈作用力。     

面向人机工程学测试的虚拟手仿真模型

以人机工程学测试为目标提出虚拟手仿真模型,建立虚拟手运动学模型。提出一种曲线拟合的校准方法,以获取精确的数据,更好地控制虚拟手的运动。建立弹簧模型计算手与被抓取物体之间的受力,以视觉渲染的形式将其反馈给用户。实验结果表明,虚拟手可自然地抓取三维虚拟物体,视觉反馈的使用可免于购买昂贵的力反馈设备。     

虚拟手抓持力觉生成算法真实性的评价

目的 虽然许多学者研发了多种虚拟手交互触力觉生成算法,但是如何评价虚拟手交互触力觉生成算法的真实性是一个富有挑战性的新问题,值得深入研究.方法 构建手指抓持力测量平台,设计3种抓持姿态下指尖静力抓持球体实验内容,测得各指尖作用力的实测值;通过虚拟手静力抓持力觉生成算法,求得这3种抓持姿态下各手指作用力的理论值;对实测值进行统计和分析,并与理论值进行对比和讨论;结果 日常抓持经验和实测值是完全相符的,实测值和理论值很接近且偏差均在可接受范围之内.单个手指作用力或多个手指合力的实测值与理论值的偏差均在1%6%.结论 本文实现了一种基于物理的实验方法,评价和分析了虚拟手静力抓持力觉生成算法的真实性,证实此算法可以逼真地生成虚拟手抓持力,可应用于具有力反馈的自然的虚拟手交互.     

虚拟手抓持规则融合策略研究

群体虚拟手抓持规则是虚拟手和虚拟物体进行抓持操作的交互规则,用于判定虚拟手是否能够成功抓持物体。对基于几何的虚拟手抓持规则和基于物理的虚拟手抓持规则分别进行了研究,针对基于几何的虚拟手抓持规则规则简单、仿真效果较差,基于物理模型的虚拟手抓持规则计算复杂、难以实现实时仿真的问题:（1）改进基于几何的虚拟手抓持规则,通过接触点位置、法矢和抓持面法矢制定抓持规则,使其效果逼近力封闭虚拟手抓持规则;（2）利用力封闭计算中抓持接触点和法矢不变的特性,通过内力配比避免了抓持操作中的非线性规划求解,使抓持操作阶段实现实时仿真;（3）通过几何约束进行初始抓持判断-力封闭计算校正-内力配比力封闭计算的策略,实现了完整的抓持过程实时仿真。设计的交互实验说明该抓持规则能实现高沉浸感和实时性的抓持仿真,可以应用到虚拟训练、虚拟装配等仿真平台。     

增强现实系统中基于视觉感知的抓取识别

增强现实系统采用单目光学透视式头盔时必然会使虚拟物体呈现在真实物体之前,使用户无法得知应该在何处抓取虚拟物体,并且系统所得到的抓取数据也会受硬件本身、人和环境等因素影响而产生偏差.从人的视觉感知习惯入手,利用机器人学和光线几何理论实现基于视觉感知的碰撞检测;通过分析抓取过程,得出3条抓取知识规则,并将其与模糊理论结合应用于证据推理.实验结果表明,采用文中方法,用户能轻松地实现抓取,整个交互过程相当自然.     

Virtual grasp release method and evaluation

We address a “sticking object” problem for the release of whole-hand virtual grasps. The problem occurs when grasping techniques require fingers to be moved outside an object's boundaries after a user's (real) fingers interpenetrate virtual objects due to a lack of physical motion constraints. This may be especially distracting for grasp techniques that introduce mismatches between tracked and visual hand configurations to visually prevent interpenetration. Our method includes heuristic analysis of finger motion and a transient incremental motion metaphor to manage a virtual hand during grasp release. We integrate the method into a spring model for whole-hand virtual grasping to maintain the physically-based pickup and manipulation behavior of such models. We show that the new spring model improves release speed and accuracy based on pick-and-drop, targeted ball-drop, and cube-alignment experiments. In contrast to a standard spring-based grasping method, measured release quality does not depend notably on object size. Users subjectively prefer the new approach and it can be tuned to avoid potential side effects such as increased drops or visual distractions. We further investigated a convergence speed parameter to find the subjectively good range and to better understand tradeoffs in subjective artifacts on the continuum between pure incremental motion and rubber-band-like convergence behavior.     

Immersive manipulation of virtual objects through glove-based hand gesture interaction

Immersive visualisation is increasingly being used for comprehensive and rapid analysis of objects in 3D and object dynamic behaviour in 4D. Challenges are therefore presented to provide natural user interaction to enable effortless virtual object manipulation. Presented in this paper is the development and evaluation of an immersive human?Ccomputer interaction system based on stereoscopic viewing and natural hand gestures. For the development, it is based on the integration of a back-projection stereoscopic system for object and hand display, a hybrid inertial and ultrasonic tracking system to provide the absolute positions and orientations of the user??s head and hands, as well as a pair of high degrees-of-freedom data gloves to provide the relative positions and orientations of digit joints and tips on both hands. For the evaluation, it is based on a two-object scene with a virtual cube and a CT (computed tomography) volume created for demonstration of real-time immersive object manipulation. The system is shown to provide a correct user view of objects and hands in 3D with depth, as well as to enable a user to use a number of simple hand gestures to perform basic object manipulation tasks involving selection, release, translation, rotation and scaling. Also included in the evaluation are some quantitative tests of the system performance in terms of speed and latency.     

The Role of Visual Feedback and Age When Grasping,Transferring and Passing Objects in Virtual Environments

ABSTRACT

Visual feedback about one’s own movements can be important for effective performance in natural and computer-generated environments. Previous work suggested that, for young adults, performance in virtual environments was influenced by the presence of a crude representation of the hand in a task-specific fashion. The current study was performed to determine whether this pattern of results holds across the lifespan. Specifically, we were interested in determining whether a representation of the hand is useful for movement performance in children, middle adults, and older adults when they reach to grasp an object, transfer it between their two hands, or receive it from a partner. Surprisingly, visual feedback condition had very little effect on performance in the current experiment, with the exception that participants in all three age groups altered how wide they opened their hand when grasping depending on visual condition. Patterns of reach and grasp performance changed across task depending on age, with older adults displaying different patterns than children and middle-aged adults. These results suggest that feedforward planning and the use of feedback are modulated by both age and task.     

虚拟拆卸中多方式交互技术的应用

通过分析虚拟环境下不同人-机交互方式的特点，结合对产品装配体进行人工拆卸的操作活动，提出面向虚拟拆卸的人-机-零件间的多方式交互操作方法，在虚拟环境下研究和利用手抓取操作、手势操作、语音命令和3D虚拟菜单等多种交互方式对产品零部件进行自然、直观的人工拆卸，人机结合进行产品的装配／拆卸序列规划。     

A Comprehensive Review of Redirected Walking Techniques: Taxonomy,Methods, and Future Directions

Virtual reality (VR) allows users to explore and experience a computer-simulated virtual environment so that VR users can be immersed in a totally artificial virtual world and interact with arbitrary virtual objects. However, the limited physical tracking space usually restricts the exploration of large virtual spaces, and VR users have to use special locomotion techniques to move from one location to another. Among these techniques, redirected walking (RDW) is one of the most natural locomotion techniques to solve the problem based on near-natural walking experiences. The core idea of the RDW technique is to imperceptibly guide users on virtual paths, which might vary from the paths they physically walk in the real world. In a similar way, some RDW algorithms imperceptibly change the structure and layout of the virtual environment such that the virtual environment fits into the tracking space. In this survey, we first present a taxonomy of existing RDW work. Based on this taxonomy, we compare and analyze both contributions and shortcomings of the existing methods in detail, and find view manipulation methods offer satisfactory visual effect but the experience can be interrupted when users reach the physical boundaries, while virtual environment manipulation methods can provide users with consistent movement but have limited application scenarios. Finally, we discuss possible future research directions, indicating combining artificial intelligence with this area will be effective and intriguing.     

Manipulating Objects in Virtual Worlds: Categorization and Empirical Evaluation of Interaction Techniques

The acceptance of virtual environment (VE) technology requires scrupulous optimization of the most basic interactions to maximize user performance and provide efficient and enjoyable virtual interfaces. Motivated by insufficient understanding of human factors implications in the design of interaction techniques for object manipulation in virtual worlds, this paper presents results of a formal study that evaluated two basic interaction metaphors for virtual manipulation—virtual pointer and virtual hand—in object selection and positioning tasks. In this work, we survey and categorize current virtual manipulation techniques according to their basic design metaphors, conduct experimental studies of the most basic techniques, and derive guidelines to aid designers in the practical development of VE applications.     

基于虚拟力引导的人机协同目标抓取方法

针对力反馈遥操作中传统人工势场法无法适应于机械臂整体的避障以及在作业过程中操作者难以控制机械臂到达所需位姿的问题,提出了一种基于虚拟力引导的人机协同目标抓取方法。力反馈设备向操作者提供力觉交互。通过结合人工势场法和虚拟夹具,构建管道形虚拟力场,生成实时虚拟力引导,实现协助操作者完成从端机器人的整体避障任务并在完成避障后引导机器人返回预定义路径并趋近目标点。当进行抓取任务时,构建锥形虚拟力场,实现协助操作者操作机械臂到达目标位置和姿态。此外,提出了一种机器人运动限制方法以降低操作者的操作失误对抓取任务的影响。实验证明,该方法能有效提高目标抓取操作的成功率和操作效率。     

Networked multiplayer cooperative interaction using decoupled motion control method in a shared virtual environment with haptic,visual and movement feedback

This paper focuses on multiplayer cooperative interaction in a shared haptic environment based on a local area network. Decoupled motion control, which allows one user to manipulate a haptic interface to control only one‐dimensional movement of an avatar, is presented as a new type haptic‐based cooperation among multiple users. Users respectively move an avatar along one coordinate axis so that the motion of the avatar is the synthesis of movements along all axes. It is different from previous haptic cooperation where all users can apply forces on an avatar along any direction to move it, the motion of which completely depends on the resultant force. A novel concept of movement feedback is put forward where one user can sense other users’ hand motions through his or her own haptic interface. The concept can also be explained wherein one person who is required to move a virtual object along only one axis can also feel the motions of the virtual object along other axes. Movement feedback, which is a feeling of motion, differs from force feedback, such as gravity, collision force and resistance. A spring‐damper force model is proposed for the computation of motion feedback to implement movement transmission among users through haptic devices. Experimental results validate that movement feedback is beneficial for performance enhancement of such kind of haptic‐based cooperation, and the effect of movement feedback in performance improvement is also evaluated by all subjects.Copyright © 2012 John Wiley & Sons, Ltd.     

An instrumented glove for grasp specification invirtual-reality-based point-and-direct telerobotics

Hand posture and force, which define aspects of the way an object is grasped, are features of robotic manipulation. A means for specifying these grasping “flavors” has been developed that uses an instrumented glove equipped with joint and force sensors. The new grasp specification system will be used at the Pennsylvania State University (Penn State) in a Virtual Reality based Point-and-Direct (VR-PAD) robotics implementation. Here, an operator gives directives to a robot in the same natural way that human may direct another. Phrases such as “put that there” cause the robot to define a grasping strategy and motion strategy to complete the task on its own. In the VR-PAD concept, pointing is done using virtual tools such that an operator can appear to graphically grasp real items in live video. Rather than requiring full duplication of forces and kinesthetic movement throughout a task as is required in manual telemanipulation, hand posture and force are now specified only once. The grasp parameters then become object flavors. The robot maintains the specified force and hand posture flavors for an object throughout the task in handling the real workpiece or item of interest     

Robust object manipulation for tactile-based blind grasping

Tactile-based blind grasping addresses realistic robotic grasping in which the hand only has access to proprioceptive and tactile sensors. The robotic hand has no prior knowledge of the object/grasp properties, such as object weight, inertia, and shape. There exists no manipulation controller that rigorously guarantees object manipulation in such a setting. Here, a robust control law is proposed for object manipulation in tactile-based blind grasping. The analysis ensures semi-global asymptotic and exponential stability in the presence of model uncertainties and external disturbances that are neglected in related work. Simulation and hardware results validate the effectiveness of the proposed approach.     

Influence of contextual objects on spatial interactions and viewpoints sharing in virtual environments

Collaborative virtual environments (CVEs) are 3D spaces in which users share virtual objects, communicate, and work together. To collaborate efficiently, users must develop a common representation of their shared virtual space. In this work, we investigated spatial communication in virtual environments. In order to perform an object co-manipulation task, the users must be able to communicate and exchange spatial information, such as object position, in a virtual environment. We conducted an experiment in which we manipulated the contents of the shared virtual space to understand how users verbally construct a common spatial representation of their environment. Forty-four students participated in the experiment to assess the influence of contextual objects on spatial communication and sharing of viewpoints. The participants were asked to perform in dyads an object co-manipulation task. The results show that the presence of a contextual object such as fixed and lateralized visual landmarks in the virtual environment positively influences the way male operators collaborate to perform this task. These results allow us to provide some design recommendations for CVEs for object manipulation tasks.     

基于双目视觉的人手定位与手势识别系统研究

提出了一种新的人手特征点提取方法,该方法将人手的质心作为匹配点,根据双目视觉定位数学模型计算目标位置信息,同时通过图像分割获取人手轮廓,利用轮廓凸包点特征来识别不同手势.在此基础上,研究设计了一种光学人手定位与手势识别系统,该系统在实时定位空间人手三维位置的同时,能够识别出相应的手势,可将其作为虚拟手的驱动接口,实现对虚拟物体的抓取、移动和释放操作.     

非结构环境下基于人机合作技术的抓取作业研究

介绍了一种利用人机合作技术在非结构环境引导机械手抓取静态目标的方法．分别介绍了将激光—CCD摄像机系统与操作者的经验相结合获得抓取目标位置的方法,及将虚拟现实技术与操作者的经验相结合获得抓取目标姿态的方法．继而利用基于模型的视觉引导技术,引导手臂完成抓取操作．     

Super-resolution for an omnidirectional vision sensor

A prominent emerging theory of sensorimotor development in biological systems proposes that control knowledge is encoded in the dynamics of physical interaction with the world. From this perspective, the musculoskeletal system is coupled through sensor feedback and neurological structure to a non-stationary world. Control is derived by reinforcing and learning to predict constructive patterns of interaction. We have adopted the traditions of dynamic pattern theory in which behavior is an artifact of coupled dynamical systems with a number of controllable degrees of freedom. For grasping and manipulation, we propose a closed-loop control process that is parametric in the number and identity of contact resources. We have shown previously that this controller generates a necessary condition for force closure grasps. In this paper, we will show how control decisions can be made by estimating patterns of membership in a family of prototypical dynamic models. A grasp controller can thus be tuned on-line to optimize performance over a variety of object geometries. This same process can be used to match the dynamics to several previously acquired haptic categories. We will illustrate how a grasping policy can be acquired that is incrementally optimal for several objects using our Salisbury hand with tactile sensor feedback.