基于TCP/IP的远程触觉交互系统力反馈数据传输方法

时延、丢包是影响远程触觉交互性能和用户体验的重要因素。以实时传输和真实再现力反馈数据为目的,提出了一种基于TCP/IP的实时力反馈数据传输方法,该方法通过时延处理和插值预处理降低网络通信过程中对远程触觉感知造成的不利影响实验结果证明,该方法有效地解决了远程触觉交互系统中实时力反馈的信息再现与信息交互问题,在远程医疗、远程教学、虚拟现实等领域具有重要的应用价值。     

一种胆道虚拟手术仿真系统*

设计了一种用于外科医生仿真训练的具有实时力反馈的胆道虚拟手术仿真系统。主要讨论了本系统的结构设计、三维重建、碰撞检测、软组织的弹性变形和力反馈等方面的技术实现。通过对胆囊取石手术进行仿真模拟,分析该系统的应用情况和性能指标。实验结果表明,该系统能有效地进行胆道手术仿真训练,提供的力反馈功能具有较好的稳定性和实时性。     

一种基于物理意义的快速力反馈形变模型及实时力觉响应算法

虚拟物体在受力作用时的形变建模是虚拟环境中力/触觉人机交互的关键.文中提出了一种新的基于物理意义的形变建模方法,不仅计算速度快,满足力反馈的实时性要求,而且能够同时保证接触力和形变的计算具有较高的精度,适用于具有较大变形量的柔性物体的力反馈计算,满足精细作业对虚拟现实系统的要求.     

用于触摸屏交互的可穿戴指端力反馈系统设计

力触觉再现技术在增强用户与触摸屏交互的真实感和沉浸感方面发挥着越来越重要的作用,设计了一种面向触摸屏图像信息感知的可穿戴指端力反馈装置。设计基于线绳牵引驱动的指端力反馈机构,将装置分为驱动和可穿戴部分,使其具备小巧、轻便、易佩戴的特点。设计测控系统,实现了输出反馈力的检测和控制。研究图像信息提取和力触觉建模算法,获取图像中物体的三维特征信息,配合力反馈装置和测控系统实现图像纹理高度、外形轮廓信息再现反馈。实验结果表明,该装置最大可以提供5.3 N的反馈力,能够帮助用户感知触摸屏中图像的高度和形状信息。     

气动力反馈数据手套设计与验证

针对目前面向大型复杂机电产品的虚拟装配环境中力反馈装置所能提供的触觉力偏小的问题,提出一种基于微型低摩擦气缸的内置式力反馈数据手套.首先介绍了手套结构及其工作原理;然后搭建了实验系统,基于牛顿迭代法进行运动学方程组求解,验证了手套的角度测量和力反馈功能;最后开发了手套的人机交互控制系统,采用D-H理论来描述虚拟手的空间位置,基于虚拟墙的弹簧-质点模型计算虚拟力.实验结果表明,所设计的数据手套可实现大触觉力的反馈,满足虚拟装配系统虚拟抓取的需求.     

基于力反馈的虚拟脊柱手术模拟系统的构建

针对医学手术排练演习和手术教学开发了虚拟脊柱手术模拟平台。在该平台上能够根据手术的真实环境,模拟常用手术器械和脊柱结构,实现人机实时交互。详细介绍了系统的硬件环境参数,并基于3DS MAX的建模和力触觉渲染引擎CHAI 3D进行仿真。仿真实验结果表明,基于力反馈的虚拟脊柱手术模拟系统可以有效地模拟脊柱手术时的力反馈状态,帮助医师进行脊柱手术术前训练,并适用于高校的医学教学和医疗机构人员的培训。     

基于STM32的力觉反馈装置及用于主从遥操作实现

在遥操作系统中为了增强现实及实现本地力觉信号再现功能以提高精细化操作的目的 ,设计了用于人机交互功能的力反馈装置;该装置为单自由度结构,基于步进电机驱动;利用STM32微控制器采集触觉力信号以及关节位移信号,通过设计基于力误差的控制律调整位置变量实现输出力信号与标准力信号的匹配;为了验证该力反馈装置进行了标准力信号再现实验;并且利用该力反馈装置作为主机械手与单自由度从机械手搭建遥操作装置,进行了力、位置双边跟踪实验验证,实现了主、从机械手力、位置协同一致的目的.     

带有力觉和触觉临场感的灵巧手主从系统的设计

针对遥控作业中控制者操作时缺乏力觉和触觉临场感等问题，介绍了设计的带有力觉和触觉临场感主、从灵巧手系统，讨论了在从机械手上触觉和力觉的感知以及在主机械手上触觉和力觉的再现等问题，提出了利用模糊控制实现触觉再现以及改进的力反馈－位置型结构来实现力觉再现的新方法，最后进行了实验验证．     

面向上颌骨骨折复位手术的虚拟系统设计

基于3D计算机触觉视觉交互(CHAI3D)和开放图形库(Open GL)等开源软件,设计了针对上颌骨复位手术的仿真系统。使用真实病例的CT图像搭建虚拟场景,通过Geomagic力反馈设备对虚拟模型进行三维操作并输出触觉反馈。在原有单点碰撞算法的基础上,提出了使用多个中介代理的多点碰撞算法,避免了虚拟手术工具的手柄插入虚拟器官的不实仿真;通过力反馈设备对头颅骨模型进行选择、移动和旋转,模拟手术中对头颅骨的移动和放置。系统可用于训练医学院学生,也可用于复杂手术的术前规划。     

立体显示技术在力反馈仿真手术的应用

本文介绍了立体显示技术,并且结合触觉编程开发包OpenHaptics,应用于以PHANToM作为力反馈设备构建的虚拟手术仿真系统上。使仿真系统能够实时进行具有立体感的多种手术操作,提供真实的立体视觉反馈和力反馈,具有良好的沉浸感和交互性。     

用于实时柔性触觉再现的平行菱形链连接模型

精度高且实时性好的柔性触觉变形模型是实现触觉再现系统的关键。提出了一种新的基于物理意义的平行菱形链连接触觉变形模型,系统中各个链结构单元相对位移的叠加对外等效为物体表面的变形,与之相连的弹簧弹性力的合力等效为物体表面的接触力。使用Delta 6-DOF手控器,建立了触觉再现实验系统,对柔性体的接触变形和实时虚拟触觉反馈进行仿真, 实验结果表明所提出的模型不仅计算简单,而且能够保证触觉接触力和形变计算具有较高精度,满足虚拟现实系统对精细作业和实时性的要求。     

Effects of vision and friction on haptic perception

OBJECTIVE: Two experiments were conducted to examine the effects of vision and masking friction on contact perception and compliance differentiation thresholds in a simulated tissue-probing task. BACKGROUND: In minimally invasive surgery, the surgeon receives limited haptic feedback because of the current design of the instrumentation and relies on visual feedback to judge the amount of force applied to the tissues. It is suggested that friction forces inherent in the instruments contribute to errors in surgeons' haptic perception. This paper investigated the psychophysics of contact detection and cross-modal sensory processing in the context of minimally invasive surgery. METHOD: A within-subjects repeated measures design was used, with friction, vision, tissue softness, and order of presentation as independent factors, and applied force, detection time, error, and confidence as dependent measures. Eight participants took part in each experiment, with data recorded by a custom force-sensing system. RESULTS: In both detection and differentiation tasks, higher thresholds, longer detection times, and more errors were observed when vision was not available. The effect was more pronounced when haptic feedback was masked by friction forces in the surgical device (p < .05). CONCLUSION: Visual and haptic feedback were equally important for tissue compliance differentiation. APPLICATION: A frictionless endoscopic instrument can be designed to restore critical haptic information to surgeons without having to create haptic feedback artificially.     

Energy balance method for modelling of soft tissue deformation

This paper presents a novel methodology for modelling of soft tissue deformation, from the standpoint of work–energy balance based on the law of conservation of energy. The work done by an external force is always balanced against the strain energy due to the internal force of the object. A position-based incremental approach is established, in which the work–energy balance is achieved via an iterative position increment process for the new equilibrium state of the object. The position-based incremental approach is further combined with non-rigid mechanics of motion to govern the dynamics of soft tissue deformation. The proposed method employs nonlinear geometric and material formulations to account for the nonlinear soft tissue deformation. Soft tissue material properties can be accommodated by specifying strain energy density functions. Integration with a haptic device is also achieved for soft tissue deformation with haptic feedback for surgical simulation. Experimental results demonstrate that the deformations by the proposed method are in good agreement with those by a commercial package of finite element analysis. Isotropic and anisotropic deformations, as well as soft tissue viscoelastic behaviours, can be accommodated by the proposed methodology via strain energy density functions.     

带有力感觉的显微外科手术血管的仿真研究

随着虚拟现实技术的进步,带有力感觉的仿真研究迅速发展,特别是在医学上得到了广泛应用。利用虚拟现实技术取代效率低下的常规培训,可以大大减少所需的培训时间和昂贵的动物实验费用。针对复杂的显微外科手术的血管缝合,作者建立了可用于大夫培训、手术规划的虚拟现实系统。其中的交互设备采用了具有力反馈功能的PHANTOM—Desktop,使用与之配套的GHOSTSDK软件开发包,建立虚拟环境。考虑到柔性体仿真的实时性和真实性要求,采用了质量一弹簧模型的建模方法。在完成了对虚拟环境中柔性平面仿真的基础上,成功实现了带有力感觉的显微外科血管穿刺的虚拟仿真。     

Haptic feedback and control of a flexible surgical endoscopic robot

A flexible endoscope could reach the potential surgical site via a single small incision on the patient or even through natural orifices, making it a very promising platform for surgical procedures. However, endoscopic surgery has strict spatial constraints on both tool-channel size and surgical site volume. It is therefore very challenging to deploy and control dexterous robotic instruments to conduct surgical procedures endoscopically. Pioneering endoscopic surgical robots have already been introduced, but the performance is limited by the flexible neck of the robot that passes through the endoscope tool channel. In this article we present a series of new developments to improve the performance of the robot: a force transmission model to address flexibility, elongation study for precise position control, and tissue property modeling for haptic feedback. Validation experiment results are presented for each sector. An integrated control architecture of the robot system is given in the end.     

Force modeling for tooth preparation in a dental training system

Feedback force is very important for novices to simulate tooth preparation by using the haptic interaction system (dental training system) in a virtual environment. In the process of haptic simulation, the fidelity of generated forces by a haptic device decides whether the simulation is successful. A force model computes feedback force, and we present an analytical force model to compute the force between a tooth and a dental pin during tooth preparation. The force between a tooth and a dental pin is modeled in two parts: (1) force to resist human’s operation and (2) friction to resist the rotation of the dental engine. The force to resist the human’s operation is divided into three parts in the coordinates that are constructed on the bottom center of the dental pin. In addition, we also consider the effects of dental-pin type, tooth stiffness, and contact geometry in the force model. To determine the parameters of the force model, we construct a measuring system by using machine vision and a force/torque sensor to track the human’s operations and measure the forces between the dental pins and teeth. Based on the measuring results, we construct the relation between the force and the human’s operation. The force model is implemented in the prototype of a dental training system that uses the Phantom as the haptic interface. Dentists performing virtual operations have confirmed the fidelity of feedback force.     

Development scheme of haptic-based system for interactive deformable simulation

This paper aims to develop a novel scheme for interactively deformable simulation with haptic feedback. All design modules are packaged and implemented, and the experiments are conducted to study the effects of interactive deformation. The study compares the experimental results of haptic feedback by different force propagation methods, according to the Hounsfield unit (HU) of volume data. Additionally, the estimation of optimal propagation depth is illustrated by using a mass-spring model. Finally, by using the proposed scheme, a haptic-based medical simulation system for brain surgery is investigated. The integration test results with haptic feedback scenarios show that the proposed development scheme can certainly comply with the design modules, and the deformable simulation and haptic force reach a good agreement.     

Soft tissue deformation with reaction-diffusion process for surgery simulation

This paper presents a new methodology to conduct modelling and analysis of soft tissue deformation from the physicochemical viewpoint of soft tissues for surgery simulation. The novelty of this methodology is that soft tissue deformation is converted into a reaction-diffusion process coupled with a mechanical load, and thus reaction-diffusion of mechanical load and non-rigid mechanics of motion are combined to govern the dynamics of soft tissue deformation. The mechanical load applied to a soft tissue to cause a deformation is incorporated into the reaction-diffusion system and consequently distributed among mass points of the soft tissue. An improved reaction-diffusion model is developed to describe the distribution of the mechanical load in the tissue. A generic finite difference scheme is presented for construction of the reaction-diffusion model on a 3D tissue surface. A gradient method is established for derivation of internal forces from the distribution of the mechanical load. Real-time interactive deformation of virtual human organs with haptic feedback has been achieved by the proposed methodology for surgery simulation. The proposed methodology not only accommodates isotropic, anisotropic and inhomogeneous materials by simply modifying diffusion coefficients, but also accepts local and large-range deformations simultaneously.     

Effect of kinesthetic and tactile haptic feedback on the quality of experience of edutainment applications

Haptic technologies and applications have received enormous attention in the last decade. The incorporation of haptic modality into multimedia applications adds excitement and enjoyment to an application. It also adds a more natural feel to multimedia applications, that otherwise would be limited to vision and audition, by engaging as well the user’s sense of touch, giving a more intrinsic feel essential for ambient intelligent applications. However, the improvement of an application’s Quality of Experience (QoE) by the addition of haptic feedback is still not completely understood. The research presented in this paper focuses on the effect of haptic feedback and what it potentially adds to the experience of the user as opposed to the traditional visual and auditory feedback. In essence, it investigates certain issues regarding stylus-based haptic education applications and haptic-enhanced entertainment videos. To this end, we used two haptic applications: the haptic handwriting learning tool to experiment with force feedback haptic interaction and the tactile YouTube application for tactile haptic feedback. In both applications, our analysis shows that the addition of haptic feedback will increase the QoE in the absence of fatigue or discomfort for this category of applications. This implies that the incorporation of haptic modality (both force feedback as well as tactile feedback) has positively contributed to the overall QoE for the users.