A fingerprint pointing device utilizing the deformation of the fingertip during the incipient slip

In this paper, a novel pointing device is proposed that utilizes the deformation of the fingertip. When a fingertip is pressed and slightly slid on a rigid plate, a partial slip, called an "incipient slip", occurs on the contact surface. While the deformation around the center of the contact area is small during the incipient slip, the boundary region moves to the sliding direction of the fingertip. The deformation changes depending on the sliding distance of the fingertip and the exerted force on the contact surface. The velocity of the pointer can be determined by the estimated distance and force based on the measurement of the deformation. In this study, the correlation between the sliding distance of the fingertip and the deformation and between the exerted force and the deformation are investigated. The degree of the deformation due to the sliding motion can be estimated based on the detected fingerprint center. The group delay spectrum tracking method is proposed for the detection of the fingerprint center. A prototype pointing device is developed to evaluate the operationality of the proposed device. Comparative experiments with conventional pointing devices are conducted. The validity of the proposed device is confirmed by the experiments.     

Sensing mechanism for estimation of the physical properties of an object avoiding failure of the sensors

Active sensing, in which a robot pushes an object and senses the reaction force or joint angle by means of the force sensor at the point of the contact or on the joint, is one of the effective approaches to estimate the physical properties of an object, such as its compliance. A compliant joint driven by elastic actuators has an advantage over a rigid joint driven by a motor with a high gear ratio in that it absorbs the reaction force, and thus avoids any joint damage during active sensing. However, this approach is not suitable for either rigid joint or a compliant joint because the sensors attached to the contact point and the joint tend to break, owing to iterative contact or an excessive force. Here, this paper adopts a one-degree-of-freedom joint mechanism driven by elastic pneumatic actuators, and focuses on the passivity of the elastic pneumatic actuator, in which the pressure is changed when force is applied, after which it is deformed. By utilizing the passivity of the actuators under a number of conditions, this paper derives multiple regression models of the force and the angle, using the pressures before and after force is applied to the joint mechanism. Experimental results present that the contact information can be estimated from the pressure values and that the joint mechanism can detect the elasticity of an object using the regression models. We also observe the range of the elasticity of the object by tuning the joint compliance. This approach provides a robot hand that can estimate the contact information, including the force and joint displacement, avoiding the failure of the sensors.     

Tactile data modeling and interpretation for stable grasping and manipulation

This paper is devoted to present the latest results on the exploitation of the force/tactile sensor developed by the authors in terms of modeling and interpretation of the data provided by the device. An analytical nonlinear model of the elastically deformable sensor is derived and validated, which allows to reconstruct the position and orientation of the surface in contact with a rigid object on the basis of the sensor signals. The reconstruction is performed via an Extended Kalman Filter able to counteract the measurement noise and to handle the nonlinearity of the model at the same time. The contact plane position and orientation information together with the contact force vector measured by the sensor are used to estimate the physical parameter most relevant to manipulation control purposes: the friction coefficient. A slippage control algorithm is presented which exploits the estimated friction and a novel slipping detection algorithm is proposed to cope with the unavoidable uncertainties of the real world and its effectiveness is experimentally proved in comparison with the existing techniques.     

Contact and Deformation Modeling for Interactive Environments

Contact and deformation modeling for interactive environments has seen many applications, from surgical simulation and training, to virtual prototyping, to teleoperation, etc., where both visual feedback and haptic feedback are needed. High-quality feedback demands a high level of physical realism as well as a high update rate in rendering, which are often conflicting requirements. In this paper, we present a unique approach to modeling force and deformation between a rigid body and an elastic object under complex contacts, which achieves a good compromise of reasonable physical realism and real-time update rate (at least 1 kHz). We simulate contact forces based on a nonlinear physical model. We further introduce a novel approximation of material deformation suitable for interactive environments based on applying Bernoulli-Euler bending beam theory to the simulation of elastic shape deformation. Our approach is able to simulate the contact forces exerted upon the rigid body (that can be virtually held by a user via a haptic device) not only when it forms one or more than one contact with the elastic object, but also when it moves compliantly on the surface of the elastic object, taking friction into account. Our approach is also able to simulate the global and local shape deformation of the elastic object due to contact. All the simulations can be performed in a combined update rate of over 1 kHz, which we demonstrate in several examples.     

The use of active deformable models in model-based robotic visual servoing

This paper presents a new approach for visual tracking and servoing in robotics. We introduce deformable active models as a powerful means for tracking a rigid or semi-rigid (possibly partially occluded) object in movement within the manipulator's workspace. Deformable models imitate, in real-time, the dynamic behavior of elastic structures. These computer-generated models are designed to capture the silhouette of objects with well-defined boundaries, in terms of image gradient. By means of an eye-in-hand robot arm configuration, the desired motion of the end-effector is computed with the objective of keeping the target's position and shape invariant with respect to the camera frame. Optimal estimation and control techniques (LQG regulator) have been successfully implemented in order to deal with noisy measurements provided by our vision sensor. Experimental results are presented for the tracking of a rigid or semi-rigid object. The experiments performed in a real-time environment show the effectiveness and robustness of the proposed method for servoing tasks based on visual feedback.     

Fusing force and vision feedback for manipulating deformable objects

This article describes a framework that fuses vision and force feedback for the control of highly deformable objects. Deformable active contours, or snakes, are used to visually observe changes in object shape over time. Finite‐element models of object deformations are used with force feedback to predict expected visually observed deformations. Our approach improves the performance of large, complex deformable contour tracking over traditional computer vision tracking techniques. This same approach of combining deformable active contours with finite‐element material models is modified so that a vision sensor, i.e., a charge‐coupled device (CCD) camera, can be used as a force sensor. By visually tracking changes in contours on the object, material deflections can be transformed into applied stress estimates through finite element modeling. Therefore, internal object stresses due to object manipulation can be visually observed and controlled, thus creating a framework for deformable object manipulation. A pinhole camera model is used to accomplish vision and force sensor feedback assimilation from these two sensing modalities during manipulation. © 2001 John Wiley & Sons, Inc.     

分层优化PF在六维力传感器下E型膜中的应用

针对动载环境下,噪声污染导致六维力传感器测量精度急剧下降的问题,提出一种具有分层优化步骤的改进粒子滤波算法。以双E型弹性体六维力传感器下E型膜为研究对象,根据正弦激励力响应和应变的关系,建立非线性系统模型。在粒子滤波的框架下,将样本集按权值的蜕化程度分层,引入野草繁殖算法,将最新的观测信息融入高权值子集。基于Thompson-Taylor算法,通过聚合重采样将高、低权值粒子随机组合,产生中权值粒子集。将优化后的粒子滤波算法在六维力传感器动态测试系统中进行仿真研究,结果表明,该算法能以更小的估计误差贴近真实后验概率密度,在保持实时性的同时,有效地提高六维力传感器的测量精度。     

Position/force hybrid control system for high precision aligning of small gripper to ring object

A position/force hybrid control system based on impedance control scheme is designed to align a small gripper to a special ring object. The vision information provided by microscope vision system is used as the feedback to indicate the position relationship between the gripper and the ring object. Multiple image features of the gripper and the ring object are extracted to estimate the relative positions between them. The end-effector of the gripper is tracked using the extracted features to keep the gripper moving in the field of view. The force information from the force sensor serves as the feedback to ensure that the contact force between the gripper and the ring object is limited in a small safe range. Experimental results verify the effectiveness of the proposed control strategy.     

Comparisons of apparent mass responses of human subjects seated on rigid and elastic seats under vertical vibration

The apparent mass (AM) responses of human body seated on elastic seat, without and with a vertical back support, are measured using a seat pressure sensing mat under three levels of vertical vibration (0.25, 0.50 and 0.75 m/s² rms acceleration) in 0.50–20 Hz frequency range. The responses were also measured with a rigid seat using the pressure mat and a force plate in order to examine the validity of the pressure mat. The pressure mat resulted in considerably lower AM magnitudes compared to the force plate. A correction function was proposed and applied, which resulted in comparable AM from both measurement systems for the rigid seat. The correction function was subsequently applied to derive AM of subjects seated on elastic seat. The responses revealed lower peak magnitude and corresponding frequency compared to those measured with rigid seat, irrespective of back support and excitation considered.     

Slippage control in soft finger grasping and manipulation

Handling objects with robotic soft fingers without considering the odds of slippage are not realistic. Grasping and manipulation algorithms have to be tested under such conditions for evaluating their robustness. In this paper, a dynamic analysis of rigid object manipulation with slippage control is studied using a two-link finger with soft hemispherical tip. Dependency on contact forces applied by a soft finger while grasping a rigid object is examined experimentally. A power-law model combined with a linear viscous damper is used to model the elastic behavior and damping effect of the soft tip, respectively. In order to obtain precise dynamic equations governing the system, two second-order differential equations with variable coefficients have been designed to describe the different possible states of the contact forces accordingly. A controller is designed based on the rigid fingertip model using the concept of feedback linearization for each phase of the system dynamics. Numerical simulations are used to evaluate the performance of the controller. The results reveal that the designed controller shows acceptable performance for both soft and rigid finger manipulation in reducing and canceling slippage. Furthermore, simulations indicate that the applied force in the soft finger manipulation is considerably less than the rigid “one.”.     

Control of Robots with Elastic Joints Interacting with Dynamic Environment

In this paper, the control of robots with elastic joints in contact with dynamic environment is considered. It is shown how control laws synthesized for the robots with rigid joints interacting with dynamic environment can also be used in the case of robots with elastic joints. The proposed control laws are based on a robot model interacting with dynamic environment, including the dynamics of actuators and the elasticity of joints. The proposed control laws possess two feedback loops: the outer, serving for on-line calculation of the motor shaft angle based on the position error or the contact force error, and the inner one, serving for performing stabilization around the calculated motor shaft angle. Simulation results which exhibit the application of the appropriate control laws are also presented.     

A Telepresence-Guaranteed Control Scheme for Teleoperation Applications of Transferring Weight-Unknown Objects

Currently, most teleoperation work is focusing on scenarios where slave robots interact with unknown environments. However, in some fields such as medical robots or rescue robots, the other typical teleoperation application is precise object transportation. Generally, the object’s weight is unknown yet essential for both accurate control of the slave robot and intuitive perception of the human operator. However, due to high cost and limited installation space, it is unreliable to employ a force sensor to directly measure the weight. Therefore, in this paper, a control scheme free of force sensor is proposed for teleoperation robots to transfer a weight-unknown object accurately. In this scheme, the workspace mapping between master and slave robot is firstly established, based on which, the operator can generate command trajectory on-line by operating the master robot. Then, a slave controller is designed to follow the master command closely and estimate the object’s weight rapidly, accurately and robust to unmodeled uncertainties. Finally, for the sake of telepresence, a master controller is designed to generate force feedback to reproduce the estimated weight of the object. In the end, comparative experiments show that the proposed scheme can achieve better control accuracy and telepresence, with accurate force feedback generated in only 500 ms.      

弹性体力/触觉框架模型

在力/触觉建模技术中,高效的实时性与基于物理意义的准确性是与生俱来的一对矛盾.为了提高实时性与准确性的和谐程度,提出一种分析虚拟柔性体力/变形关系的框架结构模型.在柔性体内部建立一定数量并相互独立的钢架结构,运用虚功原理计算每根钢架上节点的位移;根据接触点和节点的分布,采用分水岭法将柔性体表面分割成若干区域,通过本区域内节点和接触点的位移插值计算区域内顶点的变形,进而得到柔性体的整体变形.仿真实验结果表明,该模型简便、实时性好,在具体的虚拟建模力反馈工作中可操作性强.     

Design of a slave arm of a surgical robot system to estimate the contact force at the tip of the employed instruments

Acquisition of the contact force at the instrument tip can enable better performance, e.g. transparency of the haptic feedback in the surgical robot systems. It is, however, difficult to measure the contact force directly due to technical limitations in attaching sensors to the tip of the instruments. This paper proposes a method to estimate the forces by installing the sensors away from the instrument tip. The proposed method employs specially designed mechanical parts of the slave robot, i.e. a slider cover plate for the z-axis translational force along the insertion direction, and docking clamps for the rotational pivot torques around the fulcrum point. Strain gauges are attached to specially designed places with enhancing shapes. The simulation results of the force estimation are presented to confirm the strain concentration area. The proposed method is validated with quantitative experimental results. Calibrated weights are determined upon the comparison of the strain value with a calibrated 6-axis force/torque sensor. The percentage error in the force calibration is about 5~8% calculated by the root mean square error (RMSE) of force-sensing performance. In addition, it can be computed by considering only the bending phase of each sensor although the hysteresis is observed from the calibration graph.     

Distant Collision Response in Rigid Body Simulations

We use a finite element model to predict the vibration response of objects in a rigid body simulation, such that rigid objects are augmented to provide a plausible elastic collision response between distant objects due to vibration. We start with a generalized eigenvalue decomposition of the elastic model to precompute a response to an impact at any point on an elastic object with fixed boundary conditions. Then, given a collision between objects, we generate an approximate response impulse to distribute to other objects already in contact with the colliding bodies. This can lead to distant impacts causing an object to slip, or a delicate stack of objects to fall. We also use a geodesic distance based spatial attenuation approximation for travelling waves in objects to respond to an impact at one contact with an impulse at other locations. This response ultimately allows a long distance relationship between contacts, both across a single object being struck, but also traversing the contact graph of a larger collection of objects. We qualitatively validate our approach with a ground truth simulation, and demonstrate a number of scenarios where a long distance relationship between contacts is valuable.     

Visual-Based Impedance Control of Out-of-Plane Cell Injection Systems

In this paper, a vision-based impedance control algorithm is proposed to regulate the cell injection force, based on dynamic modeling conducted on a laboratory test-bed cell injection system. The injection force is initially calibrated to derive the relationship between the force and the cell deformation utilizing a cell membrane point-load model. To increase the success rate of injection, the injector is positioned out of the focal plane of the camera, used to obtain visual feedback for the injection process. In this out-of-plane injection process, the total cell membrane deformation is estimated, based on the $X-Y$ coordinate frame deformation of the cell, as measured with a microscope, and the known angle between the injector and the $X-Y$ plane. Further, a relationship between the injection force and the injector displacement of the cell membrane, as observed with the camera, is derived. Based on this visual force estimation scheme, an impedance control algorithm is developed. Experimental results of the proposed injection method are given which validate the approach.      

Interaction model between elastic objects for haptic feedback considering collisions of soft tissue

The simulation of organ–organ interaction is indispensable for practical and advanced medical VR simulator such as open surgery and indirect palpation. This paper describes a method to represent real-time interaction between elastic objects for accurate force feedback in medical VR simulation. The proposed model defines boundary deformation of colliding elements based on temporary surface forces calculated by temporary deformation. The model produces accurate deformation and force feedback considering collisions of objects as well as prevents unrealistic overlap of objects. A prototype simulator of rectal palpation is constructed on general desktop PC with a haptic device, PHANToM. The system allows users to feel different stiffness of a rear elastic object located behind another elastic object. The results of experiments confirmed the method expresses organ–organ interaction in real-time and produces realistic and perceivable force feedback.     

基于共面圆的距离传感器与相机的相对位姿标定

近年来, 距离传感器与摄像机的组合系统标定在无人车环境感知中得到了广泛的研究与应用, 其中基于平面特征的方法简单易行而被广泛采用. 然而, 目前多数方法基于点匹配进行, 易错且鲁棒性较低. 本文提出了一种基于共面圆的距离传感器与相机的组合系统相对位姿估计方法. 该方法使用含有两个共面圆的标定板, 可以获取相机与标定板间的位姿, 以及距离传感器与标定板间的位姿. 此外, 移动标定板获取多组数据, 根据计算得到两个共面圆的圆心在距离传感器和相机下的坐标, 优化重投影误差与3D对应点之间的误差, 得到距离传感器与相机之间的位姿关系. 该方法不需要进行特征点的匹配, 利用射影不变性来获取相机与三维距离传感器的位姿. 仿真实验与真实数据实验结果表明, 本方法对噪声有较强的鲁棒性, 得到了精确的结果.     

A low-complexity sensor fusion algorithm based on a fiber-optic gyroscope aided camera pose estimation system

Visual tracking, as a popular computer vision technique, has a wide range of applications, such as camera pose estimation. Conventional methods for it are mostly based on vision only, which are complex for image processing due to the use of only one sensor. This paper proposes a novel sensor fusion algorithm fusing the data from the camera and the fiber-optic gyroscope. In this system, the camera acquires images and detects the object directly at the beginning of each tracking stage; while the relative motion between the camera and the object measured by the fiber-optic gyroscope can track the object coordinate so that it can improve the effectiveness of visual tracking. Therefore, the sensor fusion algorithm presented based on the tracking system can overcome the drawbacks of the two sensors and take advantage of the sensor fusion to track the object accurately. In addition, the computational complexity of our proposed algorithm is obviously lower compared with the existing approaches(86% reducing for a 0.5 min visual tracking). Experiment results show that this visual tracking system reduces the tracking error by 6.15% comparing with the conventional vision-only tracking scheme(edge detection), and our proposed sensor fusion algorithm can achieve a long-term tracking with the help of bias drift suppression calibration.     

Pseudo-interference stiffness estimation, a highly efficient numerical method for force evaluation in contact problems

This paper presents a novel method, Pseudo-Interference Stiffness Estimation (PISE), for evaluating the contact compliance and the contact load in the contacting elastic solids. The PISE method is based on the evaluation of the geometric overlap of two assumedly rigid bodies and estimation of the contact force based on this artificial overlap area (or volume). In this paper, an example of the dynamic simulation of two disk collision problem is solved both by PISE method and finite element contact model. The contact force and velocity changes during impact from both methods are shown to be in good agreement. However, PISE method is, computationally, orders of magnitude (about 3000 times in our numerical simulations) faster than finite element contact analysis. The proposed method will be of practical use in contact force approximation of contacting bodies, such as meshing of spur gear teeth, cam analysis and synthesis, robotic grabbing, and numerous other applications.