A sensor for dynamic tactile information with applications in human–robot interaction and object exploration

We present a novel tactile sensor, which is applied for dextrous grasping with a simple robot gripper. The hardware novelty consists of an array of capacitive sensors, which couple to the object by means of little brushes of fibers. These sensor elements are very sensitive (with a threshold of about 5 mN) but robust enough not to be damaged during grasping. They yield two types of dynamical tactile information corresponding roughly to two types of tactile sensor in the human skin. The complete sensor consists of a foil-based static force sensor, which yields the total force and the center of the two-dimensional force distribution and is surrounded by an array of the dynamical sensor elements. One such sensor has been mounted on each of the two gripper jaws of our humanoid robot and equipped with the necessary read-out electronics and a CAN bus interface. We describe applications to guiding a robot arm on a desired trajectory with negligible force, reflective grip improvement, and tactile exploration of objects to create a shape representation and find stable grips, which are applied autonomously on the basis of visual recognition.     

Remote Tactile Transmission with Time Delay for Robotic Master–Slave Systems

This study develops a method to compensate for the communication time delay for tactile transmission systems. For transmitting tactile information from remote sites, the communication time delay degrades the validity of feedback. However, so far time delay compensation methods for tactile transmissions have yet to be proposed. For visual or force feedback systems, local models of remote environments were adopted for compensating the communication delay. The local models cancel the perceived time delay in sensory feedback signals by synchronizing them with the users' operating movements. The objectives of this study are to extend the idea of the local model to tactile feedback systems and develop a system that delivers tactile roughness of textures from remote environments to the users of the system. The local model for tactile roughness is designed to reproduce the characteristic cutaneous deformations, including vibratory frequencies and amplitudes, similar to those that occur when a human finger scans rough textures. Physical properties in the local model are updated in real-time by a tactile sensor installed on the slave-side robot. Experiments to deliver the perceived roughness of textures were performed using the developed system. The results showed that the developed system can deliver the perceived roughness of textures. When the communication time delay was simulated, it was confirmed that the developed system eliminated the time delay perceived by the operators. This study concludes that the developed local model is effective for remote tactile transmissions.     

Biomimetic Tactile Sensor Array

The performance of robotic and prosthetic hands in unstructured environments is severely limited by their having little or no tactile information compared to the rich tactile feedback of the human hand. We are developing a novel, robust tactile sensor array that mimics the mechanical properties and distributed touch receptors of the human fingertip. It consists of a rigid core surrounded by a weakly conductive fluid contained within an elastomeric skin. The sensor uses the deformable properties of the finger pad as part of the transduction process. Multiple electrodes are mounted on the surface of the rigid core and connected to impedance-measuring circuitry safely embedded within the core. External forces deform the fluid path around the electrodes, resulting in a distributed pattern of impedance changes containing information about those forces and the objects that applied them. Here we describe means to optimize the dynamic range of individual electrode sensors by texturing the inner surface of the silicone skin. Forces ranging from 0.1 to 30 N produced impedances ranging from 5 to 1000 kΩ. Spatial resolution (below 2 mm) and frequency response (above 50 Hz) appeared to be limited only by the viscoelastic properties of the silicone elastomeric skin.     

Robust bin-picking system using tactile sensor

ABSTRACT

This paper presents a robust bin-picking system utilizing tactile sensors and a vision sensor. The object position and orientation are estimated using a fast template-matching method through the vision sensor. When a robot picks up an object, the tactile sensors detect the success or failure of the grasping, and a force sensor detects the contact with the environment. A weight sensor is also used to judge whether the lifting of the object has been successful. The robust and efficient bin-picking system presented herein is implemented through the integration of different sensors. In particular, the tactile sensors realize rope-shaped object picking that has yet to be made possible with conventional picking systems. The effectiveness of the proposed method was confirmed through grasping experiments and in a competitive event at the World Robot Challenge 2018.     

Whole-finger manipulation with a two-fingered robot hand

This paper describes a method for whole-finger rolling manipulation using a two-fingered robot hand. 'Whole-finger' refers to the use of the complete phalangeal surface during the manipulation. An example of whole-finger manipulation by the human hand is the rolling of a pen between two fingers. The proposed method is based on a two-dimensional model for modelling an object manipulation and is derived from a study of the movement of the contact line between both fingers. Also, the method uses tactile sensor information to estimate the contact point position together with the local curvature of the object. This whole-finger dexterous manipulation is demonstrated on a prototype two-fingered hand. This 5 d.o.f. hand consists of a tendon driven index and thumb, and is equipped with force and tactile sensors. The dimensions and performance of this device are 'human-sized'. A hybrid force-position control scheme is used. The hierarchical control structure is implemented on a dual transputer system. This paper first describes the kinematic model used for whole-finger manipulation. In the second part, the main emphasis is put on the mechanical design and on the transputer-based control system.     

Virtual Force/Tactile Sensors for Interactive Machines Using the User's Biological Signals

This study proposes a new approach to virtual realization of force/tactile sensors in machines equipped with no real sensors. The key of our approach is that a machine exploits the user's biological signals. Therefore, this approach is not dependent on controlled objects and is expected to be widely applicable for a variety of machines including robots. This article describes an example robotic system comprised of an industrial robot manipulator, a motion capture system and a surface electromyogram (EMG) measurement apparatus. By monitoring/recording the user's surface EMG and postural information in real-time, we show that a robot equipped with no force/tactile sensors behaved similarly to one possessing sensors over its body. Another advantage of our approach is demonstrated by a task in which a robot and a user cooperatively hold and move a heavy load.     

机器人触觉传感器柔性动态系统模型的建立

本文就仿生型多功能触觉传感器——人工皮肤(Artificial skin)从理论上进行了深入的研究.进一步揭示了用PVF_2作敏感材料模拟“人工皮肤”的可行性.介绍了这种人工皮肤进行模式识别的方法,建立了压觉感受器的数学模型,并提出了一种能感受冷、热变化及对象物的硬度、光滑度的方案。并论证了方案的可行性.此外,就这一柔性系统的结构.在总结分析以前文献的基础上,提出了新的假设.     

The development of a fault-tolerant control approach and its implementation on a flexible arm robot

This article presents a robust sensor fault-tolerant control (FTC) scheme and its implementation on a flexible arm robot. Sensor faults affect the system's performance in the closed loop when the faulty sensor readings are used to generate the control input. In this article, the non-faulty sensors are used to reconstruct the faults on the potentially faulty sensors. The reconstruction is subtracted from the faulty sensors to generate a 'virtual sensor' which (instead of the normally used faulty sensor output) is then used to generate the control input. A design method is also presented in which the virtual sensor is made insensitive to any system uncertainties (which could corrupt the fault reconstruction) that cannot fit into the framework of the model used. Two fault conditions are tested: total failure and incipient faults. Then the scheme robustness is tested and evaluated through its implementation on two flexible arm systems, one with a flexible joint and the other with a flexible link. Excellent results have been obtained for both cases (joint and link); the FTC scheme produced system performance almost identical to the fault-free scenario, whilst providing an indication that a fault is present, even for simultaneous faults.     

Sensor Fault Tolerance Method by Using a Bayesian Network for Robot Behavior

This paper presents FTBN, a new framework that performs learning autonomous mobile robot behavior and fault tolerance simultaneously. For learning behavior in the presence of a robot sensor fault this framework uses a Bayesian network. In the proposed framework, sensor data are used to detect a faulty sensor. Fault isolation is accomplished by changing the Bayesian network structure using interpreted evidence from robot sensors. Experiments including both simulation and a real robot are performed for door-crossing behavior using prior knowledge and sensor data at several maps. This paper explains the learning behavior, optimal tracking, exprimental setup and structure of the proposed framework. The robot uses laser and sonar sensors for door-crossing behavior, such that each sensor can be corrupted during the behavior. Experimental results show FTBN leads to robust behavior in the presence of a sensor fault as well as performing better compared to the conventional Bayesian method.     

'Search balls': sensor units for searching inside rubble

'Search balls' for searching inside rubble are proposed. A search ball is a small sensor unit which contains some sensors for searching for disaster victims, such as cameras and microphones, and a radio transceiver; it does not have actuators. Many balls are thrown into rubble and fall down while repeatedly colliding; they are scattered inside the rubble. The sensor information from the balls is transmitted by radio from the rubble and monitored at a safe area. In this way, rescuers can search a wide area inside the rubble rapidly. In the present paper, an impact-resistant ball structure is proposed: sensors, an electronic circuit, a radio transceiver and a battery are wrapped up in cushion material and packed into an impact-resistant outer shell. As a method of distributing balls inside rubble, the effect of the ball shape is discussed. The influence of ball shape and size on the distribution of balls inside rubble is experimentally evaluated using miniature rubble. One-to-one communication between balls and external computers for identifying the balls and acquiring the sensor information from them is discussed. Then a sphere-type search ball is developed: it contains three wireless cameras, IR LEDs, a radio receiver and a battery. The impact resistance of this ball is evaluated by experiments. Using a realistic model of rubble, it is experimentally verified that the external computer can send commands to the ball inside this rubble and the video signal from the ball can be transmitted out.     

Heterogeneous multisensor fusion for mapping dynamic environments

In this paper, we propose a heterogeneous multisensor fusion algorithm for mapping in dynamic environments. The algorithm synergistically integrates the information obtained from an uncalibrated camera and sonar sensors to facilitate mapping and tracking. The sonar data is mainly used to build a weighted line-based map via the fuzzy clustering technique. The line weight, with confidence corresponding to the moving object, is determined by both sonar and vision data. The motion tracking is primarily accomplished by vision data using particle filtering and the sonar vectors originated from moving objects are used to modulate the sample weighting. A fuzzy system is implemented to fuse the two sensor data features. Additionally, in order to build a consistent global map and maintain reliable tracking of moving objects, the well-known extended Kalman filter is applied to estimate the states of robot pose and map features. Thus, more robust performance in mapping as well as tracking are achieved. The empirical results carried out on the Pioneer 2DX mobile robot demonstrate that the proposed algorithm outperforms the methods a using homogeneous sensor, in mapping as well as tracking behaviors.     

Hybrid three-dimensional camera pose estimation using particle filter sensor fusion

To properly align objects in the real and virtual worlds in an augmented reality (AR) space it is essential to keep tracking the camera's exact three-dimensional position and orientation (camera pose). State-of-the-art analysis shows that traditional vision-based or inertial sensor-based solutions are not adequate when used individually. Sensor fusion for hybrid tracking has become an active research direction during the past few years, although how to do it in a robust and principled way is still an open problem. In this paper, we develop a hybrid camera pose-tracking system that combines vision and inertial sensor technologies. We propose to use the particle filter framework for the sensor fusion system. Particle filters are sequential Monte-Carlo methods based upon a point mass (or 'particle') representation of probability densities, which can be applied to any state space model and which generalize the traditional Kalman filtering methods. We have tested our algorithm to evaluate its performance and have compared the results obtained by the particle filter with those given by a classical extended Kalman filter. Experimental results are presented     

柔性触觉传感技术及其在医疗康复机器人的应用

柔性触觉传感器易于贴合皮肤等不规则表面,相比刚性传感器具有更强的信号感知能力、更高的精度和更佳的穿戴舒适性,在人机交互、医疗设备、可穿戴设备、健康监测等领域发挥着重要作用.鉴于此,从传感器不同工作原理出发,对柔性触觉传感器进行系统地介绍和对比,从结构优化的角度分析传感器性能优化方法,整理出微结构、结构疏松化、多模态测量...     

A Method for Accurate Map Construction Using Time Registration from a Moving SOKUIKI Sensor

Mobile robots have various sensors that are considered as their eyes and a two-dimensional laser scanner (SOKUIKI sensor) is one of them. This sensor is commonly used for constructing environment maps or detecting obstacles. It is very important to estimate accurately the sensor's position and direction when scan data is obtained in order to use correctly such a sensor for these purposes. On the other hand, scanning sensors have an essencial problem: time lag. That is, the time when the laser is transmitted to each direction is different. Thus the map has many errors if the measurement time lag problem is not resolved. This paper presents a method for laser scan data synchronization using time registration and, as a practical application, we show its effectiveness for accurate map construction.     

The development and control of a flexible-spine for a human-form robot

In this paper, the development of a robot which has a flexible spine is presented. By embedding a multi-d.o.f. soft structure into a robot body as a spine, the robot can increase its ability to absorb shock and to work in various environment such as narrow places. As a result of these abilities, the robot can expand its opportunity to work in the human environment. Moreover, its motion could be more natural. The developed full-body human-form robot has a five-jointed flexible spine. Each joint (vertebra) has 3 d.o.f. Between each vertebrae is a 'disk' made of silicone rubber. The spine is controlled by eight tendons, whose tensions can be controlled using tension sensors and locally distributed microcontrollers. This paper describes the development of the flexible spine and the control of the posture of the spine and body.     

Design and fabrication of a flexible three-axial tactile sensor array based on polyimide micromachining

This paper describes the design and fabrication of a flexible three-axial tactile sensor array using advanced polyimide micromachining technologies. The tactile sensor array is comprised of sixteen micro force sensors and it measures 13 mm × 18 mm. Each micro force sensor has a square membrane and four strain gauges, and its force capacity is 0.6 N in the three-axial directions. The optimal positions of the strain gauges are determined by the strain distribution obtained form finite element analysis (FEA). The normal and shear forces are detected by combining responses from four thin-film metal strain gauges embedded in a polyimide membrane. In order to acquire force signals from individual micro force sensors, we fabricated a PCB based on a multiplexer, operational amplifier and microprocessor with CAN network function. The sensor array is tested from the evaluation system with a three-component load cell. The developed sensor array can be applied in robots’ fingertips, as well as to other electronic applications with three-axial force measurement and flexibility keyword requirements.     

基于EIT技术的柔性触觉传感器的设计

随着机器人技术的日益发展,柔性传感器在机器人皮肤上的应用也得到了新的发展。本文提出并研究了一种基于导电聚合物压敏电阻效应的柔性触觉传感器的设计,使用由聚二甲基硅氧烷PDMS（Poly Di Methyl Siloxane）和多壁碳纳米管（MWCNTs）混合而成的导电橡胶作为传感器主体,运用EIT（Electrical Impedance Tomography）技术,设计并制作了本系统的硬件电路,并用其检测、传输导电橡胶的边缘电势数据。最后在计算机中应用工具包EIDORS进行有限元模型和图像重构技术,有效且直观的将导电橡胶上的受力位置表现出来。实验对1~3个目标分别进行了成像,证明了本设计的可行性。     

面向ERT大面积触觉传感的自适应优化成像方法

在动态的非结构化环境中,有效地感知物理接触对于智能机器人安全交互至关重要。为了能够检测各种潜在的物理交互,需要在机器人表面部署大面积触觉传感器。目前,现有的大面积触觉传感器主要是通过传感阵列方式实现的,但是大规模部署传感元件在实际应用中存在巨大挑战。电阻层析成像(Electrical Resistance Tomography, ERT)技术作为一种连续传感方式,有望克服传统触觉传感阵列的一些限制。为此,利用ERT设计了一款新型的大面积触觉传感器。在此基础上,提出了一种基于自适应感兴趣区(Region of Interest, ROI)的图像重构算法,将图像重构限制在交互区域内,以提高传感器的空间分辨率。为了验证提出成像算法的有效性,通过仿真与物理实验对其进行了全面评估。实验验证了该算法可以有效提高触觉传感器在交互区域的空间分辨率,使其具有较高的测量精度。实验结果表明,该传感器的平均定位误差为0.823 cm,能够准确地识别8种不同交互模式,其精度高达98.6%。这一研究工作表明,该传感器为机器人具身触觉传感的实现提供了一个新的解决方案。     

Development of a High-Resolution Human-Specific Breath Gas Sensor for Survivor Detection in Disaster Zones

This paper presents a new type of ultrasonic gas molecule concentration sensor for rescue robotics. This device can measure the change of gas concentration with a sampling rate of over 400 kHz. The performance is evaluated by measuring the CO₂ concentration in human respiration gas. The experiments show that the proposed sensor could detect a difference between 5% CO₂-containing air, humidified air and dry air with over 50 dB signal-to-noise, which are the main components of our respiration gas. Another important result was that our sensor could give information about the 'dead space', which is distributed from the lungs to the mouth. The 'dead space' could not be detected by previously proposed commercially distributed gas sensors because of the time needed to analyze the gas. We verified the distance dependency of the respiration detection in a open space that was considered for use for finding survivors. These results make the proposed sensor especially applicable for finding survivors in disaster zones.