实时触觉再现接口的设计与实现

提出了一种具有实时触觉反馈作用的人机接口装置的设计方法,以控制弹性梁的有效变形长度为核心,保证了触觉再现的实时性和准确性,从而可以在操作者指端大范围再现虚拟对象或远程对象的刚度.介绍了该人机触觉再现接口装置的机械设计、传感器系统、直流伺服控制系统以及数据采集系统的设计.该实时触觉再现接口具有尺寸小、刚度再现范围大等特点,可广泛应用于遥操作机器人和虚拟现实领域.     

Optical merger of direct vision with virtual images for scaled teleoperation

Scaled teleoperation is increasingly prevalent in medicine, as well as in other applications of robotics. Visual feedback in such systems is essential and should make maximal use of natural hand-eye coordination. This paper describes a new method of visual feedback for scaled teleoperation in which the operator manipulates the handle of a remote tool in the presence of a registered virtual image of the target in real time. The method adapts a concept already used successfully in a new medical device called the sonic flashlight, which permits direct in situ visualization of ultrasound during invasive procedures. The sonic flashlight uses a flat-panel monitor and a half-silvered mirror to merge the visual outer surface of a patient with a simultaneous ultrasound scan of the patient's interior. Adapting the concept to scaled teleoperation involves removing the imaging device and the target to a remote location and adding a master-slave control device. This permits the operator to see his hands, along with what appears to be the tool, and the target, merged in a workspace that preserves natural hand-eye coordination. Three functioning prototypes are described, one based on ultrasound and two on light microscopy. The limitations and potential of the new approach are discussed.     

Augmented reality-based robot teleoperation system using RGB-D imaging and attitude teaching device

Augmented reality (AR)-based programming using the demonstration method has been widely studied. However, studies on AR-based programming for remote robots are lacking because of the limitation of human–computer interaction. This paper proposes an AR-based robot teleoperation system and method using RGB-D imaging and an attitude teaching device. By sending the color and depth images of the remote robot environment to the local side, the operators can complete the teleoperation of the robot at the local side. First, the operators select key positions on the motion path of the robot endpoint from color images via a mouse, and the computer calculates the 3D coordinates of these key points in the robot base coordinate system to complete the position teaching process. In the robot attitude teaching process, the AR technology is used to superimpose the virtual robot model onto the color images of the robot teleoperation environment, so as to make the virtual robot endpoint to move along the teaching path. An operator can use the portable attitude teaching device designed in this study to control the robot movement parameters, such as the attitude and motion speed, during the movement of the virtual robot. After the position and attitude teaching processes, the robot movement trajectory can be generated. To make the base coordinate system of the virtual model consistent with that of the physical robot, we propose an online AR registration method, which does not require manually placing the AR registration marker. The proposed AR-based robot teleoperation system can quickly and easily complete robot teleoperation at the local side.     

Novel Haptic Device Using Jamming Principle for Providing Kinaesthetic Feedback in Glove-Based Control Interface

This paper presents a new type of wearable haptic device which can augment a sensor glove in various tasks of telemanipulation. We present the details of its two alternative designs jamming tubes or jamming pads, and their control system. These devices use the jamming phenomena to change the stiffness of their elements and block the hand movement when a vacuum is applied. We present results of our experiments to measure static and dynamic changes in stiffness, which can be used to change the perception of grabbing hard or soft objects. The device, at its current state is capable of resisting forces of up to 7 N with 5 mm displacement and can simulate hardness up to the hardness of a rubber. However, time necessary for a complete change of stiffness is high (time constant 0.5 s); therefore, additional cutaneous interface was added in a form of small vibration motors. Finally, we show an application of the haptic interface in our teleoperation system to provide the operator with haptic feedback in a light weight and simple form.     

Enhanced teleoperation performance using hybrid control and virtual fixture

To develop secure, natural and effective teleoperation, the perception of the slave plays a key role for the interaction of a human operator with the environment. By sensing slave information, the human operator can choose the correct operation in a process during the human–robot interaction. This paper develops an integrated scheme based on a hybrid control and virtual fixture approach for the telerobot. The human operator can sense the slave interaction condition and adjust the master device via the surface electromyographic signal. This hybrid control method integrates the proportional-derivative control and the variable stiffness control, and involves the muscle activation at the same time. It is proposed to quantitatively analyse the human operator's control demand to enhance the control performance of the teleoperation system. In addition, due to unskilful operation and muscle physiological tremor of the human operator, a virtual fixture method is developed to ensure accuracy of operation and to reduce the operation pressure on the human operator. Experimental results demonstrated the effectiveness of the proposed method for the teleoperated robot.     

Gaze direction based vehicle teleoperation method with omnidirectional image stabilization and automatic body rotation control

In robot teleoperation, a robot works as a physical agent at a remote site for a robot operator. There are mainly two tasks in robot teleoperation using camera images: environment recognition using visual information and robot control according to the recognition. In this paper, we propose a gaze direction based vehicle teleoperation method with an omnidirectional image stabilization and an automatic body rotation control. In the proposed method, we manage above two tasks in the same manner that are usually treated separately. This method is an intuitive vehicle teleoperation method where an operator do not need to have concern about vehicle body orientations and can absorb differences of vehicle driving mechanisms. That is, this method frees an operator from being bothered from controlling a vehicle and the operator can concentrate on where he/she intends to go. This method mainly consists of two technologies: an omnidirectional image stabilization technology and automatic body rotation control. The conducted experiments show effectiveness of the proposed method.     

基于领域知识的3维动态场景目标识别和配准

为了增强遥操作系统的临场感,使操作者更好地融入远程工作环境,提出了基于领域知识的3维动态场景目标识别和配准算法.首先,通过离线解析和切分虚拟样机CAD（计算机辅助设计）模型构建包含多视角点云特征和装配约束的领域知识库.其次,通过动态采集场景点云并计算CVFH（clustered viewpoint feature histogram）和FPFH（fast point feature histogram）特征,利用领域知识库中多视角点云特征和点云CVFH特征进行比对实现目标识别,并由FPFH特征经过两步配准实现目标姿态确定.最后,利用装配约束知识库实现以遥操作机器人工作状态变化为驱动的指导信息的精确配准和实时推送.实验结果表明,该系统不但可以有效地指导远程机器人完成维修操作,还可以提高遥操作的精度和效率.     

Bilateral teleoperation of a group of mobile robots for cooperative tasks

A visual and force feedback-based teleoperation scheme is proposed for cooperative tasks. The bilateral teleoperation system includes a haptic device, an overhead camera and a group of wheeled robots. The commands of formation and average velocities of the multiple robots are generated by the operator through the haptic device. The state of the multiple robots and the working environment is sent to the human operator. The received information contains the feedback force through the haptic device and visual information returned by a depth camera. The feedback force based on the difference between the desired and actual average velocities is presented. The wave variable method is employed in the bilateral teleoperation of multiple mobile robots with time delays. The effectiveness of the bilateral teleoperation system is demonstrated by experiments. The robots in the slave side are able to follow the commands from the master side to interact with the environments, including moving in different formations and pushing a box. The results show that the scheme enables the operator to manipulate a group of robots to complete cooperative tasks freely.     

基于ABB机器人的遥操作控制系统设计

工业机器人传统的编程与控制方式受距离空间限制,不便对机器人进行远程控制、控制不灵活。针对工业机器人的远程控制进行了研究,设计了基于ABB工业机器人的遥操作控制系统。在本地上位机上设计操作者界面,实现本地和远程程序的变量同步,以TCP/IP协议与ABB机器人控制柜通信,直接通过以太网对远程端的机器人进行控制和管理。在仿真和实验中,通过分析机器人对上位机给定轨迹的跟踪效果验证了系统的有效性。仿真和实验结果表明该系统能对远程ABB机器人进行有效的控制,可以让机器人代替人在恶劣的工作环境中作业,并提高工业生产效率。     

基于视觉反馈和预测仿真的Internet机器人遥操作

随着网络技术的发展,基于Internet的机器人遥操作因其广泛的应用前景而成为研究的热点,如何消除Internet引入的复杂时延对控制系统的影响是解决问题的关键.该文在分析Internet两点间时延规律的基础上,设计并实现了一个基于视觉反馈和预测仿真的机器人遥操作系统(Telemanipulator,Tman),三维仿真服务器模拟出操作现场,远程用户根据视觉反馈和本机的三维仿真及时做出控制决策;同时,为了消除时延可能导致的超调所带来的安全隐患,采取远程控制结合本地自主控制的模式,由本地服务器完成最后的小区域内的精细操作.实验结果表明仿真预测和控制模式切换有效地补偿了时延,保证了控制系统的鲁棒性和安全性.     

基于FPGA的视觉处理系统设计与实现

从计算机视觉系统的基本体系结构出发,指出了计算机视觉系统要实现的主要功能,从理论上探讨了计算机视觉系统在硬件实现层次上存在的问题.进一步以立体视觉的应用要求为例提出了以现场可编程门阵列(FPGA)为核心芯片的视觉处理系统.因为FPGA具有极强的可重构性,可承担部分原来由上位机软件完成的运算,增强了视觉处理的实时性.其应用于计算机三维立体视觉系统中,作为前端的图像采集器和视觉协处理器取得了良好的效果.     

Variable admittance force feedback device and its human-robot interaction stability

In teleoperation, a force feedback device is a medium to build a transparent interaction environment between a human and a remote robotic arm. Using force feedback devices, the users can operate the remote robotic arm intuitively and perceive remote interaction through the force channel, just as if they are in the remote environment. Compared with impedance devices, admittance devices have the advantages of large feedback output, high stiffness, high reverse driving performance, and flexible structure, which are more suitable for the teleoperation of heavy-duty and large-size robotic arms. However, the control of admittance devices is relatively complex and has some inherent limitations such as response delay, instability from high-frequency oscillation, difficulty in achieving constant speed control, etc. Errors in admittance model parameters and human physiological characteristics, such as force application fluctuations, are the root causes of these problems. In this study, we proposed a fuzzy variable damping admittance algorithm, which allows the device to identify the user's movement intention and give respond quickly and accurately. We also established a human-robot interaction (HRI) system model of an admittance master controller device and summarized the principles of the admittance parameter configuration of a stable system. For the device's high-frequency oscillation instability caused by human arm stiffness, we propose an oscillation observation and reduction algorithm. By observing the force signal change characteristics, the algorithm can quickly detect the unstable behavior caused by human hands and perform oscillation reduction. To reduce the influence on upper limb uniform motion caused by fluctuating force application, we proposed a constant velocity intention inference algorithm based on a velocity spherical cone to smooth out the device operating velocity to achieve smooth control. The method proposed in this study achieved stable control in a 6 DOF force feedback device as a master controller, and the effect has been verified by experiments.     

预估控制下的实时网络遥操作移动机器人

构建了能使操作者通过Internet远程实时控制的移动机器人系统．为了补偿网络时延和抵消其对遥操作系统的影响,基于我们以前提出的改进型Smith预估器原理,采用了预估控制策略．为了保证系统稳定性和透明性,基于主从端的传感器信息交换,设计了一个动态模型管理器,其中模型和力反馈误差调节通过模糊控制实现．除了力反馈外,为了增强遥操作的实时性,引入了预估的虚拟显示．为了精确地预测网络时延,提出了一个新颖的时钟同步算法．为了降低时延抖动,结合我们提出的两个算法,实现了数据缓冲策略．最后,通过长距离的网络遥操作实验验证了系统和控制策略的实用性和有效性．     

PD‐like controllers for delayed bilateral teleoperation of manipulators robots

This paper proposes a compensated PD‐like controller for delayed bilateral teleoperation of a manipulator robot. The scheme has a PD‐like remote controller, a damping into the master, and a compensation strategy. The proposed compensation removes part of potential energy of the user's command depending on the difference between the situation on the remote site and the situation as perceived by the human operator. In addition, the stability of the delayed teleoperation system is analyzed, and a comparison based on experiments is carried out in order to analyze the advantages of using the proposed compensation. Finally, results of a bilateral teleoperation including the proposed control scheme, where the master and slave exchange information by using a low‐cost connection of mobile Internet, are shown. Copyright © 2014 John Wiley & Sons, Ltd.     

Providing synthetic views for teleoperation using visual posetracking in multiple cameras

This paper describes a visual tool for teleoperative experimentation involving remote manipulation and contact tasks. Using modest hardware, it recovers in real time the pose of moving polyhedral objects, and presents a synthetic view of the scene to the operator of a teleoperated robot using any chosen viewpoint and viewing direction. To recover pose, the method of line tracking first introduced by Harris (1992) is extended to multiple calibrated cameras, and its dynamic performance improved using robust methods and iterative filtering. Experiments are reported which determine the static and dynamic performance of the vision system, and its use in teleoperation is illustrated in two experiments, a peg-in-hole manipulation task and an impact control task     

Force reflecting teleoperation with adaptive impedance control

Experimentation and a survey of the literature clearly show that contact stability in a force reflecting teleoperation system requires high levels of damping on the master robot. However, excessive damping increases the energy required by an operator for commanding motion. The objective of this paper is to describe a new force reflecting teleoperation methodology that reduces operator energy requirements without sacrificing stability. We begin by describing a new approach to modeling and identifying the remote environment of the teleoperation system. We combine a conventional multi-input, multi-output recursive least squares (MIMO-RLS) system identification, identifying in real-time the remote environment impedance, with a discretized representation of the remote environment. This methodology generates a time-varying, position-dependent representation of the remote environment dynamics. Next, we adapt the target impedance of the master robot with respect to the dynamic model of the remote environment. The environment estimation and impedance adaptation are executed simultaneously and in real time. We demonstrate, through experimentation, that this approach significantly reduces the energy required by an operator to execute remote tasks while simultaneously providing sufficient damping to ensure contact stability.     

Visual context and the control of movements through video display

The aim of this experiment was to test the influence of target context on adaptation to scale perturbations introduced by a video display. Participants performed pointing movements without direct vision of their moving hand, although they could see their movements on a video display. Their perceived movements could be reduced, enlarged. or displayed at their actual size. Three target contexts were compared: dark surround, illuminated frame, and familiar object. Movements were executed with or without vision of hand displacement. Results showed that target context enhanced an allocentric coding of the movement, which improved movement execution. However, the effect of target context changed whether or not the displacement of the hand was available. Overall, the results suggest that target context allowed the extraction of dynamic information about movements, which is used to program and control movements. This suggests that target context could be used efficiently to improve spatial accuracy and speed in teleoperation learning. Potential applications include the reduction of difficulties encountered during teleoperation learning through the introduction of visual context.     

基于距离传感器的双边遥操作

为了改善双边遥操作的力反馈性能,本文根据从端操作臂上的传感器检测的目标距离信息,设计了新 的PD 双边控制器．证明了系统的稳定性条件,并通过单自由度双边遥操作实验系统,对提出的控制方法进行了实 验验证．实验结果表明,当从端操作臂靠近目标时,主端操作臂产生了逐渐增大的反馈力．这种控制策略为操作者 安全实现遥操作任务提供了有效手段．     

Teleoperation of Mobile Robots With Time-Varying Delay

This paper proposes a stable control scheme for teleoperation of mobile robots with visual feedback in presence of time-varying delay. The proposed control scheme is based on using a model of the human operator to combine (on the remote site) the velocity command generated by the human operator in a delayed time instant, the received information (which stimulates the operator) in such moment, and the current state of the remote site to set the velocity reference, which is applied to the mobile robot. In addition, the proposed control scheme does not modify the information sent from the remote site to the local site. On the other hand, the proposed scheme uses estimated parameters of the human operator and a fictitious force, which is calculated using data fusion from ultrasonic sensors and optical flow field taken from panoramic images. To illustrate the performance and stability of the proposed control structure, several teleoperation experiments between Argentina and Brazil linked via Internet are shown.     

Controlling air traffic with touch sensitive screen

Modern air traffic control (ATC) systems are increasingly using computer generated radar displays to present targets, target information and other general information to control personnel. On the control side the radar operator uses devices such as rolling ball, joystick, keyboard or lightpen. A laboratory demonstration of a new touch sensitive control device in a simulated ATC system is described in some detail together with results. Ergonomic advantages and design problems are also discussed.