Human-centered control scheme for delayed bilateral teleoperation of mobile robots

Teleoperation task performance strongly depends on how well the human operator’s commands are executed. In this paper, we propose a control scheme for delayed bilateral teleoperation of mobile robots that considers user’s commands execution in order to achieve a high-performance teleoperation system in some important aspects like time to complete the task, safety, and operator dependence. We describe some evaluation metrics that allow us to address these aspects and a quantitative metric is proposed and incorporated in the control scheme to compensate wrong commands. A force feedback is applied to the master at the local site as a haptic cue. In addition, the system stability is analyzed taking into consideration the master and remote robot dynamic models and the asymmetric time-varying delays of the communication channel. Multiple human-in-the-loop simulations were carried out and the results of the evaluation metrics were discussed. Additionally, we present experiments where a user teleoperates a mobile robot via the Internet connection between Argentina and Italy.     

Control aspects of a robotic haptic interface for kinesthetic knee joint simulation

A new 6 DOF (degrees of freedom) haptic device is presented for display of the dynamic properties of a virtual human knee in orthopaedic training. In order to achieve the high impedance requirements in this application an industrial robot has been selected and extended with a PC-based high performance controller hardware and redundant safety features. Two control architectures—a force-command and motion-command control—are presented and both implemented in a 1 DOF application. With a new method for evaluation and comparison of controller performance in terms of impedance error the experimental results reveal that motion-command provides better accuracy for display of the high impedances specific to the human knee application.     

A teleoperation system for micro positioning with haptic feedback

This paper presents the research work on a 1 Degree of Freedom (DOF) macro-micro teleoperation system which enables human operator to perform complex task in micro environment such as cell insertion with the capability of haptic feedback. To reach submicron resolution, a nano-motion piezo actuator was used as the slave robot and a servo DC motor was used as the master robot. Force sensors were implemented at both ends for haptic feedback and a microscope equipped with camera was employed for real-time visual feedback. The hysteresis nonlinearity of the piezo motor was modeled using LuGre friction model and compensated for. A Sliding Mode Based Impedance Controller (SMBIC) was designed at the slave side to ensure position tracking while an impedance force controller was designed at the master side to ascertain tracking of the force. Control parameters were chosen based on Llewellyn stability criteria such that the entire system stays stable against parameter uncertainties and constant time delay. The experimental results demonstrated capability of the proposed control frameworks in desirable tracking of the position and force signals while the entire system remained stable. The results of this study can be used for complex tasks in micron environment such as cell insertion.     

Upper-body haptic system for snake robot teleoperation in pipelines

Snake robots have shown a great potential for operations in confined workplaces that are less accessible or dangerous to human workers, such as the in-pipe inspection. However, the snake robot teleoperation remains a nontrivial task due to the unique locomotion mechanism (e.g., helical motion) and the constraints of the workplaces including the low visibility and indistinguishable features. Most snake robot feedback systems are based on the live camera view only. It is hard for the human operator to develop a correct spatial understanding of the remote workplace, leading to problems such as disorientation and motion sickness in snake robot teleoperation. This study designs and evaluates an innovative haptic assistant system for snake robot teleoperation in the in-pipe inspection. An upper-body haptic suit with 40 vibrators on both the front and back sides of the human operator was developed to generate haptic feedback corresponding to the bottom and up sides of the snake robot, transferring the egocentric sensation of the snake robot to the human operator. A human-subject experiment (n = 31) was performed to evaluate the efficacy of the developed system. The results indicate that the proposed haptic assistant system outperformed other feedback systems in terms of both task performance and subjective workload and motion sickness evaluations. It inspires new control and feedback designs for the future snake robot in industrial operations.     

High-fidelity sliding mode control of a pneumatic haptic teleoperation system

For a pneumatic teleoperation system with on/off solenoid valves, sliding mode control laws for position and force ensuring low switching (open/close) activity of the valves are developed. Since each pneumatic actuator has two pneumatic chambers with a total of four on/off valves, 16 possible combinations (‘operating modes’) for the valves’ on/off positions exist, but only seven of which are both functional and unique. While previous work has focused on three-mode sliding-based position control of one pneumatic actuator, this paper develops the seven-mode sliding-based bilateral control of a teleoperation system comprising a pair of pneumatic actuators. The proposed bilateral sliding control schemes are experimentally validated on a pair of actuators utilizing position-position and force-position teleoperation architectures. The results demonstrate that leveraging the additional modes of operation leads to more efficient and smoother control of the pneumatic teleoperation system. It was found that viscous friction forces were crippling haptic feedback in the position-position architecture. Through the use of force sensors, the force-position architecture was able to compensate for the heavy viscous friction forces.     

Projected predictive Energy-Bounding Approach for multiple degree-of-freedom haptic teleoperation

This paper presents a geometrically and dynamically transparent projected predictive Energy-Bounding Approach (EBA) for multiple degree-of-freedom (m-DOF) haptic teleoperation with time delays. The straightforward extension of a single-DOF teleoperation to m-DOF teleoperation suffers from a bad force distortion problem that becomes worse with the increase in communication time delays between master and slave sites. Due to this, geometry perception of remote m-DOF objects at master site is severely deteriorated. To solve this problem, a projected predictive m-DOF EBA is proposed by combining the m-DOF Predictive EBA and the projection method that can compensate the force distortion problem. The proposed approach also includes steps to get stably the information about the contact locations (geometry) of m-DOF remote objects without geometry detection sensors. In order to validate the proposed approach, some experiments performing surface contour-following are conducted using two Phantoms: (i) 2-DOF object (bobbin), and (ii) 3-DOF semi-spherical object in the presence of large time delays.     

An environment for motor skill transfer based on wearable haptic communication

Personal and Ubiquitous Computing - We propose an environment for assisting in the transfer of motor skills by supporting nonverbal communication with wearable haptic technology. The proposed...     

Image-based haptic display via a novel pen-shaped haptic device on touch screens

Multimedia Tools and Applications - Haptic interaction is a new interactive mode in the interaction technology between human and touch screens. In this work, we present an image-based haptic...     

Detecting work-related stress with a wearable device

Excessive stress will lower work efficiency, lead to negative emotions and even various illnesses. This paper aims at detecting work-related stress based on physiological signals measured by a wearable device. Different from common binary stress detection, this study detects three levels of stress, i.e., no stress, moderate stress and high perceived stress. The Montreal Imaging Stress Task (MIST) is used to simulate the different stress conditions, including both mental stress and psychosocial stress factors that are relevant at the workplace. A sensor-based wearable device is used to acquire the electrocardiogram (ECG) and respiration (RSP) signals from 39 participants. We extract stress-related features from ECG and RSP, and the Random Forest is used to select the optimal feature combination, which is later fed to the classifier. Four classifiers are investigated about their ability to predict the three stress levels. Finally, the combination of Random Forest and Support Vector Machine (SVM) achieve the best performance. With this method, the accuracy is improved from 78% to 84% in three states classification. And in binary stress detection, the accuracy is 94%.     

Ergonomic evaluation of a wearable assistive device for overhead work

Overhead work is an important risk factor for upper extremity (UE) musculoskeletal disorders. We examined the potential of a mechanical arm and an exoskeletal vest as a wearable assistive device (WADE) for overhead work. Twelve participants completed 10 minutes of simulated, intermittent overhead work, using each of three payloads (1.1, 3.4 and 8.1 kg) and with/without the WADE. Ratings of perceived discomfort (RPDs) and electromyography (EMG) were obtained for the upper arms, shoulders and low back. Using the WADE, UE RPDs decreased by ～50% with the heavier payloads, whereas smaller (～25%) and non-significant increases in low-back RPDs were found and were relatively independent of payload. Changes in RPDs with WADE use were consistent with physical demands indicated by EMG, though EMG-based differences in fatigue were less apparent. Participants generally preferred using the WADE, particularly with heavier payloads. These results supported the potential utility of a WADE as an intervention for overhead work.     

基于时域无源性控制的六足机器人双边触觉遥操作

随着六足机器人研究工作的深入,针对其遥操作系统的开发面临诸多挑战.为了弥补松软接触条件对系统可控性及稳定性的影响,提出一种基于时域无源性控制(time-domain passivity control,TDPC)的六足机器人双边触觉遥操作方法.其主从两端采取位置-速度的交互模式,通过分析足-地柔性接触的作用机理,构建无源观测器和无源控制律以补偿足底滑移所导致环境系统的潜在有源性,采用速度跟踪模式设计基于触觉力反馈的系统控制架构,并利用Llewellyn准则确定控制律参数的稳定范围.最后,搭建半物理仿真实验平台并验证所提出的双边触觉遥操作方法在松软地形条件下能够保证六足机器人遥操作系统的稳定,且兼具较好的持续跟踪能力.     

穿戴系统中的天线设计

采用U型缝加载技术,设计了一种用于穿戴系统的双频平面倒F天线(PIFA),分别工作于GSM(全球通)的900 MHz和WLAN(无线局域网)的2.45 GHz频段,两个频段的相对阻抗带宽分别为10%和5%;在两个谐振频点处,天线的增益分别为0.4 dBi和1.7 dBi.天线具有低剖面结构、重量轻、结构简单等优点,完全满足穿戴系统中长、短距离无线通信要求.     

Human-centered design for advanced services: A multidimensional design methodology

Advanced services have caught the attention of industries and academics as a way to exploit new customer value propositions. However, the existing design methodologies for advanced services are limited to partially addressing one or some key design elements, hence causing confusion in practice. Moreover, human factors are not often addressed, even though the design for advanced services requires human-centered thinking. Aiming to advance the body of research, the current study aims to conceptually propose a multidimensional design methodology called DIMAND that captures the key design elements and their relations in a single-view structure in accordance with a human-centric approach. Specifically, DIMAND encapsulates the (i) life-cycle service design interrelated with other key design elements—(ii) stakeholder networks, (iii) new service development methods, and (iv) design skills—that must be considered to develop effective advanced service design. Based on a hybrid research design, DIMAND was conceptually developed through systematic reviews and structured analysis of existing design methodologies, as well as an elicitation of expert knowledge in the domain through the analytical hierarchy process (AHP). For validation, the average usability score of DIMAND as evaluated by 26 practitioners was 72.2, which falls into “excellence” on the simplified system usability scale (SUS), hence confirming its potential utility. As a result, DIMAND offers a novel and holistic guideline for design practitioners and engineers to obtain coherence in all the life-cycle design processes by simultaneously taking these key design elements and their relations into account, making the design of advanced services more practical.     

A method for embedding a computer vision application into a wearable device

Pattern classification applications can be found everywhere, especially the ones that use computer vision. What makes them difficult to embed is the fact that they often require a lot of computational resources. Embedded computer vision has been applied in many contexts, such as industrial or home automation, robotics, and assistive technologies. This work performs a design space exploration in an image classification system and embeds a computer vision application into a minimum resource platform, targeting wearable devices. The feature extractor and the classifier are evaluated for memory usage and computation time. A method is proposed to optimize such characteristics, leading to a reduction of over 99% in computation time and 92% in memory usage, with respect to a standard implementation. Experimental results in an ARM Cortex-M platform showed a total classification time of 0.3 s, maintaining the same accuracy as in the simulation performed. Furthermore, less than 20 KB of data memory was required, which is the most limited resource available in low-cost and low-power microcontrollers. The target application, used for the experimental evaluation, is a crosswalk detector used to help visually impaired persons.     

Energy-based control of a haptic device using brakes.

This paper proposes an energy-based control method of a haptic device with electric brakes. Unsmooth motion is frequently observed in a haptic system using brakes during a wall-following task. Since it is generally known that a haptic system using brakes is passive due to brake's characteristics, its energy behavior has seldom been investigated. However, force distribution at the end effector reveals that the unsmooth motion of a haptic system using brakes represents active behavior of the system in the specific direction. A force control scheme is proposed that computes the gain for smooth motion by considering the energy behavior of a system. Experiments show that smooth wall following is possible with a proposed force control scheme.     

A large haptic device for aircraft engine maintainability

The virtual reality for maintainability (Revima) VR system supports maintainability simulation in aeronautics. Within this project we have developed and integrated a haptic device, the large haptic interface for aeronautic maintainability (LHIfAM). We use this device to track hand movements and provide force feedback within the large geometric models that describe aircraft engines. The user movements are the same as those that occur when testing physical mock-ups. An integrated haptic device and VR system for testing aircraft engines reduces development costs and avoids the necessity of physical mock-ups formaintainability.     

Advanced teleoperation and control system for industrial robots based on augmented virtuality and haptic feedback

There are some industrial tasks that are still mainly performed manually by human workers due to their complexity, which is the case of surface treatment operations (such as sanding, deburring, finishing, grinding, polishing, etc.) used to repair defects. This work develops an advanced teleoperation and control system for industrial robots in order to assist the human operator to perform the mentioned tasks. On the one hand, the controlled robotic system provides strength and accuracy, holding the tool, keeping the right tool orientation and guaranteeing a smooth approach to the workpiece. On the other hand, the advanced teleoperation provides security and comfort to the user when performing the task. In particular, the proposed teleoperation uses augmented virtuality (i.e., a virtual world that includes non-modeled real-world data) and haptic feedback to provide the user an immersive virtual experience when remotely teleoperating the tool of the robot system to treat arbitrary regions of the workpiece surface. The method is illustrated with a car body surface treatment operation, although it can be easily extended to other surface treatment applications or even to other industrial tasks where the human operator may benefit from robotic assistance. The effectiveness of the proposed approach is shown with several experiments using a 6R robotic arm. Moreover, a comparison of the performance obtained manually by an expert and that obtained with the proposed method has also been conducted in order to show the suitability of the proposed approach.     

A modified adaptive controller design for teleoperation systems

In this paper, a new adaptive controller is proposed to ensure the stability and good performance of a teleoperation system while a wide range of time delays is considered. For this means, a feedforward compensator is designed to ensure system passivity and then a new model reference adaptive controller (MRAC) is developed to provide good performance. The developed system demonstrates good stability and force tracking capabilities. A command generator tracker (CGT) is designed for a sample teleoperation system and the results are compared with the proposed system.