A human–robot interface for mobile manipulator

This paper presents a remote manipulation method for mobile manipulator through operator’s gesture. In particular, a track mobile robot is equipped with a 4-DOF robot arm to grasp objects. Operator uses one hand to control both the motion of mobile robot and the posture of robot arm via scheme of gesture polysemy method which is put forward in this paper. A sensor called leap motion (LM), which can obtain the position and posture data of hand, is employed in this system. Two filters were employed to estimate the position and posture of human hand so as to reduce the inherent noise of the sensor. Kalman filter was used to estimate the position, and particle filter was used to estimate the orientation. The advantage of the proposed method is that it is feasible to control a mobile manipulator through just one hand using a LM sensor. The effectiveness of the proposed human–robot interface was verified in laboratory with a series of experiments. And the results indicate that the proposed human–robot interface is able to track the movements of operator’s hand with high accuracy. It is found that the system can be employed by a non-professional operator for robot teleoperation.     

Markerless human–robot interface for dual robot manipulators using Kinect sensor

Remote teleoperation of robot manipulators is often necessary in unstructured, dynamic, and dangerous environments. However, the existing mechanical and other contacting interfaces require unnatural, or hinder natural, human motions. At present, the contacting interfaces used in teleoperation for multiple robot manipulators often require multiple operators. Previous vision-based approaches have only been used in the remote teleoperation for one robot manipulator as well as require the special quantity of illumination and visual angle that limit the field of application. This paper presents a noncontacting Kinect-based method that allows a human operator to communicate his motions to the dual robot manipulators by performing double hand–arm movements that would naturally carry out an object manipulation task. This paper also proposes an innovative algorithm of over damping to solve the problem of error extracting and dithering due to the noncontact measure. By making full use of the human hand–arm motion, the operator would feel immersive. This human–robot interface allows the flexible implementation of the object manipulation task done in collaboration by dual robots through the double hand–arm motion by one operator.     

Vision-based arm gesture recognition for a long-range human–robot interaction

This paper proposes a vision-based human arm gesture recognition method for human–robot interaction, particularly at a long distance where speech information is not available. We define four meaningful arm gestures for a long-range interaction. The proposed method is capable of recognizing the defined gestures only with 320×240 pixel-sized low-resolution input images captured from a single camera at a long distance, approximately five meters from the camera. In addition, the system differentiates the target gestures from the users’ normal actions that occur in daily life without any constraints. For human detection at a long distance, the proposed approach combines results from mean-shift color tracking, short- and long-range face detection, and omega shape detection. The system then detects arm blocks using a background subtraction method with a background updating module and recognizes the target gestures based on information about the region, periodical motion, and shape of the arm blocks. From experiments using a large realistic database, a recognition rate of 97.235% is achieved, which is a sufficiently practical level for various pervasive and ubiquitous applications based on human gestures.     

Shape recognition of laser beam trace for human–robot interface

A robot navigation system using the pattern recognition of figures drawn by a laser pointer has been proposed. Typical figures are registered and assigned to individual robot commands. Each figure is identified based on the feature of its edges. This system detects trace of laser beam and calculates its optical flow vectors. Each figure has its own characteristic distribution pattern of vector inclinations. By evaluating the optical flow pattern of displayed laser spot, the system distinguishes the shape of a laser beam trace and provides the command to a robot corresponding to the drawn figure. The proposed system has been applied to the mobile robot, and shows its effectiveness by steering successfully.     

Human motion quality and accuracy measuring method for human–robot physical interactions

Intelligent Service Robotics - In human–robot collaboration (HRC), human motion capture can be considered an enabler for switching autonomy between humans and robots to create efficient and...     

Improved stability of haptic human–robot interfaces using measurement of human arm stiffness

Necessary physical contact between an operator and a force feedback haptic device creates a coupled system consisting of human and machine. This contact, combined with the natural human tendency to increase arm stiffness to attempt to stabilize its motion, can reduce the stability of the system. This paper proposes a method to increase stability on demand while maintaining speed and performance. Operator arm stiffness is not directly measurable, so controllers cannot typically account for this issue. The causes of arm end-point stiffness are examined as related to system stability, and a method for estimating changes in arm stiffness based on arm muscle activity was designed to provide a robotic controller with additional information about the operator. This was accomplished using electromyograms (EMGs) to measure muscle activities and estimating the level of arm stiffness, which was used to adjust the dynamic characteristics of an impedance controller. To support this design, the correlation between EMGs and arm stiffness was validated experimentally. Further experiments characterized the effects of the designed system on operator performance. This showed increased stability and faster, more accurate movements using the compensating system. Such a system could be used in many applications, including force assisting devices in industrial facilities.     

Gesture recognition based human–robot interactive control for robot soccer

Microsystem Technologies - This paper proposes an interaction method in human gestures for controlling robot play soccer. It aims to design a human–robot interactive control scheme let a...     

Inkjet-printed proximity sensor for human–robot interaction

Microsystem Technologies - With the rapid development of artificial intelligence (AI), the human–robot interaction (HRI) has been attracted considerable attention in recent years. The...     

Automatic domain modeling for human–robot interaction

Intelligent Service Robotics - This paper introduces an approach to automatic domain modeling for human–robot interaction. The proposed approach is symbolic and intended for semantically...     

Mental tasks-based brain–robot interface

This paper describes a Brain Computer Interface (BCI) based on electroencephalography (EEG) that allows control of a robot arm. This interface will enable people with severe disabilities to control a robot arm to assist them in a variety of tasks in their daily lives. The BCI system developed differentiates three cognitive processes, related to motor imagination, registering the brain rhythmic activity through 16 electrodes placed on the scalp. The features extraction algorithm is based on the Wavelet Transform (WT). A Linear Discriminant Analysis (LDA) based classifier has been developed in order to differentiate between the three mental tasks. The classifier combines through a score-based system four LDA-based models simultaneously. The experimental results with six volunteers performing several trajectories with a robot arm are shown in this paper.     

Incremental learning of gestures for human–robot interaction

For a robot to cohabit with people, it should be able to learn people’s nonverbal social behavior from experience. In this paper, we propose a novel machine learning method for recognizing gestures used in interaction and communication. Our method enables robots to learn gestures incrementally during human–robot interaction in an unsupervised manner. It allows the user to leave the number and types of gestures undefined prior to the learning. The proposed method (HB-SOINN) is based on a self-organizing incremental neural network and the hidden Markov model. We have added an interactive learning mechanism to HB-SOINN to prevent a single cluster from running into a failure as a result of polysemy of being assigned more than one meaning. For example, a sentence: “Keep on going left slowly” has three meanings such as, “Keep on (1)”, “going left (2)”, “slowly (3)”. We experimentally tested the clustering performance of the proposed method against data obtained from measuring gestures using a motion capture device. The results show that the classification performance of HB-SOINN exceeds that of conventional clustering approaches. In addition, we have found that the interactive learning function improves the learning performance of HB-SOINN.     

A unified multimodal control framework for human–robot interaction

In human–robot interaction, the robot controller must reactively adapt to sudden changes in the environment (due to unpredictable human behaviour). This often requires operating different modes, and managing sudden signal changes from heterogeneous sensor data. In this paper, we present a multimodal sensor-based controller, enabling a robot to adapt to changes in the sensor signals (here, changes in the human collaborator behaviour). Our controller is based on a unified task formalism, and in contrast with classical hybrid visicn–force–position control, it enables smooth transitions and weighted combinations of the sensor tasks. The approach is validated in a mock-up industrial scenario, where pose, vision (from both traditional camera and Kinect), and force tasks must be realized either exclusively or simultaneously, for human–robot collaboration.     

Human tracking and silhouette extraction for human–robot interaction systems

In this study, we propose a new integrated computer vision system designed to track multiple human beings and extract their silhouette with a pan-tilt stereo camera, so that it can assist in gesture and gait recognition in the field of Human–Robot Interaction (HRI). The proposed system consists of three modules: detection, tracking and silhouette extraction. These modules are robust to camera movements, and they work interactively in near real-time. Detection was performed by camera ego-motion compensation and disparity segmentation. For tracking, we present an efficient mean shift-based tracking method in which the tracking objects are characterized as disparity weighted color histograms. The silhouette was obtained by two-step segmentation. A trimap was estimated in advance and then effectively incorporated into the graph-cut framework for fine segmentation. The proposed system was evaluated with respect to ground truth data, and it was shown to detect and track multiple people very well and also produce high-quality silhouettes.

Nonlinear model reference adaptive impedance control for human–robot interactions

Four nonlinear Model Reference Adaptive Impedance Controllers are introduced, tested, and compared for the control of the robot impedance with uncertainties in model parameters. The intended application area is human–robot interactions (HRI), particularly rehabilitation robots and haptic interfaces. All of the controllers make the closed-loop dynamics of the robot similar to the reference impedance model besides providing asymptotic tracking of the impedance model for the robot end-effector in Cartesian coordinates. The tracking and global stability are proved using Lyapunov stability theorem. Based on the simulations and experiments on a two-DOFs robot, the effectiveness of the proposed controllers is investigated.     

Design of a quadruped robot for human–elephant conflict mitigation

Introduction

Human–elephant conflict is a major problem leading to crop damage, human death by elephants and elephants being killed by people. The surveillance and tracking of elephant herds are difficult due to their size and nature of movement.

Materials

In this article, we propose a four-wheeled quadruped robot to mitigate human–elephant conflict. The robot can detect movement of wild pachyderms in certain pockets along the forest borders through which the elephants enter into the human living areas from the forest. The robot is so designed that it can navigate with wheels on flat terrains and with legs on unfriendly rugged terrains with the help of mounted cameras.

Methods

The images of the wild elephant are captured and transmitted to the base stations and an SMS is sent to the forest officials indicating an elephant is found. We obtain a suitable kinematic model for both legs and wheels with control algorithm for the quadruped robot to move along a predetermined path.

Conclusion

The quadruped robot proposed is a solution to detect elephant movement without affecting the ecological conditions to overcome human–elephant conflict. The unpredictability of time and location of elephant arrival into the villages are considered the major issues that are resolved in this work. The results of our work contribute to elephant conservation issues and are suitable for the detection of elephants in forest border areas.     

Skill learning framework for human–robot interaction and manipulation tasks

In this article, a learning framework that enables robotic arms to replicate new skills from human demonstration is proposed. The learning framework makes use of online human motion data acquired using wearable devices as an interactive interface for providing the anticipated motion to the robot in an efficient and user-friendly way. This approach offers human tutors the ability to control all joints of the robotic manipulator in real-time and able to achieve complex manipulation. The robotic manipulator is controlled remotely with our low-cost wearable devices for easy calibration and continuous motion mapping. We believe that our approach might lead to improving the human-robot skill learning, adaptability, and sensitivity of the proposed human-robot interaction for flexible task execution and thereby giving room for skill transfer and repeatability without complex coding skills.     

An online collision-free trajectory generation algorithm for human–robot collaboration

The premise of human–robot collaboration is that robots have adaptive trajectory planning strategies in hybrid work cell. The aim of this paper is to propose a new online collision avoidance trajectory planning algorithm for moderate dynamic environments to insure human safety when sharing collaborative tasks. The algorithm contains two parts: trajectory generation and local optimization. Firstly, based on empirical Dirichlet Process Gaussian Mixture Model (DPGMM) distribution learning, a neural network trajectory planner called Collaborative Waypoint Planning network (CWP-net) is proposed to generate all key waypoints required for dynamic obstacle avoidance in joint space according to environmental inputs. These points are used to generate quintic spline smooth motion trajectories with velocity and acceleration constraints. Secondly, we present an improved Stochastic Trajectory Optimization for Motion Planning (STOMP) algorithm which locally optimizes the generated trajectories of CWP-net by constraining the trajectory optimization range and direction through the DPGMM model. Simulations and real experiments from an industrial use case of human–robot collaboration in the field of aircraft assembly testing show that the proposed algorithm can smoothly adjust the nominal path online and effectively avoid collisions during the collaboration.     

A human–robot interface using particle filter,Kalman filter,and over-damping method

This paper presents a novel human–manipulator interface which copies the hand motion to control a manipulator. In the proposed interface, an inertial measurement unit is used to measure the orientation of the human hand, and a 3D camera is employed to locate the human hand using the Camshift algorithm. Although the position and the orientation of the human can be obtained from two sensors, the measurement errors increase over time due to the noise of the devices and the tracking errors. Therefore, particle filter and Kalman filter are used to estimate the position and the orientation of the human hand. Due to the limitations of the perceptive and the motor, human operator cannot accomplish the high-precision manipulation without any assistance. An over-damping method is employed to assist the operator to improve the accuracy and reliability in determining the postures of the manipulator. The human–manipulator interface system was experimentally tested in a lab environment, and the results indicate that such an interface can successfully control a robot manipulator even when the operator is not an expert.     

Shared control architecture based on RFID to control a robot arm using a spontaneous brain–machine interface

This paper describes a shared control architecture combining a Brain–Machine Interface (BMI) with Radio-frequency Identification (RFID) technology to control a robot arm in pick and place operations. A non-invasive spontaneous BMI capable of distinguishing between three different mental tasks has been designed. Using the BMI, the user can control the robot in order to perform complex actions (e.g. pick and place operations). RFID tags have been placed in the experimental setup to give information about the position of the objects in the scene. With this information, the user is able to pick and place the objects with a robot arm by performing simple commands: move left, move right, pick or place, with the only help of the BMI. Four volunteers have successfully controlled the robot arm, and time and accuracy have been measured.