Controller design for human-robot interaction

Many robotics tasks require a robot to share the same workspace with humans. In such settings, it is important that the robot performs in such a way that does not cause distress to humans in the workspace. In this paper, we address the problem of designing robot controllers which minimize the stress caused by the robot while performing a given task. We present a novel, data-driven algorithm which computes human-friendly trajectories. The algorithm utilizes biofeedback measurements and combines a set of geometric controllers to achieve human friendliness. We evaluate the comfort level of the human using a Galvanic Skin Response (GSR) sensor. We present results from a human tracking task, in which the robot is required to stay within a specified distance without causing high stress values.

Facial expression recognition and tracking for intelligent human-robot interaction

For effective interaction between humans and socially adept, intelligent service robots, a key capability required by this class of sociable robots is the successful interpretation of visual data. In addition to crucial techniques like human face detection and recognition, an important next step for enabling intelligence and empathy within social robots is that of emotion recognition. In this paper, an automated and interactive computer vision system is investigated for human facial expression recognition and tracking based on the facial structure features and movement information. Twenty facial features are adopted since they are more informative and prominent for reducing the ambiguity during classification. An unsupervised learning algorithm, distributed locally linear embedding (DLLE), is introduced to recover the inherent properties of scattered data lying on a manifold embedded in high-dimensional input facial images. The selected person-dependent facial expression images in a video are classified using the DLLE. In addition, facial expression motion energy is introduced to describe the facial muscle’s tension during the expressions for person-independent tracking for person-independent recognition. This method takes advantage of the optical flow which tracks the feature points’ movement information. Finally, experimental results show that our approach is able to separate different expressions successfully.     

Model reference adaptive impedance control for physical human-robot interaction

This paper presents a novel enhanced human-robot interaction system based on model reference adaptive control. The presented method delivers guaranteed stability and task performance and has two control loops. A robot-specific inner loop, which is a neuroadaptive controller, learns the robot dynamics online and makes the robot respond like a prescribed impedance model. This loop uses no task information, including no prescribed trajectory. A task-specific outer loop takes into account the human operator dynamics and adapts the prescribed robot impedance model so that the combined human-robot system has desirable characteristics for task performance. This design is based on model reference adaptive control, but of a nonstandard form. The net result is a controller with both adaptive impedance characteristics and assistive inputs that augment the human operator to provide improved task performance of the human-robot team. Simulations verify the performance of the proposed controller in a repetitive point-to-point motion task. Actual experimental implementations on a PR2 robot further corroborate the effectiveness of the approach.     

Robot apology as a post-accident trust-recovery control strategy in industrial human-robot interaction

Due to safety requirements for Human-Robot Interaction (HRI), industrial robots have to meet high standards of safety requirements (ISO 10218). However, even if robots are incapable of causing serious physical harm, they still may influence people's mental and emotional wellbeing, as well as their trust, behaviour and performance in close collaboration. This work uses an HTC Vive Virtual Reality headset to study the potential of using robot control strategies to positively influence human post-accident behaviour. In the designed scenario, a virtual industrial robot first makes sudden unexpected movements, after which it either does or does not attempt to apologise for them. The results show that after the robot tries to communicate with the participants, the robot is reported to be less scary, more predictable and easier to work with. Furthermore, postural analysis shows that the participants who were the most affected by the robot's sudden movement recover 74% of their postural displacement within 60 s after the event if the robot apologised, and only 34% if it did not apologise. It is concluded, that apologies, which are commonly used as a trust-recovery strategy in social robotics, can positively influence people engaged with industrial robotics as well.Relevance to industryFindings can be used as guidelines for designing robot behaviour and trust-recovery control strategies meant to speed up human recovery after a trust-violating event in industrial Human-Robot Interaction.     

An effective method for detecting facial features and face in human-robot interaction

In this paper, an effective method of facial features detection is proposed for human-robot interaction (HRI). Considering the mobility of mobile robot, it is inevitable that any vision system for a mobile robot is bound to be faced with various imaging conditions such as pose variations, illumination changes, and cluttered backgrounds. To detecting face correctly under such difficult conditions, we focus on the local intensity pattern of the facial features. The characteristics of relatively dark and directionally different pattern can provide robust clues for detecting facial features. Based on this observation, we suggest a new directional template for detecting the major facial features, namely the two eyes and the mouth. By applying this template to a facial image, we can make a new convolved image, which we refer to as the edge-like blob map. One distinctive characteristic of this map image is that it provides the local and directional convolution values for each image pixel, which makes it easier to construct the candidate blobs of the major facial features without the information of facial boundary. Then, these candidates are filtered using the conditions associated with the spatial relationship of the two eyes and the mouth, and the face detection process is completed by applying appearance-based facial templates to the refined facial features. The overall detection results obtained with various color images and gray-level face database images demonstrate the usefulness of the proposed method in HRI applications.     

Rescuing interfaces: A multi-year study of human-robot interaction at the AAAI Robot Rescue Competition

This paper presents results from three years of studying human-robot interaction in the context of the AAAI Robot Rescue Competition. We discuss our study methodology, the competitors’ systems and performance, and suggest ways to improve human-robot interaction in urban search and rescue (USAR) as well as other remote robot operations.

Soft computing based intention reading techniques as a means of human-robot interaction for human centered system

 Friendly interaction between robot and human is vital in the design of a human centered system. Among several interaction technologies, the intention reading of the user plays an important role for the human centered system. We focus on the aspect of intention reading in rehabilitation robots, and implement its capability for the wheelchair-based robotic arm system, called KARES (KAIST Rehabilitation Engineering Service System) II. An effective intention reading scheme is proposed on the basis of several soft computing techniques to handle uncertainty of the user's intention. Two application examples of intention reading in KARES II are discussed: one is visual servoing of the user's face, and the other is emergency stop of the robot by using EMG signals of the user's arm. This work is partially supported by the Ministry of Science and Technology of Korea as a part of Critical Technology 21 Program on “Development of Service Robot Technology”.     

Human motion prediction for human-robot collaboration

In human-robot collaborative manufacturing, industrial robots would work alongside human workers who jointly perform the assigned tasks seamlessly. A human-robot collaborative manufacturing system is more customised and flexible than conventional manufacturing systems. In the area of assembly, a practical human-robot collaborative assembly system should be able to predict a human worker’s intention and assist human during assembly operations. In response to the requirement, this research proposes a new human-robot collaborative system design. The primary focus of the paper is to model product assembly tasks as a sequence of human motions. Existing human motion recognition techniques are applied to recognise the human motions. Hidden Markov model is used in the motion sequence to generate a motion transition probability matrix. Based on the result, human motion prediction becomes possible. The predicted human motions are evaluated and applied in task-level human-robot collaborative assembly.     

人机交互中的个性化情感模型

人与机器人的交互过程中,情感因素的引入能够使人机交流更加自然和谐.因此,完整的人工情感模型的建立是首要解决的问题.基于情感能量理论基础,首先,提出了心境自发转移和刺激转移模型.其次,结合情绪自发转移的马尔可夫链模型和刺激转移的HMM模型,将心境和情绪的自发和刺激转移过程统一在一个框架下.最后,将完整的人工情感模型软件化并应用于儿童玩伴机器人上,在接受非结构化环境与用户的信息输入后,个性化的情感软件模块产生输出,实现针对儿童用户的玩伴机器人个性化交互,通过应用验证了该模型的有效性.