Modern artificial intelligence for human-robot interaction

Interaction between humans and traditional robotic systems can be problematic, as the traditional robotic systems often perform repetitive actions in a restricted environment, whereas human interaction is often characterized by novel ways of interactions, which creates an ever-changing environment. Therefore, traditional robotic system methods and technologies are often difficult to apply in situations and applications where the interaction plays a major role. So we developed a new kind of user-guided behavior-based robotics and applied this method to both mobile and humanoid robots, in order to investigate how nonexpert users could develop their own complex robot behaviors within a very short time (e.g., 30-60 min) with no prior knowledge of the robot technology. In the development of the team of humanoid robots, Viki, we used a modern approach to artificial intelligence that puts emphasis on the balance between control, electronic hardware, material, sensory system, and energy. It is possible to develop simple user interfaces with the user-guided behavior-based approach, in order to allow any user to design performances with the humanoid robots. However, we should not only look at the control of robots when designing human-robot interactions, but also on the physical aspects of the robotic system. Therefore, the novel concept of "programming by building" is introduced and exemplified with the creation of a prototype system consisting of building blocks with individual processing and communication capabilities. By assembling such building blocks, the user develops both the physical aspects and the functionality of the robotic system in an easy manner with no need to use a host computer or a traditional programming language. The approach is further exemplified with the implementation of neural building blocks that can later be trained by the user.     

HERMES - a versatile personal robotic assistant

We have developed a humanoid robot, HERMES, to study several key technologies that are important for personal robots, such as robot design, sensors and perception, locomotion, localization and navigation, manipulation, human-robot communication and interaction, adaptability and learning, system architecture and integration, and dependability. The robot's skill-based system architecture was derived from a qualitative model of human information processing and insights gained from psychological literature dealing with skill acquisition, human performance and motor learning. HERMES' system architecture, several of its skills and the design principles are introduced, and some experiments carried out with the real robot are presented, including a long-term test where HERMES served in a museum, far away from its home laboratory, for more than six months up to 12 hours per day. During this period the robot and its skills were regularly demonstrated to the public by nonexpert presenters. Also, HERMES interacted with the visitors, chatted with them in English, French and German, answered questions and performed services as requested by them.     

Interactive Human Intention Reading by Learning Hierarchical Behavior Knowledge Networks for Human‐Robot Interaction

For efficient interaction between humans and robots, robots should be able to understand the meaning and intention of human behaviors as well as recognize them. This paper proposes an interactive human intention reading method in which a robot develops its own knowledge about the human intention for an object. A robot needs to understand different human behavior structures for different objects. To this end, this paper proposes a hierarchical behavior knowledge network that consists of behavior nodes and directional edges between them. In addition, a human intention reading algorithm that incorporates reinforcement learning is proposed to interactively learn the hierarchical behavior knowledge networks based on context information and human feedback through human behaviors. The effectiveness of the proposed method is demonstrated through play‐based experiments between a human and a virtual teddy bear robot with two virtual objects. Experiments with multiple participants are also conducted.     

A switching command-based whole-body operation method for humanoid robots

This paper introduces a switching command-based whole-body operation method for humanoid robots. Humanoid robots are biped machines possessing multiple degrees of freedom (DOF). Due to the complexity of their multi-DOF structure, and the difficulty in maintaining postural stability, whole-body operation of humanoid robots is fundamentally different from traditional fixed-base manipulators or stable-base mobile manipulators. By studying the shifts in locus of attention between human body joints during task execution, we developed a switching command-based operation method that allows the operator to select only the necessary points of the humanoid robot's body for manipulation. Whole-body motion satisfying the desired movements of the selected points is generated using an inverse-kinematics motion generation scheme. This switching operation method enables flexible whole-body operation of humanoid robots using simple input devices. The proposed whole-body operation method is implemented as a teleoperation system using two 3-DOF joysticks to operate a 30-DOF humanoid robot (HRP-1S) developed in the Humanoid Robotics Project (HRP) of the Ministry of Economy, Trade, and Industry of Japan. Experiments teleoperating HRP-1S confirmed the effectiveness of our method.     

Humanoid motion planning of robotic arm based on human arm action feature and reinforcement learning

The use and application of robotic arms in helping the aged and vulnerable persons are increasing gradually. In order to achieve safer and reliable human-robot interaction and its wider adoption, the requirements for the humanoid motion of robotic arms are becoming more stringent. This paper presents a humanoid motion planning method for a robotic arm based on the physics of human arm and reinforcement learning. Firstly, the humanoid motion rules are extracted by analyzing and learning the action data of human arm, which is collected using the VICON optical motion capture system. Then, according to the acquired features and rules, the corresponding reward functions are proposed and the humanoid motion training of the robotic arm is carried out by using the reinforcement learning based on Deep Deterministic Policy Gradient (DDPG) and Hindsight Experience Replay (HER) algorithm. Finally, the experiments are carried out to verify whether the robotic arm motions planned by the proposed approach are humanoid, and the observed results show its feasibility and effectiveness in planning the humanoid motion of the robotic arm.     

小型仿人机器人的设计及步态规划

针对现有仿人形机器人造价高的缺点,设计一款低成本的小型双足机器人研究平台.根据人类步行过程及人体生理结构.依据模糊控制与专家控制相结合的理论提出一种简单的双足机器人模型.并根据仿生学原理确定机器人的自由度配置及各关节的比例尺寸.然后,利用目前通用的行为规划软件对双足机器人步态规划进行仿真.并在平坦地面上进行相应行走试验.实验证明.根据人行走模式对机器人进行步态规划的算法稳定可行.为机器人的教学和科研提供了良好的实验平台.     

Kalman-Filter-Based Sensor Integration of Variable Power Assist Control Based on Human Stiffness Estimation

In applications such as robot collaborating with human operators, the robot system must operate more slowly and be more compliant to safe user interaction. Moreover, a consideration of the dynamic properties of human operators is also important for the human application. According to such requirements, this paper presents a novel sensorless force control approach for the robot-assisted motion of the human arm. A twin direct-drive motor system with a wire rope has been developed to provide a precise force sensation and safety for human-robot interaction. In order to control the wire rope tension and human interaction force, two mode designs of the force control are realized. The common mode is utilized for the control of wire rope tension. In the differential mode, the Kalman-filter-based sensor integration for the interaction force observer is proposed in this paper. By combining two motor encoders and a commercial acceleration sensor together, white Gaussian noise is reduced, and high accurate feedback of the contact force is obtained. A variable power assist control method based on a real-time estimation of the stiffness of the human arm is also introduced. By considering the stiffness in human arm movements, this method increases the efficiency of the force control system and realizes comfortable force for human-robot interaction. The effectiveness of the method is verified by experimental results.     

A Hybrid CPG–ZMP Controller for the Real-Time Balance of a Simulated Flexible Spine Humanoid Robot

Although a few researchers have started to realize the importance of a flexible spine in generating more natural-looking behaviors for humanoid robots, there has been no work on trying to make a full-body humanoid robot (that has a flexible spine) to maintain balance. The main reason for this is that it is very costly and difficult to develop and control this kind of robot. This paper serves two purposes. First, it proposes the use of a realistic 3-D robot simulator as a platform for costly flexible spine humanoid robotics research. Second, it presents a hybrid CPG-ZMP controller for the simulated robot. The biologically inspired CPG component of our controller allows the mechanical spine and feet to exhibit rhythmic motions using only two control parameters. Through monitoring the measured ZMP location, the engineering component modulates the neural activity of the CPG to allow the robot to maintain balance while it is standing and exhibiting motions on the sagittal and frontal planes in real time. The final postures of the simulated humanoid emerge automatically in real time through dynamic interactions between the neural networks, the robot itself, and its environment. Since experimental results have also demonstrated that our system is robust against disturbances from external pushing forces, our controller has the potential to be applicable to the next generation of humanoid robots.      

小型仿人机器人的设计及步态规划

针对现有仿人形机器人造价高的缺点,设计一款低成本的小型双足机器人研究平台。根据人类步行过程及人体生理结构．依据模糊控制与专家控制相结合的理论提出一种简单的双足机器人模型,并根据仿生学原理确定机器人的自由度配置及各关节的比例尺寸。然后,利用目前通用的行为规划软件对双足机器人步态规划进行仿真,并在平坦地面上进行相应行走试验。实验证明,根据人行走模式对机器人进行步态规划的算法稳定可行,为机器人的教学和科研提供了良好的实验平台。     

On activating human communications with pet-type robot AIBO

In this paper we describe effects of human interactions with a pet-type robot, especially with AIBO. First, we describe a design concept for AIBO based on how to increase its "lifelike" appearance. By introducing statistical results of marketing, and experiments involving human-robot interactions using AIBO, we show that this pet-type robot activates human emotions effectively. Furthermore, the experiments demonstrate that AIBO helps in human-human communication. We discuss the phenomena of interaction with AIBO, and attempt to explain why this happened.     

Analysis of effect of feedback current variation in CT Delta-Sigma modulators with Gm-C integrators

NAO humanoid robots are being used in many human-robot interaction applications. One of the important existing challenges is developing an accurate real-time face recognition system which does not require to have high computational cost. In this research work a real-time face recognition system by using block processing of local binary patterns of the face images captured by NAO humanoid is proposed. Majority voting and best score ensemble approaches have been used in order to boost the recognition results obtained in different colour channels of YUV colour space, which is a default colour space provided by the camera of NAO humanoid. The proposed method has been adopted on NAO humanoid and tested under real-world conditions. The recognition results were boosted in the real-time scenario by employing majority voting on the intra-sequence decisions with window size of 5. The experimental results are showing that the proposed face recognition algorithm overcomes the conventional and state-of-the-art techniques.     

Talking to Machines: Introducing Robot Perception to Resolve Speech Recognition Uncertainties

The use of spontaneous speech as a form of communication between humans and robots is a potential solution for more efficient human-robot interactions. Accuracy is one of the main problems associated with the automatic speech recognition (ASR) component of human-robot interactive systems. The standard ASR approach is based on statistical methods applied to phoneme domains. However, some problems cannot be solved with the rule-based approaches used so far; therefore, alternative strategies could be the solution. The aim of this paper is to investigate some aspects related to the use of a robot's perceptive abilities to increase the robustness of ASR components. The robot evaluative abilities are used to incrementally build knowledge that will be used during the recognition phase. This paper covers aspects concerning the use of time-warping algorithms to improve the speech recognition performance. In particular, aspects related to the accuracy and efficiency of this approach when applied to whole-sentence speech signals are discussed.     

A two-arm situated artificial communicator for human-robot cooperative assembly

We present the development of a robot system with some cognitive capabilities, as well as experimental results. We focus on two topics: assembly by two hands and understanding human instructions in nonconstrained natural language. These two features distinguish human beings from animals, and are, thus, the means leading to high-level intelligence. A typical application of such a system is a human-robot cooperative assembly. A human communicator sharing a view of the assembly scenario with the robot instructs the latter by speaking to it in the same way that he would communicate with a child whose common-sense knowledge is limited. His instructions can be underspecified, incomplete, and/or context dependent. After introducing the general purpose of our research project, we present the hardware and software components of our robots needed for interactive assembly tasks. We then discuss the control architecture of the robot system with two stationary robot arms by describing the functionalities of perception, instruction understanding, and execution. To show how our robot learns from humans, the implementations of a layered learning methodology, memory, and monitoring functions are introduced. Finally, we outline a list of future research topics related to the enhancement of such systems.     

Robot and its living space: A roadmap for robot development based on the view of living space

Robots, as the creation of humans, became an irreplaceable component in human society. With the advancement of technologies, robots have become more and more intelligent and have been widely used in many fields, such as disease diagnosis, customer services, healthcare for the older people, and so on. As robots made our lives much more convenient than ever before, they also brought many potential risks and challenges in technology, security, and ethic. To better understand the development of robots, we proposed a concept of a robot’s living space and analyzed the role of robots in our society. In this paper, we focus on setting a theoretical framework of the robot’s living space to further understand the human-robot relationship. The research in this paper contains three central aspects. First, we interpret the concept of the robot’s living space and the functions of each space. Second, we analyze and summarize the relative technologies which support robots living well in each space. Finally, we provide advice and improvement measures based on a discussion of potential problems caused by the developments of robots. With the trend of robots humanization and human-robot society integration, we should seriously consider how to collaborate with intelligent robots to achieve hybrid intelligence. To build a harmonious human-robot integrated society, studying the robot’s living space and its relationship with humans is the pre-requisite and roadmap.     

老年陪护机器人系统的设计与实现

设计了一种以引导式语音交互和肢体动作为表达形式的老年陪护机器人系统,该机器人系统采用人性化语音交互、触觉感知与交互技术,对使用者提供个性化服务.采用非特定人语音识别技术,利用有限的硬件资源,构建稳健的识别模型,实现高性能并满足实时应答要求的语音识别片上系统.在真实动物运动姿态建模的基础上,提出了一种应用欠驱动柔性结构的...     

基于体感识别的辅助倒水机器人控制系统

为了更好的给老年人及残疾人提供服务,基于体感识别技术,设计了辅助倒水机器人控制系统。首先通过Kinect摄像头获取人体景深图像,然后利用体感识别控制算法对人体姿势进行识别,最后将识别结果转换为控制命令传输到辅助倒水机器人使其执行相应动作。实验表明,辅助倒水机器人具有良好的操控及执行性能,能较好地满足老年人及残疾人的需求,对服务型机器人领域的发展研究具有很好的参考价值。     

How to achieve the huggable behavior of the social robot Probo? A reflection on the actuators

Most robots have a mechanical look or are covered with plastic or metallic shells. Their actuators are stiff which gives them not only an unnatural look, but also an unnatural touch. The goal of the huggable robot Probo is to serve as robotic research platform for human-robot interaction (HRI) studies with a special focus on children. Since not only cognitive interaction, but also physical interaction is targeted a new mechatronic design must be developed. To give Probo a huggable and safe behavior a new set of actuators is developed together with a triple layered protection cover which is presented in this paper. Probo’s soft touch is introduced, on the one side by use of novel passive compliant actuators, Compliant Bowden Cable Driven Actuators (CBCDAs), and on the other side by combining custom made servo motors, Non Back Drivable Servos (NBDSs), with flexible components and materials such as springs, silicon and foam. The working principle of the novel CBCDA is extensively described, together with experiments in order to determine its level of compliance and its bandwidth.     

An Artificial Motion and Tactile Receptor Constructed by Hyperelastic Double Physically Cross-Linked Silk Fibroin Ionoelastomer

Ionotronic artificial motion and tactile receptor (i-AMTR) is essential to realize an interactive human-machine interface. However, an i-AMTR that effectively mimics the composition, structure, mechanics, and multi-functionality of human skin, called humanoid i-AMTR, is yet to be developed. To bridge this technological gap, this study proposes a strategy that combines molecular structure design and function integration to construct a humanoid i-AMTR. Herein, a silk fibroin ionoelastomer (SFIE) with double cross-linked molecular structure is designed to mimic the composition and structure of human skin, thereby resolving the conflict of stretchability, softness, and resilience, suffered by many previously reported ionotronics. Functionally, electromechanical sensing and triboelectricity-based tactile perception are integrated into SFIE, to enable simultaneous perception of both motion and tactile inputs. By further leveraging the machine learning and Internet of Things (IoT) techniques, the proposed SFIE-based humanoid i-AMTR precisely senses the movement of human body and accurately sortball objects made of different materials. Notably, the success rate for 610 sorting tests reaches as high as 92.3%. These promising results essentially demonstrate a massive potential of humanoid i-AMTR in the fields of sorting robots, rehabilitation medicine, and augmented reality.     

A behavior-based mobile robot with a visual landmark-recognition system

In this paper, based on behavior-based artificial intelligence we have built a fully autonomous mobile robot. Several modules are developed for the mobile robot to implement different levels of competences and behaviors, where each module itself generates behaviors. New modules can be easily added to the robot system to improve in the competence without changing any existing modules. A vision-based landmark recognition system for robot navigation is developed as the highest layer in the subsumption architecture. A genetic-algorithm-based search method for pattern recognition of digital images is proposed and implemented to recognize artificial landmarks by searching all the predefined patterns. The vision layer is capable of generating the desired behaviors corresponding to various landmarks. A combination of eight ultrasonic sensors is designed to implement obstacle-avoidance behaviors through a set of fuzzy rules. The effectiveness of this behavior-based mobile robot is demonstrated by experimental studies.