Design of walking gaits for Tao-Pie-Pie,a small humanoid robot

A new way for multi-axis robot trajectory planning using a single cubic spline incorporating velocity and acceleration clipping is presented. Equations for velocity and acceleration clipping employing the cubic spline function for a single axis are derived. A robot tool-tip velocity vector magnitude clipping algorithm is proposed. Implementation for a fly-by and contour following trajectory control is discussed.     

Hybrid autonomous control for multi mobile robots

Reinforcement learning can be an adaptive and flexible control method for autonomous system. It does not need a priori knowledge; behaviors to accomplish given tasks are obtained automatically by repeating trial and error. However, with increasing complexity of the system, the learning costs are increased exponentially. Thus, application to complex systems, like a many redundant d.o.f. robot and multi-agent system, is very difficult. In the previous works in this field, applications were restricted to simple robots and small multi-agent systems, and because of restricted functions of the simple systems that have less redundancy, effectiveness of reinforcement learning is restricted. In our previous works, we had taken these problems into consideration and had proposed new reinforcement learning algorithm, 'Q-learning with dynamic structuring of exploration space based on GA (QDSEGA)'. Effectiveness of QDSEGA for redundant robots has been demonstrated using a 12-legged robot and a 50-link manipulator. However, previous works on QDSEGA were restricted to redundant robots and it was impossible to apply it to multi mobile robots. In this paper, we extend our previous work on QDSEGA by combining a rule-based distributed control and propose a hybrid autonomous control method for multi mobile robots. To demonstrate the effectiveness of the proposed method, simulations of a transportation task by 10 mobile robots are carried out. As a result, effective behaviors have been obtained.     

Efficient Behavior Learning Based on State Value Estimation of Self and Others

The existing reinforcement learning methods have been seriously suffering from the curse of the dimension problem, especially when they are applied to multiagent dynamic environments. One of the typical examples is a case of RoboCup competitions since other agents and their behavior easily cause state and action space variations. This paper presents a method of modular learning in a multiagent environment by which the learning agent can acquire cooperative behavior with its teammates and competitive behavior against its opponents. The key ideas to resolve the issue are as follows. First, a two-layer hierarchical system with multilearning modules is adopted to reduce the size of the sensor and action spaces. The state space of the top layer consists of the state values from the lower level and the macro actions are used to reduce the size of the physical action space. Second, the state of the other, to what extent it is close to its own goal, is estimated by observation and used as a state variable in the top layer state space to realize the cooperative/competitive behavior. The method is applied to a four (defense team)-on-five (offense team) game task and the learning agent (a passer of the offense team) successfully acquired the teamwork plays (pass and shoot) within much shorter learning time.     

morph3: a compact-size humanoid robot system capable of acrobatic behavior

morph3 is a compact-size humanoid robot that is able to generate acrobatic motions such as somersaults and gymnastic motion. A new mechanism and control system, i. e. a real-time joint compliance control system and tactile sensing shells on its body, make acrobatic motion possible. In this paper, the design concepts and construction of morph3 are described.     

Reinforcement learning of humanoid rhythmic walking parameters based on visual information

This paper presents a method for learning the parameters of rhythmic walking to generate purposive humanoid motions. The controller consists of the two layers: rhythmic walking is realized by the lower layer, which adjusts the speed of the phase on the desired trajectory depending on sensory information, and the upper layer learns (i) the feasible parameter sets that enable stable walking, (ii) the causal relationship between the walking parameters to be given to the lower-layer controller and the change in the sensory information and (iii) the feasible rhythmic walking parameters by reinforcement learning so that a robot can reach the goal based on visual information. The experimental results show that a real humanoid learns to reach the ball and to shoot it into the goal in the context of the RoboCup soccer competition, and the further issues are discussed.     

Position-based impedance control of a biped humanoid robot

This paper describes position-based impedance control for biped humanoid robot locomotion. The impedance parameters of the biped leg are adjusted in real-time according to the gait phase. In order to reduce the impact/contact forces generated between the contacting foot and the ground, the damping coefficient of the impedance of the landing foot is increased largely during the first half double support phase. In the last half double support phase, the walking pattern of the leg changed by the impedance control is returned to the desired walking pattern by using a polynomial. Also, the large stiffness of the landing leg is given to increase the momentum reduced by the viscosity of the landing leg in the first half single support phase. For the stability of the biped humanoid robot, a balance control that compensates for moments generated by the biped locomotion is employed during a whole walking cycle. For the confirmation of the impedance and balance control, we have developed a life-sized humanoid robot, WABIAN-RIII, which has 43 mechanical d.o.f. Through dynamic walking experiments, the validity of the proposed controls is verified.     

Motion capture-based wearable interaction system and its application to a humanoid robot,AMIO

In this paper, we present a wearable interaction system to enhance interaction between a human user and a humanoid robot. The wearable interaction system assists the user and enhances interaction with the robot by intuitively imitating the user motion while expressing multimodal commands to the robot and displaying multimodal sensory feedback. AMIO, the biped humanoid robot of the AIM Laboratory, was used in experiments to confirm the performance and effectiveness of the proposed system, including the overall performance of motion tracking. Through an experimental application of this system, we successfully demonstrated human and humanoid robot interactions.     

Development of soft and distributed tactile sensors and the application to a humanoid robot

This paper describes a comprehensive tactile sensor system which can cover wide areas of full-body robots. Based on design criteria which are introduced from requirements, we develop two types of tactile sensor elements. One is a multi-valued touch sensor which has multi-level pressure thresholds. It is capable of covering wide areas of robot surfaces. The other is made of soft, conductive gel, which has the advantage of compliance compared with other sheet-type tactile sensors. With these two types sensors, we develop the tactile sensor system on the full-body robot 'H4'. Details of the sensor system on the robot and some experiments using tactile information are described.     

Designing a sociable humanoid robot for interdisciplinary research

This paper presents the humanoid robot BARTHOC and the smaller, but system-equal twin, BARTHOC Junior. Both robots have been developed to study human–robot interaction. The main focus of BARTHOC's design was to realize the expression and behavior of the robot to be as human-like as possible. This allows us to apply the platform to manifold research and demonstration areas. With its human-like look and mimic possibilities it differs from other platforms like ASIMO or QRIO and enables experiments even close to Mori's 'uncanny valley'. The paper describes details of the mechanical and electrical design of BARTHOC together with its PC control interface and an overview of the interaction architecture. Its humanoid appearance allows limited imitation of human behavior. The basic interaction software running on BARTHOC has been completely ported from a mobile robot except for some functionalities that could not be used due to hardware differences such as the lack of mobility. Based on these components, the robot's human-like appearance will enable us to study embodied interaction and to explore theories of human intelligence.     

Learning to Acquire Whole-Body Humanoid Center of Mass Movements to Achieve Dynamic Tasks

This paper presents a novel approach for acquiring dynamic whole-body movements on humanoid robots focused on learning a control policy for the center of mass (CoM). In our approach, we combine both a model-based CoM controller and a model-free reinforcement learning (RL) method to acquire dynamic whole-body movements in humanoid robots. (i) To cope with high dimensionality, we use a model-based CoM controller as a basic controller that derives joint angular velocities from the desired CoM velocity. The balancing issue can also be considered in the controller. (ii) The RL method is used to acquire a controller that generates the desired CoM velocity based on the current state. To demonstrate the effectiveness of our approach, we apply it to a ball-punching task on a simulated humanoid robot model. The acquired whole-body punching movement was also demonstrated on Fujitsu's Hoap-2 humanoid robot.     

Experimental realization of dynamic walking of the biped humanoid robot KHR-2 using zero moment point feedback and inertial measurement

This paper describes a novel control algorithm for dynamic walking of biped humanoid robots. For the test platform, we developed KHR-2 (KAIST Humanoid Robot-2) according to our design philosophy. KHR-2 has many sensory devices analogous to human sensory organs which are particularly useful for biped walking control. First, for the biped walking motion, the motion control architecture is built and then an appropriate standard walking pattern is designed for the humanoid robots by observing the human walking process. Second, we define walking stages by dividing the walking cycle according to the characteristics of motions. Third, as a walking control strategy, three kinds of control schemes are established. The first scheme is a walking pattern control that modifies the walking pattern periodically based on the sensory information during each walking cycle. The second scheme is a real-time balance control using the sensory feedback. The third scheme is a predicted motion control based on a fast decision from the previous experimental data. In each control scheme, we design online controllers that are capable of maintaining the walking stability with the control objective by using force/torque sensors and an inertial sensor. Finally, we plan the application schedule of online controllers during a walking cycle according to the walking stages, accomplish the walking control algorithm and prove its effectiveness through experiments with KHR-2.     

Final-state control of a two-link cat robot

In this study, we deal with the twisting motion of a falling cat robot by means of two torque inputs around her waist. The cat robot consists of two rigid columns and has two internal actuators at the joint to generate torque inputs around normal coordinates. This system is a nonholonomic system whose angular momentum is conserved. We formulate the state equation that has torque inputs to the joint by using the nonholonomic constraint and the Lagrange-d'Alembert principle. Then, we transform the system into a linear parameter varying system. In order to improve error learning of a final-state control method, we provide the initial inputs in order to determine the appropriate rotation direction in the early stage of the twisting motion. Next, we introduce the method of the artificial potential function to the final-state control in order to make the maximum bending angle small. The feedforward torque inputs can be obtained by the final-state control in order to bring the system from the initial state to the final state in the desired time. In simulations, we also demonstrate that the twolink cat robot can land on her feet by using the 2-d.o.f. control system even when her waist damping coefficient varies.     

Online tracking and mimicking of human movements by a humanoid robot

This paper describes a humanoid robot system that can capture and mimic the motion of human body parts in real-time. The underlying vision system is able to automatically detect and track human body parts such as hands and faces in images captured by the robot's eyes. It is based on a probabilistic approach that can detect and track multiple blobs in a 60-Hz stereo image stream on a standard dual processor PC. A random jerk model is employed to approximate the observed human motion and a Kalman filter is used to estimate its parameters (three-dimensional positions, velocities and accelerations). This enables the system to realistically mimic the perceived motion in real-time.     

Self-Learning for a Humanoid Robotic Ping-Pong Player

Imitating the learning process of a human playing ping-pong is extremely complex. This work proposes a suitable learning strategy. First, an inverse kinematics solution is presented to obtain the smooth joint angles of a redundant anthropomorphic robot arm in order to imitate the paddle motion of a human ping-pong player. As humans instinctively determine which posture is suitable for striking a ball, this work proposes two novel processes: (i) estimating ball states and predicting trajectory using a fuzzy adaptive resonance theory network, and (ii) self-learning the behavior for each strike using a self-organizing map-based reinforcement learning network that imitates human learning behavior. Experimental results demonstrate that the proposed algorithms work effectively when applied to an actual humanoid robot playing ping-pong.     

An Anthropomorphic Robotic Platform for Progressive and Adaptive Sensorimotor Learning

In recent years, advances and improvements in engineering and robotics have in part been due to strengthened interactions with the biological sciences. Robots that mimic the complexity and adaptability of biological systems have become a central goal in research and development in robotics. Usually, such a collaboration is addressed to a 2-fold perspective of (i) setting up anthropomorphic platforms as test beds for studies in neuroscience and (ii) promoting new mechatronic and robotic technologies for the development of bio-inspired or humanoid high-performance robotic platforms. This paper provides a brief overview of recent studies on sensorimotor coordination in human motor control and proposes a novel paradigm of adaptive learning for sensorimotor control, based on a multi-network high-level control architecture. The proposed neurobiologically inspired model has been applied to a robotic platform, purposely designed to provide anthropomorphic solutions to neuroscientific requirements. The goal of this work is to use the bio-inspired robotic platform as a test bed for validating the proposed model of high-level sensorimotor control, with the aim of demonstrating adaptive and modular control based on acquired competences, with a higher degree of flexibility and generality than conventional robotic controllers, while preserving their robustness. To this purpose, a set of object-dependent, visually guided reach-and-grasp tasks and the associated training phases were first implemented in a multi-network control architecture in simulation. Subsequently, the offline learning realized in simulation was used to produce the input command of reach-and-grasp to the low-level position control of the robotic platform. Experimental trials demonstrated that the adaptive and modular high-level control allowed reaching and grasping of objects located at different positions and objects of variable size, shape and orientation. A future goal would be to address autonomous and progressive learning based on growing competences.     

The development and control of a flexible-spine for a human-form robot

In this paper, the development of a robot which has a flexible spine is presented. By embedding a multi-d.o.f. soft structure into a robot body as a spine, the robot can increase its ability to absorb shock and to work in various environment such as narrow places. As a result of these abilities, the robot can expand its opportunity to work in the human environment. Moreover, its motion could be more natural. The developed full-body human-form robot has a five-jointed flexible spine. Each joint (vertebra) has 3 d.o.f. Between each vertebrae is a 'disk' made of silicone rubber. The spine is controlled by eight tendons, whose tensions can be controlled using tension sensors and locally distributed microcontrollers. This paper describes the development of the flexible spine and the control of the posture of the spine and body.     

Policy gradient learning for a humanoid soccer robot

In humanoid robotic soccer, many factors, both at low-level (e.g., vision and motion control) and at high-level (e.g., behaviors and game strategies), determine the quality of the robot performance. In particular, the speed of individual robots, the precision of the trajectory, and the stability of the walking gaits, have a high impact on the success of a team. Consequently, humanoid soccer robots require fine tuning, especially for the basic behaviors. In recent years, machine learning techniques have been used to find optimal parameter sets for various humanoid robot behaviors. However, a drawback of learning techniques is time consumption: a practical learning method for robotic applications must be effective with a small amount of data. In this article, we compare two learning methods for humanoid walking gaits based on the Policy Gradient algorithm. We demonstrate that an extension of the classic Policy Gradient algorithm that takes into account parameter relevance allows for better solutions when only a few experiments are available. The results of our experimental work show the effectiveness of the policy gradient learning method, as well as its higher convergence rate, when the relevance of parameters is taken into account during learning.     

Mechanical design of the humanoid robot platform,HUBO

The Korea Advanced Institute of Science and Technology (KAIST) humanoid robot-1 (KHR-1) was developed for the purpose of researching the walking action of bipeds. KHR-1, which has no hands or head, has 21 d.o.f.: 12 d.o.f. in the legs, 1 d.o.f. in the torso and 8 d.o.f. in the arms. The second version of this humanoid robot, KHR-2 (which has 41 d.o.f.) can walk on a living-room floor; it also moves and looks like a human. The third version, KHR-3 (HUBO), has more human-like features, a greater variety of movements and a more human-friendly character. We present the mechanical design of HUBO, including the design concept, the lower-body design, the upper-body design and the actuator selection of joints. Previously we developed and published details of KHR-1 and KHR-2. The HUBO platform, which is based on KHR-2, has 41 d.o.f., stands 125 cm tall and weighs 55 kg. From a mechanical point of view, HUBO has greater mechanical stiffness and a more detailed frame design than KHR-2. The stiffness of the frame was increased, and the detailed design around the joints and link frame was either modified or fully redesigned. We initially introduced an exterior art design concept for KHR-2 and that concept was implemented in HUBO at the mechanical design stage.     

Vision-based hybrid control scheme for autonomous parking of a mobile robot

This paper presents a novel vision-based hybrid controller for parking of mobile robots. Parking or docking is an essential behavioral unit for autonomous robots. The proposed hybrid controller comprises a discrete event controller to change the direction of travel and a pixel error-driven proportional controller to generate motion commands to achieve the continuous motion. At the velocity control level, the robot is driven using a built-in PID control system. The feedback system uses image plane measurements in pixel units to perform image-based visual servoing (IBVS). The constraints imposed due to the non-holonomic nature of the robot and the limited field of view of the camera are taken into account in designing the IBVS-based controller. The controller continuously compares the current view of the parking station against the reference view until the desired parking condition is achieved. A comprehensive analysis is provided to prove the convergence of the proposed method. Once the parking behavior is invoked, the robot has the ability to start from any arbitrary position to achieve successful parking given that initially the parking station is in the robot's field of view. As the method is purely based on vision the hybrid controller does not require any position information (or localization) of the robot. Using the Pioneer 3AT robot, several experiments are carried out to authenticate the method. The experimental system has the ability to achieve the parking state and align laterally within ±0.5 cm of the target pose.