Predicting Object Dynamics From Visual Images Through Active Sensing Experiences

Prediction of dynamic features is an important task for determining the manipulation strategies of an object. This paper presents a technique for predicting dynamics of objects relative to the robot's motion from visual images. During the training phase, the authors use the recurrent neural network with parametric bias (RNNPB) to self-organize the dynamics of objects manipulated by the robot into the PB space. The acquired PB values, static images of objects and robot motor values are input into a hierarchical neural network to link the images to dynamic features (PB values). The neural network extracts prominent features that each induce object dynamics. For prediction of the motion sequence of an unknown object, the static image of the object and robot motor value are input into the neural network to calculate the PB values. By inputting the PB values into the closed loop RNNPB, the predicted movements of the object relative to the robot motion are calculated recursively. Experiments were conducted with the humanoid robot Robovie-IIs pushing objects at different heights. The results of the experiment predicting the dynamics of target objects proved that the technique is efficient for predicting the dynamics of the objects.     

Self-organization of Dynamic Object Features Based on Bidirectional Training

This paper presents a method to self-organize object features that describe object dynamics using bidirectional training. The model is composed of a dynamics learning module and a feature extraction module. Recurrent Neural Network with Parametric Bias (RNNPB) is utilized for the dynamics learning module, learning and self-organizing the sequences of robot and object motions. A hierarchical neural network is linked to the input of RNNPB as the feature extraction module for self-organizing object features that describe the object motions. The two modules are simultaneously trained through bidirectional training using image and motion sequences acquired from the robot's active sensing with objects. Experiments are performed with the robot's pushing motion with a variety of objects to generate sliding, falling over, bouncing and rolling motions. The results have shown that the model is capable of self-organizing object dynamics based on the self-organized features.     

Classification of Known and Unknown Environmental Sounds Based on Self-Organized Space Using a Recurrent Neural Network

Our goal is to develop a system to learn and classify environmental sounds for robots working in the real world. In the real world, two main restrictions pertain in learning. (i) Robots have to learn using only a small amount of data in a limited time because of hardware restrictions. (ii) The system has to adapt to unknown data since it is virtually impossible to collect samples of all environmental sounds. We used a neuro-dynamical model to build a prediction and classification system. This neuro-dynamical model can self-organize sound classes into parameters by learning samples. The sound classification space, constructed by these parameters, is structured for the sound generation dynamics and obtains clusters not only for known classes, but also unknown classes. The proposed system searches on the basis of the sound classification space for classifying. In the experiment, we evaluated the accuracy of classification for both known and unknown sound classes.     

Learning to Walk by Imitation in Low-Dimensional Subspaces

In this paper, we provide the first demonstration that a humanoid robot can learn to walk directly by imitating a human gait obtained from motion capture (mocap) data without any prior information of its dynamics model. Programming a humanoid robot to perform an action (such as walking) that takes into account the robot's complex dynamics is a challenging problem. Traditional approaches typically require highly accurate prior knowledge of the robot's dynamics and environment in order to devise complex (and often brittle) control algorithms for generating a stable dynamic motion. Training using human mocap is an intuitive and flexible approach to programming a robot, but direct usage of mocap data usually results in dynamically unstable motion. Furthermore, optimization using high-dimensional mocap data in the humanoid full-body joint space is typically intractable. We propose a new approach to tractable imitation-based learning in humanoids without a robot's dynamic model. We represent kinematic information from human mocap in a low-dimensional subspace and map motor commands in this low-dimensional space to sensory feedback to learn a predictive dynamic model. This model is used within an optimization framework to estimate optimal motor commands that satisfy the initial kinematic constraints as best as possible while generating dynamically stable motion. We demonstrate the viability of our approach by providing examples of dynamically stable walking learned from mocap data using both a simulator and a real humanoid robot.     

Coordinated search for a lost target in a Bayesian world

This paper describes a decentralized Bayesian approach to the problem of coordinating multiple autonomous sensor platforms searching for a single non-evading target. In this architecture, each decision maker builds an equivalent representation of the probability density function (PDF) of the target state through a general decentralized Bayesian sensor network, enabling them to coordinate their actions without exchanging any information about their plans. The advantage of the approach is that a high degree of scalability and real-time adaptability can be achieved. The framework is implemented on a real-time high-fidelity multi-vehicle simulator system. The effectiveness of the method is demonstrated in different scenarios for a team of airborne search vehicles looking for both a stationary and a drifting target lost at sea.     

The ETL-Humanoid system—a high-performance full-body humanoid system for versatile real-world interaction

This paper presents the final stage of development of a humanoid system, ETL-Humanoid. It is a full-scale humanoid system with 46 d.o.f., with the height and weight of an average Japanese person. It was designed as an experimental platform to explore the general principle of controls of complex embodied systems. The complete system, the mechanical configuration of the system and the low-level network-based control system will be presented. The final system possesses properties of compactness, modularity and is light in weight. The mechanical system is high in performance, back-drivable and compliant, allowing the possibility of a wide range of motions and capabilities. The strength and power of the system is demonstrated through an experiment of 'chin up'. The humanlike configuration and compliant aspects of the system are demonstrated via a session of physical interaction. A running-like motion was generated to show the speed of the system. Aside from its human-like physical characteristics, the system is also capable of performing higher-level interaction involving perceptions and actions.     

Reinforcement learning of a continuous motor sequence with hidden states

Reinforcement learning is the scheme for unsupervised learning in which robots are expected to acquire behavior skills through self-explorations based on reward signals. There are some difficulties, however, in applying conventional reinforcement learning algorithms to motion control tasks of a robot because most algorithms are concerned with discrete state space and based on the assumption of complete observability of the state. Real-world environments often have partial observablility; therefore, robots have to estimate the unobservable hidden states. This paper proposes a method to solve these two problems by combining the reinforcement learning algorithm and a learning algorithm for a continuous time recurrent neural network (CTRNN). The CTRNN can learn spatio-temporal structures in a continuous time and space domain, and can preserve the contextual flow by a self-organizing appropriate internal memory structure. This enables the robot to deal with the hidden state problem. We carried out an experiment on the pendulum swing-up task without rotational speed information. As a result, this task is accomplished in several hundred trials using the proposed algorithm. In addition, it is shown that the information about the rotational speed of the pendulum, which is considered as a hidden state, is estimated and encoded on the activation of a context neuron.     

Sensor system of a small biped entertainment robot

SDR-4X II is the latest prototype model of a small biped entertainment robot. It is the improved model of SDR-4X. In this paper we report on the sensing system of this robot, which is important and essential for a small biped entertainment robot which will be used in the home environment. One technology is the design of the motion sensing system, i.e. the inclination sensor system and the force sensor system which obtains the inclination of the trunk and the foot with force. Another technology is the real-world sensing system. One aspect is the touch sensing system. The robot is used in a normal home environment, so we should strongly consider the safety aspects for human. Another is the vision sensor system. The configuration and the distance image acquisition are explained. Next is the audio sensor system which obtains the sound and the voice information. The hardware system and the direction recognition are explained. These sensing systems are the key to making the biped robot walking and dynamic motion highly stable, and understanding the real-world around the robot.     

Open-end human–robot interaction from the dynamical systems perspective: mutual adaptation and incremental learning

In this paper, we experimentally investigated the open-end interaction generated by the mutual adaptation between humans and robot. Its essential characteristic, incremental learning, is examined using the dynamical systems approach. Our research concentrated on the navigation system of a specially developed humanoid robot called Robovie and seven human subjects whose eyes were covered, making them dependent on the robot for directions. We used the usual feed-forward neural network (FFNN) without recursive connections and the recurrent neural network (RNN) for the robot control. Although the performances obtained with both the RNN and the FFNN improved in the early stages of learning, as the subject changed the operation by learning on its own, all performances gradually became unstable and failed. Next, we used a 'consolidation-learning algorithm' as a model of the hippocampus in the brain. In this method, the RNN was trained by both new data and the rehearsal outputs of the RNN not to damage the contents of current memory. The proposed method enabled the robot to improve performance even when learning continued for a long time (open-end). The dynamical systems analysis of RNNs supports these differences and also showed that the collaboration scheme was developed dynamically along with succeeding phase transitions.     

Real-Time Pose-Invariant Face Recognition Using the Efficient Second-Order Minimization and the Pose Transforming Matrix

We propose a real-time pose-invariant face recognition algorithm from a gallery of frontal images only. (i) We modified the second-order minimization method for the active appearance model (AAM). This allows the AAM to have the ability of correct convergence with little loss of frame rate. (ii) We proposed a pose transforming matrix that can eliminate warping artifacts of the warped face image from AAM fitting. This makes it possible to train a neural network as the face recognizer with one frontal face image of each person in the gallery set. (iii) We propose a simple method for pose recognition by using neural networks to select the proper pose transforming matrix. The proposed algorithm was evaluated on a set of 2000 facial images of 10 people (200 images for each person obtained at various poses), achieving a great improvement in recognition.     

Hardware implementation of neuromodulated neural network for a CPU-less autonomous mobile robot

In order to construct truly autonomous mobile robots, the concept of 'packaging' is indispensable; in packaging, all parts such as controllers, power systems and batteries should be embedded inside a finite physical space, i.e., a robot's body. Therefore, implementing a controller on hardware is one of the most promising ways, since this contributes to low power consumption, miniaturization, etc. Another crucial requirement in the field of autonomous mobile robots is robustness, i.e., autonomous mobile robots have to cope with their unpredictably changing environment in real-time. In this study, to meet these requirements, the concept of a dynamically rearrangeable electrical circuit (DREC) is proposed and we implement this onto field progammable gate arrays as physical electronic circuits by borrowing the idea from neuromodulation widely observed in biological nervous systems through the diffusion-reaction mechanism of neuromodulators. We developed the DREC for the peg-pushing task as a practical example. We confirmed that the physical DREC can successfully regulate the behavior according to the situation encountered by changing its properties in real-time.     

Bi-criteria Velocity Minimization of Robot Manipulators Using a Linear Variational Inequalities-Based Primal-Dual Neural Network and PUMA560 Example

In this paper, to diminish discontinuity points arising in the infinity-norm velocity minimization scheme, a bi-criteria velocity minimization scheme is presented based on a new neural network solver (i.e., a primal-dual neural network based on linear variational inequalities (LVI)). Such a kinematic control scheme of redundant manipulators can incorporate joint physical limits such as joint limits and joint velocity limits simultaneously. Moreover, the presented kinematic control scheme can be formulated as a quadratic programming (QP) problem. As a real-time QP solver, the LVI-based primal-dual neural network is established with a simple piecewise linear structure and higher computational efficiency. Computer simulations performed based on a PUMA560 manipulator are presented to illustrate the validity and advantages of such a bi-criteria neural control scheme for redundant robots.     

Online tracking and mimicking of human movements by a humanoid robot

This paper describes a humanoid robot system that can capture and mimic the motion of human body parts in real-time. The underlying vision system is able to automatically detect and track human body parts such as hands and faces in images captured by the robot's eyes. It is based on a probabilistic approach that can detect and track multiple blobs in a 60-Hz stereo image stream on a standard dual processor PC. A random jerk model is employed to approximate the observed human motion and a Kalman filter is used to estimate its parameters (three-dimensional positions, velocities and accelerations). This enables the system to realistically mimic the perceived motion in real-time.     

Designing a sociable humanoid robot for interdisciplinary research

This paper presents the humanoid robot BARTHOC and the smaller, but system-equal twin, BARTHOC Junior. Both robots have been developed to study human–robot interaction. The main focus of BARTHOC's design was to realize the expression and behavior of the robot to be as human-like as possible. This allows us to apply the platform to manifold research and demonstration areas. With its human-like look and mimic possibilities it differs from other platforms like ASIMO or QRIO and enables experiments even close to Mori's 'uncanny valley'. The paper describes details of the mechanical and electrical design of BARTHOC together with its PC control interface and an overview of the interaction architecture. Its humanoid appearance allows limited imitation of human behavior. The basic interaction software running on BARTHOC has been completely ported from a mobile robot except for some functionalities that could not be used due to hardware differences such as the lack of mobility. Based on these components, the robot's human-like appearance will enable us to study embodied interaction and to explore theories of human intelligence.     

Development of Biped Robot MARI-3 for Jumping

A biped robot, MARI-3, for jumping is developed, of which the ultimate objective is fast walking and running. Its mechanical structure including the joint configuration and specification, the knee joint, and the speed reduction mechanism are described in detail. A specific control system RON (RObot Network) for MARI-3 that is a serial and distributed network and consists of a microcontroller, host unit, servo units, sensor units and servo amplifiers is presented as well as the sensor system. With the developed biped robot MARI-3, one-leg jumping of 110 ms jumping time and 4.0 cm jumping height was implemented as initiative and verification experiments. Furthermore, by comparison of MARI-3 with other jumping or running robots, MARI-3's potential ability for fast walking, jumping and running becomes clear.     

iCub: the design and realization of an open humanoid platform for cognitive and neuroscience research

The development of robotic cognition and the advancement of understanding of human cognition form two of the current greatest challenges in robotics and neuroscience, respectively. The RobotCub project aims to develop an embodied robotic child (iCub) with the physical (height 90 cm and mass less than 23 kg) and ultimately cognitive abilities of a 2.5-year-old human child. The iCub will be a freely available open system which can be used by scientists in all cognate disciplines from developmental psychology to epigenetic robotics to enhance understanding of cognitive systems through the study of cognitive development. The iCub will be open both in software, but more importantly in all aspects of the hardware and mechanical design. In this paper the design of the mechanisms and structures forming the basic 'body' of the iCub are described. The papers considers kinematic structures dynamic design criteria, actuator specification and selection, and detailed mechanical and electronic design. The paper concludes with tests of the performance of sample joints, and comparison of these results with the design requirements and simulation projects.     

Modular Architecture for Humanoid Walking Pattern Prototyping and Experiments

In this paper we describe the use of design patterns as a basis for creating humanoid walking pattern generator software having a modular architecture. This architecture enabled the rapid porting of several novel walking algorithms on a full-size humanoid robot, HRP-2. The body of work currently available allows extracting a general software architecture usable with inter-exchange between simulations and real experiments. The proposed architecture with the associated design patterns is described together with several applications: a pattern generator for a HRP-2 with passive toe joints, a pattern for dynamically stepping over large obstacles and a new quadratic problem (QP) formulation for the generation of the reference zero-momentum point. Thanks to the versatility and the modularity of the proposed framework, the QP method has been implemented and experienced within 4 days only.     

Mechanical design of the humanoid robot platform,HUBO

The Korea Advanced Institute of Science and Technology (KAIST) humanoid robot-1 (KHR-1) was developed for the purpose of researching the walking action of bipeds. KHR-1, which has no hands or head, has 21 d.o.f.: 12 d.o.f. in the legs, 1 d.o.f. in the torso and 8 d.o.f. in the arms. The second version of this humanoid robot, KHR-2 (which has 41 d.o.f.) can walk on a living-room floor; it also moves and looks like a human. The third version, KHR-3 (HUBO), has more human-like features, a greater variety of movements and a more human-friendly character. We present the mechanical design of HUBO, including the design concept, the lower-body design, the upper-body design and the actuator selection of joints. Previously we developed and published details of KHR-1 and KHR-2. The HUBO platform, which is based on KHR-2, has 41 d.o.f., stands 125 cm tall and weighs 55 kg. From a mechanical point of view, HUBO has greater mechanical stiffness and a more detailed frame design than KHR-2. The stiffness of the frame was increased, and the detailed design around the joints and link frame was either modified or fully redesigned. We initially introduced an exterior art design concept for KHR-2 and that concept was implemented in HUBO at the mechanical design stage.     

Production of Various Vocal Cord Vibrations Using a Mechanical Model for an Anthropomorphic Talking Robot

We developed a three-dimensional mechanical vocal cord model for Waseda Talker No. 7 (WT-7), an anthropomorphic talking robot, for generating speech sounds with various voice qualities. The vocal cord model is a cover model that has two thin folds made of thermoplastic material. The model self-oscillates by airflow exhausted from the lung model and generates the glottal sound source, which is fed into the vocal tract for generating the speech sound. Using the vocal cord model, breathy and creaky voices, as well as the modal (normal) voice, were produced in a manner similar to the human laryngeal control. The breathy voice is characterized by a noisy component mixed with the periodic glottal sound source and the creaky voice is characterized by an extremely low-pitch vibration. The breathy voice was produced by adjusting the glottal opening and generating the turbulence noise by the airflow just above the glottis. The creaky voice was produced by adjusting the vocal cord tension, the sub-glottal pressure and the vibration mass so as to generate a double-pitch vibration with a long pitch interval. The vocal cord model used to produce these voice qualities was evaluated in terms of the vibration pattern as measured by a high-speed camera, the glottal airflow and the acoustic characteristics of the glottal sound source, as compared to the data for a human.