类人足球机器人决策系统的设计

类人机器人足球比赛是机器人足球比赛的最高赛事.类人足球机器人的决策系统是基于独立视觉的自主决策系统,很大程度上决定着比赛的胜败.介绍了自主研发的类人足球机器人决策系统的架构及实现方法,并在此基础上运用有限状态机理论,对单个机器人的自主进攻策略进行了详细分析和研究,真实环境中的实验及比赛结果证明了其有效性.该决策系统的设计及研究工作对基于自主决策的多智能体协作以及服务性机器人决策系统的研究都具有重要的价值.     

Self-modeling in humanoid soccer robots

In this paper we discuss the applicability, potential benefits, open problems and expected contributions that an emerging set of self-modeling techniques might bring on the development of humanoid soccer robots. The idea is that robots might continuously generate, validate and adjust physical models of their sensorimotor interaction with the world. These models are exploited for adapting behavior in simulation, enhancing the learning skills of a robot with the regular transference of controllers developed in simulation to reality. Moreover, these simulations can be used to aid the execution of complex sensorimotor tasks, speed up adaptation and enhance task planning. We present experiments on the generation of behaviors for humanoid soccer robots using the Back-to-Reality algorithm. General motivations are presented, alternative algorithms are discussed and, most importantly, directions of research are proposed.     

Direction-changing fall control of humanoid robots: theory and experiments

Humanoid robots are expected to share human environments in the future and it is important to ensure the safety of their operation. A serious threat to safety is the fall of such robots, which can seriously damage the robot itself as well as objects in its surrounding. Although fall is a rare event in the life of a humanoid robot, the robot must be equipped with a robust fall strategy since the consequences of fall can be catastrophic. In this paper we present a strategy to change the default fall direction of a robot, during the fall. By changing the fall direction the robot may avoid falling on a delicate object or on a person. Our approach is based on the key observation that the toppling motion of a robot necessarily occurs at an edge of its support area. To modify the fall direction the robot needs to change the position and orientation of this edge vis-a-vis the prohibited directions. We achieve this through intelligent stepping as soon as the fall is predicted. We compute the optimal stepping location which results in the safest fall. Additional improvement to the fall controller is achieved through inertia shaping, which is a principled approach aimed at manipulating the robot’s centroidal inertia, thereby indirectly controlling its fall direction. We describe the theory behind this approach and demonstrate our results through simulation and experiments of the Aldebaran NAO H25 robot. To our knowledge, this is the first implementation of a controller that attempts to change the fall direction of a humanoid robot.     

Adequate motion simulation and collision detection for soccer playing humanoid robots

In this paper a humanoid robot simulator based on the multi-robot simulation framework (MuRoSimF) is presented. Among the unique features of this simulator is the scalability in the level of physical detail in both the robot’s motion and sensing systems. It facilitates the development of control software for humanoid robots which is demonstrated for several scenarios from the RoboCup Humanoid Robot League.Different requirements exist for a humanoid robot simulator. E.g., testing of algorithms for motion control and postural stability require high fidelity of physical motion properties whereas testing of behavior control and role distribution for a robot team requires only a moderate level of detail for real-time simulation of multiple robots. To meet such very different requirements often different simulators are used which makes it necessary to model a robot multiple times and to integrate different simulations with high-level robot control software.MuRoSimF provides the capability of exchanging the simulation algorithms used for each robot transparently, thus allowing a trade-off between computational performance and fidelity of the simulation. It is therefore possible to choose different simulation algorithms which are adequate for the needs of a given simulation experiment, for example, motion simulation of humanoid robots based on kinematical, simplified dynamics or full multi-body system dynamics algorithms. In this paper also the sensor simulation capabilities of MuRoSimF are revised. The methods for motion simulation and collision detection and handling are presented in detail including an algorithm which allows the real-time simulation of the full dynamics of a 21 DOF humanoid robot. Merits and drawbacks of the different algorithms are discussed in the light of different simulation purposes. The simulator performance is measured and illustrated in various examples, including comparison with experiments of a physical humanoid robot.     

A dynamic simulator for humanoid robots

Computer simulation is an essential step in the design and construction of various mechanical structures, including biped robots, because it enables rapid testing and virtual prototyping during the construction phase. Although many different simulators are available, this article gives an overview and a motivation for building a new dynamic multibody simulator. The simulator is especially adapted to humanoid robot Archie, developed at the IHRT Institute at the Technical University of Vienna. In addition, it is shown how the simulator can be used not only in the controller design, but also in the online control loop to extend the available sensors: a virtual sensors principle. This work was presented in part at the First European Workshop on Artificial Life and Robtics, Vienna, Austria, July 12–13, 2007     

Clock-turning gait synthesis for humanoid robots

Turning gait is a basic motion for humanoid robots. This paper presents a method for humanoid tuming, i.e. clock-turning. The objective of clock-turning is to change robot direction at a stationary spot. The clock-turning planning consists of four steps： ankle trajectory generation, hip trajectory generation, knee trajectory generation, and inverse kinematics calculation. Our proposed method is based on a typical humanoid structure with 12 DOFs （degrees of freedom）. The final output of clock-turning planning is 12 reference trajectories, which are used to control a humanoid robot with 12 DOFs. ZMP （zero moment point） is used as stability criterion for the planning. Simulation experiments are conducted to verify the effectiveness of our proposed clock-turuing method.     

Intelligent proximal-policy-optimization-based decision-making system for humanoid robots

With the advancements in technology, robots have gradually replaced humans in different aspects. Allowing robots to handle multiple situations simultaneously and perform different actions depending on the situation has since become a critical topic. Currently, training a robot to perform a designated action is considered an easy task. However, when a robot is required to perform actions in different environments, both resetting and retraining are required, which are time-consuming and inefficient. Therefore, allowing robots to autonomously identify their environment can significantly reduce the time consumed. How to employ machine learning algorithms to achieve autonomous robot learning has formed a research trend in current studies. In this study, to solve the aforementioned problem, a proximal policy optimization algorithm was used to allow a robot to conduct self-training and select an optimal gait pattern to reach its destination successfully. Multiple basic gait patterns were selected, and information-maximizing generative adversarial nets were used to generate gait patterns and allow the robot to choose from numerous gait patterns while walking. The experimental results indicated that, after self-learning, the robot successfully made different choices depending on the situation, verifying this approach’s feasibility.     

Reflexive stability control framework for humanoid robots

In this paper we propose a general control framework for ensuring stability of humanoid robots, determined through a normalized zero-moment-point (ZMP). The proposed method is based on the modified prioritized kinematic control, which allows smooth and continuous transition between priorities. This, as long as the selected criterion is met, allows arbitrary joint movement of a robot without any regard of the consequential movement of the ZMP. On the other hand, it constrains the movement when the criterion approaches a critical condition. The critical condition thus triggers a reflexive, subconscious behavior, which has a higher priority than the desired, conscious movement. The transition between the two is smooth and reversible. Furthermore, the switching is encapsulated in a single modified prioritized task control equation. We demonstrate the properties of the algorithm on two human-inspired robots developed in our laboratory; a human-inspired leg-robot used for imitating human movement and a skiing robot.     

Robo-Erectus: a low-cost autonomous humanoid soccer robot

An exoskeleton robot can replace the wearer's motion function by operating the human's body. The purpose of this study is to propose a power assist method of walking, standing up and going up stairs based on autonomous motion of the exoskeleton robot suit, HAL (Hybrid assistive Limb), and verify the effectiveness of this method by experiment. In order to realize power assist of tasks (walking, standing up and going up stairs) autonomically, we used the Phase Sequence control which generates a task by transiting some simple basic motions called Phases. A task was divided into some Phases on the basis of the task performed by a normal person. The joint moving modes were categorized into active, passive and free modes according to the characteristic of the muscle force conditions. The autonomous motions which HAL generates in each Phase were designed corresponding to one of the categorized modes. The power assist experiments were performed by using the autonomous motion with a focus on the active mode. The experimental results showed that the wearer's muscle activation levels in each Phase were significantly reduced. With this, we confirmed the effectiveness of the proposed assist method.     

Policy gradient learning for a humanoid soccer robot

In humanoid robotic soccer, many factors, both at low-level (e.g., vision and motion control) and at high-level (e.g., behaviors and game strategies), determine the quality of the robot performance. In particular, the speed of individual robots, the precision of the trajectory, and the stability of the walking gaits, have a high impact on the success of a team. Consequently, humanoid soccer robots require fine tuning, especially for the basic behaviors. In recent years, machine learning techniques have been used to find optimal parameter sets for various humanoid robot behaviors. However, a drawback of learning techniques is time consumption: a practical learning method for robotic applications must be effective with a small amount of data. In this article, we compare two learning methods for humanoid walking gaits based on the Policy Gradient algorithm. We demonstrate that an extension of the classic Policy Gradient algorithm that takes into account parameter relevance allows for better solutions when only a few experiments are available. The results of our experimental work show the effectiveness of the policy gradient learning method, as well as its higher convergence rate, when the relevance of parameters is taken into account during learning.     

Whole-body hierarchical motion and force control for humanoid robots

Robots acting in human environments usually need to perform multiple motion and force tasks while respecting a set of constraints. When a physical contact with the environment is established, the newly activated force task or contact constraint may interfere with other tasks. The objective of this paper is to provide a control framework that can achieve real-time control of humanoid robots performing both strict and non strict prioritized motion and force tasks. It is a torque-based quasi-static control framework, which handles a dynamically changing task hierarchy with simultaneous priority transitions as well as activation or deactivation of tasks. A quadratic programming problem is solved to maintain desired task hierarchies, subject to constraints. A generalized projector is used to quantitatively regulate how much a task can influence or be influenced by other tasks through the modulation of a priority matrix. By the smooth variations of the priority matrix, sudden hierarchy rearrangements can be avoided to reduce the risk of instability. The effectiveness of this approach is demonstrated on both a simulated and a real humanoid robot.     

Towards stratified model-based environmental visual perception for humanoid robots

An autonomous environmental visual perception approach for humanoid robots is presented. The proposed framework exploits the available model information and the context acquired during global localization by establishing a vision-model coupling in order to overcome the limitations of purely data-driven approaches in object recognition and surrounding status assertion. The exploitation of the model-vision coupling through the properceptive components is the key element to solve complex visual assertion-queries with proficient performance. An experimental evaluation with the humanoid robot ARMAR-IIIa is presented.     

Cooperative transportation system for humanoid robots using simulation-based learning

In this paper, an approach to the behavior acquisition required for humanoid robots to carry out a cooperative transportation task is proposed. In the case of object transportation involving two humanoid robots, mutual position shifts may occur due to the body swinging of the robots. Therefore, it is necessary to correct the position in real-time. Developing the position shift correction system requires a great deal of effort. Solution to the problem of learning the required behaviors is obtained by using the Classifier System and Q-Learning. The successful cooperation of two HOAP-1 humanoid robots in the transportation task has been confirmed by several experimental results.     

A new active visual system for humanoid robots.

In this paper, a new active visual system is developed, which is based on bionic vision and is insensitive to the property of the cameras. The system consists of a mechanical platform and two cameras. The mechanical platform has two degrees of freedom of motion in pitch and yaw, which is equivalent to the neck of a humanoid robot. The cameras are mounted on the platform. The directions of the optical axes of the two cameras can be simultaneously adjusted in opposite directions. With these motions, the object's images can be located at the centers of the image planes of the two cameras. The object's position is determined with the geometry information of the visual system. A more general model for active visual positioning using two cameras without a neck is also investigated. The position of an object can be computed via the active motions. The presented model is less sensitive to the intrinsic parameters of cameras, which promises more flexibility in many applications such as visual tracking with changeable focusing. Experimental results verify the effectiveness of the proposed methods.     

Policy gradient learning for quadruped soccer robots

In real-world robotic applications, many factors, both at low level (e.g., vision, motion control and behaviors) and at high level (e.g., plans and strategies) determine the quality of the robot performance. Consequently, fine tuning of the parameters, in the implementation of the basic functionalities, as well as in the strategic decisions, is a key issue in robot software development. In recent years, machine learning techniques have been successfully used to find optimal parameters for typical robotic functionalities. However, one major drawback of learning techniques is time consumption: in practical applications, methods designed for physical robots must be effective with small amounts of data. In this paper, we present a method for concurrent learning of best strategy and optimal parameters using policy gradient reinforcement learning algorithm. The results of our experimental work in a simulated environment and on a real robot show a very high convergence rate.     

Biomimetic pupils for augmenting eye emulation in humanoid robots

Contemporary approaches in the development of humanoid robots continually neglect holistic nuances, particularly in ocular prosthetic design. The standard solid glass and acrylic eye construction techniques implemented in humanoid robot design present the observer with an inaccurate representation of the natural human eye by utilising hardened synthetic materials which prohibit pupillary dynamics. Precise eye emulation is an essential factor in the development of a greater realistic humanoid robot as misrepresentation in ocular form and function will appear distinctly prevalent during proximity face to face communication as eye contact is the primary form of interpersonal communicative processing. This paper explores a new material approach in the development of a more accurate humanoid robotic eye construction by employing natural compounds similar in structure to that found in the organic human eye to replace the traditional glass and acrylic modelling techniques. Furthermore, this paper identifies a gap in current ocular system design as no robotic eye model can accurately replicate all the natural operations of the human iris simultaneously in reaction to light and emotive responsivity. This paper offers a new system design approach to augment future humanoid robot eye construction towards achieving a greater accurate and naturalistic eye emulation.     

A combined approach for improving humanoid robots autonomous cognitive capabilities

Recent technologies advancements promise to change our lives dramatically in the near future. A new different living society is progressively emerging, witnessed from the conception of novel digital ecosystems, where humans are expected to share their own spaces and habits with machines. Humanoid robots are more and more being developed and provided with enriched functionalities; however, they are still lacking in many ways. One important goal in this sense is to enrich their cognitive capabilities, to make them more “intelligent” in order to better support humans in both daily and special activities. The goal of this research is to set a step in bridging the gap between symbolic AI and connectionist approaches in the context of knowledge acquisition and conceptualization. Hence, we present a combined approach based on semantics and machine learning techniques for improving robots cognitive capabilities. This is part of a wider framework that covers several aspects of knowledge management, from representation and conceptualization, to acquisition, sharing and interaction with humans. Our focus in this work is in particular on the development and implementation of techniques for knowledge acquisition. Such techniques are discussed and validated through experiments, carried out on a real robotic platform, showing the effectiveness of our approach. The results obtained confirmed that the combination of the approaches gives superior performance with respect to when they are considered individually.

     

Evolutionary design of Sugeno-type fuzzy systems for modelling humanoid robots

An evolutionary design of Sugeno-type fuzzy systems for modelling humanoid robots is presented in this article, and issues related to the determination of the antecedent and consequent structures of the fuzzy model are addressed. In the design of the fuzzy model, determination of the type, the number of membership functions assigned to the input variables, the types of consequent equations for the fuzzy rules, the optimal number of input variables, and the dominant input variables among the input candidates are carried out using evolutionary algorithms. Using these algorithms, proper structures are evolved for the antecedent and the consequent of the Sugeno-type fuzzy model. Simulations are performed to show the effectiveness of the developed method when applied to a humanoid robot system with strong nonlinearities that have 10 input candidates.     

A novel compound biped locomotion algorithm for humanoid robots to realize biped walking

In this paper, a compound biped locomotion algorithm for a humanoid robot under development is presented. This paper is organized in two main parts. In the first part, it mainly focuses on the structural design for the humanoid. In the second part, the compound biped locomotion algorithm is presented based on the reference motion and reference Zero Moment Point （ZMP）. This novel algorithm includes calculation of the upper body motion and trajectory of the Center of Gravity （COG） of the robot. First, disturbances from the environment are eliminated by the compensational movement of the upper body; then based on the error between a reference ZMP and the real ZMP as well as the relation between ZMP and CoG, the CoG error is calculated, thus leading to the CoG trajectory. Then, the motion of the robot converges to its reference motion, generating stable biped walking. Because the calculation of upper body motion and trajectory of CoG both depend on the reference motion, they can work in parallel, thus providing double insurances against the robot＇s collapse. Finally, the algorithm is validated by different kinds of simulation experiments.