A comparison of three insect-inspired locomotion controllers

This paper compares three insect inspired controllers which were implemented on an autonomous hexapod robot. There is a growing interest in using insect locomotion schemes to control walking robots. Researchers' interest in insect-based controllers ranges from understanding the biological basis of locomotion control in insects to building real-time walking machines which require relatively little computational power. Several models for insect locomotion exist, and robotics researchers tend to adopt one approach and experiment with it.

In contrast, this paper offers a comparison of three insect inspired controllers — all of which were implemented and tested on the same autonomous hexapod robot. Some of the controllers used reflex-based mechanisms whereas others used pattern-based mechanisms. Reflexive controllers exploit sensory stimulus and response reactions to produce leg motion and gait coordination. In contrast, pattern-based controllers depend more upon pre-programmed patterns of behavior which may be influenced by external events. Typically, these pre-programmed patterns of behavior are implemented using central pattern generators (CPGs).

In this work, we compare gait coordination performance of three controllers on flat terrain. We extend the comparison to include leg loading considerations, disabled leg compensation, and externally applied leg perturbations. We discuss the differences between controllers with respect to inconsistent leg retraction velocities, leg design issues, sensing requirements, and computational issues. The robot performed quite differently under varying experimental conditions depending upon which controller was used. We found that controller performance was the most sensitive to robot design parameters. For our case, we had the most success with pattern-based mechanisms given the leg design of our robot and its limitations in controlling the retraction velocity of its legs. The pattern-based mechanisms allowed the robot to remain stable over a variety of gaits while the robot was subjected to loading the legs, disabling a leg, and physically disturbing the legs. The reflexive mechanisms were less successful at maintaining stability when the robot's legs were increasingly disrupted.     

仿生机器人运动步态控制:强化学习方法综述

仿生机器人是一类典型的多关节非线性欠驱动系统，其步态控制是一个非常具有挑战性的问题。对于该问题，传统的控制和规划方法需要针对具体的运动任务进行专门设计，需要耗费大量时间和精力，而且所设计出来的控制器往往没有通用性。基于数据驱动的强化学习方法能对不同的任务进行自主学习，且对不同的机器人和运动任务具有良好的通用性。因此，近年来这种基于强化学习的方法在仿生机器人运动步态控制方面获得了不少应用。针对这方面的研究，本文从问题形式化、策略表示方法和策略学习方法3个方面对现有的研究情况进行了分析和总结，总结了强化学习应用于仿生机器人步态控制中尚待解决的问题，并指出了后续的发展方向。     

Learning CPG-based biped locomotion with a policy gradient method

In this paper, we propose a learning framework for CPG-based biped locomotion with a policy gradient method. We demonstrate that appropriate sensory feedback to adjust the rhythm of the CPG (Central Pattern Generator) can be learned using the proposed method within a few hundred trials in simulations. We investigate linear stability of a periodic orbit of the acquired walking pattern considering its approximated return map. Furthermore, we apply the controllers acquired in numerical simulations to our physical 5-link biped robot in order to empirically evaluate the robustness of walking in the real environment. Experimental results demonstrate that the robot was able to successfully walk using the acquired controllers even in the cases of an environmental change by placing a seesaw-like metal sheet on the ground and a parametric change of the robot dynamics with an additional weight on a shank, which was not modeled in the numerical simulations.     

Intelligent robots as artificial living creatures

This article introduces a multi locomotion robot, MLR III, which has multiple locomotion types, i.e., not only brachiating but also walking. Conventionally we have studied dexterous locomotion robots and their controller design. One is a simplified two-link robot, Brachiator II. This is an example of an underactuated system in which a robot mechanism has more degrees of freedom than actuators. The desired motions are encoded as the output of a target dynamical system inspired by the pendulum-link motion of an apes brachiation. The other is a monkey-type robot, Brachiator III. Brachiator III achieves a dexterous motion using redundant degrees of freedom. The motion is generated in an empirical learning process on an intelligent structure, on which the learning algorithm coordinates some primitive motions to generate the desired motion. MLR III is an extended locomotion robot that has multiple types of locomotions: brachiation, bipedal walking, and quadrupedal walking, similar to a monkey or gorilla. This article introduces the mechanism and controller design for brachiating motion.This work was presented, in part, at the 8th International Symposium on Artificial Life and Robitics, Oita, Japan, January 24–26, 2003     

Minimalistic control of biped walking in rough terrain

Toward our comprehensive understanding of legged locomotion in animals and machines, the compass gait model has been intensively studied for a systematic investigation of complex biped locomotion dynamics. While most of the previous studies focused only on the locomotion on flat surfaces, in this article, we tackle with the problem of bipedal locomotion in rough terrains by using a minimalistic control architecture for the compass gait walking model. This controller utilizes an open-loop sinusoidal oscillation of hip motor, which induces basic walking stability without sensory feedback. A set of simulation analyses show that the underlying mechanism lies in the “phase locking” mechanism that compensates phase delays between mechanical dynamics and the open-loop motor oscillation resulting in a relatively large basin of attraction in dynamic bipedal walking. By exploiting this mechanism, we also explain how the basin of attraction can be controlled by manipulating the parameters of oscillator not only on a flat terrain but also in various inclined slopes. Based on the simulation analysis, the proposed controller is implemented in a real-world robotic platform to confirm the plausibility of the approach. In addition, by using these basic principles of self-stability and gait variability, we demonstrate how the proposed controller can be extended with a simple sensory feedback such that the robot is able to control gait patterns autonomously for traversing a rough terrain.     

基于深度强化学习的双足机器人斜坡步态控制方法

为提高准被动双足机器人斜坡步行稳定性,本文提出了一种基于深度强化学习的准被动双足机器人步态控制方法.通过分析准被动双足机器人的混合动力学模型与稳定行走过程,建立了状态空间、动作空间、episode过程与奖励函数.在利用基于DDPG改进的Ape-X DPG算法持续学习后,准被动双足机器人能在较大斜坡范围内实现稳定行走.仿...     

On the Improvement of Walking Performance in Natural Environments by a Compliant Adaptive Gait

It is a widespread idea that animal-legged locomotion is better than wheeled locomotion on natural rough terrain. However, the use of legs as a locomotion system for vehicles and robots still has a long way to go before it can compete with wheels and trucks, even on natural ground. This paper aims to solve two main disadvantages plaguing walking robots: their inability to react to external disturbances (which is also a drawback of wheeled robots); and their extreme slowness. Both problems are reduced here by combining: 1) a gait-parameter-adaptation method that maximizes a dynamic energy stability margin and 2) an active-compliance controller with a new term that compensates for stability variations, thus helping the robot react stably in the face of disturbances. As a result, the combined gait-adaptation approach helps the robot achieve faster, more stable compliant motions than conventional controllers. Experiments performed with the SILO4 quadruped robot show a relevant improvement in the walking gait     

仿蚯蚓移动机器人离散步态控制与相位差控制特性比较

从机器人的运动特征、稳态平均速度和波动特性3个方面,对仿蚯蚓移动机器人的离散步态控制策略和相位差控制策略进行比较研究．首先,通过学习蚯蚓的形态学特征,基于舵机－弹簧钢片复合结构,设计并制作了可以执行拮抗变形的仿蚯蚓机器人单元,并将其串联得到一个8单元仿蚯蚓移动机器人．以该机器人为平台,从理论和实验角度研究了机器人在离散步态控制和等相位差控制下的平均速度和运动特征．研究发现,对于2种控制策略,实验得到的平均速度都与理论预测定性吻合,但机器人单元在运动过程中有可能发生显著的向后滑动,使得实验得到的平均速度低于理论预测的平均速度．随后,从波传播的角度对2种控制策略进行了比较．2种控制策略都使得机器人单元的变形以波动的形式沿机器人进行传播,传播方向与机器人运动方向相反,与蚯蚓的后退蠕动波机理保持一致．对于离散步态控制,波传播的波形、波速、波长和周期都与步态参数密切相关;对于相位差控制,波形和周期都由作动规律决定,不能通过相位差进行调节,但波速和波长与相位差成反比．从控制效果来看,机器人在最优的等相位差控制模式下可以实现更高的平均速度,且与蚯蚓的连续特征保持一致,具有一定的优势．     

Overview of the Lucy Project: Dynamic Stabilization of a Biped Powered by Pneumatic Artificial Muscles

This paper gives an overview of the Lucy project. What is special is that the biped is not actuated with the classical electrical drives, but with pleated pneumatic artificial muscles. In an antagonistic setup of such muscles both the torque and the compliance are controllable. From human walking there is evidence that joint compliance plays an important role in energy-efficient walking and running. To be able to walk at different walking speeds and step lengths, a trajectory generator and joint trajectory tracking controller are combined. The first generates dynamically stable trajectories based on the objective locomotion parameters which can be changed from step to step. The joint trajectory tracking unit controls the pressure inside the muscles so the desired motion is followed. It is based on a computed torque model and takes the torque–angle relation of the antagonistic muscle setup into account. With this strategy the robot is able to walk at a speed up to 0.15 m/s. A compliance controller is developed to reduce the energy consumption by combining active trajectory control with the exploitation of the natural dynamics. A mathematical formulation was developed to find an optimal compliance setting depending on the desired trajectory and physical properties of the system. This strategy is experimentally evaluated on a single pendulum structure and not implemented on the real robot because the walking speed of the robot is currently too slow. At the end a discussion is given about the pros and cons of building a pneumatic biped, and the control architecture used.     

Energy-preserving control of a passive one-legged running robot

Fast and energy-efficient control is an increasingly important and attractive area of research in legged locomotion. In this paper, we present a new simple controller for a planar one-legged passive running robot having a springy leg and a compliant hip joint. The most distinctive advantage of the controller over previously proposed ones is it does not require any pre-planned trajectories nor target dynamics. Instead, it utilizes exact non-linear dynamics. Our results are summarized as follows. First, we propose an energy-preserving control strategy for energy-efficient and autonomous gait generation. This strategy is successfully implemented as a new touchdown controller at the flight phase. Simulation results show that the robot can hop from a wide set of initial conditions. Moreover, the running gaits generated are found to be quasi-periodic orbits, which can be seen in Hamiltonian systems. Since the controlled running gaits exist for every admissible energy level, they have some robustness against disturbances. Next, it is shown that an adaptive control of the touchdown angle, which is similar to a delayed feedback controller for a chaotic system, can asymptotically stabilize these quasi-periodic gaits to the periodic ones of the desired period, with some limitations. In particular, for one-periodic gait, by using some additional adaptive controllers, the robot eventually hops without any control inputs. Since our energy-preserving strategy is clear and implementation of the controller is straightforward, we believe it can be easily applied to a wide class of legged mechanisms.     

A Biologically Inspired Biped Locomotion Strategy for Humanoid Robots: Modulation of Sinusoidal Patterns by a Coupled Oscillator Model

Biological systems seem to have a simpler but more robust locomotion strategy than that of the existing biped walking controllers for humanoid robots. We show that a humanoid robot can step and walk using simple sinusoidal desired joint trajectories with their phase adjusted by a coupled oscillator model. We use the center-of-pressure location and velocity to detect the phase of the lateral robot dynamics. This phase information is used to modulate the desired joint trajectories. We do not explicitly use dynamical parameters of the humanoid robot. We hypothesize that a similar mechanism may exist in biological systems. We applied the proposed biologically inspired control strategy to our newly developed human-sized humanoid robot computational brain (CB) and a small size humanoid robot, enabling them to generate successful stepping and walking patterns.     

Gait Generation and Stabilization for Nearly Passive Dynamic Walking Using Auto‐distributed Impulses

We propose a state feedback control design via linearization for flexible walking on flat ground. First, we generate nearly passive limit cycles, being stable or not, using impulsive toe‐off actuations. The term ‘nearly passive’ means that the dynamics is completely passive almost everywhere except at the toe‐off moment. A feature of our gait generation method is that walking gaits are characterized only by amounts of supplied energy, and we observe that other variables, including input torques, are auto‐balanced via our method. After gait generation, we design a feedback controller considering robustness and input saturation. As a result, each limit cycle can be matched with its respective controller classified only by energy levels. We have verified that walking speeds monotonically increase by adding more energy, and the ankle joint plays a significant role in compass‐gait walking. Finally, instead of applying impulsive torques, we discuss a practical issue regarding realistic control inputs that ensure stable gait transitions as energy levels are elevated.     

Real‐time Locomotion Controller using an Inverted‐Pendulum‐based Abstract Model

In this paper, we propose a novel motion controller for the online generation of natural character locomotion that adapts to new situations such as changing user control or applying external forces. This controller continuously estimates the next footstep while walking and running, and automatically switches the stepping strategy based on situational changes. To develop the controller, we devise a new physical model called an inverted‐pendulum‐based abstract model (IPAM). The proposed abstract model represents high‐dimensional character motions, inheriting the naturalness of captured motions by estimating the appropriate footstep location, speed and switching time at every frame. The estimation is achieved by a deep learning based regressor that extracts important features in captured motions. To validate the proposed controller, we train the model using captured motions of a human stopping, walking, and running in a limited space. Then, the motion controller generates human‐like locomotion with continuously varying speeds, transitions between walking and running, and collision response strategies in a cluttered space in real time.     

Fuzzy neural network approaches for robotic gait synthesis

In this paper, a learning scheme using a fuzzy controller to generate walking gaits is developed. The learning scheme uses a fuzzy controller combined with a linearized inverse biped model. The controller provides the control signals at each control time instant. The algorithm used to train the controller is "backpropagation through time". The linearized inverse biped model provides the error signals for backpropagation through the controller at control time instants. Given prespecified constraints such as the step length, crossing clearance, and walking speed, the control scheme can generate the gait that satisfies these constraints. Simulation results are reported for a five-link biped robot.     

Formation control of underactuated bio-inspired snake robots

This paper considers formation control of snake robots. In particular, based on a simplified locomotion model, and using the method of virtual holonomic constraints, we control the body shape of the robot to a desired gait pattern defined by some pre-specified constraint functions. These functions are dynamic in that they depend on the state variables of two compensators which are used to control the orientation and planar position of the robot, making this a dynamic maneuvering control strategy. Furthermore, using a formation control strategy we make the multi-agent system converge to and keep a desired geometric formation, and enforce the formation follow a desired straight line path with a given speed profile. Specifically, we use the proposed maneuvering controller to solve the formation control problem for a group of snake robots by synchronizing the commanded velocities of the robots. Simulation results are presented which illustrate the successful performance of the theoretical approach.     

Reinforcement learning of humanoid rhythmic walking parameters based on visual information

This paper presents a method for learning the parameters of rhythmic walking to generate purposive humanoid motions. The controller consists of the two layers: rhythmic walking is realized by the lower layer, which adjusts the speed of the phase on the desired trajectory depending on sensory information, and the upper layer learns (i) the feasible parameter sets that enable stable walking, (ii) the causal relationship between the walking parameters to be given to the lower-layer controller and the change in the sensory information and (iii) the feasible rhythmic walking parameters by reinforcement learning so that a robot can reach the goal based on visual information. The experimental results show that a real humanoid learns to reach the ball and to shoot it into the goal in the context of the RoboCup soccer competition, and the further issues are discussed.     

Oscillatory motion-based miniature magnetic walking robot actuated by a rotating magnetic field

Magnetic micro-robots have been proposed for use in biomedical applications. These studies focus on locomotion control using a gradient, alternating, and rotating magnetic fields at the sub-micro scale. However, this study focuses on a basic mechanism of active locomotion for diagnostic robots. Furthermore, the digestive intestine in the human body has a complex path in which locomotion methods can become either swimming or walking according to the inner condition. Therefore, we propose a new simple mechanism for amphibious locomotion within a rotating magnetic field using the three-axis Helmholtz coil system. The proposed magnetic robot consists of NdFeB permanent spherical magnets, flexible silicone tubes, and legs. Successive changes of actuation of yaw and roll motions cause alternating and walking motions. Direction of movement is decided by rotating the direction of the magnetic field (clockwise or counter-clockwise). In addition, turning directions are decided by the plane of the rotating magnetic field. A magnetic torque between the rotating magnetic field and the magnetic moments produce a constant walking pattern similar to a trotting gait. In addition, an oscillatory motion of the flexible robot body can generate a thrust force in the liquid. Finally, through the various experiments, we evaluate the capability of the locomotion.     

Development of flexible joints for a humanoid robot that walks on an oscillating plane

In this study, we develop flexible joints for a humanoid robot that walks on an oscillating plane and discuss their effectiveness in compensating disturbances. Conventional robots have a rigid frame and are composed of rigid joints driven by geared motors. Therefore, disturbances, which may be caused by external forces from other robots, obstacles, vibration and oscillation of the surface upon which the robot is walking, and so on, are transmitted directly to the robot body, causing the robot to fall. To address this problem, we focus on a flexible mechanism. We develop flexible joints and incorporate them in the waist of a humanoid robot; the experimental task of the robot is to walk on a horizontally oscillating plane until it reaches the desired position. The robot with the proposed flexible joints, reached the goal position despite the fact that the controller was the same as that used for a conventional robot walking on a static plane. From these results, we conclude that our proposed mechanism is effective for humanoid robots that walk on an oscillating plane.     

Intelligent trajectory control using recurrent averaging learning

In this article, robotic trajectory control using artificial intelligence techniques is developed. The learning strategy is called recurrent averaging learning. It takes the average of initial states and final states after a cycle of training and sets this value as the new initial and final states for next training cycle. A three-layer neural network is used as a controller, it provides the control signals in each stage of a walking gait. A linearized inverse biped model is derived. This model calculates the error signals that will be used to back propagate to the controller in each stage. Through learning, the robot can develop skills to walk along a predefined path with specified step length, walking speed, and crossing clearance. This proposed scheme is tested with simulations of the BLR-G1 walking robot on horizontal and sloping surfaces.     

Integration of posture and rhythmic motion controls in quadrupedal dynamic walking using phase modulations based on leg loading/unloading

In this paper, we intend to show the basis of a general legged locomotion controller with the ability to integrate both posture and rhythmic motion controls and shift continuously from one control method to the other according to the walking speed. The rhythmic motion of each leg in the sagittal plane is generated by a single leg controller which controls the swing-to-stance and stance-to-swing phase transitions using respectively leg loading and unloading information. Since rolling motion induced by inverted pendulum motion during the two-legged stance phases results in the transfer of the load between the contralateral legs, leg loading/unloading involves posture information in the frontal plane. As a result of the phase modulations based on leg loading/unloading, rhythmic motion of each leg is achieved and inter-leg coordination (resulting in a gait) emerges, even without explicit coordination amongst the leg controllers, allowing to realize dynamic walking in the low- to medium-speed range. We show that the proposed method has resistance ability against lateral perturbations to some extent, but that an additional ascending coordination mechanism between ipsilateral legs is necessary to withstand perturbations decreasing the rolling motion amplitude. Even without stepping reflex using vestibular information, our control system, relying on phase modulations based on leg loading/unloading and the ascending coordination mechanism between ipsilateral legs, enables low speed dynamic walking on uneven terrain with long cyclic period, which was not realized in our former studies. Details of trajectory generation, movies of simulations and movies of preliminary experiments using a real robot are available at: http://robotics.mech.kit.ac.jp/kotetsu/.