Exploiting natural dynamics for gait generation in undulatory locomotion

ABSTRACT

Robotic vehicles inspired by animal locomotion operate via periodic body movements. The pattern of body oscillation (gait) can be mimicked from animals, but understanding the principles underlying gait generation would allow for broad, flexible applications beyond nature's design. We hypothesise that travelling-wave oscillations observed in undulatory locomotion can be characterised as a natural oscillation of the locomotion dynamics and propose a formal definition of the natural gait for locomotion systems. We identify the essential dynamics and define the mode shape of natural oscillation by the free response of an idealised system. We then use body-environment resonance to define the amplitude and frequency. Explicit formulas for the natural gait are derived to provide insight into the mechanisms underlying undulatory locomotion. Examples of a swimming leech and a fictitious swimmer illustrate how undulatory gaits similar to those observed can be produced as the natural gait and modulated to achieve different swim speeds.     

Minimalistic control of biped walking in rough terrain

Toward our comprehensive understanding of legged locomotion in animals and machines, the compass gait model has been intensively studied for a systematic investigation of complex biped locomotion dynamics. While most of the previous studies focused only on the locomotion on flat surfaces, in this article, we tackle with the problem of bipedal locomotion in rough terrains by using a minimalistic control architecture for the compass gait walking model. This controller utilizes an open-loop sinusoidal oscillation of hip motor, which induces basic walking stability without sensory feedback. A set of simulation analyses show that the underlying mechanism lies in the “phase locking” mechanism that compensates phase delays between mechanical dynamics and the open-loop motor oscillation resulting in a relatively large basin of attraction in dynamic bipedal walking. By exploiting this mechanism, we also explain how the basin of attraction can be controlled by manipulating the parameters of oscillator not only on a flat terrain but also in various inclined slopes. Based on the simulation analysis, the proposed controller is implemented in a real-world robotic platform to confirm the plausibility of the approach. In addition, by using these basic principles of self-stability and gait variability, we demonstrate how the proposed controller can be extended with a simple sensory feedback such that the robot is able to control gait patterns autonomously for traversing a rough terrain.     

Sensorimotor Interactions During Locomotion: Principles Derived from Biological Systems

Rhythmic movements in biological systems are produced in part by central circuits called central pattern generators (CPGs). For example, locomotion in vertebrates derives from the spinal CPG with activity initiated by the brain and controlled by sensory feedback. Sensory feedback is traditionally viewed as controlling CPGs cycle by cycle, with the brain commanding movements on a top down basis. We present an alternative view which in sensory feedback alters the properties of the CPG on a fast as well as a slow time scale. The CPG, in turn, provides feedforward filtering of the sensory feedback. This bidirectional interaction is widespread across animals, suggesting it is a common feature of motor systems, and, therefore, might offer a new way to view sensorimotor interactions in all systems including robotic systems. Bidirectional interactions are also apparent between the cerebral cortex and the CPG. The motor cortex doesn't simply command muscle contractions, but rather operates with the CPG to produce adaptively structured movements. To facilitate these adaptive interactions, the motor cortex receives feedback from the CPG that creates a temporal activity pattern mirroring the spinal motor output during locomotion. Thus, the activity of the motor cortical cells is shaped by the spinal pattern generator as they drive motor commands. These common features of CPG structure and function are suggested as offering a new perspective for building robotic systems. CPGs offer a potential for adaptive control, especially when combined with the principles of sensorimotor integration described here.     

Development of cutaneo-motor coordination in an autonomous robotic system

The capability of autonomously discovering relations between perceptual data and motor actions is crucial for the development of robust adaptive robotic systems intended to operate in an unknown environment. In the case of robotic tactile perception, a proper interaction between contact sensing and motor control is the basic step toward the execution of complex motor procedures such as grasping and manipulation.In this paper the autonomous development of cutaneo-motor coordination is investigated in the case of a robotic finger mounted on a robotic manipulator, for a particular class of micromovements. A neural network architecture linking changes in the sensed tactile pattern with the motor actions performed is described and experimental results are analyzed. Examples of application of the developed sensory-motor coordination in the generation of motor control procedures for the estimate of surface curvature are considered.     

Chaotic exploration and learning of locomotion behaviors

We present a general and fully dynamic neural system, which exploits intrinsic chaotic dynamics, for the real-time goal-directed exploration and learning of the possible locomotion patterns of an articulated robot of an arbitrary morphology in an unknown environment. The controller is modeled as a network of neural oscillators that are initially coupled only through physical embodiment, and goal-directed exploration of coordinated motor patterns is achieved by chaotic search using adaptive bifurcation. The phase space of the indirectly coupled neural-body-environment system contains multiple transient or permanent self-organized dynamics, each of which is a candidate for a locomotion behavior. The adaptive bifurcation enables the system orbit to wander through various phase-coordinated states, using its intrinsic chaotic dynamics as a driving force, and stabilizes on to one of the states matching the given goal criteria. In order to improve the sustainability of useful transient patterns, sensory homeostasis has been introduced, which results in an increased diversity of motor outputs, thus achieving multiscale exploration. A rhythmic pattern discovered by this process is memorized and sustained by changing the wiring between initially disconnected oscillators using an adaptive synchronization method. Our results show that the novel neurorobotic system is able to create and learn multiple locomotion behaviors for a wide range of body configurations and physical environments and can readapt in realtime after sustaining damage.     

Quantifying patterns of agent–environment interaction

This article explores the assumption that a deeper (quantitative) understanding of the information-theoretic implications of sensory–motor coordination can help endow robots not only with better sensory morphologies, but also with better exploration strategies. Specifically, we investigate by means of statistical and information-theoretic measures to what extent sensory–motor coordinated activity can generate and structure information in the sensory channels of a simulated agent interacting with its surrounding environment. The results show how the usage of correlation, entropy, and mutual information can be employed (a) to segment an observed behavior into distinct behavioral states; (b) to analyze the informational relationship between the different components of the sensory–motor apparatus; and (c) to identify patterns (or fingerprints) in the sensory–motor interaction between the agent and its local environment.     

Polychaete-Like Undulatory Robotic Locomotion in Unstructured Substrates

A biological paradigm of versatile locomotion and effective motion control is provided by the polychaete annelid worms, whose motion adapts to a large variety of unstructured environmental conditions (sand, mud, sediment, water, etc.), and could thus be of interest to replicate by robotic analogs. Their locomotion is characterized by the combination of a unique form of tail-to-head body undulations (opposite to snakes and eels), with the rowing-like action of numerous lateral appendages distributed along their long segmented body. Focusing on the former aspect of polychaete locomotion, computational models of crawling and swimming by such tail-to-head body undulations have been developed in this paper. These are based on the Lagrangian dynamics of the system and on resistive models of its interaction with the environment, and are used for simulation studies demonstrating the generation of undulatory gaits. Several biomimetic robotic prototypes have been developed, whose undulatory actuation achieves propulsion on sand and other granular unstructured environments. Extensive experimental studies demonstrate the feasibility of robot propulsion by tail-to-head body undulations in such environments, as well as the agreement of its qualitative and quantitative characteristics to the predictions of the corresponding computational models.     

Rapid and Cheap Learning by Exploiting Biarticular Muscles — A Case Study With a Two-Dimensional Serpentine Robot

This study is intended to deal with the interplay between control and mechanical systems, and to discuss the 'brain–body interaction as it should be', particularly from the viewpoint of learning. To this end, we have employed a decentralized control of a two-dimensional serpentine robot consisting of several identical body segments as a practical example. The preliminary simulation results derived indicate that the convergence of decentralized learning of locomotion control can be significantly improved, even with an extremely simple learning algorithm, i.e., a gradient method, by introducing biarticular muscles which induce long-distant physical interaction between the body segments compared to the one only with monoarticular muscles. This strongly suggests the fact that a certain amount of computation should be offloaded from the brain into its body, which allows robots to emerge various with interesting functionalities.     

Robust learning control for robotic manipulators with an extension to a class of non-linear systems

A robust learning control (RLC) scheme is developed for robotic manipulators by a synthesis of learning control and robust control methods. The non-linear learning control strategy is applied directly to the structured system uncertainties that can be separated and expressed as products of unknown but repeatable (over iterations) state-independent time functions and known state-dependent functions. The non-linear uncertain terms in robotic dynamics such as centrifugal, Coriolis and gravitational forces belong to this category. For unstructured uncertainties which may have non-repeatable factors but are limited by a set of known bounding functions as the only a priori knowledge, e.g the frictions of a robotic manipulator, robust control strategies such as variable structure control strategy can be applied to ensure global asymptotic stability. By virtue of the learning and robust properties, the new control system can easily fulfil control objectives that are difficult for either learning control or variable structure control alone to achieve satisfactorily. The proposed RLC scheme is further shown to be applicable to certain classes of non-linear uncertain systems which include robotic dynamics as asubset. Various important properties concerning learning control, such as the need for a resetting condition and derivative signals, whether using iterative control mode or repetitive control mode, are also made clear in relation to different control objectives and plant dynamics.     

Modeling sensorimotor learning with linear dynamical systems

Recent studies have employed simple linear dynamical systems to model trial-by-trial dynamics in various sensorimotor learning tasks. Here we explore the theoretical and practical considerations that arise when employing the general class of linear dynamical systems (LDS) as a model for sensorimotor learning. In this framework, the state of the system is a set of parameters that define the current sensorimotor transformation-the function that maps sensory inputs to motor outputs. The class of LDS models provides a first-order approximation for any Markovian (state-dependent) learning rule that specifies the changes in the sensorimotor transformation that result from sensory feedback on each movement. We show that modeling the trial-by-trial dynamics of learning provides a substantially enhanced picture of the process of adaptation compared to measurements of the steady state of adaptation derived from more traditional blocked-exposure experiments. Specifically, these models can be used to quantify sensory and performance biases, the extent to which learned changes in the sensorimotor transformation decay over time, and the portion of motor variability due to either learning or performance variability. We show that previous attempts to fit such models with linear regression have not generally yielded consistent parameter estimates. Instead, we present an expectation-maximization algorithm for fitting LDS models to experimental data and describe the difficulties inherent in estimating the parameters associated with feedback-driven learning. Finally, we demonstrate the application of these methods in a simple sensorimotor learning experiment: adaptation to shifted visual feedback during reaching.     

A method for determination of optimal gaits with application to a snake-like serial-link structure

In this paper, we present a method of determining optimal gaits for shape actuated locomotion systems. This method is the synthesis of techniques for computing reduced equations for robotic locomotion systems and a numerical optimal control strategy. Symmetry reduction processes induce a form of locomotion system dynamics that reveals a cyclic-like coupling between group, shape, and momenta coordinates. This form allows one to focus on designing gaits, abandoning concern over shape dynamics. Using this vantage point we indicate how a numerical optimal control method based on Gaussian quadrature may be acclimatized to periodicity, thus providing optimal gaits. The method is demonstrated by means of its application to a snake-like serial-link structure or snake robot. This application provides scientific merit to hypotheses concerning observed locomotion phenomena amongst animals employing undulatory propulsive mechanisms.     

Causal connectivity of evolved neural networks during behavior

To show how causal interactions in neural dynamics are modulated by behavior, it is valuable to analyze these interactions without perturbing or lesioning the neural mechanism. This paper proposes a method, based on a graph-theoretic extension of vector autoregressive modeling and 'Granger causality,' for characterizing causal interactions generated within intact neural mechanisms. This method, called 'causal connectivity analysis' is illustrated via model neural networks optimized for controlling target fixation in a simulated head-eye system, in which the structure of the environment can be experimentally varied. Causal connectivity analysis of this model yields novel insights into neural mechanisms underlying sensorimotor coordination. In contrast to networks supporting comparatively simple behavior, networks supporting rich adaptive behavior show a higher density of causal interactions, as well as a stronger causal flow from sensory inputs to motor outputs. They also show different arrangements of 'causal sources' and 'causal sinks': nodes that differentially affect, or are affected by, the remainder of the network. Finally, analysis of causal connectivity can predict the functional consequences of network lesions. These results suggest that causal connectivity analysis may have useful applications in the analysis of neural dynamics.     

A biomimetic reach and grasp approach for mechanical hands

Reach and grasp are the two key functions of human prehension. The Central Nervous System controls these two functions in a separate but interdependent way. The choice between different solutions to reach and grasp an object–provided by multiple and redundant degrees of freedom (dof)–depends both on the properties and on the use (affordance) of the object to be manipulated. This same control paradigm, i.e. subdivision of prehension into reach and grasp as well as the corresponding multimodal (sensory/motor) information fusion schemes, can also be applied to a mechanical hand carried by a robotic arm. The robotic arm will then be responsible for positioning the hand with respect to the object, and the hand will then grasp and manipulate the object. In this article, we present a biomimetic sensory–motor control scheme in the aim of providing an object-dependent and intelligent reach and grasp ability to such systems. The proposed model is based on a multi-network architecture which incorporates multiple Matching Units trained by a statistical learning algorithm (LWPR). Matching Units perform a multimodal signal integration by correlating sensory and motor information analogous to that observed in cerebral neuronal networks. The simulated network of multiple Matching Units provided estimations of object-dependent 5-finger grasp configurations with endpoint positional errors in the order of a few millimeters. For validation, these estimations were then applied to the control of movement kinematics on an experimental robot composed of a 6 dof robot arm carrying a 16 dof mechanical 4-finger hand. Precision of the kinematics control was such that successful reach, grasp and lift was obtained in all the tests.     

Biped Locomotion Control through a Biomimetic CPG-based Controller

Modern concepts of motor learning favour intensive training directed to the neural networks stimulation and reorganization within the spinal cord, the central pattern generator, by taking advantage of the neural plasticity. In the present work, a biomimetic controller using a system of adaptive oscillators is proposed to understand the neuronal principles underlying the human locomotion. A framework for neural control is presented, enabling the following contributions: a) robustness to external perturbations; b) flexibility to variations in the environmental constraints; and c) incorporation of volitional mechanisms for self-adjustment of gait dynamics. Phase modulation of adaptive oscillators and postural balance control are proposed as main strategies for stable locomotion. Simulations of the locomotion model with a biped robot in closed-loop control are presented to validate the implemented neuronal principles. Specifically, the proposed system for online modulation of previous learnt gait patterns was verified in terrains with different slopes. The proposed phase modulation method and postural balanced control enabled robustness enhancement considering a broader range of slope angles than recent studies. Furthermore, the system was also verified for tilted ground including different slopes in the same experiment and uneven terrain with obstacles. Adaptive Frequency Oscillators, under Dynamic Hebbian Learning Adaptation mechanism, are proposed to build a hierarchical control architecture with spinal and supra spinal centers with multiple rhythm-generating neural networks that drive the legs of a biped model. The proposed neural oscillators are based on frequency adaptation and can be entrained by sensory feedback to learn specific patterns. The proposed biomimetic controller intrinsically generates patterns of rhythmic activity that can be induced to sustain CPG function by specific training. This method provides versatile control, paving the way for the design of experimental motor control studies, optimal rehabilitation procedures and robot-assisted therapeutic outcomes.     

Vibration Based Under-Actuated Bounding Mechanism

Today’s robots are able to perform very limited locomotion tasks by consuming high energy although animals are able to carry out very complicated but stable locomotion tasks using less control inputs and energy. Therefore, it is important to understand the principles of animal locomotion in order to develop efficient legged robots. This paper presents a U-shape visco-elastic beam mechanism that is able to run like a bounding animal when it is actuated by a simple pendulum at the torsional resonance frequency of the elastic body. A simple physical model has been developed to investigate the dynamics of the mechanism and the natural body dynamics of quadrupeds. In the mechanism, a small rotating mass was attached to a DC motor which was mounted on the center of the spine. When this motor is actuated at around the torsional resonance frequency of the elastic body, the robot starts to move and it exhibits a self-organized locomotion behavior. The self-organized locomotion process of the robot does not require any central authority, sensory feedback or external element imposing a planned motion. Comparing the bounding locomotion of the beam mechanism with those of well-known quadrupeds such as a horse, greyhound and cheetah, it can be concluded that the pendulum-driven U-shaped visco-elastic beam displays kinematic behavior similar to a horse, in terms of both experimental and simulation results. Interestingly, this bounding locomotion occurs only if the shape ratio and the actuation frequencies of the beam are close to those of the fastest quadrupeds.     

Control mechanisms and local perception to support autonomous behavior in virtual animated agents

We propose basic mechanisms that can be used as support to perform computer animated agents with autonomous behavior. We provide the agents with local perception and use a control theory approach, dealing with dynamics and kinematics issues, in order to establish a well structured way to control the agent resources. While maintaining the virtual agents with physical specifications and constraints necessary for animation purposes, the mechanisms provide other realistic needs necessary for providing autonomous behavior. In this way, agents can interact with their environment independent of user interaction, learning the characteristics of its habitat and adapting to it in order to survive. We validate the mechanisms by presenting two platforms with different structures (sensors, actuators, and dynamics) which have used them. As practical results, by using the same control structure, the animated agents are able to perform different tasks like learning attention control and pattern categorization based on multi-modal sensory information, learning how to progress onto a rough terrain, and learning how to perform visual monitoring tasks of their environment.     

Distributed control of uncertain systems using superpositions of linear operators

Control in the natural environment is difficult in part because of uncertainty in the effect of actions. Uncertainty can be due to added motor or sensory noise, unmodeled dynamics, or quantization of sensory feedback. Biological systems are faced with further difficulties, since control must be performed by networks of cooperating neurons and neural subsystems. Here, we propose a new mathematical framework for modeling and simulation of distributed control systems operating in an uncertain environment. Stochastic differential operators can be derived from the stochastic differential equation describing a system, and they map the current state density into the differential of the state density. Unlike discrete-time Markov update operators, stochastic differential operators combine linearly for a large class of linear and nonlinear systems, and therefore the combined effects of multiple controllable and uncontrollable subsystems can be predicted. Design using these operators yields systems whose statistical behavior can be specified throughout state-space. The relationship to Bayesian estimation and discrete-time Markov processes is described.     

On-line expectation-based novelty detection for mobile robots

This paper presents a recurrent neural network based novelty filter where a Scitos G5 mobile robot explored the environment and built dynamic models of observed sensory–motor values, then the acquired models of normality are used to predict the expected future values of sensory–motor inputs during patrol. Novelties could be detected whenever the prediction error between models-predicted values and actual observed values exceeded a local novelty threshold. The network is trained on-line; it grows by inserting new nodes when abnormal observation is perceived from the environment; and also shrinks when the learned information is not necessary anymore. In addition, the network is also capable of learning region-specific novelty thresholds on-line continuously.To evaluate the proposed algorithm, real-world robotic experiments were conducted by fusing sensory perceptions (vision and laser sensors) and the robot motor control outputs (translational and rotational velocities). Experimental results showed that all of the novelty cases were highlighted by the proposed algorithms and it produced reliable local novelty thresholds while the robot patrols in the noisy environment. The statistical analysis showed that there was a strong correlation between the novelty filter responses and the actual novelty status. Furthermore, the filter was also compared with another novelty filter and the results showed that the proposed system performed better novelty detection.     

Modeling and System Identification of a Life-Size Brake-Actuated Manipulator

Safety is a critical factor when designing a robotic rehabilitation environment. Whole-limb or life-size haptic interaction would allow virtual robotic rehabilitation of daily living activities such as sweeping or shelving. However, it has been too dangerous to implement such an environment with conventional active robots that use motor, hydraulic, or pneumatic actuation. To address this issue, a life-size 6-degree-of-freedom (DOF) brake-actuated manipulator (BAM) was designed and constructed. This paper details the BAM's system models including mechanisms, kinematics, and dynamics, as well as detailed input and friction models. In addition, a new system-identification technique that utilizes human input to excite the robot's dynamics with unscented Kalman filtering was employed to identify system parameters. Noise sources are discussed, and the model is validated through force estimation with inverse dynamics. Model parameters and performance are compared with other commercially available haptic devices. The BAM shows a significantly larger workspace, maximum force, and stiffness over other devices exhibiting its promise toward rehabilitative applications.      

Certain Principles of Biomorphic Robots

The field of biomorphic robotics can advance as quickly as clear principles of biological systems can be identified, implemented, and tested in robotic devices. Here, we describe the implementation of three principles: (1) the prediction of the sensory consequences of movement and its role in the extraction of novelty and awareness; (2) learning affordances and the direct perception of what an agent can do at a particular instant and how it can do it; (3) exploitation of the physical dynamics of a system to simplify robot control.