Adaptive walking control of quadruped robots based on central pattern generator （CPG） and reflex

This paper presents a central pattern generator (CPG) and vestibular reflex combined control strategy for a quadruped robot. An oscillator network and a knee-to-hip mapping function are presented to realize the rhythmic motion for the quadruped robot. A two-phase parameter tuning method is designed to adjust the parameters of oscillator network. First, based on the numerical simulation, the influences of the parameters on the output signals are analyzed, then the genetic algorithm (GA) is used to evolve the phase relationships of the oscillators to realize the basic animal-like walking pattern. Moreover, the animal’s vestibular reflex mechanism is mimicked to realize the adaptive walking of the quadruped robot on a slope terrain. Coupled with the sensory feedback information, the robot can walk up and down the slope smoothly. The presented bio-inspired control method is validated through simulations and experiments with AIBO. Under the control of the presented CPG and vestibular reflex combined control method, AIBO can cope with slipping, falling down and walk on a slope successfully, which demonstrates the effectiveness of the proposed walking control method.     

Design of highly effective multilayer feedforward neural network by using genetic algorithm

This paper presents a highly effective and precise neural network method for choosing the activation functions (AFs) and tuning the learning parameters (LPs) of a multilayer feedforward neural network by using a genetic algorithm (GA). The performance of the neural network mainly depends on the learning algorithms and the network structure. The backpropagation learning algorithm is used for tuning the network connection weights, and the LPs are obtained by the GA to provide both fast and reliable learning. Also, the AFs of each neuron in the network are automatically chosen by a GA. The present study consists of 10 different functions to accomplish a better convergence of the desired input–output mapping. Test studies are performed to solve a set of two-dimensional regression problems for the proposed genetic-based neural network (GNN) and conventional neural network having sigmoid AFs and constant learning parameters. The proposed GNN has also been tested by applying it to three real problems in the fields of environment, medicine, and economics. Obtained results prove that the proposed GNN is more effective and reliable when compared with the classical neural network structure.     

Central pattern generator based reflexive control of quadruped walking robots using a recurrent neural network

This paper presents a novel Central Pattern Generator (CPG) model for controlling quadruped walking robots. The improvement of this model focuses on generating any desired waveforms along with accurate online modulation. In detail, a well-analyzed Recurrent Neural Network is used as the oscillators to generate simple harmonic periodic signals that exhibit limit cycle effects. Then, an approximate Fourier series is employed to transform those mentioned simple signals into arbitrary desired outputs under the phase constraints of several primary quadruped gaits. With comprehensive closed-form equations, the model also allows the user to modulate the waveform, the frequency and the phase constraint of the outputs online by directly setting the inner parameters without the need for any manual tuning. In addition, an associated controller is designed using leg coordination Cartesian position as the control state space based on which stiffness control is performed at sub-controller level. In addition, several reflex modules are embedded to transform the feedback of all sensors into the CPG space. This helps the CPG recognize external disturbances and utilize inner limit cycle effect to stabilize the robot motion. Finally, experiments with a real quadruped robot named AiDIN III performing several dynamic trotting tasks on several unknown natural terrains are presented to validate the effectiveness of the proposed CPG model and controller.     

New automated learning CPG for rhythmic patterns

In this paper, we suggest a new supervised learning method called Fourier based automated learning central pattern generators (FAL-CPG), for learning rhythmic signals. The rhythmic signal is analyzed with Fourier analysis and fitted with a finite Fourier series. CPG parameters are selected by direct comparison with the Fourier series. It is shown that the desired rhythmic signal is learned and reproduced with high accuracy. The resulting CPG network offers several advantages such as, modulation and robustness against perturbation. The proposed learning method is simple, straightforward and efficient. Furthermore, it is suitable for on-line applications. The effectiveness of the proposed method is shown by comparison with four other supervised learning methods as well as an industrial robotic trajectory following application.     

Tuning of the structure and parameters of a neural network using an improved genetic algorithm

This paper presents the tuning of the structure and parameters of a neural network using an improved genetic algorithm (GA). It is also shown that the improved GA performs better than the standard GA based on some benchmark test functions. A neural network with switches introduced to its links is proposed. By doing this, the proposed neural network can learn both the input-output relationships of an application and the network structure using the improved GA. The number of hidden nodes is chosen manually by increasing it from a small number until the learning performance in terms of fitness value is good enough. Application examples on sunspot forecasting and associative memory are given to show the merits of the improved GA and the proposed neural network.     

Learning model for coupled neural oscillators.

Neurophysiological experiments have shown that many motor commands in living systems are generated by coupled neural oscillators. To coordinate the oscillators and achieve a desired phase relation with desired frequency, the intrinsic frequencies of component oscillators and coupling strengths between them must be chosen appropriately. In this paper we propose learning models for coupled neural oscillators to acquire the desired intrinsic frequencies and coupling weights based on the instruction of the desired phase pattern or an evaluation function. The abilities of the learning rules were examined by computer simulations including adaptive control of the hopping height of a hopping robot. The proposed learning rule takes a simple form like a Hebbian rule. Studies on such learning models for neural oscillators will aid in the understanding of the learning mechanism of motor commands in living bodies.     

Output recurrent wavelet neural network-based adaptive backstepping controller for a class of MIMO nonlinear non-affine uncertain systems

In this paper, an adaptive backstepping control problem is proposed for a class of multiple-input-multiple-output nonlinear non-affine uncertain systems. An output recurrent wavelet neural network (ORWNN) is used to approximate the unknown nonlinear functions to develop the proposed adaptive backstepping controller. The proposed ORWNN combines the advantages of wavelet-based neural network, fuzzy neural network, and output feedback layer to achieve higher approximation accuracy and faster convergence. According to the estimation of ORWNN, the control scheme is designed by backstepping approach such that the system outputs follow the desired trajectories. Based on the Lyapunov approach, our approach guarantees that the system outputs converge to a small neighborhood of the references signals, that is, all signals of the closed-loop system are semi-globally uniformly ultimately bounded. Finally, simulation results including double pendulums system and two inverted pendulums on carts system are shown to demonstrate the performance and effectiveness of our approach.     

Nonlinear controller tuning based on a sequence of identifications of linearized time-varying models

A novel algorithm for tuning controllers for nonlinear plants is presented. The algorithm iteratively minimizes a criterion of the control performance. In each iteration one experiment is performed with a reference signal slightly different from the previous reference signal. The input–output signals of the plant are used to identify a linear time-varying model of the plant which is then used to calculate an update of the controller parameters. The algorithm requires an initial feedback controller that stabilizes the closed loop for the desired reference signal and in its vicinity, and that the closed-loop outputs are similar for the previous and current reference signals. The tuning algorithm is successfully tested on a laboratory set-up of the Furuta pendulum.     

Genetic Tuning of PID Controllers Using a Neural Network Model: A Seesaw Example

When genetic algorithms (GAs) are applied for PID parameter tuning, since the PID parameters are adjusted almost randomly, it is possible that the plant will be damaged due to abrupt changes in PID parameters. To solve this problem, a neural network will be used to model the plant and the genetic tuning procedure will be performed on the neural network instead of the plant. After determining the PID parameters in this off-line manner, these gains are then applied to the plant for on-line control. Moreover, considering that the neural network model may not be accurate enough, a method is also proposed for on-line fine-tuning of PID parameters. To show the validity of the proposed method, a seesaw system that has one input and two outputs will be used for experimental evaluation     

A new optimization algorithm for single hidden layer feedforward neural networks

Feedforward neural networks are the most commonly used function approximation techniques in neural networks. By the universal approximation theorem, it is clear that a single-hidden layer feedforward neural network (FNN) is sufficient to approximate the corresponding desired outputs arbitrarily close. Some researchers use genetic algorithms (GAs) to explore the global optimal solution of the FNN structure. However, it is rather time consuming to use GA for the training of FNN. In this paper, we propose a new optimization algorithm for a single-hidden layer FNN. The method is based on the convex combination algorithm for massaging information in the hidden layer. In fact, this technique explores a continuum idea which combines the classic mutation and crossover strategies in GA together. The proposed method has the advantage over GA which requires a lot of preprocessing works in breaking down the data into a sequence of binary codes before learning or mutation can apply. Also, we set up a new error function to measure the performance of the FNN and obtain the optimal choice of the connection weights and thus the nonlinear optimization problem can be solved directly. Several computational experiments are used to illustrate the proposed algorithm, which has good exploration and exploitation capabilities in search of the optimal weight for single hidden layer FNNs.     

MULTITUNING FUZZY CONTROL SYSTEMS DESIGN

A multituning fuzzy control system structure that involves two simple, but effective tuning mechanisms, is proposed: one is called fuzzy control rule tuning mechanism (FCRTM); the other is called dynamic scalar tuning mechanism (DSTM). In FCRTM, it is used to generate the necessary control rules with a center extension method. In DSTM, it contains three fuzzy IF-THEN rules for determining the appropriate scaling factors for the fuzzy control system. In this paper, a method based on the genetic algorithm (GA) is proposed to simultaneously choose the appropriate parameters in FCRTM and DSTM. That is, the proposed GA-based method can automatically generate the required rule base of fuzzy controller and efficiently determine the appropriate map for building the dynamic scalars of fuzzy controller. A multiobjective fitness function is proposed to determine an appropriate parameter set such that not only the selected fuzzy control structure has fewer fuzzy rules, but also the controlled system has a good control performance. Finally, an inverted pendulum control problem is given to illustrate the effectiveness of the proposed control scheme.     

Locomotion of snake-like robots using adaptive neural oscillators

This paper proposes a CPG-based control architecture using a frequency-adaptive oscillator for undulatory locomotion of snake-like robots. The control architecture consists of a network of neural oscillators that generates desired oscillatory output signals with specific phase lags. A key feature of the proposed architecture is a self-adaptation process that modulates the parameters of the CPG to adapt the motion of the robot to varying coefficients of body-ground friction. This process is based on the frequency-adaptation rule of the oscillator that is designed to learn the periodicity of sensory feedback signals. It has an important meaning of establishing a closed-loop CPG much more robust against environmental and/or system parameter changes. We verify the validity of the proposed locomotion control system employing a simulated snake-like robot moving over terrains with different friction coefficients with a constant velocity.     

How the brain generates movement

In this study, we assume that the brain uses a general-purpose pattern generator to transform static commands into basic movement segments. We hypothesize that this pattern generator includes an oscillator whose complete cycle generates a single movement segment. In order to demonstrate this hypothesis, we construct an oscillator-based model of movement generation. The model includes an oscillator that generates harmonic outputs whose frequency and amplitudes can be modulated by external inputs. The harmonic outputs drive a number of integrators, each activating a single muscle. The model generates muscle activation patterns composed of rectilinear and harmonic terms. We show that rectilinear and fundamental harmonic terms account for known properties of natural movements, such as the invariant bell-shaped hand velocity profile during reaching. We implement these dynamics by a neural network model and characterize the tuning properties of the neural integrator cells, the neural oscillator cells, and the inputs to the system. Finally, we propose a method to test our hypothesis that a neural oscillator is a central component in the generation of voluntary movement.     

Nonlinear blind source separation using higher order statistics anda genetic algorithm

This paper presents a novel method for blindly separating unobservable independent source signals from their nonlinear mixtures. The demixing system is modeled using a parameterized neural network whose parameters can be determined under the criterion of independence of its outputs. Two cost functions based on higher order statistics are established to measure the statistical dependence of the outputs of the demixing system. The proposed method utilizes a genetic algorithm (GA) to minimize the highly nonlinear and nonconvex cost functions. The GA-based global optimization technique is able to obtain superior separation solutions to the nonlinear blind separation problem from any random initial values. Compared to conventional gradient-based approaches, the GA-based approach for blind source separation is characterized by high accuracy, robustness, and convergence rate. In particular, it is very suitable for the case of limited available data. Simulation results are discussed to demonstrate that the proposed GA-based approach is capable of separating independent sources from their nonlinear mixtures generated by a parametric separation model     

Adaptive actuator failure compensation control for MIMO systems*

Two adaptive failure compensation control schemes based on MRAC are developed for a class of MIMO LTI systems with unknown actuator failures. An effective controller structure is proposed to achieve the desired plant-model output matching when implemented with matching parameters. Design conditions are specified for such nominal plant-model output matching. Two adaptive versions of the nominal controller are proposed and stable adaptive laws are derived for updating the controller parameters when plant parameters and failure parameters are unknown. All closed-loop signals are bounded and the plant outputs track the given reference outputs asymptotically, despite the uncertainties in actuator failures and plant parameters. Simulation results for an aircraft lateral dynamic model verify the desired adaptive control system performance in the presence of unknown rudder and aileron failures.     

Correlation-based tuning of decoupling multivariable controllers

The iterative method labelled correlation-based tuning (CbT) is considered in this paper for tuning linear time-invariant multivariable controllers. The approach allows one to tune some elements of the controller transfer function matrix to satisfy the desired closed-loop performance, while the other elements are tuned to mutually decouple the closed-loop outputs. Decoupling is achieved by decorrelating a given reference with the non-corresponding outputs. The controller parameters are calculated either by solving a correlation equation (decorrelation procedure) or by minimizing a cross-correlation function (correlation reduction). In addition, the preferred way of exciting a 2×2 system for CbT is investigated via the accuracy of the estimated controller parameters. It is shown that simultaneous excitation of both reference signals does not improve the accuracy of the estimated controller parameters compared to the case of sequential excitation. In fact, one must choose between low experimental cost (simultaneous excitation) and better accuracy of the estimated parameters (sequential excitation). The theoretical results are illustrated via three simulation studies.     

A hybrid genetic algorithm–adaptive network-based fuzzy inference system in prediction of wave parameters

An important issue in application of fuzzy inference systems (FISs) to a class of system identification problems such as prediction of wave parameters is to extract the structure and type of fuzzy if–then rules from an available input–output data set. In this paper, a hybrid genetic algorithm–adaptive network-based FIS (GA–ANFIS) model has been developed in which both clustering and rule base parameters are simultaneously optimized using GAs and artificial neural nets (ANNs). The parameters of a subtractive clustering method, by which the number and structure of fuzzy rules are controlled, are optimized by GAs within which ANFIS is called for tuning the parameters of rule base generated by GAs. The model has been applied in the prediction of wave parameters, i.e. wave significant height and peak spectral period, in a duration-limited condition in Lake Michigan. The data set of year 2001 has been used as training set and that of year 2004 as testing data. The results obtained by the proposed model are presented and analyzed. Results indicate that GA–ANFIS model is superior to ANFIS and Shore Protection Manual (SPM) methods in terms of their prediction accuracy.     

Controlling chaos by GA-based reinforcement learning neural network

Proposes a TD (temporal difference) and GA (genetic algorithm) based reinforcement (TDGAR) neural learning scheme for controlling chaotic dynamical systems based on the technique of small perturbations. The TDGAR learning scheme is a new hybrid GA, which integrates the TD prediction method and the GA to fulfil the reinforcement learning task. Structurally, the TDGAR learning system is composed of two integrated feedforward networks. One neural network acts as a critic network for helping the learning of the other network, the action network, which determines the outputs (actions) of the TDGAR learning system. Using the TD prediction method, the critic network can predict the external reinforcement signal and provide a more informative internal reinforcement signal to the action network. The action network uses the GA to adapt itself according to the internal reinforcement signal. This can usually accelerate the GA learning since an external reinforcement signal may only be available at a time long after a sequence of actions have occurred in the reinforcement learning problems. By defining a simple external reinforcement signal. the TDGAR learning system can learn to produce a series of small perturbations to convert chaotic oscillations of a chaotic system into desired regular ones with a periodic behavior. The proposed method is an adaptive search for the optimum control technique. Computer simulations on controlling two chaotic systems, i.e., the Henon map and the logistic map, have been conducted to illustrate the performance of the proposed method.     

Entrainment-enhanced neural oscillator for rhythmic motion control

We propose a new neural oscillator model to attain rhythmic movements of robotic arms that features enhanced entrainment property. It is known that neural oscillator networks could produce rhythmic commands efficiently and robustly under the changing task environment. However, when a quasi-periodic or non-periodic signal is inputted into the neural oscillator, even the most widely used Matsuoka’s neural oscillator (MNO) may not be entrained to the signal. Therefore, most existing neural oscillator models are only applicable to a particular situation, and if they are coupled to the joints of robotic arms, they may not be capable of achieving human-like rhythmic movement. In this paper, we perform simulations of rotating a crank by a two-link planar arm whose joints are coupled to the proposed entrainment-enhanced neural oscillator (EENO). Specifically, we demonstrate the excellence of EENO and compare it with that of MNO by optimizing their parameters based on simulated annealing (SA). In addition, we show an impressive capability of self-adaptation of EENO that enables the planar arm to make adaptive changes from a circular motion into an elliptical motion. To the authors’ knowledge, this study seems to be the first attempt to enable the oscillator-coupled robotic arm to track a desired trajectory interacting with the environment.     

Reference trajectory tuning of model predictive control

An approach to minimize tuning effort of nominal Model Predictive Control algorithms is proposed. The algorithm dynamically calculates output set points to accommodate user-defined output importance, which is more intuitive than selecting values for the MPC weighing matrices. Instead of tuning the weights on the outputs deviations from their set points, weights on the input values and input increments, which are the usual tuning parameters of MPC, the desired output control performance of the MPC can be specified by performance factors. The proposed method extends the existing methods that consider a reference trajectory for the output tracking to the case of zone control and input targets. The proposed method also assumes that, as in most commercial MPC packages, the controller has two layers: a static layer and an extended dynamic layer. The method is illustrated by three case studies, contemplating both SISO and MIMO systems. It is observed that: the output set point tracking performance can be changed without modifying the MPC tuning weights, the approach is capable of achieving similar performance to conventional MPC tuned by multiobjective optimization techniques from the literature, with a fraction of computer effort, and it can be integrated with Real Time Optimization algorithms to control complex systems, always respecting output constraints.