Evolutionary design and behavior analysis of neuromodulatory neural networks for mobile robots control

Evolutionary Robotics (ER) is one of promising approaches to design robot controllers which essentially have complicated and/or complex properties. In most ER research, the sensory–motor mappings of robots are represented as artificial neural networks, and their connection weights (and sometimes the structure of the networks) can be optimized in the parameter spaces by using evolutionary computation. However, generally, the evolved neural controllers could be fragile in unexperienced environments, especially in real worlds, because the evolutionary optimization processes would be executed in idealized simulators. This is known as the gap problem between the simulated and real worlds. To overcome this, the author focused on evolving an on-line learning ability instead of weight parameters in a simulated environment. According to recent biological findings, actually, the kinds of on-line adaptation abilities can be found in real nervous systems of insects and crustaceans, and it is also known that a variety of neuromodulators (NMs) play crucial roles to regulate the network characteristics (i.e. activating/blocking/changing of synaptic connections). Based on this, a neuromodulatory neural network model was proposed and it was utilized as a mobile robot controller. In the paper, the detail behavior analysis of the evolved neuromodulatory neural network is also discussed.     

机器人视觉伺服中CMAC学习控制系统研究

根据小脑模型关联控制器（CMAC）收敛速度快,适于实时控制系统的特点,设计了一种基于CMAC学习控制方法的机器人视觉伺服系统。在该系统中,CMAC被用作前馈视觉控制器对常规反馈控制器进行补偿。所提出的CMAC控制器替代图像雅可比矩阵来获得目标图像特征和机器人关节运动之间2D/3D变换关系,通过其在线学习,可以使系统对摄像机标定误差不敏感,从而提高系统的鲁棒性。实验证明了所设计控制系统的有效性。     

Imitation learning with spiking neural networks and real-world devices

This article is about a new approach in robotic learning systems. It provides a method to use a real-world device that operates in real-time, controlled through a simulated recurrent spiking neural network for robotic experiments. A randomly generated network is used as the main computational unit. Only the weights of the output units of this network are changed during training. It will be shown, that this simple type of a biological realistic spiking neural network—also known as a neural microcircuit—is able to imitate robot controllers like that incorporated in Braitenberg vehicles. A more non-linear type controller is imitated in a further experiment. In a different series of experiments that involve temporal memory reported in Burgsteiner et al. [2005. In: Proceedings of the 18th International Conference IEA/AIE. Lecture Notes in Artificial Intelligence. Springer, Berlin, pp. 121–130.] this approach also provided a basis for a movement prediction task. The results suggest that a neural microcircuit with a simple learning rule can be used as a sustainable robot controller for experiments in computational motor control.     

Diagnostic neural adaptive control of drifting systems

A hierarchical network of neural network planning and control is employed to successfully accomplish a task such as grasping in a cluttered real world environment. In order for the individual robot joint controllers to follow their specific reference commands, information is shared with other neural network controllers and planners within the hierarchy. Each joint controller is initialized with weights that will acceptably control given a change in any of several crucial parameters across a broad operating range. When increased accuracy is needed as parameters drift, the diagnostic node fuzzy supervisor interprets the controller network's diagnostic outputs and transitions the weights to a closest fit specificchild controller. Future reference commands are in turn influenced by the diagnostic outputs of every robot joint neural network controller. The neural network controller and diagnostics are demonstrated for linear and nonlinear plants.     

A novel fused neural network controller for lateral control of autonomous vehicles

We present a novel fused feed-forward neural network controller inspired by the notion of task decomposition principle. The controller is structurally simple and can be applied to a class of control systems that their control requires manipulation of two input variables. The benchmark problem of inverted pendulum is such example that its control requires availability of the angle as well as the displacement. We demonstrate that the lateral control of autonomous vehicles belongs to this class of systems and successfully apply the proposed controller to this problem. The parameters of the controller are encoded into real value chromosomes for genetic algorithm (GA) optimization. The neural network controller contains three neurons and six connection weights implying a small search space implying faster optimization time due to few controller parameters. The controller is also tested on two benchmark control problems of inverted pendulum and the ball-and-beam system. In particular, we apply the controller to lateral control of a prototype semi-autonomous vehicle. Simulation results suggest a good performance for all the tested systems. To demonstrate the robustness of the controller, we conduct Monte-Carlo evaluations when the system is subjected to random parameter uncertainty. Finally experimental studies on the lateral control of a prototype autonomous vehicle with different speed of operation are included. The simulation and experimental studies suggest the feasibility of this controller for numerous applications.     

基于神经网络的微分对策控制器设计

采用伴随－BP技术，将微分对策的两点边值求解问题转化两个神经网络的学习问题，训练后的两个神经网络分别作为对策双方的最优控制器在线使用，避免了直接求解复杂的两点边值问题，对追逃微分对策问题的仿真结果表明，该方法对初始条件和噪声具有较好的鲁棒性。     

Evolving an interval type-2 fuzzy PID controller for the redundant robotic manipulator

Robotic manipulators are a multi-input multi-output, dynamically coupled, highly time-varying, complex and highly nonlinear systems wherein the external disturbances, parameter variations, and random noise adversely affects the performance of the robotic system. Therefore, in order to deal with such complexities, however, an intriguing task for control researchers, these systems require an efficient and robust controller. In this paper, a novel application of genetic algorithms (GA) optimization approach to optimize the scaling factors of interval type-2 fuzzy proportional derivative plus integral (IT2FPD+I) controllers is proposed for 5-DOF redundant robot manipulator for trajectory tracking task. All five controllers' parameters are optimized simultaneously. Further, a procedure for selecting appropriate initial search space is also demonstrated. In order to make a fair comparison between different controllers, the tuning of each of the controllers' parameters is done with GA. This optimization technique uses the time domain optimal tuning while minimizing the fitness function as the sum of integral of multiplication of time with square error (ITSE) for each joint. To ascertain the effectiveness of IT2FPID controller, it is compared against type-1 fuzzy PID (T1FPID) and conventional PID controllers. Furthermore, robustness testing of developed IT2FPID controller for external disturbances, parameter variations, and random noise rejection is also investigated. Finally, the experimental study leads us to claim that our proposed controller can not only assure best trajectory tracking in joint and Cartesian space, but also improves the robustness of the systems for external disturbances, parameter variations, and random noise.     

基于神经网络的机器人自学习控制器

提出一种神经网络与PID控制相结合的机器人自学习控制器.为加快神经网络的 学习收敛性,研究了有效的优化学习算法.以两关节机器人为对象的仿真表明,该控制器使机 器人跟踪希望轨迹,其系统响应、跟踪精度和鲁棒性优于常规的控制策略.     

ARTIFICIAL INTELLIGENCE AND GRAPH THEORY TOOLS FOR DESCRIBING SWITCHED LINEAR CONTROL SYSTEMS

This paper develops a representation of multi-model based controllers using artificial intelligence techniques. These techniques will be graph theory, neural networks, genetic algorithms, and fuzzy logic. Thus, graph theory is used to describe in a formal and concise way the switching mechanism between the various plant parameterizations of the switched system. Moreover, the interpretation of multi-model controllers in an artificial intelligence frame will allow the application of each specific technique to the design of improved multi-model based controllers. The obtained artificial intelligence-based multi-model controllers are compared with classic single model-based ones. It is shown through simulation examples that a transient response improvement can be achieved by using multi-estimation based techniques. Furthermore, a method for synthesizing multi-model-based neural network controllers from already designed single model-based ones is presented, extending the applicability of this kind of technique to a more general type of controller. Also, some applications of genetic algorithms and fuzzy logic to multi-model controller design are proposed. In particular, the mutation operation from genetic algorithms inspires a robustness test, which consists of a random modification of the estimates which is used to select the one leading to the better identification performance towards parameterizing online the adaptive controller. Such a test is useful for plants operating in a noisy environment. The proposed robustness test improves the selection of the plant model used to parameterize the adaptive controller in comparison to classic multi-model schemes where the controller parameterization choice is basically taken based on the identification accuracy of each model. Moreover, the fuzzy logic approach suggests new ideas to the design of multi-estimation structures, which can be applied to a broad variety of adaptive controllers such as robotic manipulator controller design.     

On line discrete-time control of industrial robots

A recent discrete-time robotic model is employed and linearized to develop three different LQ-type robot controllers. These controllers are: optimal time-varying linear-quadratic controller (OTVLQ), OTVLQ controller with steady-state error elimination, and one-step ahead optimal LQ controller. These controllers require moderate computational effort and are offered for microprocessor-based implementation. Experimental results on a KUKA IR 160/15 robotic model verified the success and efficiency of the controllers.     

多变量非线性自整定PID控制器 *

本文提出一种基于神经网络的多变量非线性自整定ＰＩＤ控制器，通过神经网络权值的学习在线自动整定控制器参数，将其用于某水浴系统的温度多变量控制，仿真结果令人满意。该控制器的设计无需对象模型，具有响应豆腐快，抗干扰能力强和鲁棒性好等特点，控制器不仅算法简单，实现简易，而且适用范围广。     

Adaptive Control of Space Structures via Recurrent Neural Networks

The paper presents a study on the adaptive control of large space structures, based on the use of real time recurrent neural networks. The controller relies on two interconnected neural networks. One is used for system identification and works in parallel with the real structure. The other performs the actual control task, and feeds back onto the structure and the identification net. The two networks are trained on line, with their synaptic weights adapting to time varying system configurations. A series of numerical examples on the model of a large structure provides the basis for understanding the performances and robustness of the method proposed, and indicates the feasibility of real on line applications. This revised version was published online in July 2006 with corrections to the Cover Date.     

Dissolved oxygen control for activated sludge processes

Controllers for dissolved oxygen reference trajectory tracking for activated sludge processes are proposed and investigated. A nonlinear model predictive controller and a direct reference adaptive controller are investigated. Both the nutrient and the phosphorous removal from a wastewater by its biological treatment using an activated sludge technology are considered. An approach to the controller design utilises a structure of the dissolved oxygen dynamics and its two time scales: fast and slow. The predictive controllers offer good tracking performance and robustness. The direct model reference adaptive controller is much simpler to implement. However, it is more difficult to compromise between tracking accuracy and rate of change and magnitudes of the control actions. The controllers are validated by simulation using real data sets and an ASM2d model of the biological reactor.     

Performance analysis of fractional order fuzzy PID controllers applied to a robotic manipulator

A two-link robotic manipulator is a Multi-Input Multi-Output (MIMO), highly nonlinear and coupled system. Therefore, designing an efficient controller for this system is a challenging task for the control engineers. In this paper, the Fractional Order Fuzzy Proportional-Integral-Derivative (FOFPID) controller for a two-link planar rigid robotic manipulator for trajectory tracking problem is investigated. Robustness testing of FOFPID controller for model uncertainties, disturbance rejection and noise suppression is also investigated. To study the effectiveness of FOFPID controller, its performance is compared with other three controllers namely Fuzzy PID (FPID), Fractional Order PID (FOPID) and conventional PID. For tuning of parameters of all the controllers, Cuckoo Search Algorithm (CSA) optimization technique was used. Two performance indices namely Integral of Absolute Error (IAE) and Integral of Absolute Change in Controller Output (IACCO) having equal weightage for both the links are considered for minimization. Numerical simulation results clearly indicate the superiority of FOFPID controller over the other controllers for trajectory tracking, model uncertainties, disturbance rejection and noise suppression.     

Neuroevolution of Inverted Pendulum Control: A Comparative Study of Simulation Techniques

The inverted pendulum control problem is a classical benchmark in control theory. Amongst the approaches to developing control programs for an inverted pendulum, the evolution of Artificial Neural Network (ANN) based controllers has received some attention. The authors have previously shown that Evolutionary Robotics (ER) can successfully be used to evolve inverted pendulum stabilization controllers in simulation and that these controllers can transfer successfully from simulation to real-world robotic hardware. During this process, use was made of robotic simulators constructed from empirically-collected data and based on ANNs. The current work aims to compare this method of simulator construction with the more traditional method of building robotic simulators based on physics equations governing the robotic system under consideration. In order to compare ANN-based and physics-based simulators in the evolution of inverted pendulum controllers, a real-world wheeled inverted pendulum robot was considered. Simulators based on ANNs as well as on a system of ordinary differential equations describing the dynamics of the robot were developed. These two simulation techniques were then compared by using each in the simulation-based evolution of controllers. During the evolution process, the effects of injecting different levels of noise into the simulation was furthermore studied. Encouraging results were obtained, with controllers evolved using ANN-based simulators and realistic levels of noise outperforming those evolved using the physics-based simulators.     

Robust reinforcement learning control with static and dynamic stability

Robust control theory is used to design stable controllers in the presence of uncertainties. This provides powerful closed‐loop robustness guarantees, but can result in controllers that are conservative with regard to performance. Here we present an approach to learning a better controller through observing actual controlled behaviour. A neural network is placed in parallel with the robust controller and is trained through reinforcement learning to optimize performance over time. By analysing nonlinear and time‐varying aspects of a neural network via uncertainty models, a robust reinforcement learning procedure results that is guaranteed to remain stable even as the neural network is being trained. The behaviour of this procedure is demonstrated and analysed on two control tasks. Results show that at intermediate stages the system without robust constraints goes through a period of unstable behaviour that is avoided when the robust constraints are included. Copyright © 2001 John Wiley & Sons, Ltd.     

采掘机器人的模糊监督——神经网络控制器技术

介绍一种基于规则的自学习神经网络控制器在采掘机器人上的应用。它根据实时执行的结果，采用多步学习－模糊监督学习方法，修正神经网络的教师信号，使控制算法简化，提高了计算的实时性，加快了学习速度实验验证了采用该方法取得的一些结果。     

An analog CMOS central pattern generator for interlimb coordination in quadruped locomotion.

This paper proposes a neuromorphic analog CMOS controller for interlimb coordination in quadruped locomotion. Animal locomotion, such as walking, running, swimming, and flying, is based on periodic rhythmic movements. These rhythmic movements are driven by the biological neural network, called the central pattern generator (CPG). In recent years, many researchers have applied CPG to locomotion controllers in robotics. However, most of these have been developed with digital processors and, thus, have several problems, such as high power consumption. In order to overcome such problems, a CPG controller with analog CMOS circuit is proposed. Since the CMOS transistors in the circuit operate in their subthreshold region and under low supply voltage, the controller can reduce power consumption. Moreover, low-cost production and miniaturization of controllers are expected. We have shown through computer simulation, such circuit has the capability to generate several periodic rhythmic patterns and transitions between their patterns promptly.     

Evolutionary robotics: the Sussex approach

We give an overview of evolutionary robotics research at Sussex over the last five years. We explain and justify our distinctive approaches to (artificial) evolution, and to the nature of robot control systems that are evolved. Results are presented from research with evolved controllers for autonomous mobile robots, simulated robots, co-evolved animats, real robots with software controllers, and a real robot with a controller directly evolved in hardware.     

A new class of nonlinear PID controllers with robotic applications

This paper introduces a new class of simple nonlinear PID controllers and provides a formal treatment of their stability analysis. These controllers are comprised of a sector-bounded nonlinear gain in cascade with a linear fixed-gain P, PD, PI, or PID controller. Three simple nonlinear gains are proposed: the sigmoidal function, the hyperbolic function, and the piecewise–linear function. The systems to be controlled are assumed to be modeled or approximated by second-order transfer functions, which can represent many robotic applications. The stability of the closed-loop systems incorporating nonlinear P, PD, PI, and PID controllers are investigated using the Popov stability criterion. It is shown that for P and PD controllers, the nonlinear gain is unbounded for closed-loop stability. For PI and PID controllers, simple expressions are derived that relate the controller gains and system parameters to the maximum allowable nonlinear gain for stability. A numerical example is given for illustration. The stability of partially-nonlinear PID controllers is also discussed. Finally, the nonlinear PI controller is implemented as a force controller on a robotic arm and experimental results are presented. These results demonstrate the superior performance of the nonlinear PI controller relative to a fixed-gain PI controller. © 1998 John Wiley & Sons, Inc. 15: 161–181, 1998