Indirect adaptive control for synchronous Generator: comparison of MLP/RBF neural networks approach with Lyapunov stability analysis

This paper compares two indirect adaptive neurocontrollers, namely a multilayer perceptron neurocontroller (MLPNC) and a radial basis function neurocontroller (RBFNC) to control a synchronous generator. The different damping and transient performances of two neurocontrollers are compared with those of conventional linear controllers, and analyzed based on the Lyapunov direct method.     

Comparison of heuristic dynamic programming and dual heuristicprogramming adaptive critics for neurocontrol of a turbogenerator

This paper presents the design of an optimal neurocontroller that replaces the conventional automatic voltage regulator (AVR) and the turbine governor for a turbogenerator connected to the power grid. The neurocontroller design uses a novel technique based on the adaptive critic designs (ACDs), specifically on heuristic dynamic programming (HDP) and dual heuristic programming (DHP). Results show that both neurocontrollers are robust, but that DHP outperforms HDP or conventional controllers, especially when the system conditions and configuration change. This paper also shows how to design optimal neurocontrollers for nonlinear systems, such as turbogenerators, without having to do continually online training of the neural networks, thus avoiding risks of instability.     

Neuromorphic learning of continuous-valued mappings fromnoise-corrupted data

The effect of noise on the learning performance of the backpropagation algorithm is analyzed. A selective sampling of the training set is proposed to maximize the learning of control laws by backpropagation, when the data have been corrupted by noise. The training scheme is applied to the nonlinear control of a cart-pole system in the presence of noise. The neural computation provides the neurocontroller with good noise-filtering properties. In the presence of plant noise, the neurocontroller is found to be more stable than the teacher. A novel perspective on the application of neural network technology to control engineering is presented.     

Implementation of adaptive critic-based neurocontrollers for turbogenerators in a multimachine power system

This paper presents the design and practical hardware implementation of optimal neurocontrollers that replace the conventional automatic voltage regulator (AVR) and the turbine governor of turbogenerators on multimachine power systems. The neurocontroller design uses a powerful technique of the adaptive critic design (ACD) family called dual heuristic programming (DHP). The DHP neurocontrollers' training and testing are implemented on the Innovative Integration M67 card consisting of the TMS320C6701 processor. The measured results show that the DHP neurocontrollers are robust and their performance does not degrade unlike the conventional controllers even when a power system stabilizer (PSS) is included, for changes in system operating conditions and configurations. This paper also shows that it is possible to design and implement optimal neurocontrollers for multiple turbogenerators in real time, without having to do continually online training of the neural networks, thus avoiding risks of instability.     

Learning rules for neuro-controller via simultaneous perturbation

This paper describes learning rules using simultaneous perturbation for a neurocontroller that controls an unknown plant. When we apply a direct control scheme by a neural network, the neural network must learn an inverse system of the unknown plant. In this case, we must know the sensitivity function of the plant using a kind of the gradient method as a learning rule of the neural network. On the other hand, the learning rules described here do not require information about the sensitivity function. Some numerical simulations of a two-link planar arm and a tracking problem for a nonlinear dynamic plant are shown.     

Local control gradients criterion for selection of neuroemulators for model reference adaptive neurocontrol

We discuss neural identification and control of nonlinear dynamic plant. We analyze a problem of selection of a proper neuroemulator for training neurocontrollers, and we propose a new effective criterion based on the analysis of local gradients of neuroemulator??s input neurons. We present results of numerical simulations of neurocontroller training by a gradient descent method and by an Extended Kalman Filter method.     

Genetic reinforcement learning through symbiotic evolution forfuzzy controller design

An efficient genetic reinforcement learning algorithm for designing fuzzy controllers is proposed in this paper. The genetic algorithm (GA) adopted in this paper is based upon symbiotic evolution which, when applied to fuzzy controller design, complements the local mapping property of a fuzzy rule. Using this Symbiotic-Evolution-based Fuzzy Controller (SEFC) design method, the number of control trials, as well as consumed CPU time, are considerably reduced when compared to traditional GA-based fuzzy controller design methods and other types of genetic reinforcement learning schemes. Moreover, unlike traditional fuzzy controllers, which partition the input space into a grid, SEFC partitions the input space in a flexible way, thus creating fewer fuzzy rules. In SEFC, different types of fuzzy rules whose consequent parts are singletons, fuzzy sets, or linear equations (TSK-type fuzzy rules) are allowed. Further, the free parameters (e.g., centers and widths of membership functions) and fuzzy rules are all tuned automatically. For the TSK-type fuzzy rule especially, which put the proposed learning algorithm in use, only the significant input variables are selected to participate in the consequent of a rule. The proposed SEFC design method has been applied to different simulated control problems, including the cart-pole balancing system, a magnetic levitation system, and a water bath temperature control system. The proposed SEFC has been verified to be efficient and superior from these control problems, and from comparisons with some traditional GA-based fuzzy systems.     

Learning Dynamic Obstacle Avoidance for a Robot Arm Using Neuroevolution

Neural networks can be evolved to control robot manipulators in tasks like target tracking and obstacle avoidance in complex environments. Neurocontrollers are robust to noise and can be adapted to different environments and robot configurations. In this paper, neurocontrollers were evolved to position the end effector of a robot arm close to a target in three different environments: environments without obstacles, environments with stationary obstacles, and environments with moving obstacles. The evolved neurocontrollers perform qualitatively like inverse kinematic controllers in environments with no obstacles and like path-planning controllers based on Rapidly-exploring random trees in environments with obstacles. Unlike inverse kinematic controllers and path planners, the approach reliably generalizes to environments with moving obstacles, making it possible to use it in natural environments.     

Training Recurrent Neurocontrollers for Robustness With Derivative-Free Kalman Filter

We are interested in training neurocontrollers for robustness on discrete-time models of physical systems. Our neurocontrollers are implemented as recurrent neural networks (RNNs). A model of the system to be controlled is known to the extent of parameters and/or signal uncertainties. Parameter values are drawn from a known distribution. For each instance of the model with specified parameters, a recurrent neurocontroller is trained by evaluating sensitivities of the model outputs to perturbations of the neurocontroller weights and incrementally updating the weights. Our training process strives to minimize a quadratic cost function averaged over many different models. In the end, the process yields a robust recurrent neurocontroller, which is ready for deployment with fixed weights. We employ a derivative-free Kalman filter algorithm proposed by Norgaard and extended by Feldkamp (2001) and Feldkamp (2002) to neural network training. Our training algorithm combines effectiveness of a second-order training method with universal applicability to both differentiable and nondifferentiable systems. Our approach is that of model reference control, and it extends significantly the capabilities proposed by Prokhorov (2001). We illustrate it with two examples     

Training recurrent neurocontrollers for real-time applications.

In this paper, we introduce a new approach to train recurrent neurocontrollers for real-time applications. We begin with training a recurrent neurocontroller for robustness on high-fidelity models of physical systems. For training, we use a recently developed derivative-free Kalman filter method which we enhance for controller training. After training, we fix weights of our recurrent neurocontroller and deploy it in an embedded environment. Then, we carry out additional training of the neurocontroller by adapting in real time its internal state (short-term memory), rather than its weights (long-term memory). Such real-time training is done with a new combination of simultaneous perturbation stochastic approximation (SPSA) and adaptive critic. Our critic is also a recurrent neural network (RNN), and it is trained by stochastic meta-descent (SMD) for increased efficiency. Our approach is applied to two important practical problems, electronic throttle control and hybrid electric vehicle control, with apparent performance improvement.     

Environment-Optimized Speech Enhancement

In this paper, we present a training-based approach to speech enhancement that exploits the spectral statistical characteristics of clean speech and noise in a specific environment. In contrast to many state-of-the-art approaches, we do not model the probability density function (pdf) of the clean speech and the noise spectra. Instead, subband-individual weighting rules for noisy speech spectral amplitudes are separately trained for speech presence and speech absence from noise recordings in the environment of interest. Weighting rules for a variety of cost functions are given; they are parameterized and stored as a table look-up. The speech enhancement system simply works by computing the weighting rules from the table look-up indexed by the a posteriori signal-to-noise ratio (SNR) and the a priori SNR for each subband computed on a Bark scale. Optimized for an automotive environment, our approach outperforms known-environment-independent-speech enhancement techniques, namely the a priori SNR-driven Wiener filter and the minimum mean square error (MMSE) log-spectral amplitude estimator, both in terms of speech distortion and noise attenuation.     

Self-organizing fuzzy control of active suspension systems

A self-organizing fuzzy controller (SOFC) is proposed to control an active suspension system and evaluate its control performance. In complicated nonlinear system control, the SOFC continually updates the learning strategy in the form of fuzzy rules during the control process. The learning rate and the weighting distribution value of the controller are hard to regulate, so its fuzzy control rules may be excessively modified such that the system response generally causes an oscillatory phenomenon. Two fuzzy-logic controllers were designed according to the system output error and the error change, and introduced to the SOFC to determine the appropriate parameters of the learning rate and the weighting distribution, to eliminate this oscillation. This new modifying self-organizing fuzzy-control approach can effectively improve the control performance of the system, reduce the time consumed to establish a suitable fuzzy rule table, and support practically convenient fuzzy-controller applications in an active suspension control system, as verified experimentally.     

Self-adaptation of learning rate in XCS working in noisy and dynamic environments

An extended classifier system (XCS) is an adaptive rule-based technique that uses evolutionary search and reinforcement learning to evolve complete, accurate, and maximally general payoff map of an environment. The payoff map is represented by a set of condition-action rules called classifiers. Despite this insight, till now parameter-setting problem associated with LCS/XCS has important drawbacks. Moreover, the optimal values of some parameters are strongly influenced by properties of the environment like its complexity, changeability, and the level of noise. The aim of this paper is to overcome some of these difficulties by a self-adaptation of a learning rate parameter, which plays a key role in reinforcement learning, since it is used for updates of classifier parameters: prediction, prediction error, fitness, and action set estimation. Self-adaptive control of prediction learning rate is investigated in the XCS, whereas the fitness and error learning rates remain fixed. Simultaneous self-adaptation of prediction learning rate and mutation rate also undergo experiments. Self-adaptive XCS solves one-step problems in noisy and dynamic environments.     

基于自适应评价UPFC神经网络控制器设计

对含UPFC(统一潮流控制器)的电力系统提出一种新型的非线性最优神经网络控制器。启发式动态规划(HDP)是自适应评价设计(ACDs)体系中的一员,采用HDP来设计UPFC神经网络控制器。和传统的PI控制器相比,这种神经网络控制器能够提供非线性最优控制。仿真结果表明,此种控制器具有很好的控制效果。     

CSTR系统的基于CMFC神经元网络的学习控制研究

CMAC(Cerebellar Model Articulation Controller或Cerebellar Model Arithmetic Computer)神经元网络是由Albus提出的一种表达复杂非线性函数的表格查询的自适应系统。本文将CMAC应用到具体的连续搅拌反应釜(CSTR)系统的学习控制研究中,仿真结果表明,该学习控制策略具有较强的自学习能力且容易实现,对于改善非线性控制的性能,不失为一种有益的尝试。     

Behavioral control through evolutionary neurocontrollers for autonomous mobile robot navigation

This paper deals with the study of scaling up behaviors in evolutive robotics (ER). Complex behaviors were obtained from simple ones. Each behavior is supported by an artificial neural network (ANN)-based controller or neurocontroller. Hence, a method for the generation of a hierarchy of neurocontrollers, resorting to the paradigm of Layered Evolution (LE), is developed and verified experimentally through computer simulations and tests in a Khepera^® micro-robot. Several behavioral modules are initially evolved using specialized neurocontrollers based on different ANN paradigms. The results show that simple behaviors coordination through LE is a feasible strategy that gives rise to emergent complex behaviors. These complex behaviors can then solve real-world problems efficiently. From a pure evolutionary perspective, however, the methodology presented is too much dependent on user’s prior knowledge about the problem to solve and also that evolution take place in a rigid, prescribed framework. Mobile robot’s navigation in an unknown environment is used as a test bed for the proposed scaling strategies.     

Fynesse: An architecture for integrating prior knowledge in autonomously learning agents

Reinforcement learning is an optimisation technique for applications like control or scheduling problems. It is used in learning situations, where success and failure of the system are the only training information. Unfortunately, we have to pay a price for this powerful ability: long training times and the instability of the learning process are not tolerable for industrial applications with large continuous state spaces. From our point of view, the integration of prior knowledge is a key mechanism for making autonomous learning practicable for industrial applications. The learning control architecture Fynesse provides a unified view onto the integration of prior control knowledge in the reinforcement learning framework. In this way, other approaches in this area can be embedded into Fynesse. The key features of Fynesse are (1) the integration of prior control knowledge like linear controllers, control characteristics or fuzzy controllers, (2) autonomous learning of control strategies and (3) the interpretation of learned strategies in terms of fuzzy control rules. The benefits and problems of different methods for the integration of a priori knowledge are demonstrated on empirical studies.The research project F ynesse was supported by the Deutsche Forschungsgemeinschaft (DFG).     

Designing Decentralized Controllers for Distributed-Air-Jet MEMS-Based Micromanipulators by Reinforcement Learning

Distributed-air-jet MEMS-based systems have been proposed to manipulate small parts with high velocities and without any friction problems. The control of such distributed systems is very challenging and usual approaches for contact arrayed system don’t produce satisfactory results. In this paper, we investigate reinforcement learning control approaches in order to position and convey an object. Reinforcement learning is a popular approach to find controllers that are tailored exactly to the system without any prior model. We show how to apply reinforcement learning in a decentralized perspective and in order to address the global-local trade-off. The simulation results demonstrate that the reinforcement learning method is a promising way to design control laws for such distributed systems.     

Controlling fast spring-legged locomotion with artificial neural networks

Controlling the model of an one-legged robot is investigated. The model consists merely of a mass less spring attached to a point mass. The motion of this system is characterised by repeated changes between ground contact and flight phases. It can be kept in motion by active control only. Robots that are suited for fast legged locomotion require different hardware layouts and control approaches in contrast to slow moving ones. The spring mass system is a simple model that describes this principle movement of a spring-legged robot. Multi-Layer-Perceptrons (MLPs), Radial Basis Functions (RBFs) and Self-Organising Motoric Maps (SOMMs) were used to implement neurocontrollers for such a movement system. They all prove to be suitable for control of the movement. This is also shown by an experiment where the environment of the spring-mass system is changed from even to uneven ground. The neurocontroller is performing well with this additional complexity without being trained for it.     

Ant Colony Optimization Incorporated With Fuzzy Q-Learning for Reinforcement Fuzzy Control

This paper proposes the design of fuzzy controllers by ant colony optimization (ACO) incorporated with fuzzy-Q learning, called ACO-FQ, with reinforcements. For a fuzzy inference system, we partition the antecedent part a priori and then list all candidate consequent actions of the rules. In ACO-FQ, the tour of an ant is regarded as a combination of consequent actions selected from every rule. Searching for the best one among all combinations is partially based on pheromone trail. We assign to each candidate in the consequent part of the rule a corresponding Q-value. Update of the Q-value is based on fuzzy-Q learning. The best combination of consequent values of a fuzzy inference system is searched according to pheromone levels and Q-values. ACO-FQ is applied to three reinforcement fuzzy control problems: (1) water bath temperature control; (2) magnetic levitation control; and (3) truck backup control. Comparisons with other reinforcement fuzzy system design methods verify the performance of ACO-FQ.