Neural network-based flight control systems: Present and future

As the first review in this field, this paper presents an in-depth mathematical view of Intelligent Flight Control Systems (IFCSs), particularly those based on artificial neural networks. The rapid evolution of IFCSs in the last two decades in both the methodological and technical aspects necessitates a comprehensive view of them to better demonstrate the current stage and the crucial remaining steps towards developing a truly intelligent flight management unit. To this end, in this paper, we will provide a detailed mathematical view of Neural Network (NN)-based flight control systems and the challenging problems that still remain. The paper will cover both the model-based and model-free IFCSs. The model-based methods consist of the basic feedback error learning scheme, the pseudocontrol strategy, and the neural backstepping method. Besides, different approaches to analyze the closed-loop stability in IFCSs, their requirements, and their limitations will be discussed in detail. Various supplementary features, which can be integrated with a basic IFCS such as the fault-tolerance capability, the consideration of system constraints, and the combination of NNs with other robust and adaptive elements like disturbance observers, would be covered, as well. On the other hand, concerning model-free flight controllers, both the indirect and direct adaptive control systems including indirect adaptive control using NN-based system identification, the approximate dynamic programming using NN, and the reinforcement learning-based adaptive optimal control will be carefully addressed. Finally, by demonstrating a well-organized view of the current stage in the development of IFCSs, the challenging issues, which are critical to be addressed in the future, are thoroughly identified. As a result, this paper can be considered as a comprehensive road map for all researchers interested in the design and development of intelligent control systems, particularly in the field of aerospace applications.     

Fractional-order integral resonant control of collocated smart structures

This paper proposes a fractional-order integral controller, FI, which is a simple, robust and well-performing technique for vibration control in smart structures with collocated sensors and actuators. This new methodology is compared with the most relevant controllers for smart structures. It is demonstrated that the proposed controller improves the robustness of the closed-loop system to changes in the mass of the payload at the tip. The previous controllers are robust in the sense of being insensitive to spillover and maintaining the closed-loop stability when changes occur in the plant parameters. However, the phase margin of such closed-loop systems (and, therefore, their damping) may change significantly as a result of these parameter variations. In this paper the possibility of increasing the phase margin robustness by using a fractional-order controller with a very simple structure is explored. This controller has been applied to an experimental smart structure, and simulations and experiments have shown the improvement attained with this new technique in the removal of the vibration in the structure when the mass of the payload at the tip changes.     

A learning architecture based on reinforcement learning for adaptive control of the walking machine LAURON

The learning of complex control behaviour of autonomous mobile robots is one of the actual research topics. In this article an intelligent control architecture is presented which integrates learning methods and available domain knowledge. This control architecture is based on Reinforcement Learning and allows continuous input and output parameters, hierarchical learning, multiple goals, self-organized topology of the used networks and online learning. As a testbed this architecture is applied to the six-legged walking machine LAURON to learn leg control and leg coordination.     

Concepts and Facilities of a Neural Reinforcement Learning Control Architecture for Technical Process Control

The paper presents the concepts of a neural control architecture that is able to learn high quality control behaviour in technical process control from scratch. As the input to the learning system, only the control target must be specified. In the first part of the article, the underlying theoretical principles of dynamic programming methods are explained, and their adaptation to the context of technical process control is described. The second part discusses the basic capabilities of the learning system on a typical benchmark problem, where a special focus lies on the quality of the acquired control law. The application to a highly nonlinear chemical reactor and to an instable multi-output system shows the ability of the proposed neural control architecture to learn even difficult control strategies from scratch.     

Control of a nonlinear liquid level system using a new artificial neural network based reinforcement learning approach

Most industrial processes exhibit inherent nonlinear characteristics. Hence, classical control strategies which use linearized models are not effective in achieving optimal control. In this paper an Artificial Neural Network (ANN) based reinforcement learning (RL) strategy is proposed for controlling a nonlinear interacting liquid level system. This ANN-RL control strategy takes advantage of the generalization, noise immunity and function approximation capabilities of the ANN and optimal decision making capabilities of the RL approach. Two different ANN-RL approaches for solving a generic nonlinear control problem are proposed and their performances are evaluated by applying them to two benchmark nonlinear liquid level control problems. Comparison of the ANN-RL approach is also made to a discretized state space based pure RL control strategy. Performance comparison on the benchmark nonlinear liquid level control problems indicate that the ANN-RL approach results in better control as evidenced by less oscillations, disturbance rejection and overshoot.     

基于自适应评价的非线性系统神经网络控制

针对一类非线性系统,提出了一种自适应评价方法．该方法可以控制系统输出对参考信号进行跟踪,其评价函数可直接解析求出．该方法只需一个动作网络用于产生控制动作,并且方法中的网络权值初始化可随机选取．使用Lyapunov方法对整个系统的动态性能进行分析,证明了在一定条件下此方法能保证闭环误差及网络权值一致最终有界．仿真结果与理论分析相一致,证明了所提出方法的有效性．     

Enhancing neural control systems by fuzzy logic and evolutionary reinforcement

Neural networks are widely used for system modelling and control because of their ability to approximate complex non-linear functions. Fuzzy systems, similarly, have been shown to be able to approximate or model any nonlinear system. Fuzzy-logic and neural systems, however, have very contrasting application requirements and it has been said that their integration offers a facility to bridge symbolic knowledge processing and connectionist learning. The significance of the integration becomes more apparent by considering their disparities. Neural networks do not provide a strong scheme for knowledge representation, while fuzzy systems do not possess capabilities for automated learning. On the other hand, another learning method has emerged recently, as an alternative to inductive techniques used with neural networks, namely, genetic or evolutionary learning. This paper will present a technique for the fusion of the three paradigms in a learning control context. It will describe a type of learning, known as Evolutionary Algorithm Reinforcement Learning (EARL), which is used to optimise a fuzzy neural control system. An application case study is also presented.     

An active vibration absorber of smart panel by using a decomposed parallel fuzzy control structure

This study proposes novel development of piezoelectric actuators as elements of smart structures. The primary goal of this work is to control actively the vibrations of smart structures by using a decomposed parallel fuzzy control approach. This study attempts to demonstrate the general methodology by decomposing a large-scale system into smaller subsystems in a parallel structure so that the proposed fuzzy control methodology developed here can be applied for studying a complex system. The paper shows a very promising application field, while the results obtained show that the fuzzy approach can be much better than other conventional control methods used till now. Consequently, the creation of new control structures is very important for new application possibilities of fuzzy control because they enable one to minimize the completeness of the problem solved as well as to utilize the capacity of computational technologies in full range.     

Intelligent fault-tolerant system of vibration control for flexible structures

This paper describes an intelligent fault-tolerant control method for vibration control of flexible structures. We consider a case where the fault phenomena of the control system for flexible structures can be treated as a change of system parameters. Therefore, the adaptive control method can be applied to a vibration control system for flexible structures with a fault. In this paper, a neural network (NN) adaptive control system is used to compensate for the change in the parameters of a plant with a fault. When the characteristics of the plant and of a nominal model have been agreed by a NN adaptive control system, the control method designed for the nominal model, such as decoupling feedback control or linearizing feedback control, can be used even if the change in the system parameters has been caused by a fault. To confirm the effectiveness of the proposed fault-tolerant control method, the simulational results from a 5-link robotic arm are shown at the end of the paper. This work was presented, in part, at the Fourth International Symposium on Artificial Life and Robotics, Oita, Japan, January 19–22, 1999     

A message efficient intersection control algorithm for intelligent transportation in smart cities

IoV based traffic control at intersections has been recently studied widely to realize intelligent transportation. However, existing solutions usually suffer from high communication cost, which may cause serious packet interference and long time delays. In this paper, we design a new algorithm to realize intersection control via vehicular ad hoc networking. We basically adopt the approach of mutual exclusion, which can let vehicles at an intersection compete for the privilege of passing via message exchange. Different from existing works, we adopt a group based privilege competition design. By letting only group head handling requests from other lanes, message cost can be significantly reduced. Reducing communication cost is a significant issue in IoV based intersection control because high communication cost will cause packet interference and packet losses, which further result in safety problems. The key challenges lie in deterring group size and recognizing group head. Compared similar works, our new algorithm can conduct intersection control with much less message cost, and its advantage is validated by simulations using ns3.     

A middleware framework for application-aware and user-specific energy optimization in smart mobile devices

Mobile battery-operated devices are becoming an essential instrument for business, communication, and social interaction. In addition to the demand for an acceptable level of performance and a comprehensive set of features, users often desire extended battery lifetime. In fact, limited battery lifetime is one of the biggest obstacles facing the current utility and future growth of increasingly sophisticated “smart” mobile devices. This paper proposes a novel application-aware and user-interaction aware energy optimization middleware framework (AURA) for pervasive mobile devices. AURA optimizes CPU and screen backlight energy consumption while maintaining a minimum acceptable level of performance. The proposed framework employs a novel Bayesian application classifier and management strategies based on Markov Decision Processes and Q-Learning to achieve energy savings. Real-world user evaluation studies on Google Android based HTC Dream and Google Nexus One smartphones running the AURA framework demonstrate promising results, with up to 29% energy savings compared to the baseline device manager, and up to 5×savings over prior work on CPU and backlight energy co-optimization.     

Reinforcement learning-based event-triggered optimal control for unknown nonlinear systems with input delay

The optimal control issue of discrete-time nonlinear unknown systems with time-delay control input is the focus of this work. In order to reduce communication costs, a reinforcement learning-based event-triggered controller is proposed. By applying the proposed control method, closed-loop system's asymptotic stability is demonstrated, and a maximum upper bound for the infinite-horizon performance index can be calculated beforehand. The event-triggered condition requires the next time state information. In an effort to forecast the next state and achieve optimal control, three neural networks (NNs) are introduced and used to approximate system state, value function, and optimal control. Additionally, a M NN is utilized to cope with the time-delay term of control input. Moreover, taking the estimation errors of NNs into account, the uniformly ultimately boundedness of state and NNs weight estimation errors can be guaranteed. Ultimately, the validity of proposed approach is illustrated by simulations.     

Deep learning for pedestrian collective behavior analysis in smart cities: A model of group trajectory outlier detection

This paper introduces a new model to identify collective abnormal human behaviors from large pedestrian data in smart cities. To accurately solve the problem, several algorithms have been proposed in this paper. These can be split into two categories. First, algorithms based on data mining and knowledge discovery, which study the different correlation among human behavioral data, and identify the collective abnormal human behavior from knowledge extracted. Secondly, algorithms exploring convolution deep neural networks, which learn different features of historical data to determine the collective abnormal human behaviors. Experiments on an actual human behaviors database have been carried out to demonstrate the usefulness of the proposed algorithms. The results show that the deep learning solution outperforms both data mining as well as the state-of-the-art solutions in terms of runtime and accuracy performance. In particular, for large datasets, the accuracy of the deep learning solution reaches 88%, however other solutions do not exceed 81%. Additionally, the runtime of the deep learning solution is below 50 seconds, whereas other solutions need more than 80 seconds for analyzing the same database.     

A review of neural networks for statistical process control

This paper aims to take stock of the recent research literature on application of Neural Networks (NNs) to the analysis of Shewhart's traditional Statistical Process Control (SPC) charts. First appearing in the late 1980s, most of the literature claims success, great or small, in applying NNs for SPC (NNSPC). These efforts are viewed in this paper as useful steps towards automatic on-line SPC for continuous improvement of quality and for real-time manufacturing process control. A standard NN approach that can parallel the universality of the traditional Shewhart charts has not yet been developed or adopted, although knowledge in this area is rapidly increasing. This paper attempts to provide a practical insight into the issues involved in application of NNs to SPC with the hope of advancing the use of NN techniques and facilitating their adoption as a new and useful aspect of SPC. First, a brief review of control chart analysis prior to the introduction of NN technology is presented. This is followed by an examination and classification of the NNSPC existing literature. Next, an extensive discussion of implementation issues with reference to significant research papers is presented. Finally, after summarising the survey, a set of general guidelines for future applications of NNs to SPC is outlined.     

Evaluation of robust control by universal learning networks

The characteristics of control system design using a universal learning network (ULN) are such that both the controlled systems and their controller are represented in a unified framework, and that the learning stage of the ULN can be executed by using not only first-order derivatives (gradient) but also the higher order derivatives of the criterion function with respect to parameters. ULNs have the same generalization ability as neural networks. So the ULN controller is able to control the system in a favorable way under an environment which is little different from the environment of the control system at the learning stage. However, stability cannot be sufficiently realized. In this paper, we propose a robust control method using a ULN and second-order derivatives of that ULN. Robust control, as considered here, is defined as follows. Even though the initial values of the node outputs are very different from those at the learning stage, the control system is able to reduce its influence to other node outputs and can control the system as in the case of no variation. In order to realize such robust control, a new term concerning the variation is added to the usual criterion function, and the parameters are adjusted so as to minimize the above-mentioned criterion function using second-order derivatives of the criterion function with respect to the parameters. Finally, it is shown that the ULN controller constructed by the proposed method works effectively in a simulation study of a non-linear crane system. This work was presented, in part, at the International Symposium on Artificial Life and Robotics, Oita, Japan, February 18–20, 1996     

Adaptive control for autonomous ships with uncertain model and unknown propeller dynamics

Motion control is one of the most critical aspects in the design of autonomous ships. During maneuvering, the dynamics of propellers as well as the craft hydrodynamical specifications experience severe uncertainties. In this paper, an adaptive control approach is proposed to control the motion and trajectory tracking of an autonomous vessel by adopting neural networks that is used for estimating the dynamics of the propellers and handling hydrodynamical uncertainties. Considering that the maneuvering model of a vessel resemble a nonlinear non-affine-in-control system, the proposed neural-based adaptive control algorithm is designed to estimate the nonlinear influence of the input function which in this case is the dynamics of propellers and thrusters. It is also shown that the proposed methodology is capable of handling state dependent uncertainties within the ship maneuvering model. A Lyapunov-based technique and Uniform Ultimate Boundedness are used to prove the correctness of the algorithm. To assess the method’s performance, several experiments are considered including trajectory tracking simulations in the port of Rotterdam.     

Finite-horizon neuro-optimal tracking control for a class of discrete-time nonlinear systems using adaptive dynamic programming approach

In this paper, a finite-horizon neuro-optimal tracking control strategy for a class of discrete-time nonlinear systems is proposed. Through system transformation, the optimal tracking problem is converted into designing a finite-horizon optimal regulator for the tracking error dynamics. Then, with convergence analysis in terms of cost function and control law, the iterative adaptive dynamic programming (ADP) algorithm via heuristic dynamic programming (HDP) technique is introduced to obtain the finite-horizon optimal tracking controller which makes the cost function close to its optimal value within an ?-error bound. Three neural networks are used as parametric structures to implement the algorithm, which aims at approximating the cost function, the control law, and the error dynamics, respectively. Two simulation examples are included to complement the theoretical discussions.     

Deep learning-empowered digital twin for visualized weld joint growth monitoring and penetration control

This paper presents an innovative digital twin to monitor and control complex manufacturing processes by integrating deep learning which offers strong feature extraction and analysis abilities. Taking welding manufacturing as a case study, a deep learning-empowered digital twin is developed as the visualized digital replica of the physical welding for joint growth monitoring and penetration control. In such a system, the information available directly from sensors including weld pool images, arc images, welding current and arc voltage is collected in pulsed gas tungsten arc welding (GTAW-P). Then, the undirect information charactering the weld joint geometry and determining the welding quality, including the weld joint top-side bead width (TSBW) and back-side bead width (BSBW), is computed/estimated by traditional image processing methods and deep convolutional neural networks (CNNs) respectively. Compared with single image source, weld pool image or arc image, the CNN model performs better when taking the 2-channel composite image combined by both as the input and the state-of-the-art accuracy in BSBW prediction with mean square error (MSE) as 0.047 mm² is obtained. Then, a decision-making strategy is developed to control the welding penetration to meet the quality requirement and applied successfully in various welding conditions. By modeling the weld joint cross section as an ellipse, the developed digital twin is visualized to offer a graphical user interface (GUI) for users perceiving the weld joint growth intuitively and effectively.