A fast learning algorithm of neural network with tunable activation function

This paper presents a modified structure of a neural network with tunable activation function and provides a new learning algorithm for the neural network training. Simulation results of XOR problem, Feigenbaum function, and Henon map show that the new algorithm has better performance than BP (back propagation) algorithm in terms of shorter convergence time and higher convergence accuracy. Further modifications of the structure of the neural network with the faster learning algorithm demonstrate simpler structure with even faster convergence speed and better convergence accuracy.     

NNF and NNPrF—Fuzzy Petri Nets Based on Neural Network for Knowledge Representation, Reasoning and Learning

This paper proposes NNF-a fuzzy Petri Net system based on neural network for proposition logic repesentation,and gives the formal definition of NNF.For the NNF model,forward reasoning algorithm,backward reasoning algorithm and knowledge learning algorithm are discussed based on weight training algorithm of neural network-Back Propagation algorithm.Thus NNF is endowed with the ability of learning a rule.The paper concludes with a discussion on extending NNF to predicate logic,forming NNPrF,and proposing the formal definition and a reasoning algorithm of NNPrF.     

Neural Network Algorithm for Designing FIR Filters Utilizing Frequency-Response Masking Technique

This paper presents a new joint optimization method for the design of sharp linear-phase finite-impulse response (FIR) digital filters which are synthesized by using basic and multistage frequency-response-masking(FRM) techniques.The method is based on a batch back-propagation neural network algorithm with a variable learning rate mode.We propose the following two-step optimization technique in order to reduce the complexity.At the first step,an initial FRM filter is designed by alternately optimizing th...     

Heuristic dynamic programming-based learning control for discrete-time disturbed multi-agent systems

Owing to extensive applications in many fields, the synchronization problem has been widely investigated in multi-agent systems. The synchronization for multi-agent systems is a pivotal issue, which means that under the designed control policy, the output of systems or the state of each agent can be consistent with the leader. The purpose of this paper is to investigate a heuristic dynamic programming (HDP)-based learning tracking control for discrete-time multi-agent systems to achieve synchronization while considering disturbances in systems. Besides, due to the difficulty of solving the coupled Hamilton– Jacobi–Bellman equation analytically, an improved HDP learning control algorithm is proposed to realize the synchronization between the leader and all following agents, which is executed by an action-critic neural network. The action and critic neural network are utilized to learn the optimal control policy and cost function, respectively, by means of introducing an auxiliary action network. Finally, two numerical examples and a practical application of mobile robots are presented to demonstrate the control performance of the HDP-based learning control algorithm.     

Construction of a new adaptive wavelet network and its learning algorithm

A new adaptive learning algorithm for constructing and training wavelet networks is proposed based on the time-frequency localization properties of wavelet frames and the adaptive projection algorithm. The exponential convergence of the adaptive projection algorithm in finite-dimensional Hilbert spaces is constructively proved, with exponential decay ratios given with high accuracy. The learning algorithm can sufficiently utilize the time-frequency information contained in the training data, iteratively determines the number of the hidden layer nodes and the weights of wavelet networks, and solves the problem of structure optimization of wavelet networks. The algorithm is simple and efficient, as illustrated by examples of signal representation and denoising.     

Adaptive air-fuel ratio control with MLP network

This paper presents an application of adaptive neural network model-based predictive control (MPC) to the air-fuel ratio of an engine simulation. A multi-layer perceptron (MLP) neural network is trained using two on-line training algorithms: a back propagation algorithm and a recursive least squares (RLS) algorithm. It is used to model parameter uncertainties in the nonlinear dynamics of internal combustion (IC) engines. Based on the adaptive model, an MPC strategy for controlling air-fuel ratio is realized, and its control performance compared with that of a traditional PI controller. A reduced Hessian method, a newly developed sequential quadratic programming (SQP) method for solving nonlinear programming (NLP) problems, is implemented to speed up nonlinear optimization in the MPC.     

Nonlinear optimized adaptive trajectory control of helicopter

This paper attempts to develop an optimized adaptive trajectory control system for helicopters based on the dynamic inversion method. This control algorithm is implemented by three time-scale separation architectures. Pseudo control hedging (PCH) is used to protect the adaptive element from actuator saturation nonlinearities and also from the inner-outer-loop interaction. In addition, to augment the attitude control system, two online adaptive architectures that employ a neural network are used. By tuning the neural network based on the system model, a better and faster learning will be achieved, but this is a frustrating and time consuming process. Due to complexity in accurate tuning of neural network, this paper introduces a non-dominated sorting genetic algorithm II (NSGA-II) for off-line optimization of the neural network. Thus, in the proposed method, the neural network can compensate model inversion error caused by the deficiency of full knowledge of helicopter dynamics more accurately. The effectiveness of proposed method is demonstrated by numerical simulations.     

A Wavelet Neural Network Based Non-linear Model Predictive Controller for a Multi-variable Coupled Tank System

In this paper, a novel real time non-linear model predictive controller(NMPC) for a multi-variable coupled tank system(CTS) is designed. CTSs are highly non-linear and can be found in many industrial process applications. The involvement of multi-input multi-output(MIMO) system makes the design of an effective controller a challenging task. MIMO systems have inherent couplings,interactions in-between the process input-output variables and generally have an complex internal structure. The aim of this paper is to design, simulate, and implement a novel real time constrained NMPC for a multi-variable CTS with the aid of intelligent system techniques. There are two major formidable challenges hindering the success of the implementation of a NMPC strategy in the MIMO case. The first is the difficulty of obtaining a good non-linear model by training a non-convex complex network to avoid being trapped in a local minimum solution. The second is the online real time optimisation(RTO) of the manipulated variable at every sampling time.A novel wavelet neural network(WNN) with high predicting precision and time-frequency localisation characteristic was selected for an MIMO model and a fast stochastic wavelet gradient algorithm was used for initial training of the network. Furthermore, a genetic algorithm was used to obtain the optimised parameters of the WNN as well as the RTO during the NMPC strategy. The proposed strategy performed well in both simulation and real time on an MIMO CTS. The results indicated that WNN provided better trajectory regulation with less mean-squared-error and average control energy compared to an artificial neural network. It is also shown that the WNN is more robust during abnormal operating conditions.     

Design of robotic visual servo control based on neural network and genetic algorithm

A new visual servo control scheme for a robotic manipulator is presented in this paper, where a back propagation (BP) neural network is used to make a direct transition from image feature to joint angles without requiring robot kinematics and camera calibration. To speed up the convergence and avoid local minimum of the neural network, this paper uses a genetic algorithm to find the optimal initial weights and thresholds and then uses the BP algorithm to train the neural network according to the data given. The proposed method can effectively combine the good global searching ability of genetic algorithms with the accurate local searching feature of BP neural network. The Simulink model for PUMA560 robot visual servo system based on the improved BP neural network is built with the Robotics Toolbox of Matlab. The simulation results indicate that the proposed method can accelerate convergence of the image errors and provide a simple and effective way of robot control.     

A stable learning algorithm for block-diagonal recurrent neural networks: application to the analysis of lung sounds.

A novel learning algorithm, the Recurrent Neural Network Constrained Optimization Method (RENNCOM) is suggested in this paper, for training block-diagonal recurrent neural networks. The training task is formulated as a constrained optimization problem, whose objective is twofold: (1) minimization of an error measure, leading to successful approximation of the input/output mapping and (2) optimization of an additional functional, the payoff function, which aims at ensuring network stability throughout the learning process. Having assured the network and training stability conditions, the payoff function is switched to an alternative form with the scope to accelerate learning. Simulation results on a benchmark identification problem demonstrate that, compared to other learning schemes with stabilizing attributes, the RENNCOM algorithm has enhanced qualities, including, improved speed of convergence, accuracy and robustness. The proposed algorithm is also applied to the problem of the analysis of lung sounds. Particularly, a filter based on block-diagonal recurrent neural networks is developed, trained with the RENNCOM method. Extensive experimental results are given and performance comparisons with a series of other models are conducted, underlining the effectiveness of the proposed filter.     

Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network

In recent years, a recurrent neural network called projection neural network was proposed for solving monotone variational inequalities and related convex optimization problems. In this paper, we show that the projection neural network can also be used to solve pseudomonotone variational inequalities and related pseudoconvex optimization problems. Under various pseudomonotonicity conditions and other conditions, the projection neural network is proved to be stable in the sense of Lyapunov and globally convergent, globally asymptotically stable, and globally exponentially stable. Since monotonicity is a special case of pseudomononicity, the projection neural network can be applied to solve a broader class of constrained optimization problems related to variational inequalities. Moreover, a new concept, called componentwise pseudomononicity, different from pseudomononicity in general, is introduced. Under this new concept, two stability results of the projection neural network for solving variational inequalities are also obtained. Finally, numerical examples show the effectiveness and performance of the projection neural network     

A general projection neural network for solving monotone variational inequalities and related optimization problems

Recently, a projection neural network for solving monotone variational inequalities and constrained optimization problems was developed. In this paper, we propose a general projection neural network for solving a wider class of variational inequalities and related optimization problems. In addition to its simple structure and low complexity, the proposed neural network includes existing neural networks for optimization, such as the projection neural network, the primal-dual neural network, and the dual neural network, as special cases. Under various mild conditions, the proposed general projection neural network is shown to be globally convergent, globally asymptotically stable, and globally exponentially stable. Furthermore, several improved stability criteria on two special cases of the general projection neural network are obtained under weaker conditions. Simulation results demonstrate the effectiveness and characteristics of the proposed neural network.     

Neural network for quadratic optimization with bound constraints

A recurrent neural network is presented which performs quadratic optimization subject to bound constraints on each of the optimization variables. The network is shown to be globally convergent, and conditions on the quadratic problem and the network parameters are established under which exponential asymptotic stability is achieved. Through suitable choice of the network parameters, the system of differential equations governing the network activations is preconditioned in order to reduce its sensitivity to noise and to roundoff errors. The optimization method employed by the neural network is shown to fall into the general class of gradient methods for constrained nonlinear optimization and, in contrast with penalty function methods, is guaranteed to yield only feasible solutions.     

改进的粒子群神经网络检测种蛋成活性

提出了一种基于改进粒子群神经网络进行孵化种蛋成活性自动检测的方法.提取HSI图像的H分量作为孵化种蛋表面颜色特征,通过主成分分析,找到了6个主成分特征向量,减少了神经网络输入节点数.利用改进粒子群算法优化多层前馈神经网络的拓扑结构,提高了神经网络的学习质量和速度.训练集的样本具有足够代表性和全面性,提高了网络的泛化能力.实验证明,该方法检测准确性较高,具有鲁棒性和高效率.     

基于改进遗传算法的RBF神经网络结构优化研究

针对RBF神经网络隐含层节点数过多导致网络结构复杂的问题,提出了一种基于改进遗传算法(IGA)的RBF神经网络优化算法。利用IGA优化基于正交最小二乘法的RBF神经网络结构,通过对隐含层输出矩阵的列向量进行全局寻优,从而设计出结构更优的基于IGA的RBF神经网络(IGA-RBF)。将IGA-RBF神经网络的学习算法应用于电子元器件贮存环境温湿度预测模型,与基于正交最小二乘法的RBF神经网络进行比较的结果表明:IGA-RBF神经网络设计出来的网络训练步数减少了44步,隐含层节点数减少了34个,且预测模型得到的温湿度误差较小,拟合精度大于0.95,具有更高的预测精度。     

基于Hopfield网络的多变量动态矩阵控制及其在精馏塔系统中的应用

利用Hopfield网络的优化计算功能求解有约束多变量动态知阵控制问题,算法简单,收敛性好,即或以用硬件实时实现,也可用数值积分求解,对Wood-Berry精馏塔的仿真表明了算法的有效性。     

具有时变时滞的递归神经网络的渐近稳定性分析

当神经网络应用于最优化计算时,理想的情形是只有一个全局渐近稳定的平衡点,并且以指数速度趋近于平衡点,从而减少神经网络所需计算时间.研究了带时变时滞的递归神经网络的全局渐近稳定性.首先将要研究的模型转化为描述系统模型,然后利用Lyapunov-Krasovskii稳定性定理、线性矩阵不等式(LMI)技术、S过程和代数不等式方法,得到了确保时变时滞递归神经网络渐近稳定性的新的充分条件,并将它应用于常时滞神经网络和时滞细胞神经网络模型,分别得到了相应的全局渐近稳定性条件.理论分析和数值模拟显示,所得结果为时滞递归神经网络提供了新的稳定性判定准则.     

复杂运动目标的学习与识别

针对复杂运动目标识别问题,提出了一个基于反馈型随机神经网络的运动认脸与物体的自动识别系统,该系统具有强大学习能力,运动目标检测与识别快速准确等特点,在对该的核心－反馈型二元网络进行深入分析的基础上,提出了一种适合于该神经网络模型的高效渐进式Boltzmann学习算法,实验结果表明,该识别系统性能优异,在几个方面超过了eTrue公司的TrueFace人脸识别系统。     

Second-order neural nets for constrained optimization

Analog neural nets for constrained optimization are proposed as an analogue of Newton's algorithm in numerical analysis. The neural model is globally stable and can converge to the constrained stationary points. Nonlinear neurons are introduced into the net, making it possible to solve optimization problems where the variables take discrete values, i.e., combinatorial optimization.