A Simplified Dual Neural Network for Quadratic Programming With Its KWTA Application

The design, analysis, and application of a new recurrent neural network for quadratic programming, called simplified dual neural network, are discussed. The analysis mainly concentrates on the convergence property and the computational complexity of the neural network. The simplified dual neural network is shown to be globally convergent to the exact optimal solution. The complexity of the neural network architecture is reduced with the number of neurons equal to the number of inequality constraints. Its application to k-winners-take-all (KWTA) operation is discussed to demonstrate how to solve problems with this neural network     

A discrete-time dynamic K-winners-take-all neural circuit

A discrete-time mathematical model of K-winners-take-all (KWTA) neural circuit that can quickly identify the K-winning from N neurons, where 1K<N, whose input signals are larger than that of remaining N−K neurons, is given and analyzed. A functional block scheme of the circuit is presented. For N competitors, such circuit is composed of N feedforward and one feedback hard-limiting neurons that are used to determine the dynamic shift of input signals. The circuit has low computational and hardware implementation complexity, high speed of signal processing, can process signals of any finite range, possesses signal order preserving property and does not require resetting and corresponding supervisory circuit that additionally increases a speed of signal processing.     

一种具有small world 特性的ESN 结构分析与设计

针对回声状态网络（ESN）结构设计复杂、参数选择难度大的问题,提出一种具有small world特性的ESN（SWESN）.首先采用神经元空间增长算法在平面区域生成small world拓扑网络;然后根据网络节点与基准点的Euclidean距离将网络节点进行重新排序,并将平面上的物理节点及其连接映射为SWESN的内部神经元连接矩阵,从而使动态神经元池具有small world特性.实验表明,SWESN动力学特性比常规ESN更为丰富,在鲁棒性、抗干扰能力等方面均优于常规的ESN.     

Dynamic neural cortical ensembles model

One-layered model of cortical neurons as a set of overlapping ensembles, each with a structure similar to Hopfield network, is proposed. Ensemble equilibrium equation is solved and formulas for connections weights calculation for given set of attractors are obtained. Concept of dynamic attractors that consists of consequent recalling of stored patterns with moving activity through the network is introduced. Role of dynamic attractors in long-term memory is discussed and mechanism for memory recovery after destruction of some neurons is proposed. Results of experiments on associative memory recovery after partial removal of neurons are shown.     

基于NARX神经网络航空发动机参数动态辨识模型

针对航空发动机参数非线性动态特性,提出一种基于外部输入非线性自回归（NARX）神经网络的发动机参数动态辨识模型。主要思路是根据NARX网络的非线性时序预测特性,结合发动机参数的稳态和动态参数,提出一种基于偏稳态差值预测的NARX参数动态模型结构。设计了SP-P辨识结构,整定了模型内部结构参数并建立N1（低压转子转速）、N2（高压转子转速）、EGT（涡轮后排气温度）参数非线性差分预测模型。最后依据某发动机试车样本,对推杆加减速时N1、N2、EGT动态辨模型进行仿真。仿真结果表明,N2相对误差小于0.2%,N1相对误差小于0.3%,EGT相对误差小于[1℃],满足发动机试车仿真需要。最后,将所建模型应用于某A320机务维修训练器的发动机仿真系统。     

基于半监督学习的动态神经网络结构设计

针对神经网络初始结构的设定依赖于工作者的经验、自适应能力较差等问题,提出一种基于半监督学习(SSL)算法的动态神经网络结构设计方法。该方法采用半监督学习方法利用已标记样例和无标记样例对神经网络进行训练,得到一个性能较为完善的初始网络结构,之后采用全局敏感度分析法(GSA)对网络隐层神经元输出权值进行分析,判断隐层神经元对网络输出的影响程度,即其敏感度值大小,适时地删减敏感度值很小的神经元或增加敏感度值较大的神经元,实现动态神经网络结构的优化设计,并给出了网络结构变化过程中收敛性的证明。理论分析和Matlab仿真实验表明,基于SSL算法的神经网络隐层神经元会随训练时间而改变,实现了网络结构动态设计。在液压厚度自动控制(AGC)系统应用中,大约在160 s时系统输出达到稳定,输出误差大约为0.03 mm,与监督学习(SL)方法和无监督学习(USL)方法相比,输出误差分别减小了0.03 mm和0.02 mm,这表明基于SSL算法的动态网络在实际应用中能有效提高系统输出的准确性。     

On evil twin networks and the value of limited randomized routing

A dynamic two-stage Delta network (N inputs and outputs) is introduced and analyzed for permutation routing. The notion of evil twins is introduced and a deterministic procedure is given to route any permutation in no more than 2/sup 4//spl radic/N network cycles. Two limited randomized routing schemes are then analyzed. The first called Single Randomization yields on average at most N!+1 (N!=O(logN/loglogN)/sup 1/ and is the greatest integer such that (N!)!/spl les/N) network cycles and the second called Multiple Randomization yields on average at most upper bound [log(logN+1)]+2+1/N network cycles for any input permutation. The probability of any permutation requiring at least c network cycles more than the above average bounds is then shown to be at most 1/(c+1) for Single Randomization and 1/N/sup r/ for Multiple Randomization, respectively. It is then shown how the dynamic two-stage network can be physically realized as a three-stage network. Both the evil twin and Multiple Randomization algorithms have been integrated into an off-the-shelf ASIC from PMC-Sierra, Inc. (PM-73488) which has been designed as a building block for such a three-stage implementation. These routing schemes are also adapted to run on a recirculating network. Recirculation is used to effect a reshuffling of data as in the dynamic network, but with a considerable reduction in network cost.     

Cognitive enhancement mediated through postsynaptic actions of norepinephrine on ongoing cortical activity

We propose two distinct types of norepinephrine (NE)-neuromodulatory systems: an enhanced-excitatory and enhanced-inhibitory (E-E/E-I) system and a depressed-excitatory and enhanced-inhibitory (D-E/E-I) system. In both systems, inhibitory synaptic efficacies are enhanced, but excitatory ones are modified in a contradictory manner: the E-E/E-I system enhances excitatory synaptic efficacies, whereas the D-E/E-I system depresses them. The E-E/E-I and D-E/E-I systems altered the dynamic property of ongoing (background) neuronal activity and greatly influenced the cognitive performance (S/N ratio) of a cortical neural network. The E-E/E-I system effectively enhanced S/N ratio for weaker stimuli with lower doses of NE, whereas the D-E/E-I system enhanced stronger stimuli with higher doses of NE. The neural network effectively responded to weaker stimuli if brief gamma-bursts were involved in ongoing neuronal activity that is controlled under the E-E/E-I neuromodulatory system. If the E-E/E-I and the D-E/E-I systems interact within the neural network, depressed neurons whose activity is depressed by NE application have bimodal property. That is, S/N ratio can be enhanced not only for stronger stimuli as its original property but also for weaker stimuli, for which coincidental neuronal firings among enhanced neurons whose activity is enhanced by NE application are essential. We suggest that the recruitment of the depressed neurons for the detection of weaker (subthreshold) stimuli might be advantageous for the brain to cope with a variety of sensory stimuli.     

联想记忆神经网络的训练

提出了一种联想记忆神经网络的优化训练方案,说明网络的样本吸引域可用阱深参数作 一定程度的控制,使网络具有尽可能好的容错性.计算表明,训练网络可达到α＜1(α=M/ N,N是神经元数,M是贮存样本数),而仍有良好的容错性,明显优于外积法、正交化外积法、 赝逆法等常用方案.文中还对训练网络的对称性与收敛性问题进行了讨论.     

动态突触型Hopfield神经网络的动态特性研究

提出一种基于动态突触的离散型Hoppfield神经网（DSDNN）模型,给出了DSDNN的连接权值的动态演化模型及其神经元的状态更新模型．证明了DSDNN的平衡点与常规离散型Hopfield神经网络的平衡点具有一一对应的关系,分析了平衡点的稳定性．最后通过仿真分析了DSDNN的动态演化特性与其参数的关系。     

IMPROVED CONVERGENCE THROUGH DYNAMIC PROGRAMMING APPROACH

New hybrid methods for solving the multiplayer perceptron optimization problem are proposed which use the computation capabilities of Bellman's dynamic programming (DP) method. To solve the neural network optimization problem, we consider the case of output neurons differently from that of hidden neurons. For the neurons of the output layer we apply the conventional DP and for the hidden neurons we apply a method based on gradient approach. Computer simulation shows that the new hybrid methods outperform the gradient-based optimization methods in converging speed and avoiding the local minimum.     

A generalized Takagi–Sugeno–Kang recurrent fuzzy-neural filter for adaptive noise cancelation

This paper presents a recurrent fuzzy-neural filter for adaptive noise cancelation. The cancelation task is transformed to a system-identification problem, which is tackled by use of the dynamic neuron-based fuzzy neural network (DN-FNN). The fuzzy model is based on Takagi–Sugeno–Kang fuzzy rules, whose consequent parts consist of linear combinations of dynamic neurons. The orthogonal least squares method is employed to select the number of rules, along with the number and kind of dynamic neurons that participate in each rule. Extensive simulation results are given and performance comparison with a series of other dynamic fuzzy and neural models is conducted, underlining the effectiveness of the proposed filter and its superior performance over its competing rivals.     

A Novel Pruning Algorithm for Optimizing Feedforward Neural Network of Classification Problems

Optimizing the structure of neural networks is an essential step for the discovery of knowledge from data. This paper deals with a new approach which determines the insignificant input and hidden neurons to detect the optimum structure of a feedforward neural network. The proposed pruning algorithm, called as neural network pruning by significance (N2PS), is based on a new significant measure which is calculated by the Sigmoidal activation value of the node and all the weights of its outgoing connections. It considers all the nodes with significance value below the threshold as insignificant and eliminates them. The advantages of this approach are illustrated by implementing it on six different real datasets namely iris, breast-cancer, hepatitis, diabetes, ionosphere and wave. The results show that the proposed algorithm is quite efficient in pruning the significant number of neurons on the neural network models without sacrificing the networks performance.     

A Proposed Hybrid Recurrent Neural Control System for Two Co-operating Robots

This paper discusses a model refernce adaptive (MRAC) position/force controller using proposed neural networks for two co-operating planar robots. The proposed neural network is a recurrent hybrid network. The recurrent networks have feedback connections and thus an inherent memory for dynamics, which makes them suitable for representing dynamic systems. A feature of the networks adopted is their hybrid hidden layer, which includes both linear and nonlinear neurons. On the other hand, the results of the case of a single robot under position control alone are presented for comparison. The results presented show the superior ability of the proposed neural network based model reference adaptive control scheme at adapting to changes in the dynamics parameters of robots.     

Multifeedback-Layer Neural Network

The architecture and training procedure of a novel recurrent neural network (RNN), referred to as the multifeedback-layer neural network (MFLNN), is described in this paper. The main difference of the proposed network compared to the available RNNs is that the temporal relations are provided by means of neurons arranged in three feedback layers, not by simple feedback elements, in order to enrich the representation capabilities of the recurrent networks. The feedback layers provide local and global recurrences via nonlinear processing elements. In these feedback layers, weighted sums of the delayed outputs of the hidden and of the output layers are passed through certain activation functions and applied to the feedforward neurons via adjustable weights. Both online and offline training procedures based on the backpropagation through time (BPTT) algorithm are developed. The adjoint model of the MFLNN is built to compute the derivatives with respect to the MFLNN weights which are then used in the training procedures. The Levenberg-Marquardt (LM) method with a trust region approach is used to update the MFLNN weights. The performance of the MFLNN is demonstrated by applying to several illustrative temporal problems including chaotic time series prediction and nonlinear dynamic system identification, and it performed better than several networks available in the literature     

An asynchronous recurrent linear threshold network approach to solving the traveling salesman problem

In this paper, an approach to solving the classical Traveling Salesman Problem (TSP) using a recurrent network of linear threshold (LT) neurons is proposed. It maps the classical TSP onto a single-layered recurrent neural network by embedding the constraints of the problem directly into the dynamics of the network. The proposed method differs from the classical Hopfield network in the update of state dynamics as well as the use of network activation function. Furthermore, parameter settings for the proposed network are obtained using a genetic algorithm, which ensure a stable convergence of the network for different problems. Simulation results illustrate that the proposed network performs better than the classical Hopfield network for optimization.     

具有二级模式的多状态神经元联想记忆网络

提出了一种新型的联想记忆神经网络,神经元的状态为向量,含N个神经元的网络中存储的模式由N个具有M个分量的二级模式组成,每个模式存储在一个由N个连接组成的“模式环”中,连接由“连接状态”和“禁止路径”组成,前者用于存储二级模式,后者用于消除假模式,回忆时允许输入不完整的模式,记忆容量为(N-1)！。     

Augmented Hopfield network for mixed-integer programming

Watta and Hassoun (1996) proposed a coupled gradient neural network for mixed integer programming. In this network continuous neurons were used to represent discrete variables. For the larger temporal problem they attempted many of the solutions found were infeasible. This paper proposes an augmented Hopfield network which is similar to the coupled gradient network proposed by Watta and Hassoun. However, in this network truly discrete neurons are used. It is shown that this network can be applied to mixed integer programming. Results illustrate that feasible solutions are now obtained for the larger temporal problem.     

A neural network approach to complete coverage path planning.

Complete coverage path planning requires the robot path to cover every part of the workspace, which is an essential issue in cleaning robots and many other robotic applications such as vacuum robots, painter robots, land mine detectors, lawn mowers, automated harvesters, and window cleaners. In this paper, a novel neural network approach is proposed for complete coverage path planning with obstacle avoidance of cleaning robots in nonstationary environments. The dynamics of each neuron in the topologically organized neural network is characterized by a shunting equation derived from Hodgkin and Huxley's (1952) membrane equation. There are only local lateral connections among neurons. The robot path is autonomously generated from the dynamic activity landscape of the neural network and the previous robot location. The proposed model algorithm is computationally simple. Simulation results show that the proposed model is capable of planning collision-free complete coverage robot paths.     

Modelling of a magneto-rheological damper by evolving radial basis function networks

This paper presents an approach to approximate the forward and inverse dynamic behaviours of a magneto-rheological (MR) damper using evolving radial basis function (RBF) networks. Due to the highly nonlinear characteristics of MR dampers, modelling of MR dampers becomes a very important problem to their applications. In this paper, an alternative representation of the MR damper in terms of evolving RBF networks, which have a structure of four input neurons and one output neuron to emulate the forward and inverse dynamic behaviours of an MR damper, respectively, is developed by combining the genetic algorithms (GAs) to search for the network centres with other standard learning algorithms. Training and validating of the evolving RBF network models are achieved by using the data generated from the numerical simulation of the nonlinear differential equations proposed for the MR damper. It is shown by the validation tests that the evolving RBF networks can represent both forward and inverse dynamic behaviours of the MR damper satisfactorily.