A Modified Backpropagation Training Algorithm for Feedforward Neural Networks*

In this paper, a new efficient learning procedure for training single hidden layer feedforward network is proposed. This procedure trains the output layer and the hidden layer separately. A new optimization criterion for the hidden layer is proposed. Existing methods to find fictitious teacher signal for the output of each hidden neuron, modified standard backpropagation algorithm and the new optimization criterion are combined to train the feedforward neural networks. The effectiveness of the proposed procedure is shown by the simulation results. *The work of P. Thangavel is partially supported by UGC, Government of India sponsored project.     

A Modified Semi-Supervised Learning Algorithm on Laplacian Eigenmaps

In this letter, a modified algorithm is proposed to extend 2-class semi-supervised learning on Laplacian eigenmaps to multi-class learning problems. The modified algorithm significantly increases its learning speed, and at the same time attains a satisfactory classification performance that is not lower than the original algorithm.     

Privacy-Preserving Backpropagation Neural Network Learning

With the development of distributed computing environment , many learning problems now have to deal with distributed input data. To enhance cooperations in learning, it is important to address the privacy concern of each data holder by extending the privacy preservation notion to original learning algorithms. In this paper, we focus on preserving the privacy in an important learning model, multilayer neural networks. We present a privacy-preserving two-party distributed algorithm of backpropagation which allows a neural network to be trained without requiring either party to reveal her data to the other. We provide complete correctness and security analysis of our algorithms. The effectiveness of our algorithms is verified by experiments on various real world data sets.     

Towards the Optimal Learning Rate for Backpropagation

A backpropagation learning algorithm for feedforward neural networks withan adaptive learning rate is derived. The algorithm is based uponminimising the instantaneous output error and does not include anysimplifications encountered in the corresponding Least Mean Square (LMS)algorithms for linear adaptive filters. The backpropagation algorithmwith an adaptive learning rate, which is derived based upon the Taylorseries expansion of the instantaneous output error, is shown to exhibitbehaviour similar to that of the Normalised LMS (NLMS) algorithm. Indeed,the derived optimal adaptive learning rate of a neural network trainedby backpropagation degenerates to the learning rate of the NLMS for a linear activation function of a neuron. By continuity, the optimal adaptive learning rate for neural networks imposes additional stabilisationeffects to the traditional backpropagation learning algorithm.     

Multilayer Feedforward Neural Network Based on Multi-valued Neurons (MLMVN) and a Backpropagation Learning Algorithm

A multilayer neural network based on multi-valued neurons (MLMVN) is considered in the paper. A multi-valued neuron (MVN) is based on the principles of multiple-valued threshold logic over the field of the complex numbers. The most important properties of MVN are: the complex-valued weights, inputs and output coded by the kth roots of unity and the activation function, which maps the complex plane into the unit circle. MVN learning is reduced to the movement along the unit circle, it is based on a simple linear error correction rule and it does not require a derivative. It is shown that using a traditional architecture of multilayer feedforward neural network (MLF) and the high functionality of the MVN, it is possible to obtain a new powerful neural network. Its training does not require a derivative of the activation function and its functionality is higher than the functionality of MLF containing the same number of layers and neurons. These advantages of MLMVN are confirmed by testing using parity n, two spirals and “sonar” benchmarks and the Mackey–Glass time series prediction.     

一类改进的最小距离分类器的增量学习算法

提出一种基于改进的最小距离分类器的增量学习算法,消除增量学习过程中产生的分类器内部结构的相互干扰,使分类器既能记住已学习的知识,又能学习新知识.增量学习需要对分类器结构进行调整,必须使用有代表性的已学习样本帮助分类器在学习新知识时复习旧知识.针对正态分布的样本集提出一种筛选算法,只保留有代表性的少量样本,大大减少存储消耗和重新训练的计算开销.实验结果证明该算法对样本的识别准确率高,在有效识别新样本的同时对以前学习的样本也保持较高的识别率,消耗存储空间小.     

基于改进的字典学习算法的图像去噪方法

提出对基于MOD和K-SVD字典学习算法的图像去噪的两个方面的改进。在字典更新阶段,采用一种新的字典更新方式,在保持支集完备的同时寻找字典和表示法。在稀疏编码阶段,根据前一次追踪过程产生的部分系数进行修正和更新。分别对这两种改进进行了验证,并说明了如何进行更快速的训练以及取得更好的结果,实验结果证实了论文方法的有效性。     

A Hammerstein Recurrent Neurofuzzy Network With an Online Minimal Realization Learning Algorithm

This paper presents a Hammerstein recurrent neurofuzzy network associated with an online minimal realization learning algorithm for dealing with nonlinear dynamic applications. We fuse the concept of states in linear systems into a neurofuzzy framework so that the whole structure can be expressed by a state-space representation. An online minimal realization learning algorithm has been developed to find a controllable and observable state-space model of minimal size from the input–output measurements of a given system. Such an idea can simultaneously resolve the problem of the determination of a minimal structure and the difficulty of network stability analysis. The advantages of our approach include: 1) our recurrent network is capable of translating the complicated dynamic behavior of a nonlinear system into a minimal set of linguistic fuzzy dynamical rules and into state-space representation as well and 2) an online minimal realization learning algorithm unifies an order determination algorithm, a hybrid parameter initialization method, and a recursive recurrent learning algorithm into a systematic procedure to identify a minimal structure with satisfactory performance. Performance evaluations on benchmark examples as well as real-world applications have successfully validated the effectiveness of our approach.      

Global Convergence of GHA Learning Algorithm With Nonzero-Approaching Adaptive Learning Rates

The generalized Hebbian algorithm (GHA) is one of the most widely used principal component analysis (PCA) neural network (NN) learning algorithms. Learning rates of GHA play important roles in convergence of the algorithm for applications. Traditionally, the learning rates of GHA are required to converge to zero so that its convergence can be analyzed by studying the corresponding deterministic continuous-time (DCT) equations. However, the requirement for learning rates to approach zero is not a practical one in applications due to computational roundoff limitations and tracking requirements. In this paper, nonzero-approaching adaptive learning rates are proposed to overcome this problem. These proposed adaptive learning rates converge to some positive constants, which not only speed up the algorithm evolution considerably, but also guarantee global convergence of the GHA algorithm. The convergence is studied in detail by analyzing the corresponding deterministic discrete-time (DDT) equations. Extensive simulations are carried out to illustrate the theory.     

Analysis of the Initial Values in Split-Complex Backpropagation Algorithm

When a multilayer perceptron (MLP) is trained with the split-complex backpropagation (SCBP) algorithm, one observes a relatively strong dependence of the performance on the initial values. For the effective adjustments of the weights and biases in SCBP, we propose that the range of the initial values should be greater than that of the adjustment quantities. This criterion can reduce the misadjustment of the weights and biases. Based on the this criterion, the suitable range of the initial values can be estimated. The results show that the suitable range of the initial values depends on the property of the used communication channel and the structure of the MLP (the number of layers and the number of nodes in each layer). The results are studied using the equalizer scenarios. The simulation results show that the estimated range of the initial values gives significantly improved performance.      

A New Incremental PCA Algorithm With Application to Visual Learning and Recognition

This paper proposes a new mean-shifting Incremental PCA (IPCA) method based on the autocorrelation matrix. The dimension of the updated matrix remains constant instead of increasing with the number of input data points. Comparing to some previous batch and iterative PCA algorithms, the proposed IPCA requires lower computational time and storage capacity owing to the two transformations designed. The experiment results show the efficiency and accuracy of the proposed IPCA method in applications of the on-line visual learning and recognition.     

基于改进多核学习的语音情感识别算法

提出一种基于改进多核学习的语音情感识别算法.算法以高斯径向基核函数为基准,通过采样不同的样本,采用不同的评价标准并获得不同的参数,来提高分类性能.此外,通过引入多核技术,将得到的高斯核函数构建多核学习的基核,并通过利用松弛因子构建的软间隔多核学习的目标函数改善了学习效率.对比仿真实验结果表明,本文提出的基于多核学习语音情感识别算法有效提高了语音情感识别性能.     

A New Adaptive Backpropagation Algorithm Based on Lyapunov Stability Theory for Neural Networks

A new adaptive backpropagation (BP) algorithm based on Lyapunov stability theory for neural networks is developed in this paper. It is shown that the candidate of a Lyapunov function V(k) of the tracking error between the output of a neural network and the desired reference signal is chosen first, and the weights of the neural network are then updated, from the output layer to the input layer, in the sense that DeltaV(k)=V(k)-V(k-1)<0. The output tracking error can then asymptotically converge to zero according to Lyapunov stability theory. Unlike gradient-based BP training algorithms, the new Lyapunov adaptive BP algorithm in this paper is not used for searching the global minimum point along the cost-function surface in the weight space, but it is aimed at constructing an energy surface with a single global minimum point through the adaptive adjustment of the weights as the time goes to infinity. Although a neural network may have bounded input disturbances, the effects of the disturbances can be eliminated, and asymptotic error convergence can be obtained. The new Lyapunov adaptive BP algorithm is then applied to the design of an adaptive filter in the simulation example to show the fast error convergence and strong robustness with respect to large bounded input disturbances     

配电网重构的改进同步开关法

★ 引言 配电网重构是近年来电力系统领域一个引人注目的研究方向。通过网络重构来实现负荷转移以达到均衡负荷、消除过载、降低网损、提高供电电压质量和系统运行经济性的效果。最优配电网络重构技术最早是由Merlin和Back于1975年提出来的[1],之后不断有研究成果发表,提出了多种方法,其中,对以网损最小为目标函数的配电网络重构研究最多。理论上,在满足各种运行条件约束下,以网损最小为目标的配电网络重构问题是一个非线性组合优化问题。目前网络重构的算法很多,但是大多数的算法都是基于支路交换的[2][3][4],即先合上一个联络开关,再找…     

一种改进的波束域ESPRIT算法

高分辨方位估计技术在声纳、雷达、通信、地质探测等众多应用领域中占有十分重要的地位.ESPRIT算法作为一种成熟的高分辨算法有运算量小的优点,但其减小了阵列的有效孔径,在现有的高分辨处理设备中,由于空间的限制,阵列不可能随意增大,因此要最大限度的扩大阵列的有效孔径.通过对均匀线列阵下波束域ESPRIT算法(BmESPRIT)的研究,提出了一种基于ESPRIT的新算法,并给出了转换成波束域处理的构造方法.该算法利用后向平移技术,使两子阵列与原阵列具有相同的有效孔径.有效的利用了阵列资源.仿真结果表明在相同条件下该算法的估计性能明显提高.     

一种改进的鲸鱼优化算法

针对鲸鱼优化算法(WOA)收敛速度慢、收敛精度低的问题,在提升性能的基础上保留WOA的简单性,提出一种改进的WOA。利用分段Logistic混沌映射产生混沌序列对种群位置进行初始化,以维持全局搜索时初始种群的多样性。考虑算法的非线性优化过程和搜索过程中个体状态的差异性,在WOA中引入非线性自适应权重策略,以协调全局探索和局部开发能力。通过仿真测试比较改进算法和WOA在求解6个典型基准函数时的性能,实验结果表明,改进算法在寻优过程中能够保持初始种群多样性,且具有更快的收敛速度和更优的收敛精度。     

一种改进的人工免疫算法

简要介绍了遗传算法、人工免疫算法的工作原理。为了克服传统人工免疫算法存在的不足,该文提出一种改进的人工免疫算法,并利用Rosenbrock函数对3种算法进行了试验比较。实验结果表明,该文提出的改进的人工免疫算法较遗传算法、传统的人工免疫算法更为有效。     

一种改进的克隆选择优化算法

人工免疫系统是基于生物免疫系统特性而发展的新兴智能系统。论文利用免疫系统的克隆选择机制,提出一种用于函数优化的改进克隆选择算法。算法的主要特点是采用克隆和自适应变异等操作,提高收敛速度和种群的多样性。仿真程序表明,该算法能以较快速度完成给定范围的搜索和全局优化任务。     

基于李雅普诺夫函数的BP神经网络算法的收敛性分析

针对前馈神经网络应时变输入的自学习机制，采用李雅普诺夫函数来分析权值的收敛性，从而揭示BP神经网络算法朝最小误差方向调整权值的内在因素，并在分析单参数BP算法收敛性基础上，提出单参数变调整法则的离散型BP神经网络算法．