A Modified Backpropagation Training Algorithm for Feedforward Neural Networks*

In this paper, a new efficient learning procedure for training single hidden layer feedforward network is proposed. This procedure trains the output layer and the hidden layer separately. A new optimization criterion for the hidden layer is proposed. Existing methods to find fictitious teacher signal for the output of each hidden neuron, modified standard backpropagation algorithm and the new optimization criterion are combined to train the feedforward neural networks. The effectiveness of the proposed procedure is shown by the simulation results. *The work of P. Thangavel is partially supported by UGC, Government of India sponsored project.     

综合反向传播算法

提出一种用于多层前向神经网络的综合反向传播 算法。该算法使用了综合考虑了绝对误上对误差的广义指标函数,采用了在网络输出空间搜索的反传技术。     

A Modified Semi-Supervised Learning Algorithm on Laplacian Eigenmaps

In this letter, a modified algorithm is proposed to extend 2-class semi-supervised learning on Laplacian eigenmaps to multi-class learning problems. The modified algorithm significantly increases its learning speed, and at the same time attains a satisfactory classification performance that is not lower than the original algorithm.     

Privacy-Preserving Backpropagation Neural Network Learning

With the development of distributed computing environment , many learning problems now have to deal with distributed input data. To enhance cooperations in learning, it is important to address the privacy concern of each data holder by extending the privacy preservation notion to original learning algorithms. In this paper, we focus on preserving the privacy in an important learning model, multilayer neural networks. We present a privacy-preserving two-party distributed algorithm of backpropagation which allows a neural network to be trained without requiring either party to reveal her data to the other. We provide complete correctness and security analysis of our algorithms. The effectiveness of our algorithms is verified by experiments on various real world data sets.     

Towards the Optimal Learning Rate for Backpropagation

A backpropagation learning algorithm for feedforward neural networks withan adaptive learning rate is derived. The algorithm is based uponminimising the instantaneous output error and does not include anysimplifications encountered in the corresponding Least Mean Square (LMS)algorithms for linear adaptive filters. The backpropagation algorithmwith an adaptive learning rate, which is derived based upon the Taylorseries expansion of the instantaneous output error, is shown to exhibitbehaviour similar to that of the Normalised LMS (NLMS) algorithm. Indeed,the derived optimal adaptive learning rate of a neural network trainedby backpropagation degenerates to the learning rate of the NLMS for a linear activation function of a neuron. By continuity, the optimal adaptive learning rate for neural networks imposes additional stabilisationeffects to the traditional backpropagation learning algorithm.     

一种学习稀疏BN最优结构的改进K均值分块学习算法

目前贝叶斯网络(Bayesian networks, BN)的传统结构学习算法在处理高维数据时呈现出计算负担过大、在合理时间内难以得到期望精度结果的问题.为了在高维数据下学习稀疏BN的最优结构, 本文提出了一种学习稀疏BN最优结构的改进K均值分块学习算法.该算法采用分而治之的策略, 首先采用互信息作为节点间距离度量, 利用融合互信息的改进K均值算法对网络分块; 其次, 使用MMPC (Max-min parent and children)算法得到整个网络的架构, 根据架构找到块间所有边的可能连接方向, 从而找到所有可能的图结构; 之后, 对所有图结构依次进行结构学习; 最终利用评分找到最优BN.实验证明, 相比现有分块结构学习算法, 本文提出的算法不仅习得了网络的精确结构, 且学习速度有一定提高; 相比非分块经典结构学习算法, 本文提出的算法在保证精度基础上, 学习速度大幅提高, 解决了非分块经典结构学习算法无法在合理时间内处理高维数据的难题.     

Multilayer Feedforward Neural Network Based on Multi-valued Neurons (MLMVN) and a Backpropagation Learning Algorithm

A multilayer neural network based on multi-valued neurons (MLMVN) is considered in the paper. A multi-valued neuron (MVN) is based on the principles of multiple-valued threshold logic over the field of the complex numbers. The most important properties of MVN are: the complex-valued weights, inputs and output coded by the kth roots of unity and the activation function, which maps the complex plane into the unit circle. MVN learning is reduced to the movement along the unit circle, it is based on a simple linear error correction rule and it does not require a derivative. It is shown that using a traditional architecture of multilayer feedforward neural network (MLF) and the high functionality of the MVN, it is possible to obtain a new powerful neural network. Its training does not require a derivative of the activation function and its functionality is higher than the functionality of MLF containing the same number of layers and neurons. These advantages of MLMVN are confirmed by testing using parity n, two spirals and “sonar” benchmarks and the Mackey–Glass time series prediction.     

一类改进的最小距离分类器的增量学习算法

提出一种基于改进的最小距离分类器的增量学习算法,消除增量学习过程中产生的分类器内部结构的相互干扰,使分类器既能记住已学习的知识,又能学习新知识.增量学习需要对分类器结构进行调整,必须使用有代表性的已学习样本帮助分类器在学习新知识时复习旧知识.针对正态分布的样本集提出一种筛选算法,只保留有代表性的少量样本,大大减少存储消耗和重新训练的计算开销.实验结果证明该算法对样本的识别准确率高,在有效识别新样本的同时对以前学习的样本也保持较高的识别率,消耗存储空间小.     

BP神经网络中自适应学习率的研究

l引言~[1] 图1是一个典型的三层神经网络BP算铸示意图.Z是输入向量,Y是隐层输出向量0是网络输出向量,V及W分别为层间权向量。逆传播(Backprop-     

基于BP神经网络的指纹模板分类器分类算法

一、引言计算机指纹识别技术是一门综合性的技术,21世纪的今天,它已成为可靠的个人身份鉴定的方法之一。在所有生物识别技术中,指纹识别技术是应用最广泛最普及的,例如它可以应用于金融、保险、证券行业的身份认证,安防业,人力资源管理等等。但随着应用广泛性的增强,人工对比指纹往往会出现效率低、速度慢的现象,所以近年来人们对指纹识别技术的可靠性、实时性的要求变得越来越高了。针对这样的要求,本文提出了一种基于BP神经网络的、对已建好的指纹模板库进行快速分类的算法。     

Global Convergence of GHA Learning Algorithm With Nonzero-Approaching Adaptive Learning Rates

The generalized Hebbian algorithm (GHA) is one of the most widely used principal component analysis (PCA) neural network (NN) learning algorithms. Learning rates of GHA play important roles in convergence of the algorithm for applications. Traditionally, the learning rates of GHA are required to converge to zero so that its convergence can be analyzed by studying the corresponding deterministic continuous-time (DCT) equations. However, the requirement for learning rates to approach zero is not a practical one in applications due to computational roundoff limitations and tracking requirements. In this paper, nonzero-approaching adaptive learning rates are proposed to overcome this problem. These proposed adaptive learning rates converge to some positive constants, which not only speed up the algorithm evolution considerably, but also guarantee global convergence of the GHA algorithm. The convergence is studied in detail by analyzing the corresponding deterministic discrete-time (DDT) equations. Extensive simulations are carried out to illustrate the theory.     

Analysis of the Initial Values in Split-Complex Backpropagation Algorithm

When a multilayer perceptron (MLP) is trained with the split-complex backpropagation (SCBP) algorithm, one observes a relatively strong dependence of the performance on the initial values. For the effective adjustments of the weights and biases in SCBP, we propose that the range of the initial values should be greater than that of the adjustment quantities. This criterion can reduce the misadjustment of the weights and biases. Based on the this criterion, the suitable range of the initial values can be estimated. The results show that the suitable range of the initial values depends on the property of the used communication channel and the structure of the MLP (the number of layers and the number of nodes in each layer). The results are studied using the equalizer scenarios. The simulation results show that the estimated range of the initial values gives significantly improved performance.      

基于改进的字典学习算法的图像去噪方法

提出对基于MOD和K-SVD字典学习算法的图像去噪的两个方面的改进。在字典更新阶段,采用一种新的字典更新方式,在保持支集完备的同时寻找字典和表示法。在稀疏编码阶段,根据前一次追踪过程产生的部分系数进行修正和更新。分别对这两种改进进行了验证,并说明了如何进行更快速的训练以及取得更好的结果,实验结果证实了论文方法的有效性。     

A Hammerstein Recurrent Neurofuzzy Network With an Online Minimal Realization Learning Algorithm

This paper presents a Hammerstein recurrent neurofuzzy network associated with an online minimal realization learning algorithm for dealing with nonlinear dynamic applications. We fuse the concept of states in linear systems into a neurofuzzy framework so that the whole structure can be expressed by a state-space representation. An online minimal realization learning algorithm has been developed to find a controllable and observable state-space model of minimal size from the input–output measurements of a given system. Such an idea can simultaneously resolve the problem of the determination of a minimal structure and the difficulty of network stability analysis. The advantages of our approach include: 1) our recurrent network is capable of translating the complicated dynamic behavior of a nonlinear system into a minimal set of linguistic fuzzy dynamical rules and into state-space representation as well and 2) an online minimal realization learning algorithm unifies an order determination algorithm, a hybrid parameter initialization method, and a recursive recurrent learning algorithm into a systematic procedure to identify a minimal structure with satisfactory performance. Performance evaluations on benchmark examples as well as real-world applications have successfully validated the effectiveness of our approach.      

A New Incremental PCA Algorithm With Application to Visual Learning and Recognition

This paper proposes a new mean-shifting Incremental PCA (IPCA) method based on the autocorrelation matrix. The dimension of the updated matrix remains constant instead of increasing with the number of input data points. Comparing to some previous batch and iterative PCA algorithms, the proposed IPCA requires lower computational time and storage capacity owing to the two transformations designed. The experiment results show the efficiency and accuracy of the proposed IPCA method in applications of the on-line visual learning and recognition.     

A New Adaptive Backpropagation Algorithm Based on Lyapunov Stability Theory for Neural Networks

A new adaptive backpropagation (BP) algorithm based on Lyapunov stability theory for neural networks is developed in this paper. It is shown that the candidate of a Lyapunov function V(k) of the tracking error between the output of a neural network and the desired reference signal is chosen first, and the weights of the neural network are then updated, from the output layer to the input layer, in the sense that DeltaV(k)=V(k)-V(k-1)<0. The output tracking error can then asymptotically converge to zero according to Lyapunov stability theory. Unlike gradient-based BP training algorithms, the new Lyapunov adaptive BP algorithm in this paper is not used for searching the global minimum point along the cost-function surface in the weight space, but it is aimed at constructing an energy surface with a single global minimum point through the adaptive adjustment of the weights as the time goes to infinity. Although a neural network may have bounded input disturbances, the effects of the disturbances can be eliminated, and asymptotic error convergence can be obtained. The new Lyapunov adaptive BP algorithm is then applied to the design of an adaptive filter in the simulation example to show the fast error convergence and strong robustness with respect to large bounded input disturbances     

基于改进多核学习的语音情感识别算法

提出一种基于改进多核学习的语音情感识别算法.算法以高斯径向基核函数为基准,通过采样不同的样本,采用不同的评价标准并获得不同的参数,来提高分类性能.此外,通过引入多核技术,将得到的高斯核函数构建多核学习的基核,并通过利用松弛因子构建的软间隔多核学习的目标函数改善了学习效率.对比仿真实验结果表明,本文提出的基于多核学习语音情感识别算法有效提高了语音情感识别性能.     

基于李雅普诺夫函数的BP神经网络算法的收敛性分析

针对前馈神经网络应时变输入的自学习机制，采用李雅普诺夫函数来分析权值的收敛性，从而揭示BP神经网络算法朝最小误差方向调整权值的内在因素，并在分析单参数BP算法收敛性基础上，提出单参数变调整法则的离散型BP神经网络算法．     

一种改进的克隆选择优化算法

人工免疫系统是基于生物免疫系统特性而发展的新兴智能系统。论文利用免疫系统的克隆选择机制,提出一种用于函数优化的改进克隆选择算法。算法的主要特点是采用克隆和自适应变异等操作,提高收敛速度和种群的多样性。仿真程序表明,该算法能以较快速度完成给定范围的搜索和全局优化任务。