多层结构神经网络的等误差范围逼近与收缩学习方法及其应用

本文提出了一种用于前馈型多层神经网络学习的等误差范围逼近与收缩学习方法,这种方法仅仅要求网络的实际输出落在理想模式输出的一个事先给定的误差范围之内,从而可以大大提高网络的学习速度,且运算量小,而且通过适当选择等误差范围,它还可以提高网络在模式识别中的推广性能.如果网络用于模式联想等方面时,通过误差范围的逐步收缩,这种方法还可以以很小的额外代价提高网络学习的逼近精度;另外,它还可以避免传统方法中经常出现的训练模式反转等局域极小状态和过学习现象的出现.最后,文中给出了以这种方法训练的网络用于脑电波癫痫信号识别中的实验结果及其分析.     

对称神经元网络：渐近稳定性和学习

本文在分析对称神经元网络的渐近稳定性基础上,提出了一种非约束优化学习算法,保证训练样本成为稳定吸引子,具有一定大小的吸引域.理论上,我们证明了算法的收敛性以及形成的吸引域下界.计算机实验结果充分说明了学习算法的优越性.     

A CASCADED MODEL OF NEURAL NETWORK FOR PATTERN RECOGNITION

A cascaded model of neural network and its learning algorithm suitable for opticalimplementation are proposed.Computer simulations have shown that this model may successfullybe applied to an error-tolerance pattern recognitions of multiple 3-D targets with arbitrary spatialorientations.     

一种改进的CP网络学习算法

本文针对学习CP网络在训练过程中出现的不稳定性提出了一种改进的CP网络学习算法。由于这种改进的学习算法在训练过程中将希望输出的信息传递给第一层权值，因此它有效地克服了CP网络训练过程中的不稳定性，避免了网络在训练过程中陷于局部最小点。本文最后给出了一个实例，通过这一实例可以看出改进的学习算法明显地优于传统的学习算法。     

雷达高分辨距离像分类器的参数自适应学习算法

雷达高分辨率距离像具有目标姿态敏感性的特点,在识别时的一种解决方法是对目标不同角域建立不同的统计模型。在给定系统参数条件下,选择目标划分角域个数及每个角域覆盖范围是影响识别器运算量及识别性能的关键。该文给出了一种基于数据的自适应学习上述分类器参数的算法,基于联合高斯分布的数据模型通过迭代算法来确定数据划分边界,并自动确定目标角域个数。与等间隔数据划分方法相比,本文方法在降低识别运算量的同时,可以提高识别性能。基于实测数据的实验结果表明该方法是有效的。     

A vector neural network for emitter identification

This paper proposes a three-layer vector neural network (VNN) with a supervised learning algorithm suitable for signal classification in general, and for emitter identification (EID) in particular. The VNN can accept interval-value input data as well as scalar input data. The input features of the EID problems include the radio frequency, pulse width, and pulse repetition interval of a received emitter signal. Since the values of these features vary in interval ranges in accordance with a specific radar emitter, the VNN is proposed to process interval-value data in the EID problem. In the training phase, the interval values of the three features are presented to the input nodes of VNN. A new vector-type backpropagation learning algorithm is derived from an error function defined by the VNN's actual output and the desired output indicating the correct emitter type of the corresponding feature intervals. The algorithm can tune the weights of VNN optimally to approximate the nonlinear mapping between a given training set of feature intervals and the corresponding set of desired emitter types. After training, the VNN can be used to identify the sensed scalar-value features from a real-time received emitter signal. A number of simulations are presented to demonstrate the effectiveness and identification capability of VNN, including the two-EID problem and the multi-EID problem with/without additive noise. The simulated results show that the proposed algorithm cannot only accelerate the convergence speed, but it can help avoid getting stuck in bad local minima and achieve higher classification rate.     

30 years of adaptive neural networks: perceptron, Madaline, andbackpropagation

Fundamental developments in feedforward artificial neural networks from the past thirty years are reviewed. The history, origination, operating characteristics, and basic theory of several supervised neural-network training algorithms (including the perceptron rule, the least-mean-square algorithm, three Madaline rules, and the backpropagation technique) are described. The concept underlying these iterative adaptation algorithms is the minimal disturbance principle, which suggests that during training it is advisable to inject new information into a network in a manner that disturbs stored information to the smallest extent possible. The two principal kinds of online rules that have developed for altering the weights of a network are examined for both single-threshold elements and multielement networks. They are error-correction rules, which alter the weights of a network to correct error in the output response to the present input pattern, and gradient rules, which alter the weights of a network during each pattern presentation by gradient descent with the objective of reducing mean-square error (averaged over all training patterns)     

一种投票权值调整的神经网络集成增量学习方法

针对神经网络集成增量学习中集成输出投票权值的设定问题,给出了一种投票权值调整的神经网络集成增量学习方法。该方法定义了神经网络集成中子神经网络训练集的类核函数,通过计算待识样本与类核函数之间的核函数距离得到集成输出中子神经网络的投票权值。这种投票权值设定方法可以根据子神经网络分类器对待识样本的分类性能自适应地调整集成输出的投票权值,是一种更加合理的集成输出投票权值设定方法。仿真实验表明,这种投票权值调整的神经网络集成增量学习方法比投票权值固定的方法增量学习性能更优。      

基于Grassmann流形几何深度网络的图像集识别方法

近年来,深度学习以其强大的非线性计算能力在目标检测和识别任务中取得了巨大的突破。现有的深度学习网络几乎都是以数据的欧氏结构为前提,而在计算机视觉中许多数据都具有严格的流形结构,如图像集可表示为Grassmann流形。基于数据的流形几何结构来设计深度学习网络,将微分几何理论与深度学习理论相结合,提出一种基于Grassmann流形的深度图像集识别网络。同时在模型训练过程中,使用基于矩阵链式法则的反向传播算法来更新模型,并将权值的优化过程转换为Grassmann流形上的黎曼优化问题。实验结果表明:该方法不仅在结果上识别准确率得到了提高,同时在训练和测试速度上也有一个数量级的提升。     

基于深度残差网络的雷达目标数量估计方法

传统雷达目标检测方法一般将单个距离单元的目标当成单目标进行检测,而不会估计距离单元内目标的数量.针对该研究空缺,提出一种基于深度残差网络的雷达目标数量估计方法.该方法将雷达信号转换成时频图并输入至训练好的深度残差网络.残差网络根据单个目标与多个目标对应时频图的差异即可准确得到雷达目标数量的估计值.仿真表明该方法能有效地...     

基于GA与L-M优化算法的变压器故障诊断研究

利用MATLAB环境建立一个用于变压器故障诊断的BP网络模型。首先利用具有全局寻优功能的遗传算法对BP神经网络的初始权值和阈值进行优化,然后采用L-M(Levenberg-Marquardt)优化算法对BP神经网络进行训练,从而达到加快网络训练速度,避免训练过程陷入局部极小点的目的。最后,详细记录网络的实际输出,并与期望输出做对比研究,最终证实了此网络达到了设计要求,可用于变压器的故障诊断。     

利用卷积-循环神经网络的串行序列空时分组码识别方法

针对多输入多输出(Multiple Input Multiple Output, MIMO)系统中的空时分组码识别(Space-Time Block Code, STBC)问题,本文提出了一种利用卷积-循环神经网络的串行序列空时分组码识别方法。将一维接收信号的实部和虚部分离后输入网络,利用卷积神经网络(CNN)提取其空间特征,结合循环神经网络(RNN)提取其深层时序特征,提高网络的特征表达能力;网络训练过程采用反向传播方法,通过计算输出与目标值的误差,将误差反向传回网络中并更新权值,完成网络的训练过程;将测试集数据输入训练好的网络中,实现对空时分组码的识别和区分。该方法将深度学习算法运用到串行序列空时分组码识别当中,训练完的网络可直接对单接收天线下的空时分组码进行识别,不需要重复计算信号的统计特征,避免了人为设计特征参数和检测阈值。该方法不需要知道信道和噪声的先验信息,适用于电子侦查等非协作通信情况。仿真实验表明,该算法能够有效地对串行序列空时分组码进行识别,并且在低信噪比下有较好的识别性能。      

Incremental augmented complex adaptive IIR algorithm for training widely linear ARMA model

In this paper, we propose a distributed adaptive learning algorithm to train the coefficients of a widely linear autoregressive moving average model by measurements collected by the nodes of a network. We assume that each node uses the augmented complex adaptive infinite impulse response (ACA-IIR) filter as the learning rule, and nodes interact with each other under an incremental mode of cooperation. To derive the proposed algorithm, called the incremental ACAIIR (IACA-IIR), we firstly formulate the distributed adaptive learning problem as an unconstrained minimization problem. Then, we apply stochastic gradient optimization argument to solve it and derive the proposed algorithm. We further find the step size range where the stability of the proposed algorithm is guaranteed. We also introduce a reduced-complexity version of the IACA-IIR algorithm. Since the proposed algorithm relies on the augmented complex statistics, it can be used to model both types of complex-valued signals (proper and improper signals). To evaluate the performance of the proposed algorithm, we use both synthetic and real-world complex signals in our simulations. The results exhibit superior performance of the proposed algorithm over the non-cooperative ACA-IIR algorithm.     

深度复极限学习机在雷达HRRP目标识别中的应用

传统雷达高分辨一维距离像(High-resolution Range Profile,HRRP)目标识别方法只利用目标幅度信息而丢失其相位信息,这势必会造成信息不完备。为解决此问题,提出将深度极限学习机从实数域扩展到复数域,以有效提取复HRRP序列的深层潜在结构信息。同时为更好地保持数据间的邻域信息,将流形正则化引入到网络模型训练过程中,提出流形正则深度复极限学习机。在雷达暗室测量数据上的实验结果表明,所提算法相比常用的深度学习模型具有更好的识别效果和更快的训练速度,验证了算法的有效性。     

基于主动学习和SVM方法的网络协议识别技术

针对未知网络协议数据流的获取与标记工作主要依赖于领域专家。然而,样本数据量的增加会导致人工成本超过实际负荷。提出了一种新颖的未知网络协议识别方法。该方法基于主动学习算法,仅依靠原始网络数据流的载荷部分实现对未知网络协议的有效识别。实验结果表明,采用该方法设计的识别系统在保证识别准确率和召回率的前提下,能够有效地降低学习过程中标记的样本数目,更适用于实际的网络应用环境。     

改进的BP神经网络在交通流量预测中应用

针对传统BP学习算法收敛速度慢,对步长依赖明显等缺点,提出一种利用搜索较优步长的BP算法.在网络训练中,能够在每次迭代中搜索出一个相对合理的步长,从而使步长的选择对学习速度的影响大大降低.对交通流量预测仿真结果表明,新算法对步长选择的依赖性小于传统BP算法.     

基于粒子群优化的神经网络训练算法研究

本文提出了基于连接结构优化的粒子群优化算法(SPSO)用于神经网络训练,该算法在训练神经网络权值的同时优化其连接结构,删除冗余连接,使神经网络获得与模式分类问题匹配的信息处理能力.经SPSO训练的神经网络应用于Iris,Ionosphere以及Breast cancer模式分类问题,能够部分消除冗余分类参数及冗余连接结构对分类性能的影响.与BP算法及遗传算法比较,该算法在提高分类误差精度的同时可加快训练收敛的速度.仿真结果表明,SPSO是有效的神经网络训练算法.     

Function Approximation Based on a Network with Kernel Functions of Bounds and Locality: an Approach of Non-Parametric Estimation

This paper presents function approximation based on nonparametric estimation. As an estimation model of function approximation, a three layered network composed of input, hidden and output layers is considered. The input and output layers have linear activation units while the hidden layer has nonlinear activation units or kernel functions which have the characteristics of bounds and locality. Using this type of network, a many-to-one function is synthesized over the domain of the input space by a number of kernel functions. In this network, we have to estimate the necessary number of kernel functions as well as the parameters associated with kernel functions. For this purpose, a new method of parameter estimation in which linear learning rule is applied between hidden and output layers while nonlinear (piecewise-linear) learning rule is applied between input and hidden layers, is considered. The linear learning rule updates the output weights between hidden and output layers based on the Linear Minimization of Mean Square Error (LMMSE) sense in the space of kernel functions while the nonlinear learning rule updates the parameters of kernel functions based on the gradient of the actual output of network with respect to the parameters (especially, the shape) of kernel functions. This approach of parameter adaptation provides near optimal values of the parameters associated with kernel functions in the sense of minimizing mean square error. As a result, the suggested nonparametric estimation provides an efficient way of function approximation from the view point of the number of kernel functions as well as learning speed.     

Optimizing the Area Under a Receiver Operating Characteristic Curve With Application to Landmine Detection

A common approach to training neural network classifiers in a supervised learning setting is to minimize the mean-square error (mse) between the network output for each labeled training sample and some desired output. In the context of landmine detection and discrimination, although the performance of an algorithm is correlated with the mse, it is ultimately evaluated by using receiver operating characteristic (ROC) curves. In general, the larger the area under the ROC curve (AUC), the better. We present a new method for maximizing the AUC. Desirable properties of the proposed algorithm are derived and discussed that differentiate it from previously proposed algorithms. A hypothesis test is used to compare the proposed algorithm to an existing algorithm. The false alarm rate achieved by the proposed algorithm is found to be less than that of the existing algorithm with 95% confidence     

Medical diagnosis with C4.5 rule preceded by artificial neural network ensemble

Comprehensibility is very important when machine learning techniques are used in computer-aided medical diagnosis. Since an artificial neural network ensemble is composed of multiple artificial neural networks, its comprehensibility is worse than that of a single artificial neural network. In this paper, C4.5 Rule-PANE, which combines an artificial neural network ensemble with rule induction by regarding the former as a preprocess of the latter, is proposed. At first, an artificial neural network ensemble is trained. Then, a new training data set is generated by feeding the feature vectors of original training instances to the trained ensemble and replacing the expected class labels of original training instances with the class labels output from the ensemble. Additional training data may also be appended by randomly generating feature vectors and combining them with their corresponding class labels output from the ensemble. Finally, a specific rule induction approach, i.e., C4.5 Rule, is used to learn rules from the new training data set. Case studies on diabetes, hepatitis , and breast cancer show that C4.5 Rule-PANE could generate rules with strong generalization ability, which benefits from an artificial neural network ensemble, and strong comprehensibility, which benefits from rule induction.