Improving the Performance of Feedforward Neural Networks by Noise Injection into Hidden Neurons

The generalization ability of feedforward neural networks (NNs) depends on the size of training set and the feature of the training patterns. Theoretically the best classification property is obtained if all possible patterns are used to train the network, which is practically impossible. In this paper a new noise injection technique is proposed, that is noise injection into the hidden neurons at the summation level. Assuming that the test patterns are drawn from the same population used to generate the training set, we show that noise injection into hidden neurons is equivalent to training with noisy input patterns (i.e., larger training set). The simulation results indicate that the networks trained with the proposed technique and the networks trained with noisy input patterns have almost the same generalization and fault tolerance abilities. The learning time required by the proposed method is considerably less than that required by the training with noisy input patterns, and it is almost the same as that required by the standard backpropagation using normal input patterns.     

Evolving Multilayer Perceptrons

This paper proposes a new version of a method (G-Prop, genetic backpropagation) that attempts to solve the problem of finding appropriate initial weights and learning parameters for a single hidden layer Multilayer Perceptron (MLP) by combining an evolutionary algorithm (EA) and backpropagation (BP). The EA selects the MLP initial weights, the learning rate and changes the number of neurons in the hidden layer through the application of specific genetic operators, one of which is BP training. The EA works on the initial weights and structure of the MLP, which is then trained using QuickProp; thus G-Prop combines the advantages of the global search performed by the EA over the MLP parameter space and the local search of the BP algorithm. The application of the G-Prop algorithm to several real-world and benchmark problems shows that MLPs evolved using G-Prop are smaller and achieve a higher level of generalization than other perceptron training algorithms, such as QuickPropagation or RPROP, and other evolutive algorithms. It also shows some improvement over previous versions of the algorithm.     

人工神经网络泛化问题研究综述*

从理论、方法（思想）和技术三个层次回顾了以往工作,讨论了模型复杂度、样本复杂度及两者之间关系的相关研究;在实际中,通过控制模型复杂度、调整样本等具体技术可以在一定程度上提高神经网络的泛化能力,但这些技术仍然存在一些问题没有解决。最后提出了对今后研究的展望。     

Generalization-oriented Road Line Classification by Means of an Artificial Neural Network

In line generalization, a first goal to achieve is the classification of features previous to the selection of processes and parameters. A feed forward backpropagation artificial neural network (ANN) is designed for classifying a set of road lines through a supervised learning process, attempting to emulate a classification performed by a human expert for cartographic generalization purposes. The main steps of the process are presented in this paper: (a) experimental data selection; (b) segmentation of lines into homogeneous sections, (c) sections enrichment through a set of quantitative measures derived from a principal component analysis, and qualitative information derived from road network and road type; (d) expert classification of the sections; and finally (e) the ANN design, training and validation. The quality of results is analyzed by means of error matrices after a cross-validation process giving a goodness, or percentage of agreement, over 83%.

数值优化改进的BP网络的模式分类对比

研究了共轭梯度算法、拟牛顿算法、LM算法三类常用的数值优化改进算法,基于这三类数值优化算法分别对BP神经网络进行改进,并构建了相应的BP神经网络分类模型,将构建的分类模型应用于二维向量模式的分类,并进行了泛化能力测试,将不同BP网络分类模型的分类结果进行对比. 仿真结果表明,对于中小规模的网络而言,LM数值优化算法改进的BP网络的分类结果最为精确,收敛速度最快,分类性能最优;共轭梯度数值优化算法改进的BP网络的分类结果误差最大,收敛速度最慢,分类性能最差;拟牛顿数值优化算法改进的BP网络的分类结果误差值、收敛速度及分类性能介于上述两种算法之间.     

Crashworthiness optimization of helicopter subfloor based on decomposition and global approximation

This work deals with the development of an optimization procedure under crashworthiness requirements applied to a typical helicopter subfloor. The difficulties due to the nonlinear design space and structural behaviour are overcome by developing an optimization procedure based on decomposition, where the structure to be optimized is converted into a set of smaller and linked substructures. To evaluate the response of each substructure, global approximation strategies, based on neural networks, are used. Size variables (dimensions, thickness) and geometrical variables (element number and position) are considered in order to maximize the global crashworthiness performance. The energy absorbed per unit mass by the subfloor is chosen as objective function and acceleration constraints are considered. Genetic algorithms are used to find the optimal configuration. The optimization allowed an increase in the crush force efficiency of 12% and a decrease in the subfloor mass of 4%. A significant CPU time saving was also obtained.     

A Meta-representation for Integrating Operational Research and Artificial Intelligence in an Intelligent Decision Support Paradigm

Operational research (OR) and artificial intelligence (AI) models are primary contributors to the area of intelligent decision support systems (IDSS). Constraint logic programming (CLP) has been used successfully to substantiate the integration of OR and AI. We present a meta-level modular representation for integrating OR and AI models using CLP in an IDSS framework. The use of this representation is illustrated using a CLP-like meta-language, and the potential usefulness of this language is demonstrated using an example from the dairy industry.     

基于粒子群优化算法的BP网络学习研究

文章提出了基于粒子群优化的BP网络学习算法。在该算法中,用粒子群优化算法替代了传统BP算法中的梯度下降法,使得改进后的算法具有不易陷入局部极小、泛化性能好等特点。并将该算法应用在了高速公路动态称重系统的设计中,实验证明:这种算法能够明显减少迭代次数、提高收敛精度,其泛化性能也优于传统BP算法。     

基于神经网络-遗传算法的CT-191合成工艺优化

本文用锌粉还原N-亚硝基二苯胺的产物直接与2-甲基-4（N,N-二苄基）氨基苯甲醛缩合合成了空穴传输材料2-甲基-4（N,N-二苄基）氨基苯甲醛-1,1-二苯腙（CT-191）,采用均匀设计制定试验方案获取原始数据,应用BP人工神经网络对合成过程中工艺参数和一次产品收率的关系建立了模型,并用遗传算法进行优化得到最佳工艺条件：原料2-甲基-4-（N,N-二苄基）氨基苯甲醛：N-亚硝基二苯胺约为1：2．5,还原时间为1h,缩合时间为2h,预测收率为96．28％。验证实验的结果为95．98％．和预测值基本吻合。为化学生产工艺的优化探索了一条新途径。     

基于样本个体差异性的深度神经网络训练方法

深度神经网络目前在许多任务中的表现已经达到甚至超越了人类的水平,但是其泛化能力和人类相比还是相去甚远.如何提高网络的泛化性,一直是重要的研究方向之一.围绕这个方向开展的大量卓有成效的研究,从扩展增强训练数据、通过正则化抑制模型复杂度、优化训练策略等角度,提出了很多行之有效的方法.这些方法对于训练数据集来说都是某种全局性质的策略,每一个样本数据都会被平等的对待.但是,每一个样本数据由于其携带的信息量、噪声等的不同,在训练过程中,对模型的拟合性能和泛化性能的影响也应该是有差异性的.针对是否一些样本在反复的迭代训练中更倾向于使得模型过度拟合,如何找到这些样本,是否可以通过对不同的样本采用差异化的抗过拟合策略使得模型获得更好的泛化性能等问题,提出了一种依据样本数据的差异性来训练深度神经网络的方法,首先使用预训练模型对每一个训练样本进行评估,判断每个样本对该模型的拟合效果;然后依据评估结果将训练集分为易使得模型过拟合的样本和普通的样本两个子集;最后,再使用两个子集的数据对模型进行交替训练,过程中对易使得模型过拟合的子集采用更强有力的抗过拟合策略.通过在不同的数据集上对多种深度模型进行的一系列实验...     

基于人工免疫算法的最小二乘支持向量机参数优化算法*

针对最小二乘支持向量机（LSSVM）处理大数据集时确定最优模型参数耗时长、占内存大的问题,提出了一种基于人工免疫算法的参数寻优方法。通过分析LSSVM模型参数对分类准确率的影响发现,存在多种参数组合,使得分类准确率相同;当其中一个参数固定,另外一个参数在某些范围内变化取值时,它们的组合并不影响分类的准确率。将LSSVM模型参数作为抗体的基因设计了抗体的编码方案,利用人工免疫算法对LSSVM参数优化搜索。仿真结果表明,与使用交叉验证和网格搜索方法相比,提出的LSSVM参数优化算法在不降低分类准确率的前提下,寻优效率大大提高。     

A relaxation algorithm for estimating the domain of validity of feedforward neural networks

We expand on a recent paper by Courrieu which introduces three algorithms for determining the distance between any point and the interpolation domain associated with a feedforward neural network. This has been shown to have a significant relation with the network's generalization capability. A further neural-like relaxation algorithm is presented here, which is proven to naturally solve the problem originally posed by Courrieu. The algorithm is based on a powerful result developed in the context of Markov chain theory, and turns out to be a special case of a more general relaxation model which has long become a standard technique in the machine vision domain. Some experiments are presented which confirm the validity of the proposed approach.     

A Fast Heuristic Global Learning Algorithm for Multilayer Neural Networks

This paper presents a novel Heuristic Global Learning (HER-GBL) algorithm for multilayer neural networks. The algorithm is based upon the least squares method to maintain the fast convergence speed, and the penalized optimization to solve the problem of local minima. The penalty term, defined as a Gaussian-type function of the weight, is to provide an uphill force to escape from local minima. As a result, the training performance is dramatically improved. The proposed HER-GBL algorithm yields excellent results in terms of convergence speed, avoidance of local minima and quality of solution.     

神经网络集成

神经网络集成通过训练多个神经网络并将成结论进行合成,可以显著地提高学习系统的泛化能力。它不仅有助于科学家对机器学习和神经的深入研究,还有助于普通工程技术人员利用神经网络技术来解决真实世界中的问题。因此,它被视为一种广阔应用前景的工程化神经计算技术,已经成为机器学习和神经计算领域的研究热点。该文从实现方法、理论分析和应用成果等三个方面综述了神经网络集成的国际研究现状,并对该领域值得进一步研究的一些问题进行了讨论。