基于特征选择的神经网络集成研究

本文提出了一种基于交叉验证和ReliefF的神经网络集成学习算法(CVRNNEn算法),首先利用交叉验证选取个体网络的训练数据集,然后再对每个训练集进行特征选择,来降低数据集的规模,减少相关性低的特征的对个体网络预测结果的干扰,提高了个体网络的预测精度和个体网络之间的差异度。算法代码在weka3.5.6平台上实现,通过在UCI数据集上仿真实验,和单个RBF网络的预测结果进行比较,得出CVRNNEn算法预测性能更优,从实验上证实了该算法在预测性能上的优势。     

基于Cross-Validation的电机故障诊断振动数据处理方法

针对牵引电机故障诊断研究中所采用的神经网络方法,提出在模型训练阶段引入K折交叉验证。该方法在划分训练集与测试集期间,使验证集能够遍历所有数据集,从多方向开始学习,从而在一定程度上避免了局部极小的问题。训练完成后,以神经网络作为分类器进行故障识别。神经网络学习算法采用随机梯度下降的方法,每次投入一组数据集进行训练,大大提高了训练速度。Eclipse+Anaconda仿真结果证明:与传统神经网络电机故障诊断方法相比,该方法可以在一定程度上避免过拟合现象,同时避免局部极小。此外,在Matlab环境下,单独比较支持向量机采用交叉验证前后的故障分类效果。对比结果表明:交叉验证方法从多方向开始学习,对于提升故障诊断的准确率有较好作用。     

功耗分析攻击中机器学习模型选择研究

根据密码芯片功耗曲线的特性,对支持向量机、随机森林、K最近邻、朴素贝叶斯4种机器学习算法进行分析研究,从中选择用于功耗分析攻击的最优算法。对于机器学习算法的数据选取问题,使用多组数量相同但组成元素不同的数据集的十折交叉验证结果进行模型选择,提高测试公平性及测试结果的泛化能力。为避免十折交叉验证过程中出现测试集误差不足以近似泛化误差的问题,采用Friedman检验及Nemenyi后续检验相结合的方法对4种机器学习算法进行评估,结果表明支持向量机是适用于功耗分析攻击的最优机器学习算法。     

基于跨通道交叉融合和跨模块连接的轻量级卷积神经网络

针对传统卷积神经网络参数量过多、计算复杂度高的问题,提出了基于跨通道交叉融合和跨模块连接的轻量级卷积神经网络架构C-Net。首先,提出了跨通道交叉融合的方法,它在一定程度上克服了分组卷积中各分组之间存在缺乏信息流动的问题,简单高效地实现了不同分组之间的信息通信;其次,提出了一种跨模块连接的方法,它克服了传统轻量级架构中各基本构建块之间彼此独立的缺点,实现了同一阶段内具有相同分辨率特征映射的不同模块之间的信息融合,从而增强了特征提取能力;最后,基于提出的两种方法设计了一种新型的轻量级卷积神经网络架构C-Net。C-Net在Food_101数据集上的准确率为69.41%,在Caltech_256数据集上的准确率为63.93%。实验结果表明,与目前先进的轻量级卷积神经网络模型相比,C-Net降低了存储开销和计算复杂度。在Cifar_10数据集上的消融实验验证了所提出的两种方法的有效性。     

化学计量学建模软件的开发及应用

本文利用数据库技术,针对Y=F(X)数学模型,编写集模型管理、数据样本管理和建模、预报为一体的通用性处理软件。软件自动适应不同分析体系中,自变量数和目标函数的变化,并支持在模型中对每个自变量进行定义,及其物理意义的显示。软件中提供了MLR、PCR、PLS、神经网络和遗传算法-PLS联用5种建模方法,并将交叉验证方法嵌入PCR、PLS方法中,用交叉验证法检验样本中的奇异样本点。利用模拟数据对软件进行了检验。     

基于概率神经网络的乳腺癌计算机辅助诊断

由于癌细胞多形性,细针吸取细胞学检查存在局限性,辩识能力不足,造成误诊。为进一步提高乳腺癌辅助诊断的准确性,提出了一种基于概率神经网络的乳腺癌辅助诊断方法。首先建立基于概率神经网络的分类模型,其次确定网络的训练集和测试集,接着找出最优的径向基函数分布密度,最后计算5-折交叉验证的测试准确度,并对仿真方法和结果进行了检验。将仿真结果和检验结果与已有文献中所得出的结果进行对比分析,表明用概率神经网络进行乳腺癌的辅助诊断,具有准确度高,诊断用时少,易于实现等优点,说明了其在乳腺癌计算机辅助诊断方面的可行性和优越性。     

融合排序与回归的卷积神经网络用于表情强度估计

表情强度估计是面部表情分析的重要组成部分,是实现人机自然情感交互的关键技术.表情强度估计面临的主要挑战在于缺乏大量的有标签数据,难以通过有监督的方法来估计表情强度.尽管基于排序的方法能够解决这一问题,但是排序方法只能估计表情的相对强度,无法估计表情的绝对强度.为了解决上述问题,提出了一种融合排序与回归的卷积神经网络用于表情强度估计.其中,排序卷积神经网络采用孪生网络结构,用于学习序列中任意两帧图像的相对强弱关系;孪生网络的每一个子网采用回归卷积神经网络,用于学习有强度标签的样本,从而估计表情的绝对强度.为了验证方法的有效性,在公共数据集PAIN和CK+上进行了实验.实验结果表明,提出的方法在弱监督的条件估计表情强度的各项结果(PAIN数据集上PCC, ICC和MAE分别为0.655 1, 0.529 3和0.924 1, CK+数据集上PCC, ICC和MAE分别为0.739 1, 0.721 6和0.187 5),均优于现有的方法.     

基于跨通道交叉融合和跨模块连接的轻量级卷积神经网络

针对传统卷积神经网络参数量过多、计算复杂度高的问题，提出了基于跨通道交叉融合和跨模块连接的轻量级卷积神经网络架构C-Net。首先，提出了跨通道交叉融合的方法，它在一定程度上克服了分组卷积中各分组之间存在缺乏信息流动的问题，简单高效地实现了不同分组之间的信息通信；其次，提出了一种跨模块连接的方法，它克服了传统轻量级架构中各基本构建块之间彼此独立的缺点，实现了同一阶段内具有相同分辨率特征映射的不同模块之间的信息融合，从而增强了特征提取能力；最后，基于提出的两种方法设计了一种新型的轻量级卷积神经网络架构C-Net。C-Net在Food_101数据集上的准确率为69.41%，在Caltech_256数据集上的准确率为63.93%。实验结果表明，与目前先进的轻量级卷积神经网络模型相比，C-Net降低了存储开销和计算复杂度。在Cifar_10数据集上的消融实验验证了所提出的两种方法的有效性。     

基于改进的NSGA-II多目标优化方法研究*

针对传统多目标优化算法在其领域存在的多个子目标不能同时取优的问题,提出了一种基于改进的非支配排序遗传算法（Non-dominated Sorting Genetic Algorithm-II, NSGA-II）多目标优化方法,以多目标优化遗传算法为基础,多输入多输出的反向传播（Back-Propagation, BP）神经网络为适应度函数评价体系,保证算法快速收敛并搜索到全局最优解集,该算法在建模前对实验数据进行主成分分析,降低了运算时间和算法难度,通过在遗传进化过程中引进正态分布交叉算子（Normal Distribution Crossover, NDX）和改进的自适应调整变异算子,实现了多个目标同时取优,保证Pareto最优解集快速、准确地获取。仿真实验使用UCI数据集,通过与其他常用的多目标优化算法对比,验证了改进的NSGA-II算法精确度更高、收敛速度更快、稳定性更强。     

基于正则化KL距离的交叉验证折数K的选择

在机器学习中,K折交叉验证方法常常通过把数据分成多个训练集和测试集来进行模型评估与选择,然而其折数K的选择一直是一个公开的问题。注意到上述交叉验证数据划分的一个前提假定是训练集和测试集的分布一致,但是实际数据划分中,往往不是这样。因此,可以通过度量训练集和测试集的分布一致性来进行K折交叉验证折数K的选择。直观地,KL(Kullback-Leibler)距离是一种合适的度量方法,因为它度量了两个分布之间的差异。然而直接基于KL距离进行K的选择时,从多个数据实验结果发现随着K的增加KL距离也在增大,显然这是不合适的。为此,提出了一种基于正则化KL距离的K折交叉验证折数K的选择准则,通过最小化此正则KL距离来选择合适的折数K。进一步多个真实数据实验验证了提出准则的有效性和合理性。     

Weights and structure determination of multiple-input feed-forward neural network activated by Chebyshev polynomials of Class 2 via cross-validation

Differing from conventional improvements on backpropagation (BP) neural network, a novel neural network is proposed and investigated in this paper to overcome the BP neural-network weaknesses, which is called the multiple-input feed-forward neural network activated by Chebyshev polynomials of Class 2 (MINN-CP2). In addition, to obtain the optimal number of hidden-layer neurons and the optimal linking weights of the MINN-CP2, the paper develops an algorithm of weights and structure determination (WASD) via cross-validation. Numerical studies show the effectiveness and superior abilities (in terms of approximation and generalization) of the MINN-CP2 equipped with the algorithm of WASD via cross-validation. Moreover, an application to gray image denoising demonstrates the effective implementation and application prospect of the proposed MINN-CP2 equipped with the algorithm of WASD via cross-validation.     

Gap-based estimation: choosing the smoothing parameters for probabilistic and general regression neural networks

Probabilistic neural networks (PNN) and general regression neural networks (GRNN) represent knowledge by simple but interpretable models that approximate the optimal classifier or predictor in the sense of expected value of the accuracy. These models require the specification of an important smoothing parameter, which is usually chosen by cross-validation or clustering. In this article, we demonstrate the problems with the cross-validation and clustering approaches to specify the smoothing parameter, discuss the relationship between this parameter and some of the data statistics, and attempt to develop a fast approach to determine the optimal value of this parameter. Finally, through experimentation, we show that our approach, referred to as a gap-based estimation approach, is superior in speed to the compared approaches, including support vector machine, and yields good and stable accuracy.     

Bayesian multioutput feedforward neural networks comparison: a conjugate prior approach

A Bayesian method for the comparison and selection of multioutput feedforward neural network topology, based on the predictive capability, is proposed. As a measure of the prediction fitness potential, an expected utility criterion is considered which is consistently estimated by a sample-reuse computation. As opposed to classic point-prediction-based cross-validation methods, this expected utility is defined from the logarithmic score of the neural model predictive probability density. It is shown how the advocated choice of a conjugate probability distribution as prior for the parameters of a competing network, allows a consistent approximation of the network posterior predictive density. A comparison of the performances of the proposed method with the performances of usual selection procedures based on classic cross-validation and information-theoretic criteria, is performed first on a simulated case study, and then on a well known food analysis dataset.     

Bayesian model assessment and comparison using cross-validation predictive densities

In this work, we discuss practical methods for the assessment, comparison, and selection of complex hierarchical Bayesian models. A natural way to assess the goodness of the model is to estimate its future predictive capability by estimating expected utilities. Instead of just making a point estimate, it is important to obtain the distribution of the expected utility estimate because it describes the uncertainty in the estimate. The distributions of the expected utility estimates can also be used to compare models, for example, by computing the probability of one model having a better expected utility than some other model. We propose an approach using cross-validation predictive densities to obtain expected utility estimates and Bayesian bootstrap to obtain samples from their distributions. We also discuss the probabilistic assumptions made and properties of two practical cross-validation methods, importance sampling and k-fold cross-validation. As illustrative examples, we use multilayer perceptron neural networks and gaussian processes with Markov chain Monte Carlo sampling in one toy problem and two challenging real-world problems.     

SVD-NET: an algorithm that automatically selects network structure

An algorithm is developed for training feedforward neural networks that uses singular value decomposition (SVD) to identify and eliminate redundant hidden nodes. Minimizing redundancy gives smaller networks, producing models that generalize better and thus eliminate the need of using cross-validation to avoid overfitting. The method is demonstrated by modeling a chemical reactor.     

Comparative study of artificial neural network for classification of hot and cold recombination regions in Saccharomyces cerevisiae

At the chromosomal level of evolution, recombination is a major factor for genetic variations. However, recombination does not occur with equal frequency at various regions of genome. The recombination has the tendency to occur at specific region with higher frequency and with low frequency at other regions, and former regions are named as hot recombination regions whereas later are called cold regions for recombination. In this paper, we have developed supervised machine learning-based models using artificial neural network, support vector machine and Naïve Bayes for efficient and effective classification of such hot and cold recombination regions based on the nucleotide composition of sequences. All models were validated and tested using tenfold cross-validation. Furthermore, neural network model was validated using leave one out and random sampling techniques in addition to tenfold cross-validation. Moreover, models were evaluated using receiver-operating curve. Our results indicate that artificial neural network achieves the best result.

     

人工神经网络结合正交变换方法研究

本文提出一和中运用人工神经网络结合正交换变换的方法，即通过正交变换滤除噪声，通过交叉验证确定网络最佳构型，以充分发挥正交变换和神经网络各自的长处，即免了同过拟合，实现更准确的预后。     

自底向上优化神经网络的方法

引荐了一种自动优化神经网络的新方法。这种启迪方法综合采用了相关有效算法,通过快速自底向上构造神经网络算法,可以获得优化结构的神经网络,即时选定参数算法动态优化神经网络的学习参数,并且快速交叉校验算法为解决过度适应问题提供了捷径。实验证明,这种启迪方法能自动有效地优化神经网络,与其它算法相较而言,具有更好的归纳性能、优化的网络结构和更快的学习速度。     

神经网络模式识别用于金属间化合物三元填隙d88结构形成条件的判别

本文将神经网络模式识别用于金属间化合物3元填隙d88结构形成条件的判别;神经网络由3层组成,训练用后向传播算法。为了评价所得模型的行为,使用了交叉验证法。计算结果表明,当适当的键参数作为输入时,71种化合物能被正确地分类,且隐层节点数经优选后可以提高分类的准确率,减少计算时间。     

加权交叉验证神经网络在水质预测中的应用

在之前的研究中使用人工神经网络进行水质指标预测已经取得一定效果,在此基础上将交叉验证应用于人工神经网络的训练,获得更加准确的预测结果。以澧水某监测站的水质实测数据作为样本,选取总磷、总氮、溶解氧等6个指标,建立水质预测模型。在运用Levenberg-Marquardt优化算法对学习样本进行优化的基础上,采用加权的k-fold交叉验证方法来构建神经网络集合,构建集合时采取三种不同的混合方式：平均值、中间值和加权累积。针对不同的指标,进行了一系列的实验,总的来说,新的预测方法与简单0倍验证相比有更好的预测结果,在所有指标中氨氮和溶解氧含量预测准确率比其他指标高。