基于粒子群算法的神经网络短期降水预报建模研究

用多样性粒子群算法优化神经网络的网络结构和连接权，获得神经网络集成个体；进一步用二次规划方法，计算各集成个体的最优非负权系数进行组合集成，生成神经网络集成的输出结论，进行短期降水预报建模研究．以广西全区的月降水量实例分析，结果表明该方法能有效提高系统的泛化能力．     

基于模糊进化规划和分层方法的神经网络设计方法

本文提出一种模糊进化规划，用于前向神经网络的设计．该方法通过对神经元的部分解群体的进化，缩短了个体的编码长度，显著地减轻了计算量，同时这种方法不但能够在很大程度上简化适应值的计算，更重要的是能够降低适应值空间的复杂性，从而能够加速进化算法收敛到全局最优点．仿真结果显示，本文提出的算法能够有效抑制进化规划算法初期收敛的发生，有效地提高多层前向神经网络收敛精度，并可获得更为简洁的网络结构．     

一种基于粒子群优化算法和差分进化算法的新型混合全局优化算法

提出一种基于粒子群算法(PSO)和差分进化算法(DE)相结合的新型混合全局优化算法——PSODE．该算法基于一种双种群进化策略，一个种群中的个体由粒子群算法进化而来，另一种群的个体由差分操作进化而来．此外，通过采用一种信息分享机制，在算法执行过程中两个种群中的个体可以实现协同进化．为了进一步提高PSODE算法的性能，摆脱陷入局部最优点，还采用了一种变异机制．通过4个标准测试函数的测试并与PSO和DE算法进行比较，证明本文提出的PSODE算法是一种收敛速度快、求解精度高、鲁棒性较强的全局优化算法．     

交互式遗传算法基于NN的个体适应度分阶段估计

针对交互式遗传算法中人的疲劳问题,提出一种基于神经网络(NN)的个体适应度分阶段估计方法,给出了神经网络估计进化个体适应度与人的评价之问的转换策略以及神经网络学习效果的评价指标,并分析了算法的复杂性．实例结果验证了该方法的有效性。     

基于多种群协同进化微粒群算法的径向基神经网络设计

神经网络结构和权值的联合设计一直是神经网络进化设计的一个研究方向.本文根据基本微粒群算法的特点,借鉴递阶编码的思想,构造出一种多种群协同进化微粒群算法.该算法具有种群内个体微粒自由运动特征分量与种群运动特征分量分层递阶进化的特征,克服了标准微粒群算法在多峰函数寻优时出现的微粒“早熟”现象.应用该算法进行径向基神经网络隐层结构和径向基函数参数联合自适应设计,在非线性系统辨识中显示了比较好的收敛性和训练精度,同时也使网络的泛化能力和逼近精度这一对矛盾得到了比较好的协调统一.     

基于小生境技术的神经网络进化集成

针对目前神经网络集成方法中生成个体网络差异度小、集成泛化能力较差等缺点,提出一种基于小生境技术的神经网络进化集成方法。利用小生境技术在增加进化群体的多样性、提高进化局部搜索能力方面的良好性能,通过个体间相似程度的共享函数来调整神经网络集成中个体网络的适应度,再依据调整后的新适应度进行选择,以维护群体的多样性,得到多样性的个体网络。理论分析和实验结果表明,该方法能有效生成差异度较大的个体网络,提高神经网络集成系统的泛化能力与计算精度。     

用遗传算法优化神经网络结构

本文介绍了一种用遗传算法对神经网络和连接权值同时优化的方法，该神经网络的神经元节点可以一定程度地反馈连接，再通过基因链矩阵把神经网络的结构完整地表示，遗传进化学习后，最优个体是结构最优的神经网络，使用该方法可以设计出结构未知的神经网络，本文最后对ＸＯＲ问题进行了计算。     

TSK模糊模型的协同进化学习方法

针对TSK模糊模型的学习是多约束和多目标优化问题,提出TSK模糊模型分解为两类不同的种群,协作共同进化的模型学习方法．论述了所涉及的相关问题,包括各种群的编码及其不同的进化计算,各种群个体的合作及其适应值评估策略,模型的后件参数估计方法．该方法要求先验知识少,收敛速度快,能形成简洁的模糊模型,最后以函数近似为例说明了该方法的有效性．     

基于统计分析的分阶段进化神经网络方法

基于统计分析和分阶段进化，提出一种新的进化神经网络设计方法．本文方法 的进化过程分三个阶段：第一阶段，首先按训练样本统计特性设计较小规模的神经网络；第 二阶段，引入所有训练样本，在第一阶段的基础上，逐步扩展网络结构，新添加的神经元总 是单独训练并以抵消原网络的输出误差为其训练目标，直至训练网络达到误差要求．第三阶 段，利用统计方法，将网络中非线性变换作用相似的神经元合并，简化网络结构．本文方法 一方面减轻了进化算法的压力,另一方面指出了网络进化的方向使得进化网络的学习过程不 再是黑箱问题．计算机仿真实验表明，该方法是有效的．     

基于粒子群和投影寻踪的样本重构神经网络集成模型在降水预报中的应用

首先利用粒子群算法和投影寻踪技术构造神经网络的学习矩阵,基于负相关学习的样本重构方法生成神经网络集成个体,进一步用粒子群算法和投影寻踪回归方法对集成个体集成,生成神经网络集成的输出结论,建立基于粒子群算法-投影寻踪的样本重构神经网络集成模型。该方法应用于广西全区的月降水量预报,结果表明该方法在降水预报中能有效从众多天气因子中构造神经网络的学习矩阵,而且集成学习预测精度高、稳定性好,具有一定的推广能力。     

基于聚类分析的综合神经网络集成算法

研究神经网络集成是一种有效实用的分类方法,权值是影响神经网络集成性能的重要因素。为了克服神经网络集成固定权值的缺陷,提出一种基于聚类分析的综合神经网络集成算法。算法首先将样本分类,每类样本中加入其他样本类一定数量的中心样本,不同的神经网络学习不同类的样本。根据输入数据与样本类别之间的相关程度自适应调整集成权值。算法不仅用于自适应调整集成权值,而且是一种产生个体神经网络的训练方法。四个数据集上的仿真试验证实了算法的有效性。     

Bagging and boosting negatively correlated neural networks.

In this paper, we propose two cooperative ensemble learning algorithms, i.e., NegBagg and NegBoost, for designing neural network (NN) ensembles. The proposed algorithms incrementally train different individual NNs in an ensemble using the negative correlation learning algorithm. Bagging and boosting algorithms are used in NegBagg and NegBoost, respectively, to create different training sets for different NNs in the ensemble. The idea behind using negative correlation learning in conjunction with the bagging/boosting algorithm is to facilitate interaction and cooperation among NNs during their training. Both NegBagg and NegBoost use a constructive approach to automatically determine the number of hidden neurons for NNs. NegBoost also uses the constructive approach to automatically determine the number of NNs for the ensemble. The two algorithms have been tested on a number of benchmark problems in machine learning and NNs, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, satellite, soybean, and waveform problems. The experimental results show that NegBagg and NegBoost require a small number of training epochs to produce compact NN ensembles with good generalization.     

一种动态性神经网络的集成方法

提出一种动态性神经网络集成方法，该方法针对若干不同的神经网络，采用加权最近邻技术收集它们的泛化误差信息构成性能矩阵，在此基础上动态选择泛化误差较小的神经网络，经过动态平均形成集成的最终输出结果。实验表明，与其它方法相比，该方法具有令人满意的性能。     

A constructive algorithm for training cooperative neural network ensembles

Presents a constructive algorithm for training cooperative neural-network ensembles (CNNEs). CNNE combines ensemble architecture design with cooperative training for individual neural networks (NNs) in ensembles. Unlike most previous studies on training ensembles, CNNE puts emphasis on both accuracy and diversity among individual NNs in an ensemble. In order to maintain accuracy among individual NNs, the number of hidden nodes in individual NNs are also determined by a constructive approach. Incremental training based on negative correlation is used in CNNE to train individual NNs for different numbers of training epochs. The use of negative correlation learning and different training epochs for training individual NNs reflect CNNEs emphasis on diversity among individual NNs in an ensemble. CNNE has been tested extensively on a number of benchmark problems in machine learning and neural networks, including Australian credit card assessment, breast cancer, diabetes, glass, heart disease, letter recognition, soybean, and Mackey-Glass time series prediction problems. The experimental results show that CNNE can produce NN ensembles with good generalization ability.     

基于混沌PSO算法的选择性神经网络集成方法

提出基于十进制粒子群优化算法（DePSO）和二进制PSO算法（BiPSO）的选择性神经网络集成(NNE)方法,通过PSO算法合理选择组成神经网络集成的各个神经网络,使个体间保持较大的差异度,减小"多维共线性"和样本噪声的影响。为有效保证PSO算法的粒子多样性,在迭代过程中加入混沌变异。试验表明,混沌PSO算法是组合优化权值的有效方法,同已有方法比较可以有效提高神经网络集成的泛化能力。     

Evolutionary ensembles with negative correlation learning

Based on negative correlation learning and evolutionary learning, this paper presents evolutionary ensembles with negative correlation learning (EENCL) to address the issues of automatic determination of the number of individual neural networks (NNs) in an ensemble and the exploitation of the interaction between individual NN design and combination. The idea of EENCL is to encourage different individual NNs in the ensemble to learn different parts or aspects of the training data so that the ensemble can learn better the entire training data. The cooperation and specialization among different individual NNs are considered during the individual NN design. This provides an opportunity for different NNs to interact with each other and to specialize. Experiments on two real-world problems demonstrate that EENCL can produce NN ensembles with good generalization ability.     

Improving combination method of NCL experts using gating network

Negative Correlation Learning (NCL) is a popular combining method that employs special error function for the simultaneous training of base neural network (NN) experts. In this article, we propose an improved version of NCL method in which the capability of gating network, as the combining part of Mixture of Experts method, is used to combine the base NNs in the NCL ensemble method. The special error function of the NCL method encourages each NN expert to learn different parts or aspects of the training data. Thus, the local competence of the experts should be considered in the combining approach. The gating network provides a way to support this needed functionality for combining the NCL experts. So the proposed method is called Gated NCL. The improved ensemble method is compared with the previous approaches were used for combining NCL experts, including winner-take-all (WTA) and average (AVG) combining techniques, in solving several classification problems from UCI machine learning repository. The experimental results show that our proposed ensemble method significantly improved performance over the previous combining approaches.     

Combining features of negative correlation learning with mixture of experts in proposed ensemble methods

Both theoretical and experimental studies have shown that combining accurate neural networks (NNs) in the ensemble with negative error correlation greatly improves their generalization abilities. Negative correlation learning (NCL) and mixture of experts (ME), two popular combining methods, each employ different special error functions for the simultaneous training of NNs to produce negatively correlated NNs. In this paper, we review the properties of the NCL and ME methods, discussing their advantages and disadvantages. Characterization of both methods showed that they have different but complementary features, so if a hybrid system can be designed to include features of both NCL and ME, it may be better than each of its basis approaches. In this study, two approaches are proposed to combine the features of both methods in order to solve the weaknesses of one method with the strength of the other method, i.e., gated-NCL (G-NCL) and mixture of negatively correlated experts (MNCE). In the first approach, G-NCL, a dynamic combiner of ME is used to combine the outputs of base experts in the NCL method. The suggested combiner method provides an efficient tool to evaluate and combine the NCL experts by the weights estimated dynamically from the inputs based on the different competences of each expert regarding different parts of the problem. In the second approach, MNCE, the capability of a control parameter for NCL is incorporated in the error function of ME, which enables the training algorithm of ME to efficiently adjust the measure of negative correlation between the experts. This control parameter can be regarded as a regularization term added to the error function of ME to establish better balance in bias–variance–covariance trade-offs and thus improves the generalization ability. The two proposed hybrid ensemble methods, G-NCL and MNCE, are compared with their constituent methods, ME and NCL, in solving several benchmark problems. The experimental results show that our proposed methods preserve the advantages and alleviate the disadvantages of their basis approaches, offering significantly improved performance over the original methods.     

Identifying source(s) of out-of-control signals in multivariate manufacturing processes using selective neural network ensemble

In multivariate statistical process control (MSPC), most multivariate quality control charts are shown to be effective in detecting out-of-control signals based upon an overall statistic. But these charts do not relieve the need for pinpointing source(s) of the out-of-control signals. Neural networks (NNs) have excellent noise tolerance and high pattern identification capability in real time, which have been applied successfully in MSPC. This study proposed a selective NN ensemble approach DPSOEN, where several selected NNs are jointly used to classify source(s) of out-of-control signals in multivariate processes. The immediate location of the abnormal source(s) can greatly narrow down the set of possible assignable causes, facilitating rapid analysis and corrective action by quality operators. The performance of DPSOEN is analyzed in multivariate processes. It shows improved generalization performance that outperforms those of single NNs and Ensemble All approach. The investigation proposed a heuristic approach for applying the DPSOEN-based model as an effective and useful tool to identify abnormal source(s) in bivariate statistical process control (SPC) with potential application for MSPC in general.     

A Hybrid Neurogenetic Approach for Stock Forecasting

In this paper, we propose a hybrid neurogenetic system for stock trading. A recurrent neural network (NN) having one hidden layer is used for the prediction model. The input features are generated from a number of technical indicators being used by financial experts. The genetic algorithm (GA) optimizes the NN's weights under a 2-D encoding and crossover. We devised a context-based ensemble method of NNs which dynamically changes on the basis of the test day's context. To reduce the time in processing mass data, we parallelized the GA on a Linux cluster system using message passing interface. We tested the proposed method with 36 companies in NYSE and NASDAQ for 13 years from 1992 to 2004. The neurogenetic hybrid showed notable improvement on the average over the buy-and-hold strategy and the context-based ensemble further improved the results. We also observed that some companies were more predictable than others, which implies that the proposed neurogenetic hybrid can be used for financial portfolio construction