基于改进遗传算法的深度神经网络优化研究

深度前馈神经网络在分类和回归问题上得到了很好的应用,但网络性能极大程度上受到其结构和超参数影响.为了获得高性能的神经网络,首先对遗传算法的选择策略进行改进,之后利用该改进遗传算法,采用二进制编码与实数编码的混合编码策略对深度前馈神经网络层数、每层节点量以及学习率和权重进行优化.改进的选择策略,在最优保存策略的基础上从父代和子代合并的2n个个体中,以一定的概率选择部分适应值较差个体作为新父代,以增加种群多样性,避免陷入局部最优.同时引入dropout方法减少网络过拟合训练数据.使用Ring、Breast cancer、Twonorm、Heart、Blood、Ionosphere、Monk共7个数据集进行数值实验,并与其他相关文献中的算法比较,仿真结果表明,改进的遗传算法能搜索到较高性能的神经网络.

Optimizing deep neural networks using a modified genetic algorithm

Deep feed-forward neural networks are well applied in classification and regression problems, but network performance is greatly affected by their structure and hyper-parameters. To achieve high performance neural networks, a modified genetic algorithm is designed firstly, which modifies the selection strategy. Then, the modified genetic algorithm is employed to optimize the number of network layers, the number of nodes in each layer, and the learning rate and weights, which are coded by binary coding and real number coding strategy respectively. For the modified selection strategy, in 2n indivi- duals from the combination of parent population with offspring population, some top fitness individuals are selected and some worse fitness individuals with a high probability are also selected to achieve better diversity and avoid falling into local optimum. dropout method is introduced to avoid the overfitting training data of network. Seven datasets (Ring, Breast cancer，Twonorm, Heart，Blood，Ionosphere，Monk) are used in the experiments. The results show that, compared with the algorithms in related literatures, the modified genetic algorithm has higher performance neural networks.