贝叶斯优化算法的发展综述

介绍了贝叶斯优化算法,并针对不同的优化问题,结合经典优化方法提出的层次BOA算法、多目标层次BOA算法以及递进BOA算法,对贝叶斯优化算法的算法设计、理论分析和应用研究做了全面的总结.深入地探讨了贝叶斯优化算法计算量大,难以建立精确概率模型及扩展应用领域等问题.     

基于改进贝叶斯优化算法的CNN超参数优化方法

CNN框架中,如何对其模型的超参数进行自动化获取一直是一个重要问题。提出一种基于改进的贝叶斯优化算法的CNN超参数优化方法。该方法使用改进的汤普森采样方法作为采集函数,利用改进的马尔可夫链蒙特卡罗算法加速训练高斯代理模型。该方法可以在超参数空间不同的CNN框架下进行超参数优化。利用CIFAR-10、MRBI和SVHN测试集对算法进行性能测试,实验结果表明,改进后的CNN超参数优化算法比同类超参数优化算法具有更好的性能。     

一种混合粒子群算法的贝叶斯网络优化模型

传统的粒子群优化算法通过群体中粒子间的合作和竞争进行群体智能指导优化搜索,算法收敛速度快,但较易陷入局部较优值,进入早熟状态。为了解决这个问题,提出了一种混合粒子群算法的贝叶斯网络优化模型,它可以通过当前所选择的较优解群构造一个贝叶斯网络和联合概率分布模型,利用这个模型进行采样得到更优解,用其可随机替换掉PSO中的一些粒子或个体最优解;同时利用粒子群算法对当前选择出的较优解群进行深度搜索,并将得到的最优解融入到较优解群中。分析可知,该方法可以提高算法有效性和可靠性。     

基于贝叶斯优化的无标签网络剪枝算法

针对深度神经网络（DNN）的参数和计算量过大问题,提出一种基于贝叶斯优化的无标签网络剪枝算法。首先,利用全局剪枝策略来有效避免以逐层方式修剪而导致的模型次优压缩率;其次,在网络剪枝过程中不依赖数据样本标签,并通过最小化剪枝网络与基线网络输出特征的距离对网络每层的压缩率进行优化;最后,利用贝叶斯优化算法寻找网络每一层的最优剪枝率,以提高子网搜索的效率和精度。实验结果表明,使用所提算法在CIFAR-10数据集上对VGG-16网络进行压缩,参数压缩率为85.32%,每秒浮点运算次数（FLOPS）压缩率为69.20%,而精度损失仅为0.43%。可见,所提算法可以有效地压缩DNN模型,且压缩后的模型仍能保持良好的精度。     

一种改进的基于遗传禁忌优化的NB算法*

针对传统朴素贝叶斯分类模型应用过程中存在的特征项冗余问题,使用遗传禁忌算法对特征项集进行优化,并在此优化结果的基础上,提出了一种改进的朴素贝叶斯分类方法来解决用户模板中存在的单类别词汇问题。经实验证明,该方法比传统的朴素贝叶斯分类模型具有更好的鲁棒性和分类性能。     

贝叶斯优化算法的选择策略分析

针对贝叶斯优化算法的选择策略问题,对变量无关,双变量相关,多变量相关等3类典型函数分别用锦标赛选择、截断选择和比例选择以及自适应比例选择进行了实验。建立了相应的贝叶斯网络概率模型,并分析指出锦标赛选择策略能有效保持样本的多样性,并能建立起准确的网络模型。与比例选择策略和截断选择策略相比较,该选择策略更适用于贝叶斯优化算法。     

数据缺失条件下的贝叶斯优化算法

针对朴素贝叶斯算法存在的三方面约束和限制,提出一种数据缺失条件下的贝叶斯优化算法。该算法计算任两个属性的灰色相关度,根据灰色相关度完成相关属性的联合、冗余属性的删除和属性加权;根据灰色相关度执行改进EM算法完成缺失数据的填补,对经过处理的数据集用朴素贝叶斯算法进行分类。实验结果验证了该优化算法的有效性。     

一种故障诊断的贝叶斯优化算法研究*

提出一种基于改进贝叶斯优化算法的故障模式聚类算法,通过结合贝叶斯优化算法中的先验知识来提高算法的可靠性和全局收敛性。将改进的优化算法应用到高维数据最优统计聚类分析中,可快速优化聚类参数,得到全局最优解。以飞行控制系统操纵面的故障诊断为例进行仿真验证,结果表明该算法结构简单、故障识别可靠。     

基于GA-BP算法的超声波测量精度优化研究

针对复杂环境下超声波传感器测量系统测量精度问题,以罐体油位测量为例,提出一种基于神经网络遗传算法的超声波传感器测量精度优化模型,实现超声波油位测量系统的非线性误差校正。仿真结果表明,该方法可以减小超声波传感器本身结构和外界因素的干扰,提高测量精度。     

A new real-coded stochastic Bayesian optimization algorithm for continuous global optimization

Estimation of distribution algorithms are considered to be a new class of evolutionary algorithms which are applied as an alternative to genetic algorithms. Such algorithms sample the new generation from a probabilistic model of promising solutions. The search space of the optimization problem is improved by such probabilistic models. In the Bayesian optimization algorithm (BOA), the set of promising solutions forms a Bayesian network and the new solutions are sampled from the built Bayesian network. This paper proposes a novel real-coded stochastic BOA for continuous global optimization by utilizing a stochastic Bayesian network. In the proposed algorithm, the new Bayesian network takes advantage of using a stochastic structure (that there is a probability distribution function for each edge in the network) and the new generation is sampled from the stochastic structure. In order to generate a new solution, some new structure, and therefore a new Bayesian network is sampled from the current stochastic structure and the new solution will be produced from the sampled Bayesian network. Due to the stochastic structure used in the sampling phase, each sample can be generated based on a different structure. Therefore the different dependency structures can be preserved. Before the new generation is generated, the stochastic network’s probability distributions are updated according to the fitness evaluation of the current generation. The proposed method is able to take advantage of using different dependency structures through the sampling phase just by using one stochastic structure. The experimental results reported in this paper show that the proposed algorithm increases the quality of the solutions on the general optimization benchmark problems.     

Robust optimization using Bayesian optimization algorithm: Early detection of non-robust solutions

Probabilistic robustness evaluation is a promising approach to evolutionary robust optimization; however, high computational time arises. In this paper, we apply this approach to the Bayesian optimization algorithm (BOA) with a view to improving its computational time. To this end, we analyze the Bayesian networks constructed in BOA in order to extract the patterns of non-robust solutions. In each generation, the solutions that match the extracted patterns are detected and then discarded from the process of evaluation; therefore, the computational time in discovering the robust solutions decreases. The experimental results demonstrate that our proposed method reduces computational time, while increasing the robustness of solutions.     

A randomized stochastic optimization algorithm: Its estimation accuracy

For a randomized stochastic optimization algorithm, consistency conditions of estimates are slackened and the order of accuracy for a finite number of observations is studied. A new method of realization of this algorithm on quantum computers is developed.     

Holographic memory-based Bayesian optimization algorithm (HM-BOA) in dynamic environments

This paper presents a new evolutionary dynamic optimization algorithm, holographic memory-based Bayesian optimization algorithm (HM-BOA), whose objective is to address the weaknesses of sequential memory-based dynamic optimization approaches. To this end, holographic associative neural memory is applied to one of the recent successful memory-based evolutionary methods, DBN-MBOA (memory-based BOA with dynamic Bayesian networks). Holographic memory is appropriate for encoding environmental changes since its stimulus and response data are represented by a vector of complex numbers such that the phase and the magnitude denote the information and its confidence level, respectively. In the learning process in HM-BOA, holographic memory is trained by probabilistic models at every environmental change. Its weight matrix contains abstract information obtained from previous changes and is used for constructing a new probabilistic model when the environment changes. The unique features of HM-BOA are: 1) the stored information can be generalized, and 2) a small amount of memory is required for storing the probabilistic models. Experimental results adduce grounds for its effectiveness especially in random environments.     

面向粒子群优化的贝叶斯网络结构学习算法

提出了一种基于离散粒子群优化的贝叶斯网络结构学习算法——PSBN（Particle Swarm for Bayesian Network）。贝叶斯网络的结构被映射为一种符号编码,通过在迭代过程中对粒子的符号编码进行调整,从而进化得到具有更高适应度值的贝叶斯网络结构。根据贝叶斯网络的结构特点,粒子位置和速度的编码方案和基本操作被设计,使得算法对贝叶斯网络的结构学习有较好的收敛性。实验结果表明,与基于遗传算法的贝叶斯网络结构学习算法相比,PSBN算法具有较好的学习效果。     

Optimal steganography with blind detection based on Bayesian optimization algorithm

Pattern Analysis and Applications - An optimal steganography method is provided to embed the secret data into the low-order bits of host pixels. The main idea of the proposed method is that before...     

基于决策图贝叶斯的盲源信号分离算法

从混合观测数据向量中恢复不可观测的各个源信号是阵列处理和数据分析的一个典型问题。提出了一种基于决策图贝叶斯的盲源信号分离算法,该算法利用决策图贝叶斯优化算法代替JADE算法中的联合对角化操作,通过构造和学习网络来替代传统遗传算法中的交叉重组和变异等遗传算子,避免了对大量控制参数和遗传算子的人工选择和重要构造块的破坏。仿真结果表明,提出的算法比JADE算法和基于遗传算法的盲源信号分离方法均具有更高的分离精度。     

基于无约束优化和遗传算法的贝叶斯网络结构学习方法

基于无约束优化和遗传算法,提出一种学习贝叶斯网络结构的限制型遗传算法.首先构造一无约束优化问题,其最优解对应一个无向图.在无向图的基础上,产生遗传算法的初始种群,并使用遗传算法中的选择、交叉和变异算子学习得到最优贝叶斯网络结构.由于产生初始种群的空间是由一些最优贝叶斯网络结构的候选边构成,初始种群具有很好的性质.与直接使用遗传算法学习贝叶斯网络结构的效率相比,该方法的学习效率相对较高.     

A new real-coded Bayesian optimization algorithm based on a team of learning automata for continuous optimization

Estimation of distribution algorithms have evolved as a technique for estimating population distribution in evolutionary algorithms. They estimate the distribution of the candidate solutions and then sample the next generation from the estimated distribution. Bayesian optimization algorithm is an estimation of distribution algorithm, which uses a Bayesian network to estimate the distribution of candidate solutions and then generates the next generation by sampling from the constructed network. The experimental results show that the Bayesian optimization algorithms are capable of identifying correct linkage between the variables of optimization problems. Since the problem of finding the optimal Bayesian network belongs to the class of NP-hard problems, typically Bayesian optimization algorithms use greedy algorithms to build the Bayesian network. This paper proposes a new real-coded Bayesian optimization algorithm for solving continuous optimization problems that uses a team of learning automata to build the Bayesian network. This team of learning automata tries to learn the optimal Bayesian network structure during the execution of the algorithm. The use of learning automaton leads to an algorithm with lower computation time for building the Bayesian network. The experimental results reported here show the preference of the proposed algorithm on both uni-modal and multi-modal optimization problems.     

基于贝叶斯权函数的模型无关元学习算法

模型无关的元学习(MAML)是一种多任务的元学习算法,能使用不同的模型,并快速地在不同任务之间进行适应,但MAML在训练速度与准确率上还亟待提高.从高斯随机过程的角度出发对MAML的原理进行分析,提出一种基于贝叶斯权函数的模型无关元学习(BW-MAML)算法,该权函数利用贝叶斯分析设计并用于损失的加权.训练过程中,BW...     

遗传算法调整蚁群算法参数模型研究

由于蚁群算法参数取值范围的不确定性和参数之间的相互影响性,如何确定待解决问题蚁群算法的最优组合参数使得其求解性能最优成为一个难题,至今对该问题还没有完善的理论依据,大多数情况下是通过反复试验试凑得到。根据这些问题,通过平衡蚁群算法探索和开发能力,建立算法性能评价目标函数,采用遗传算法对蚁群参数进行求解,从而得到一组性能较佳的组合参数。基于经典TSP问题进行试验模拟,仿真实验结果表明,该模型能够有效地确定蚁群算法参数,为蚁群算法组合参数的选择提供了一种可行方案。