Exploiting Bivariate Dependencies to Speedup Structure Learning in Bayesian Optimization Algorithm

Bayesian optimization algorithm (BOA) is one of the successful and widely used estimation of distribution algorithms (EDAs) which have been employed to solve different optimization problems. In EDAs, a model is learned from the selected population that encodes interactions among problem variables. New individuals are generated by sampling the model and incorporated into the population. Different probabilistic models have been used in EDAs to learn interactions. Bayesian network (BN) is a well-known graphical model which is used in BOA. Learning a proper model in EDAs and particularly in BOA is distinguished as a computationally expensive task. Different methods have been proposed in the literature to improve the complexity of model building in EDAs. This paper employs bivariate dependencies to learn accurate BNs in BOA efficiently. The proposed approach extracts the bivariate dependencies using an appropriate pairwise interaction-detection metric. Due to the static structure of the underlying problems, these dependencies are used in each generation of BOA to learn an accurate network. By using this approach, the computational cost of model building is reduced dramatically. Various optimization problems are selected to be solved by the algorithm. The experimental results show that the proposed approach successfully finds the optimum in problems with different types of interactions efficiently. Significant speedups are observed in the model building procedure as well.     

Using previous models to bias structural learning in the hierarchical BOA

Estimation of distribution algorithms (EDAs) are stochastic optimization techniques that explore the space of potential solutions by building and sampling explicit probabilistic models of promising candidate solutions. While the primary goal of applying EDAs is to discover the global optimum or at least its accurate approximation, besides this, any EDA provides us with a sequence of probabilistic models, which in most cases hold a great deal of information about the problem. Although using problem-specific knowledge has been shown to significantly improve performance of EDAs and other evolutionary algorithms, this readily available source of problem-specific information has been practically ignored by the EDA community. This paper takes the first step toward the use of probabilistic models obtained by EDAs to speed up the solution of similar problems in the future. More specifically, we propose two approaches to biasing model building in the hierarchical Bayesian optimization algorithm (hBOA) based on knowledge automatically learned from previous hBOA runs on similar problems. We show that the proposed methods lead to substantial speedups and argue that the methods should work well in other applications that require solving a large number of problems with similar structure.     

Model accuracy in the Bayesian optimization algorithm

Evolutionary algorithms (EAs) are particularly suited to solve problems for which there is not much information available. From this standpoint, estimation of distribution algorithms (EDAs), which guide the search by using probabilistic models of the population, have brought a new view to evolutionary computation. While solving a given problem with an EDA, the user has access to a set of models that reveal probabilistic dependencies between variables, an important source of information about the problem. However, as the complexity of the used models increases, the chance of overfitting and consequently reducing model interpretability, increases as well. This paper investigates the relationship between the probabilistic models learned by the Bayesian optimization algorithm (BOA) and the underlying problem structure. The purpose of the paper is threefold. First, model building in BOA is analyzed to understand how the problem structure is learned. Second, it is shown how the selection operator can lead to model overfitting in Bayesian EDAs. Third, the scoring metric that guides the search for an adequate model structure is modified to take into account the non-uniform distribution of the mating pool generated by tournament selection. Overall, this paper makes a contribution towards understanding and improving model accuracy in BOA, providing more interpretable models to assist efficiency enhancement techniques and human researchers.     

A new real-coded stochastic Bayesian optimization algorithm for continuous global optimization

Estimation of distribution algorithms are considered to be a new class of evolutionary algorithms which are applied as an alternative to genetic algorithms. Such algorithms sample the new generation from a probabilistic model of promising solutions. The search space of the optimization problem is improved by such probabilistic models. In the Bayesian optimization algorithm (BOA), the set of promising solutions forms a Bayesian network and the new solutions are sampled from the built Bayesian network. This paper proposes a novel real-coded stochastic BOA for continuous global optimization by utilizing a stochastic Bayesian network. In the proposed algorithm, the new Bayesian network takes advantage of using a stochastic structure (that there is a probability distribution function for each edge in the network) and the new generation is sampled from the stochastic structure. In order to generate a new solution, some new structure, and therefore a new Bayesian network is sampled from the current stochastic structure and the new solution will be produced from the sampled Bayesian network. Due to the stochastic structure used in the sampling phase, each sample can be generated based on a different structure. Therefore the different dependency structures can be preserved. Before the new generation is generated, the stochastic network’s probability distributions are updated according to the fitness evaluation of the current generation. The proposed method is able to take advantage of using different dependency structures through the sampling phase just by using one stochastic structure. The experimental results reported in this paper show that the proposed algorithm increases the quality of the solutions on the general optimization benchmark problems.     

Introducing assignment functions to Bayesian optimization algorithms

In this paper, we improve Bayesian optimization algorithms by introducing proportionate and rank-based assignment functions. A Bayesian optimization algorithm builds a Bayesian network from a selected sub-population of promising solutions, and this probabilistic model is employed to generate the offspring of the next generation. Our method assigns each solution a relative significance based on its fitness, and this information is used in building the Bayesian network model. These assignment functions can improve the quality of the model without performing an explicit selection on the population. Numerical experiments demonstrate the effectiveness of this method compared to a conventional BOA.     

Continuous probabilistic model building genetic network programming using reinforcement learning

Recently, a novel probabilistic model-building evolutionary algorithm (so called estimation of distribution algorithm, or EDA), named probabilistic model building genetic network programming (PMBGNP), has been proposed. PMBGNP uses graph structures for its individual representation, which shows higher expression ability than the classical EDAs. Hence, it extends EDAs to solve a range of problems, such as data mining and agent control. This paper is dedicated to propose a continuous version of PMBGNP for continuous optimization in agent control problems. Different from the other continuous EDAs, the proposed algorithm evolves the continuous variables by reinforcement learning (RL). We compare the performance with several state-of-the-art algorithms on a real mobile robot control problem. The results show that the proposed algorithm outperforms the others with statistically significant differences.     

Effective Linkage Learning Using Low-Order Statistics and Clustering

The adoption of probabilistic models for selected individuals is a powerful approach for evolutionary computation. Probabilistic models based on high-order statistics have been used by estimation of distribution algorithms (EDAs), resulting better effectiveness when searching for global optima for hard optimization problems. This paper proposes a new framework for evolutionary algorithms, which combines a simple EDA based on order 1 statistics and a clustering technique in order to avoid the high computational cost required by higher order EDAs. The algorithm uses clustering to group genotypically similar solutions, relying that different clusters focus on different substructures and the combination of information from different clusters effectively combines substructures. The combination mechanism uses an information gain measure when deciding which cluster is more informative for any given gene position, during a pairwise cluster combination. Empirical evaluations effectively cover a comprehensive range of benchmark optimization problems.      

On the Scalability of Real-Coded Bayesian Optimization Algorithm

Estimation of distribution algorithms (EDAs) are major tools in evolutionary optimization. They have the ability to uncover the hidden regularities of problems and then exploit them for effective search. Real-coded Bayesian optimization algorithm (rBOA) which brings the power of discrete BOA to bear upon the continuous domain has been regarded as a milestone in the field of numerical optimization. It has been empirically observed that the rBOA solves, with subquadratic scaleup behavior, numerical optimization problems of bounded difficulty. This underlines the scalability of rBOA (at least) in practice. However, there is no firm theoretical basis for this scalability. The aim of this paper is to carry out a theoretical analysis of the scalability of rBOA in the context of additively decomposable problems with real-valued variables. The scalability is measured by the growth of the number of fitness function evaluations (in order to reach the optimum) with the size of the problem. The total number of evaluations is computed by multiplying the population size for learning a correct probabilistic model (i.e., population complexity) and the number of generations before convergence, (i.e., convergence time complexity). Experimental results support the scalability model of rBOA. The rBOA shows a subquadratic (in problem size) scalability for uniformly scaled decomposable problems.     

A new real-coded Bayesian optimization algorithm based on a team of learning automata for continuous optimization

Estimation of distribution algorithms have evolved as a technique for estimating population distribution in evolutionary algorithms. They estimate the distribution of the candidate solutions and then sample the next generation from the estimated distribution. Bayesian optimization algorithm is an estimation of distribution algorithm, which uses a Bayesian network to estimate the distribution of candidate solutions and then generates the next generation by sampling from the constructed network. The experimental results show that the Bayesian optimization algorithms are capable of identifying correct linkage between the variables of optimization problems. Since the problem of finding the optimal Bayesian network belongs to the class of NP-hard problems, typically Bayesian optimization algorithms use greedy algorithms to build the Bayesian network. This paper proposes a new real-coded Bayesian optimization algorithm for solving continuous optimization problems that uses a team of learning automata to build the Bayesian network. This team of learning automata tries to learn the optimal Bayesian network structure during the execution of the algorithm. The use of learning automaton leads to an algorithm with lower computation time for building the Bayesian network. The experimental results reported here show the preference of the proposed algorithm on both uni-modal and multi-modal optimization problems.     

Protein Folding in Simplified Models With Estimation of Distribution Algorithms

Simplified lattice models have played an important role in protein structure prediction and protein folding problems. These models can be useful for an initial approximation of the protein structure, and for the investigation of the dynamics that govern the protein folding process. Estimation of distribution algorithms (EDAs) are efficient evolutionary algorithms that can learn and exploit the search space regularities in the form of probabilistic dependencies. This paper introduces the application of different variants of EDAs to the solution of the protein structure prediction problem in simplified models, and proposes their use as a simulation tool for the analysis of the protein folding process. We develop new ideas for the application of EDAs to the bidimensional and tridimensional (2-d and 3-d) simplified protein folding problems. This paper analyzes the rationale behind the application of EDAs to these problems, and elucidates the relationship between our proposal and other population-based approaches proposed for the protein folding problem. We argue that EDAs are an efficient alternative for many instances of the protein structure prediction problem and are indeed appropriate for a theoretical analysis of search procedures in lattice models. All the algorithms introduced are tested on a set of difficult 2-d and 3-d instances from lattice models. Some of the results obtained with EDAs are superior to the ones obtained with other well-known population-based optimization algorithms.     

Space complexity of estimation of distribution algorithms

In this paper, we investigate the space complexity of the Estimation of Distribution Algorithms (EDAs), a class of sampling-based variants of the genetic algorithm. By analyzing the nature of EDAs, we identify criteria that characterize the space complexity of two typical implementation schemes of EDAs, the factorized distribution algorithm and Bayesian network-based algorithms. Using random additive functions as the prototype, we prove that the space complexity of the factorized distribution algorithm and Bayesian network-based algorithms is exponential in the problem size even if the optimization problem has a very sparse interaction structure.     

Linkage problem, distribution estimation, and Bayesian networks

This paper proposes an algorithm that uses an estimation of the joint distribution of promising solutions in order to generate new candidate solutions. The algorithm is settled into the context of genetic and evolutionary computation and the algorithms based on the estimation of distributions. The proposed algorithm is called the Bayesian Optimization Algorithm (BOA). To estimate the distribution of promising solutions, the techniques for modeling multivariate data by Bayesian networks are used. The BOA identifies, reproduces, and mixes building blocks up to a specified order. It is independent of the ordering of the variables in strings representing the solutions. Moreover, prior information about the problem can be incorporated into the algorithm, but it is not essential. First experiments were done with additively decomposable problems with both nonoverlapping as well as overlapping building blocks. The proposed algorithm is able to solve all but one of the tested problems in linear or close to linear time with respect to the problem size. Except for the maximal order of interactions to be covered, the algorithm does not use any prior knowledge about the problem. The BOA represents a step toward alleviating the problem of identifying and mixing building blocks correctly to obtain good solutions for problems with very limited domain information.     

Optimization based on phylogram analysis

Estimation of Distribution Algorithms (EDAs) is evolutionary algorithms with relevant performance in handling complex problems. Nevertheless, their efficiency and effectiveness directly depends on how accurate the deployed probabilistic models are, which in turn depend on methods of model building. Although the best models found in the literature are often built by computationally complex methods, whose corresponding EDAs require high running time, these methods may evaluate a lesser number of points in the search space. In order to find a better trade-off between running time (efficiency) and the number of evaluated points (effectiveness), this work uses probabilistic models built by algorithms of phylogenetic reconstruction, since some of them are able to efficiently produce accurate models. Then, an EDA, namely, Optimization based on Phylogram Analysis, and a new search technique, namely, Composed Exhaustive Search, are developed and proposed to find solutions for combinatorial optimization problems with different levels of difficulty. Experimental results show that the proposed new EDA features an interesting trade-off between running time and number of evaluated points, attaining solutions near to the best results found in the literature for each one of such performance measures.     

Holographic memory-based Bayesian optimization algorithm (HM-BOA) in dynamic environments

This paper presents a new evolutionary dynamic optimization algorithm, holographic memory-based Bayesian optimization algorithm (HM-BOA), whose objective is to address the weaknesses of sequential memory-based dynamic optimization approaches. To this end, holographic associative neural memory is applied to one of the recent successful memory-based evolutionary methods, DBN-MBOA (memory-based BOA with dynamic Bayesian networks). Holographic memory is appropriate for encoding environmental changes since its stimulus and response data are represented by a vector of complex numbers such that the phase and the magnitude denote the information and its confidence level, respectively. In the learning process in HM-BOA, holographic memory is trained by probabilistic models at every environmental change. Its weight matrix contains abstract information obtained from previous changes and is used for constructing a new probabilistic model when the environment changes. The unique features of HM-BOA are: 1) the stored information can be generalized, and 2) a small amount of memory is required for storing the probabilistic models. Experimental results adduce grounds for its effectiveness especially in random environments.     

Automated global structure extraction for effective local building block processing in XCS

Learning Classifier Systems (LCSs), such as the accuracy-based XCS, evolve distributed problem solutions represented by a population of rules. During evolution, features are specialized, propagated, and recombined to provide increasingly accurate subsolutions. Recently, it was shown that, as in conventional genetic algorithms (GAs), some problems require efficient processing of subsets of features to find problem solutions efficiently. In such problems, standard variation operators of genetic and evolutionary algorithms used in LCSs suffer from potential disruption of groups of interacting features, resulting in poor performance. This paper introduces efficient crossover operators to XCS by incorporating techniques derived from competent GAs: the extended compact GA (ECGA) and the Bayesian optimization algorithm (BOA). Instead of simple crossover operators such as uniform crossover or one-point crossover, ECGA or BOA-derived mechanisms are used to build a probabilistic model of the global population and to generate offspring classifiers locally using the model. Several offspring generation variations are introduced and evaluated. The results show that it is possible to achieve performance similar to runs with an informed crossover operator that is specifically designed to yield ideal problem-dependent exploration, exploiting provided problem structure information. Thus, we create the first competent LCSs, XCS/ECGA and XCS/BOA, that detect dependency structures online and propagate corresponding lower-level dependency structures effectively without any information about these structures given in advance.     

Extended artificial chromosomes genetic algorithm for permutation flowshop scheduling problems

In our previous researches, we proposed the artificial chromosomes with genetic algorithm (ACGA) which combines the concept of the Estimation of Distribution Algorithms (EDAs) with genetic algorithms (GAs). The probabilistic model used in the ACGA is the univariate probabilistic model. We showed that ACGA is effective in solving the scheduling problems. In this paper, a new probabilistic model is proposed to capture the variable linkages together with the univariate probabilistic model where most EDAs could use only one statistic information. This proposed algorithm is named extended artificial chromosomes with genetic algorithm (eACGA). We investigate the usefulness of the probabilistic models and to compare eACGA with several famous permutation-oriented EDAs on the benchmark instances of the permutation flowshop scheduling problems (PFSPs). eACGA yields better solution quality for makespan criterion when we use the average error ratio metric as their performance measures. In addition, eACGA is further integrated with well-known heuristic algorithms, such as NEH and variable neighborhood search (VNS) and it is denoted as eACGA_hybrid to solve the considered problems. No matter the solution quality and the computation efficiency, the experimental results indicate that eACGA_hybrid outperforms other known algorithms in literature. As a result, the proposed algorithms are very competitive in solving the PFSPs.     

一种混沌贝叶斯优化算法

为了减少贝叶斯优化算法的计算量,该文提出了一种混沌贝叶斯优化算法。用混沌随机序列产生贝叶斯优化算法的初始群体,利用混沌随机性、遍历性和对初始条件的敏感性的特点,提供给贝叶斯网络变量空间丰富的信息,有利于建立接近最优的贝叶斯网络。为增加群体的多样性同时减少贝叶斯网络的建立次数,采用混沌搜索方法对贝叶斯网络产生的新解进行变异寻优,以此为基础再建立贝叶斯网络。实验结果表明,与贝叶斯优化算法相比,混沌贝叶斯优化算法能有效减少计算量。     

Graphical model based continuous estimation of distribution algorithm

In this paper, a new estimation of distribution algorithm is introduced. The goal is to propose a method that avoids complex approximations of learning a probabilistic graphical model and considers multivariate dependencies between continuous random variables. A parallel model of some subgraphs with a smaller number of variables is learned as the probabilistic graphical model. In each generation, the joint probability distribution of the selected solutions is estimated using a Gaussian Mixture model. Then, learning the graphical model of dependencies among random variables and sampling are done separately for each Gaussian component. In the learning step, using the selected solutions of each Gaussian mixture component, the structure of a Markov network is learned. This network is decomposed to maximal cliques and a clique graph. Then, complete Bayesian network structures are learned for these subgraphs using an optimization algorithm. The proposed optimization problem is a 0–1 constrained quadratic programming which finds the best permutation of variables. Then, sampling is done from each Bayesian network of each Gaussian component. The introduced method is compared with the other network-based estimation of distribution algorithms for optimization of continuous numerical functions.     

BAIS: A Bayesian Artificial Immune System for the effective handling of building blocks

Significant progress has been made in theory and design of Artificial Immune Systems (AISs) for solving hard problems accurately. However, an aspect not yet widely addressed by the research reported in the literature is the lack of ability of the AISs to deal effectively with building blocks (partial high-quality solutions coded in the antibody). The available AISs present mechanisms for evolving the population that do not take into account the relationship among the variables of the problem, potentially causing the disruption of high-quality partial solutions. This paper proposes a novel AIS with abilities to identify and properly manipulate building blocks in optimization problems. Instead of using cloning and mutation to generate new individuals, our algorithm builds a probabilistic model representing the joint probability distribution of the promising solutions and, subsequently, uses this model for sampling new solutions. The probabilistic model used is a Bayesian network due to its capability of properly capturing the most relevant interactions among the variables. Therefore, our algorithm, called Bayesian Artificial Immune System (BAIS), represents a significant attempt to improve the performance of immune-inspired algorithms when dealing with building blocks, and hence to solve efficiently hard optimization problems with complex interactions among the variables. The performance of BAIS compares favorably with that produced by contenders such as state-of-the-art Estimation of Distribution Algorithms.     

分布估计算法综述

分布估计算法是进化计算领域新兴起的一类随机优化算法,是当前国际进化计算领域的研究热点. 分布估计算法是遗传算法和统计学习的结合,通过统计学习的手段建立解空间内个体分布的概率模型,然后对概率模型随机采样产生新的群体,如此反复进行,实现群体的进化. 分布估计算法中没有传统的交叉、变异等遗传操作,是一种全新的进化模式;这种优化技术能够通过概率图模型对变量之间的关系进行建模,从而能有效的解决多变量相关的优化问题. 根据概率模型的复杂性,本文按照变量无关、双变量相关、多变量相关等三类分别介绍相应的分布估计算法. 作为一篇综述性文章,本文旨在全面系统的向国内读者介绍这一新技术,并总结分布估计算法的研究现状和未来的研究方向.