Lipschitz global optimization methods in control problems

Many control problems involve the search for the global extremum in the space of states or the parameters of the system under study, which leads to the necessity of using effective methods of global finite-dimensional optimization. For this purpose use can be made of the geometric algorithms of Lipschitz global optimization, which are developed by the authors. A brief review of these algorithms is presented and they are compared with some algorithms of global search that are often used in technical practice. Numerical experiments are performed on a few known examples of applied multiextremal problems.     

Combining Argumentation and Bayesian Nets for Breast Cancer Prognosis

We present a new framework for combining logic with probability, and demonstrate the application of this framework to breast cancer prognosis. Background knowledge concerning breast cancer prognosis is represented using logical arguments. This background knowledge and a database are used to build a Bayesian net that captures the probabilistic relationships amongst the variables. Causal hypotheses gleaned from the Bayesian net in turn generate new arguments. The Bayesian net can be queried to help decide when one argument attacks another. The Bayesian net is used to perform the prognosis, while the argumentation framework is used to provide a qualitative explanation of the prognosis.     

Multi-information source constrained Bayesian optimization

Structural and Multidisciplinary Optimization - Design decisions for complex systems often can be made or informed by a variety of information sources. When optimizing such a system, the evaluation...     

Bayesian forecaster using class-based optimization

Suppose that several forecasters exist for the problem in which class-wise accuracies of forecasting classifiers are important. For such a case, we propose to use a new Bayesian approach for deriving one unique forecaster out of the existing forecasters. Our Bayesian approach links the existing forecasting classifiers via class-based optimization by the aid of an evolutionary algorithm (EA). To show the usefulness of our Bayesian approach in practical situations, we have considered the case of the Korean stock market, where numerous lag-l forecasting classifiers exist for monitoring its status.     

贝叶斯优化算法的发展综述

介绍了贝叶斯优化算法,并针对不同的优化问题,结合经典优化方法提出的层次BOA算法、多目标层次BOA算法以及递进BOA算法,对贝叶斯优化算法的算法设计、理论分析和应用研究做了全面的总结.深入地探讨了贝叶斯优化算法计算量大,难以建立精确概率模型及扩展应用领域等问题.     

Stability analysis of particle swarm optimization without Lipschitz constraint

There are some adjustable parameters which directly influence the performance and stability of Particle Swarm Optimization algorithm. In this paper, stabilities of PSO with constant parameters and time-varying parameters are analyzed without Lipschitz constraint. Necessary and sufficient stability conditions for acceleration factor φ and inertia weight w are presented. Experiments on benchmark functions show the good performance of PSO satisfying the stability condition, even without Lipschitz constraint . And the inertia weight w value is enhanced to ( - 1,1).     

融合知识和数据的贝叶斯网络构造方法

从数据中学习贝叶斯网络往往会因为搜索空间庞大而耗费大量时间.由于贝叶斯网络固有的因果语义,领域专家往往能够凭借自己的经验确定节点之间的因果关系.本文方法充分收集专家的意见,并利用证据理论进行综合,去除无意义的网络结构,然后利用常用的学习算法从数据中继续学习.这种融合知识和数据的贝叶斯网络构造方法利用专家知识来缩小学习算法的搜索空间,避免了盲目搜索,同时也避免了单个专家知识的主观性.实验表明该方法能够有效提高学习效率.     

Combining Auditory Preprocessing and Bayesian Estimation for Robust Formant Tracking

We present a framework for estimating formant trajectories. Its focus is to achieve high robustness in noisy environments. Our approach combines a preprocessing based on functional principles of the human auditory system and a probabilistic tracking scheme. For enhancing the formant structure in spectrograms we use a Gammatone filterbank, a spectral preemphasis, as well as a spectral filtering using Difference-of-Gaussians (DoG) operators. Finally, a contrast enhancement mimicking a competition between filter responses is applied. The probabilistic tracking scheme adopts the mixture modeling technique for estimating the joint distribution of formants. In conjunction with an algorithm for adaptive frequency range segmentation as well as Bayesian smoothing an efficient framework for estimating formant trajectories is derived. Comprehensive evaluations of our method on the VTR–Formant database emphasize its high precision and robustness. We obtained superior performance compared to existing approaches for clean as well as echoic noisy speech. Finally, an implementation of the framework within the scope of an online system using instantaneous feature-based resynthesis demonstrates its applicability to real-world scenarios.      

Combining convergence and diversity in evolutionary multiobjective optimization

Over the past few years, the research on evolutionary algorithms has demonstrated their niche in solving multiobjective optimization problems, where the goal is to find a number of Pareto-optimal solutions in a single simulation run. Many studies have depicted different ways evolutionary algorithms can progress towards the Pareto-optimal set with a widely spread distribution of solutions. However, none of the multiobjective evolutionary algorithms (MOEAs) has a proof of convergence to the true Pareto-optimal solutions with a wide diversity among the solutions. In this paper, we discuss why a number of earlier MOEAs do not have such properties. Based on the concept of epsilon-dominance, new archiving strategies are proposed that overcome this fundamental problem and provably lead to MOEAs that have both the desired convergence and distribution properties. A number of modifications to the baseline algorithm are also suggested. The concept of epsilon-dominance introduced in this paper is practical and should make the proposed algorithms useful to researchers and practitioners alike.     

Bayesian optimization with approximate set kernels

Machine Learning - We propose a practical Bayesian optimization method over sets, to minimize a black-box function that takes a set as a single input. Because set inputs are permutation-invariant,...     

一种混合粒子群算法的贝叶斯网络优化模型

传统的粒子群优化算法通过群体中粒子间的合作和竞争进行群体智能指导优化搜索,算法收敛速度快,但较易陷入局部较优值,进入早熟状态。为了解决这个问题,提出了一种混合粒子群算法的贝叶斯网络优化模型,它可以通过当前所选择的较优解群构造一个贝叶斯网络和联合概率分布模型,利用这个模型进行采样得到更优解,用其可随机替换掉PSO中的一些粒子或个体最优解;同时利用粒子群算法对当前选择出的较优解群进行深度搜索,并将得到的最优解融入到较优解群中。分析可知,该方法可以提高算法有效性和可靠性。     

Model accuracy in the Bayesian optimization algorithm

Evolutionary algorithms (EAs) are particularly suited to solve problems for which there is not much information available. From this standpoint, estimation of distribution algorithms (EDAs), which guide the search by using probabilistic models of the population, have brought a new view to evolutionary computation. While solving a given problem with an EDA, the user has access to a set of models that reveal probabilistic dependencies between variables, an important source of information about the problem. However, as the complexity of the used models increases, the chance of overfitting and consequently reducing model interpretability, increases as well. This paper investigates the relationship between the probabilistic models learned by the Bayesian optimization algorithm (BOA) and the underlying problem structure. The purpose of the paper is threefold. First, model building in BOA is analyzed to understand how the problem structure is learned. Second, it is shown how the selection operator can lead to model overfitting in Bayesian EDAs. Third, the scoring metric that guides the search for an adequate model structure is modified to take into account the non-uniform distribution of the mating pool generated by tournament selection. Overall, this paper makes a contribution towards understanding and improving model accuracy in BOA, providing more interpretable models to assist efficiency enhancement techniques and human researchers.     

Introducing assignment functions to Bayesian optimization algorithms

In this paper, we improve Bayesian optimization algorithms by introducing proportionate and rank-based assignment functions. A Bayesian optimization algorithm builds a Bayesian network from a selected sub-population of promising solutions, and this probabilistic model is employed to generate the offspring of the next generation. Our method assigns each solution a relative significance based on its fitness, and this information is used in building the Bayesian network model. These assignment functions can improve the quality of the model without performing an explicit selection on the population. Numerical experiments demonstrate the effectiveness of this method compared to a conventional BOA.     

A new real-coded stochastic Bayesian optimization algorithm for continuous global optimization

Estimation of distribution algorithms are considered to be a new class of evolutionary algorithms which are applied as an alternative to genetic algorithms. Such algorithms sample the new generation from a probabilistic model of promising solutions. The search space of the optimization problem is improved by such probabilistic models. In the Bayesian optimization algorithm (BOA), the set of promising solutions forms a Bayesian network and the new solutions are sampled from the built Bayesian network. This paper proposes a novel real-coded stochastic BOA for continuous global optimization by utilizing a stochastic Bayesian network. In the proposed algorithm, the new Bayesian network takes advantage of using a stochastic structure (that there is a probability distribution function for each edge in the network) and the new generation is sampled from the stochastic structure. In order to generate a new solution, some new structure, and therefore a new Bayesian network is sampled from the current stochastic structure and the new solution will be produced from the sampled Bayesian network. Due to the stochastic structure used in the sampling phase, each sample can be generated based on a different structure. Therefore the different dependency structures can be preserved. Before the new generation is generated, the stochastic network’s probability distributions are updated according to the fitness evaluation of the current generation. The proposed method is able to take advantage of using different dependency structures through the sampling phase just by using one stochastic structure. The experimental results reported in this paper show that the proposed algorithm increases the quality of the solutions on the general optimization benchmark problems.     

High-dimensional Bayesian optimization using low-dimensional feature spaces

Bayesian optimization (BO) is a powerful approach for seeking the global optimum of expensive black-box functions and has proven successful for fine tuning hyper-parameters of machine learning models. However, BO is practically limited to optimizing 10–20 parameters. To scale BO to high dimensions, we usually make structural assumptions on the decomposition of the objective and/or exploit the intrinsic lower dimensionality of the problem, e.g. by using linear projections. We could achieve a higher compression rate with nonlinear projections, but learning these nonlinear embeddings typically requires much data. This contradicts the BO objective of a relatively small evaluation budget. To address this challenge, we propose to learn a low-dimensional feature space jointly with (a) the response surface and (b) a reconstruction mapping. Our approach allows for optimization of BO’s acquisition function in the lower-dimensional subspace, which significantly simplifies the optimization problem. We reconstruct the original parameter space from the lower-dimensional subspace for evaluating the black-box function. For meaningful exploration, we solve a constrained optimization problem.

     

Bayesian optimization for learning gaits under uncertainty

Designing gaits and corresponding control policies is a key challenge in robot locomotion. Even with a viable controller parametrization, finding near-optimal parameters can be daunting. Typically, this kind of parameter optimization requires specific expert knowledge and extensive robot experiments. Automatic black-box gait optimization methods greatly reduce the need for human expertise and time-consuming design processes. Many different approaches for automatic gait optimization have been suggested to date. However, no extensive comparison among them has yet been performed. In this article, we thoroughly discuss multiple automatic optimization methods in the context of gait optimization. We extensively evaluate Bayesian optimization, a model-based approach to black-box optimization under uncertainty, on both simulated problems and real robots. This evaluation demonstrates that Bayesian optimization is particularly suited for robotic applications, where it is crucial to find a good set of gait parameters in a small number of experiments.     

一种混沌贝叶斯优化算法

为了减少贝叶斯优化算法的计算量,该文提出了一种混沌贝叶斯优化算法。用混沌随机序列产生贝叶斯优化算法的初始群体,利用混沌随机性、遍历性和对初始条件的敏感性的特点,提供给贝叶斯网络变量空间丰富的信息,有利于建立接近最优的贝叶斯网络。为增加群体的多样性同时减少贝叶斯网络的建立次数,采用混沌搜索方法对贝叶斯网络产生的新解进行变异寻优,以此为基础再建立贝叶斯网络。实验结果表明,与贝叶斯优化算法相比,混沌贝叶斯优化算法能有效减少计算量。     

Combining technical trading rules using particle swarm optimization

Technical trading rules have been utilized in the stock market to make profit for more than a century. However, only using a single trading rule may not be sufficient to predict the stock price trend accurately. Although some complex trading strategies combining various classes of trading rules have been proposed in the literature, they often pick only one rule for each class, which may lose valuable information from other rules in the same class. In this paper, a complex stock trading strategy, namely performance-based reward strategy (PRS), is proposed. PRS combines the two most popular classes of technical trading rules – moving average (MA) and trading range break-out (TRB). For both MA and TRB, PRS includes various combinations of the rule parameters to produce a universe of 140 component trading rules in all. Each component rule is assigned a starting weight, and a reward/penalty mechanism based on rules’ recent profit is proposed to update their weights over time. To determine the best parameter values of PRS, we employ an improved time variant particle swarm optimization (TVPSO) algorithm with the objective of maximizing the annual net profit generated by PRS. The experiments show that PRS outperforms all of the component rules in the testing period. To assess the significance of our trading results, we apply bootstrapping methodology to test three popular null models of stock return: the random walk, the AR(1) and the GARCH(1, 1). The results show that PRS is not consistent with these null models and has good predictive ability.     

Combining genetic algorithms with BESO for topology optimization

This paper proposes a new algorithm for topology optimization by combining the features of genetic algorithms (GAs) and bi-directional evolutionary structural optimization (BESO). An efficient treatment of individuals and population for finite element models is presented which is different from traditional GAs application in structural design. GAs operators of crossover and mutation suitable for topology optimization problems are developed. The effects of various parameters used in the proposed GA on the optimization speed and performance are examined. Several 2D and 3D examples of compliance minimization problems are provided to demonstrate the efficiency of the proposed new approach and its capability of obtaining convergent solutions. Wherever possible, the numerical results of the proposed algorithm are compared with the solutions of other GA methods and the SIMP method.     

Combining multi-core and GPU computing for solving combinatorial optimization problems

In this paper, we revisit the design and implementation of Branch-and-Bound (B&B) algorithms for solving large combinatorial optimization problems on GPU-enhanced multi-core machines. B&B is a tree-based optimization method that uses four operators (selection, branching, bounding and pruning) to build and explore a highly irregular tree representing the solution space. In our previous works, we have proposed a GPU-accelerated approach in which only a single CPU core is used and only the bounding operator is performed on the GPU device. Here, we extend the approach (LL-GB&B) in order to minimize the CPU–GPU communication latency and thread divergence. Such an objective is achieved through a GPU-based fine-grained parallelization of the branching and pruning operators in addition to the bounding one. The second contribution consists in investigating the combination of a GPU with multi-core processing. Two scenarios have been explored leading to two approaches: a concurrent (RLL-GB&B) and a cooperative one (PLL-GB&B). In the first one, the exploration process is performed concurrently by the GPU and the CPU cores. In the cooperative approach, the CPU cores prepare and off-load to GPU pools of tree nodes using data streaming while the GPU performs the exploration. The different approaches have been extensively experimented on the Flowshop scheduling problem. Compared to a single CPU-based execution, LL-GB&B allows accelerations up to (×$\times$160) for large problem instances. Moreover, when combining multi-core and GPU, we figure out that using RLL-GB&B is not beneficial while PLL-GB&B enables an improvement up to 36% compared to LL-GB&B.