Lipschitz global optimization methods in control problems

Many control problems involve the search for the global extremum in the space of states or the parameters of the system under study, which leads to the necessity of using effective methods of global finite-dimensional optimization. For this purpose use can be made of the geometric algorithms of Lipschitz global optimization, which are developed by the authors. A brief review of these algorithms is presented and they are compared with some algorithms of global search that are often used in technical practice. Numerical experiments are performed on a few known examples of applied multiextremal problems.     

Combining Argumentation and Bayesian Nets for Breast Cancer Prognosis

We present a new framework for combining logic with probability, and demonstrate the application of this framework to breast cancer prognosis. Background knowledge concerning breast cancer prognosis is represented using logical arguments. This background knowledge and a database are used to build a Bayesian net that captures the probabilistic relationships amongst the variables. Causal hypotheses gleaned from the Bayesian net in turn generate new arguments. The Bayesian net can be queried to help decide when one argument attacks another. The Bayesian net is used to perform the prognosis, while the argumentation framework is used to provide a qualitative explanation of the prognosis.     

Stability analysis of particle swarm optimization without Lipschitz constraint

There are some adjustable parameters which directly influence the performance and stability of Particle Swarm Optimization algorithm. In this paper, stabilities of PSO with constant parameters and time-varying parameters are analyzed without Lipschitz constraint. Necessary and sufficient stability conditions for acceleration factor φ and inertia weight w are presented. Experiments on benchmark functions show the good performance of PSO satisfying the stability condition, even without Lipschitz constraint . And the inertia weight w value is enhanced to ( - 1,1).     

Bayesian forecaster using class-based optimization

Suppose that several forecasters exist for the problem in which class-wise accuracies of forecasting classifiers are important. For such a case, we propose to use a new Bayesian approach for deriving one unique forecaster out of the existing forecasters. Our Bayesian approach links the existing forecasting classifiers via class-based optimization by the aid of an evolutionary algorithm (EA). To show the usefulness of our Bayesian approach in practical situations, we have considered the case of the Korean stock market, where numerous lag-l forecasting classifiers exist for monitoring its status.     

贝叶斯优化算法的发展综述

介绍了贝叶斯优化算法,并针对不同的优化问题,结合经典优化方法提出的层次BOA算法、多目标层次BOA算法以及递进BOA算法,对贝叶斯优化算法的算法设计、理论分析和应用研究做了全面的总结.深入地探讨了贝叶斯优化算法计算量大,难以建立精确概率模型及扩展应用领域等问题.     

Multi-information source constrained Bayesian optimization

Structural and Multidisciplinary Optimization - Design decisions for complex systems often can be made or informed by a variety of information sources. When optimizing such a system, the evaluation...     

Combining Auditory Preprocessing and Bayesian Estimation for Robust Formant Tracking

We present a framework for estimating formant trajectories. Its focus is to achieve high robustness in noisy environments. Our approach combines a preprocessing based on functional principles of the human auditory system and a probabilistic tracking scheme. For enhancing the formant structure in spectrograms we use a Gammatone filterbank, a spectral preemphasis, as well as a spectral filtering using Difference-of-Gaussians (DoG) operators. Finally, a contrast enhancement mimicking a competition between filter responses is applied. The probabilistic tracking scheme adopts the mixture modeling technique for estimating the joint distribution of formants. In conjunction with an algorithm for adaptive frequency range segmentation as well as Bayesian smoothing an efficient framework for estimating formant trajectories is derived. Comprehensive evaluations of our method on the VTR–Formant database emphasize its high precision and robustness. We obtained superior performance compared to existing approaches for clean as well as echoic noisy speech. Finally, an implementation of the framework within the scope of an online system using instantaneous feature-based resynthesis demonstrates its applicability to real-world scenarios.      

融合知识和数据的贝叶斯网络构造方法

从数据中学习贝叶斯网络往往会因为搜索空间庞大而耗费大量时间.由于贝叶斯网络固有的因果语义,领域专家往往能够凭借自己的经验确定节点之间的因果关系.本文方法充分收集专家的意见,并利用证据理论进行综合,去除无意义的网络结构,然后利用常用的学习算法从数据中继续学习.这种融合知识和数据的贝叶斯网络构造方法利用专家知识来缩小学习算法的搜索空间,避免了盲目搜索,同时也避免了单个专家知识的主观性.实验表明该方法能够有效提高学习效率.     

Combining convergence and diversity in evolutionary multiobjective optimization

Over the past few years, the research on evolutionary algorithms has demonstrated their niche in solving multiobjective optimization problems, where the goal is to find a number of Pareto-optimal solutions in a single simulation run. Many studies have depicted different ways evolutionary algorithms can progress towards the Pareto-optimal set with a widely spread distribution of solutions. However, none of the multiobjective evolutionary algorithms (MOEAs) has a proof of convergence to the true Pareto-optimal solutions with a wide diversity among the solutions. In this paper, we discuss why a number of earlier MOEAs do not have such properties. Based on the concept of epsilon-dominance, new archiving strategies are proposed that overcome this fundamental problem and provably lead to MOEAs that have both the desired convergence and distribution properties. A number of modifications to the baseline algorithm are also suggested. The concept of epsilon-dominance introduced in this paper is practical and should make the proposed algorithms useful to researchers and practitioners alike.     

Bayesian optimization with approximate set kernels

Machine Learning - We propose a practical Bayesian optimization method over sets, to minimize a black-box function that takes a set as a single input. Because set inputs are permutation-invariant,...     

结合贝叶斯网与SFMEA技术的软件故障诊断框架

软件故障已经成为导致系统出错.失效甚至崩溃的潜在与重要根源.因此,故障诊断技术对于保证软件质量具有非常重要的意义.基于人工智能理论的故障诊断技术已经得到越来越多的关注.贝叶斯网理论具有表述方便.推理严密等优点,将其与软件失效模式及影响分析方法相结合,提出了具有四层网络结构(原因层、模式层、故障层以及观察层)的WCMF贝叶斯网诊断模型.在此基础上,提出了基于故障信息数据库及诊断数据库的智能化软件故障诊断框架.最后,针对某数传导航机载软件进行了实例应用.结果表明,提出的WCMF诊断模型及诊断框架是可行且有效的,具有故障诊断方便及时、充分利用历史信息以提高诊断效率、降低时间和资源消耗等优点.     

基于改进贝叶斯优化算法的CNN超参数优化方法

CNN框架中,如何对其模型的超参数进行自动化获取一直是一个重要问题。提出一种基于改进的贝叶斯优化算法的CNN超参数优化方法。该方法使用改进的汤普森采样方法作为采集函数,利用改进的马尔可夫链蒙特卡罗算法加速训练高斯代理模型。该方法可以在超参数空间不同的CNN框架下进行超参数优化。利用CIFAR-10、MRBI和SVHN测试集对算法进行性能测试,实验结果表明,改进后的CNN超参数优化算法比同类超参数优化算法具有更好的性能。     

一种混合粒子群算法的贝叶斯网络优化模型

传统的粒子群优化算法通过群体中粒子间的合作和竞争进行群体智能指导优化搜索,算法收敛速度快,但较易陷入局部较优值,进入早熟状态。为了解决这个问题,提出了一种混合粒子群算法的贝叶斯网络优化模型,它可以通过当前所选择的较优解群构造一个贝叶斯网络和联合概率分布模型,利用这个模型进行采样得到更优解,用其可随机替换掉PSO中的一些粒子或个体最优解;同时利用粒子群算法对当前选择出的较优解群进行深度搜索,并将得到的最优解融入到较优解群中。分析可知,该方法可以提高算法有效性和可靠性。     

Output feedback distributed optimization algorithms of second-order Lipschitz nonlinear multi-agent systems

This paper introduces output feedback distributed optimization algorithms designed specifically for second-order nonlinear multi-agent systems. The agents are allowed to have heterogeneous dynamics, characterized by distinct nonlinearities, as long as they satisfy the Lipschitz continuity condition. For the case with unknown states, nonlinear state observers are designed first for each agent to reconstruct agents' unknown states. It is proven that the agents' unknown states are estimated accurately by the developed state observers. Then, based on the agents' state estimates and the gradient of each agent local cost function, a kind of output feedback distributed optimization algorithms are proposed for the considered multi-agent systems. Under the proposed distributed optimization algorithms, all the agents' outputs asymptotically approach the minimizer of the global cost function which is the sum of all the local cost functions. By using Lyapunov stability theory, convex analysis, and input-to-state stability theory, the asymptotical convergence of the output feedback distributed optimization closed-loop system is proven. Simulations are conducted to validate the efficacy of the proposed algorithms.     

微粒群优化和视觉感应相结合的图像增强方法

对粒群优化算法进行了改进,提出了一种微粒群优化和视觉感应相结合的图像增强方法,通过微粒群算法优化灰度图像的平均明暗信息熵差值,自适应地选择图像灰度转换函数,用以实现图像的增强。该方法不仅参数个数少,优化速度快,在搜索能力上优于粒群优化算法,而且能够保证算法的全局收敛性。仿真实例证明了该方法在图像增强上的有效性和优越性。     

High-dimensional Bayesian optimization using low-dimensional feature spaces

Bayesian optimization (BO) is a powerful approach for seeking the global optimum of expensive black-box functions and has proven successful for fine tuning hyper-parameters of machine learning models. However, BO is practically limited to optimizing 10–20 parameters. To scale BO to high dimensions, we usually make structural assumptions on the decomposition of the objective and/or exploit the intrinsic lower dimensionality of the problem, e.g. by using linear projections. We could achieve a higher compression rate with nonlinear projections, but learning these nonlinear embeddings typically requires much data. This contradicts the BO objective of a relatively small evaluation budget. To address this challenge, we propose to learn a low-dimensional feature space jointly with (a) the response surface and (b) a reconstruction mapping. Our approach allows for optimization of BO’s acquisition function in the lower-dimensional subspace, which significantly simplifies the optimization problem. We reconstruct the original parameter space from the lower-dimensional subspace for evaluating the black-box function. For meaningful exploration, we solve a constrained optimization problem.

     

Bayesian optimization for learning gaits under uncertainty

Designing gaits and corresponding control policies is a key challenge in robot locomotion. Even with a viable controller parametrization, finding near-optimal parameters can be daunting. Typically, this kind of parameter optimization requires specific expert knowledge and extensive robot experiments. Automatic black-box gait optimization methods greatly reduce the need for human expertise and time-consuming design processes. Many different approaches for automatic gait optimization have been suggested to date. However, no extensive comparison among them has yet been performed. In this article, we thoroughly discuss multiple automatic optimization methods in the context of gait optimization. We extensively evaluate Bayesian optimization, a model-based approach to black-box optimization under uncertainty, on both simulated problems and real robots. This evaluation demonstrates that Bayesian optimization is particularly suited for robotic applications, where it is crucial to find a good set of gait parameters in a small number of experiments.     

Model accuracy in the Bayesian optimization algorithm

Evolutionary algorithms (EAs) are particularly suited to solve problems for which there is not much information available. From this standpoint, estimation of distribution algorithms (EDAs), which guide the search by using probabilistic models of the population, have brought a new view to evolutionary computation. While solving a given problem with an EDA, the user has access to a set of models that reveal probabilistic dependencies between variables, an important source of information about the problem. However, as the complexity of the used models increases, the chance of overfitting and consequently reducing model interpretability, increases as well. This paper investigates the relationship between the probabilistic models learned by the Bayesian optimization algorithm (BOA) and the underlying problem structure. The purpose of the paper is threefold. First, model building in BOA is analyzed to understand how the problem structure is learned. Second, it is shown how the selection operator can lead to model overfitting in Bayesian EDAs. Third, the scoring metric that guides the search for an adequate model structure is modified to take into account the non-uniform distribution of the mating pool generated by tournament selection. Overall, this paper makes a contribution towards understanding and improving model accuracy in BOA, providing more interpretable models to assist efficiency enhancement techniques and human researchers.     

Introducing assignment functions to Bayesian optimization algorithms

In this paper, we improve Bayesian optimization algorithms by introducing proportionate and rank-based assignment functions. A Bayesian optimization algorithm builds a Bayesian network from a selected sub-population of promising solutions, and this probabilistic model is employed to generate the offspring of the next generation. Our method assigns each solution a relative significance based on its fitness, and this information is used in building the Bayesian network model. These assignment functions can improve the quality of the model without performing an explicit selection on the population. Numerical experiments demonstrate the effectiveness of this method compared to a conventional BOA.     

A new real-coded stochastic Bayesian optimization algorithm for continuous global optimization

Estimation of distribution algorithms are considered to be a new class of evolutionary algorithms which are applied as an alternative to genetic algorithms. Such algorithms sample the new generation from a probabilistic model of promising solutions. The search space of the optimization problem is improved by such probabilistic models. In the Bayesian optimization algorithm (BOA), the set of promising solutions forms a Bayesian network and the new solutions are sampled from the built Bayesian network. This paper proposes a novel real-coded stochastic BOA for continuous global optimization by utilizing a stochastic Bayesian network. In the proposed algorithm, the new Bayesian network takes advantage of using a stochastic structure (that there is a probability distribution function for each edge in the network) and the new generation is sampled from the stochastic structure. In order to generate a new solution, some new structure, and therefore a new Bayesian network is sampled from the current stochastic structure and the new solution will be produced from the sampled Bayesian network. Due to the stochastic structure used in the sampling phase, each sample can be generated based on a different structure. Therefore the different dependency structures can be preserved. Before the new generation is generated, the stochastic network’s probability distributions are updated according to the fitness evaluation of the current generation. The proposed method is able to take advantage of using different dependency structures through the sampling phase just by using one stochastic structure. The experimental results reported in this paper show that the proposed algorithm increases the quality of the solutions on the general optimization benchmark problems.