On three new approaches to handle constraints within evolution strategies

Evolutionary algorithms with a self-adaptive step control mechanism like evolution strategies (ES) often suffer from premature fitness stagnation on constrained numerical optimization problems. When the optimum lies on the constraint boundary or even in a vertex of the feasible search space, a disadvantageous success probability results in premature step size reduction. We introduce three new constraint-handling methods for ES on constrained continuous search spaces. The death penalty step control evolution strategy (DSES) is based on the controlled reduction of a minimum step size depending on the distance to the infeasible search space. The two sexes evolution strategy (TSES) is inspired by the biological concept of sexual selection and pairing. At last, the nested angle evolution strategy (NAES) is an approach in which the angles of the correlated mutation of the inner ES are adapted by the outer ES. All methods are experimentally evaluated on four selected test problems and compared with existing penalty-based constraint-handling methods.     

Evolution strategies for solving discrete optimization problems

A method to solve discrete optimization problems using evolution strategies (ESs) is described. The ESs imitate biological evolution in nature and have two characteristics that differ from other conventional optimization algorithms: (a) ESs use randomized operators instead of the usual deterministic ones; (b) instead of a single design point, the ESs work simultaneously with a population of design points in the space of variables. The important operators of ESs are mutation, selection and recombination. The ESs are commonly applied for continuous optimization problems. For the application to discrete problems, several modifications on the operators mutation and recombination are suggested here. Several examples from the literature are solved with this modified ES and the results compared. The examples show that the modified ES is robust and suitable for discrete optimization problems.     

Self-adaptive genetic algorithms with simulated binary crossover

Self-adaptation is an essential feature of natural evolution. However, in the context of function optimization, self-adaptation features of evolutionary search algorithms have been explored mainly with evolution strategy (ES) and evolutionary programming (EP). In this paper, we demonstrate the self-adaptive feature of real-parameter genetic algorithms (GAs) using a simulated binary crossover (SBX) operator and without any mutation operator. The connection between the working of self-adaptive ESs and real-parameter GAs with the SBX operator is also discussed. Thereafter, the self-adaptive behavior of real-parameter GAs is demonstrated on a number of test problems commonly used in the ES literature. The remarkable similarity in the working principle of real-parameter GAs and self-adaptive ESs shown in this study suggests the need for emphasizing further studies on self-adaptive GAs.     

A comparison study of self-adaptation in evolution strategies and real-coded genetic algorithms

This paper discusses the self-adaptive mechanisms of evolution strategies (ES) and real-coded genetic algorithms (RCGA) for optimization in continuous search spaces. For multi-membered evolution strategies, a self-adaptive mechanism of mutation parameters has been proposed by Schwefel. It introduces parameters such as standard deviations of the normal distribution for mutation into the genetic code and lets them evolve by selection as well as the decision variables. In the RCGA, crossover or recombination is used mainly for search. It utilizes information on several individuals to generate novel search points, and therefore, it can generate offspring adaptively according to the distribution of parents without any adaptive parameters. The present paper discusses characteristics of these two self-adaptive mechanisms through numerical experiments. The self-adaptive characteristics such as translation, enlargement, focusing, and directing of the distribution of children generated by the ES and the RCGA are examined through experiments.     

A surrogate-assisted evolution strategy for constrained multi-objective optimization

In many real-world optimization problems, several conflicting objectives must be achieved and optimized simultaneously and the solutions are often required to satisfy certain restrictions or constraints. Moreover, in some applications, the numerical values of the objectives and constraints are obtained from computationally expensive simulations. Many multi-objective optimization algorithms for continuous optimization have been proposed in the literature and some have been incorporated or used in conjunction with expert and intelligent systems. However, relatively few of these multi-objective algorithms handle constraints, and even fewer, use surrogates to approximate the objective or constraint functions when these functions are computationally expensive. This paper proposes a surrogate-assisted evolution strategy (ES) that can be used for constrained multi-objective optimization of expensive black-box objective functions subject to expensive black-box inequality constraints. Such an algorithm can be incorporated into an intelligent system that finds approximate Pareto optimal solutions to simulation-based constrained multi-objective optimization problems in various applications including engineering design optimization, production management and manufacturing. The main idea in the proposed algorithm is to generate a large number of trial offspring in each generation and use the surrogates to predict the objective and constraint function values of these trial offspring. Then the algorithm performs an approximate non-dominated sort of the trial offspring based on the predicted objective and constraint function values, and then it selects the most promising offspring (those with the smallest predicted ranks from the non-dominated sort) to become the actual offspring for the current generation that will be evaluated using the expensive objective and constraint functions. The proposed method is implemented using cubic radial basis function (RBF) surrogate models to assist the ES. The resulting RBF-assisted ES is compared with the original ES and to NSGA-II on 20 test problems involving 2–15 decision variables, 2–5 objectives and up to 13 inequality constraints. These problems include well-known benchmark problems and application problems in manufacturing and robotics. The numerical results showed that the RBF-assisted ES generally outperformed the original ES and NSGA-II on the problems used when the computational budget is relatively limited. These results suggest that the proposed surrogate-assisted ES is promising for computationally expensive constrained multi-objective optimization.     

Multi-population differential evolution with balanced ensemble of mutation strategies for large-scale global optimization

Differential evolution (DE) is a simple, yet very effective, population-based search technique. However, it is challenging to maintain a balance between exploration and exploitation behaviors of the DE algorithm. In this paper, we boost the population diversity while preserving simplicity by introducing a multi-population DE to solve large-scale global optimization problems. In the proposed algorithm, called mDE-bES, the population is divided into independent subgroups, each with different mutation and update strategies. A novel mutation strategy that uses information from either the best individual or a randomly selected one is used to produce quality solutions to balance exploration and exploitation. Selection of individuals for some of the tested mutation strategies utilizes fitness-based ranks of these individuals. Function evaluations are divided into epochs. At the end of each epoch, individuals between the subgroups are exchanged to facilitate information exchange at a slow pace. The performance of the algorithm is evaluated on a set of 19 large-scale continuous optimization problems. A comparative study is carried out with other state-of-the-art optimization techniques. The results show that mDE-bES has a competitive performance and scalability behavior compared to the contestant algorithms.     

On the equivalences and differences of evolutionary algorithms

Evolutionary algorithms (EAs) are fast and robust computation methods for global optimization, and have been widely used in many real-world applications. We first conceptually discuss the equivalences of various popular EAs including genetic algorithm (GA), biogeography-based optimization (BBO), differential evolution (DE), evolution strategy (ES) and particle swarm optimization (PSO). We find that the basic versions of BBO, DE, ES and PSO are equal to the GA with global uniform recombination (GA/GUR) under certain conditions. Then we discuss their differences based on biological motivations and implementation details, and point out that their distinctions enhance the diversity of EA research and applications. To further study the characteristics of various EAs, we compare the basic versions and advanced versions of GA, BBO, DE, ES and PSO to explore their optimization ability on a set of real-world continuous optimization problems. Empirical results show that among the basic versions of the algorithms, BBO performs best on the benchmarks that we studied. Among the advanced versions of the algorithms, DE and ES perform best on the benchmarks that we studied. However, our main conclusion is that the conceptual equivalence of the algorithms is supported by the fact that algorithmic modifications result in very different performance levels.     

Convergence in evolutionary programs with self-adaptation

Evolutionary programs are capable of finding good solutions to difficult optimization problems. Previous analysis of their convergence properties has normally assumed the strategy parameters are kept constant, although in practice these parameters are dynamically altered. In this paper, we propose a modified version of the 1/5-success rule for self-adaptation in evolution strategies (ES). Formal proofs of the long-term behavior produced by our self-adaptation method are included. Both elitist and non-elitist ES variants are analyzed. Preliminary tests indicate an ES with our modified self-adaptation method compares favorably to both a non-adapted ES and a 1/5-success rule adapted ES.     

Modified differential evolution approach for optimization of planar parallel manipulators force capabilities

The global optimization problem is not easy to solve and is still an open challenge for researchers since an analytical optimal solution is difficult to obtain even for relatively simple application problems. Conventional deterministic numerical algorithms tend to stop the search in local minimum nearest to the input starting point, mainly when the optimization problem presents nonlinear, non-convex and non-differential functions, multimodal and nonlinear. Nowadays, the use of evolutionary algorithms (EAs) to solve optimization problems is a common practice due to their competitive performance on complex search spaces. EAs are well known for their ability to deal with nonlinear and complex optimization problems. The primary advantage of EAs over other numerical methods is that they just require the objective function values, while properties such as differentiability and continuity are not necessary. In this context, the differential evolution (DE), a paradigm of the evolutionary computation, has been widely used for solving numerical global optimization problems in continuous search space. DE is a powerful population-based stochastic direct search method. DE simulates natural evolution combined with a mechanism to generate multiple search directions based on the distribution of solutions in the current population. Among DE advantages are its simple structure, ease of use, speed, and robustness, which allows its application on several continuous nonlinear optimization problems. However, the performance of DE greatly depends on its control parameters, such as crossover rate, mutation factor, and population size and it often suffers from being trapped in local optima. Conventionally, users have to determine the parameters for problem at hand empirically. Recently, several adaptive variants of DE have been proposed. In this paper, a modified differential evolution (MDE) approach using generation-varying control parameters (mutation factor and crossover rate) is proposed and evaluated. The proposed MDE presents an efficient strategy to improve the search performance in preventing of premature convergence to local minima. The efficiency and feasibility of the proposed MDE approach is demonstrated on a force optimization problem in Robotics, where the force capabilities of a planar 3-RRR parallel manipulator are evaluated considering actuation limits and different assembly modes. Furthermore, some comparison results of MDE approach with classical DE to the mentioned force optimization problem are presented and discussed.     

On the robustness of population-based versus point-basedoptimization in the presence of noise

Practical optimization problems often require the evaluation of solutions through experimentation, stochastic simulation, sampling, or even interaction with the user. Thus, most practical problems involve noise. We address the robustness of population-based versus point-based optimization on a range of parameter optimization problems when noise is added to the deterministic objective function values. Population-based optimization is realized by a genetic algorithm and an evolution strategy. Point-based optimization is implemented as the classical Hooke-Jeeves pattern search strategy and threshold accepting as a modern local search technique. We investigate the performance of these optimization methods for varying levels of additive normally distributed fitness-independent noise and different sample sizes for evaluating individual solutions. Our results strongly favour population-based optimization, and the evolution strategy in particular     

Self-Adaptive Multimethod Search for Global Optimization in Real-Parameter Spaces

Many different algorithms have been developed in the last few decades for solving complex real-world search and optimization problems. The main focus in this research has been on the development of a single universal genetic operator for population evolution that is always efficient for a diverse set of optimization problems. In this paper, we argue that significant advances to the field of evolutionary computation can be made if we embrace a concept of self-adaptive multimethod optimization in which multiple different search algorithms are run concurrently, and learn from each other through information exchange using a common population of points. We present an evolutionary algorithm, entitled A Multialgorithm Genetically Adaptive Method for Single Objective Optimization (AMALGAM-SO), that implements this concept of self adaptive multimethod search. This method simultaneously merges the strengths of the covariance matrix adaptation (CMA) evolution strategy, genetic algorithm (GA), and particle swarm optimizer (PSO) for population evolution and implements a self-adaptive learning strategy to automatically tune the number of offspring these three individual algorithms are allowed to contribute during each generation. Benchmark results in 10, 30, and 50 dimensions using synthetic functions from the special session on real-parameter optimization of CEC 2005 show that AMALGAM-SO obtains similar efficiencies as existing algorithms on relatively simple unimodal problems, but is superior for more complex higher dimensional multimodal optimization problems. The new search method scales well with increasing number of dimensions, converges in the close proximity of the global minimum for functions with noise induced multimodality, and is designed to take full advantage of the power of distributed computer networks.     

模拟进化优化方法及其应用：遗传算法

在本世纪六十年代中期,美国、德国等国家的一些科学家开始研究用模仿生物和人类进化的方法求解复杂的优化问题的方法,这里我们统称之为模拟进化优化方法(optimi:ation method by simulatedevolution)。但这些方法在六十年代和七十年代并未受到普遍的重视,一是因为当时这些方法还不成熟,二是当时计算机发展水平低,容量小,计算速度慢,这些方法又需要较大的计算量,难以实际应用。但在这期间有一些科学家一直在进行不懈的努力.代表性人物之一为美国的著名科学家J.H.Holland,他和他的学生甲直在对他所提出的一种模拟进化优化方法—遗传算法(Genetic Algorithm,GA)进行理论研究并开拓其应用领域。八十年代初期,伴随着人工神经元网络理论和机器学习理论的发展以及计算机容量和计算速度的不断提高,遗传算法的研究也越来越受到重视而逐步成熟起来,并日益受到各学科研究人员的普遍重视。自八十年代中期开始,这种方法除了在人工智能领域。     

An adaptive neural network strategy for improving the computational performance of evolutionary structural optimization

This work is focused on improving the computational efficiency of evolutionary algorithms implemented in large-scale structural optimization problems. Locating optimal structural designs using evolutionary algorithms is a task associated with high computational cost, since a complete finite element (FE) analysis needs to be carried out for each parent and offspring design vector of the populations considered. Each of these FE solutions facilitates decision making regarding the feasibility or infeasibility of the corresponding structural design by evaluating the displacement and stress constraints specified for the structural problem at hand. This paper presents a neural network (NN) strategy to reliably predict, in the framework of an evolution strategies (ES) procedure for structural optimization, the feasibility or infeasibility of structural designs avoiding computationally expensive FE analyses. The proposed NN implementation is adaptive in the sense that the utilized NN configuration is appropriately updated as the ES process evolves by performing NN retrainings using information gradually accumulated during the ES execution. The prediction capabilities and the computational advantages offered by this adaptive NN scheme coupled with domain decomposition solution techniques are investigated in the context of design optimization of skeletal structures on both sequential and parallel computing environments.     

基于DE 和SA 的Memetic 高维全局优化算法

针对高维复杂多模态优化问题,传统的进化算法存在收敛速度慢,求解精度低等缺点,提出一种面向高维优化问题的Memetic全局优化算法。算法通过全局搜索和局部搜索结合的混合搜索策略,采用多模式并行差分进化算法进行全局搜索,基于高斯分布估计的模拟退火算法进行局部搜索。改进后的Memetic算法不仅继承了差分进化算法能发现全局最优解的优点,而且能大幅度提高搜索效率。最后,通过对4个高维多峰值Benchmark函数进行仿真实验,实验结果表明本文算法有效提高了算法的收敛速度和求解精度。     

基于压缩感知的MIMO-OFDM信道估计中的导频设计方案

多输入多输出-正交频分复用(Multiple input multiple output-orthogonal frequency division multiplexing,MIMO-OFDM)系统作为MIMO系统和OFDM系统的结合,具有很高的频带利用率并能有效地对抗无线信道的多径效应。本文研究了MIMO-OFDM系统稀疏信道估计及其导频优化,将信道估计问题转化为压缩感知(Compressed sensing,CS)理论中的稀疏信号重建问题,将最小化测量矩阵的互相关作为导频优化的目标。结合已有的随机序贯搜索(Stochastic sequential search,SSS)和扩展算法2(Extension scheme 2,ES2)算法以及导频移位机制,提出了一种快速的导频优化算法随机搜索移位算法(Stochastic sequential search-shift mechanism,SSS-SM)。此算法的运算复杂度远低于已有的ES2算法,运算时间不受发射天线数影响。将SSS-SM算法和ES2算法分别获得的导频设计结果应用于MIMO-OFDM系统的信道估计,仿真结果表明,采用SSS-SM算法可以更低的算法复杂度获得与ES2算法相同的信道估计性能;高信噪比情况下,SSS-SM算法对应的均方误差(Mean square error,MSE)比ES2平均低约3~5 dB,因此这种方法在高信噪比下更有优势。     

A multi-objective differential evolutionary algorithm for constrained multi-objective optimization problems with low feasible ratio

Most current evolutionary multi-objective optimization (EMO) algorithms perform well on multi-objective optimization problems without constraints, but they encounter difficulties in their ability for constrained multi-objective optimization problems (CMOPs) with low feasible ratio. To tackle this problem, this paper proposes a multi-objective differential evolutionary algorithm named MODE-SaE based on an improved epsilon constraint-handling method. Firstly, MODE-SaE self-adaptively adjusts the epsilon level in line with the maximum and minimum constraint violation values of infeasible individuals. It can prevent epsilon level setting from being unreasonable. Then, the feasible solutions are saved to the external archive and take part in the population evolution by a co-evolution strategy. Finally, MODE-SaE switches the global search and local search by self-switching parameters of search engine to balance the convergence and distribution. With the aim of evaluating the performance of MODE-SaE, a real-world problem with low feasible ratio in decision space and fourteen bench-mark test problems, are used to test MODE-SaE and five other state-of-the-art constrained multi-objective evolution algorithms. The experimental results fully demonstrate the superiority of MODE-SaE on all mentioned test problems, which indicates the effectiveness of the proposed algorithm for CMOPs which have low feasible ratio in search space.     

混沌优化算法的性能分析

现代优化算法主要解决全局最优问题，其本质是概率性的．借鉴多种自然现象，人们提出了许多仿生、仿物算法，如禁忌搜索算法（TABU）、模拟退火（SAA）、遗传算法（GA）、进化策略（ES）、蚁群算法（ACA）等．利用混沌的遍历性进行优化搜索就是一种很有趣的研究思路，尤其对于虫口方程人们进行了许多研究，取得了一定的研究成果．但和普通的随机搜索算法相比，其性能之不足也很明显，主要体现在：混沌的遍历性不均匀，在边界处搜索密度高，远不如随机Monte Carlo搜索方法．这就从本质上决定了其搜索性能在普适性上与Monte Carlo算法有差距．仿真计算证实了这个结论．因此对于利用虫口方程进行的混沌优化研究需要谨慎采用．     

进化策略求解Hopfield神经网络

提出了一种进化策略求解ＨＯｐｆｉｅｌｄ神经网络的方法。该进化策略分三个阶段,即第一阶段只在较小区间上求出局部优化解;然后,在此基础上,由第二阶段求出较大区间上的局部优化解;最后由第三阶段求出全局优化解。同时采用Ｈｏｐｆｉｅｌｄ神经网络动态方程指导第一阶段的局部进化策略的进化方向,因而大大加快了优化搜索速度。在分阶段的进化策略中,其第一阶段只需搜索较小区间、第二和第三阶段的搜索则建立在其前一阶段的基     

A (/spl mu/ + /spl lambda/) evolutionary algorithm for reconstruction of SPECT data

Algorithms used to reconstruct single photon emission computed tomography (SPECT) data are based on one of two principles: filtered back projection or iterative methods. In this paper, an evolution strategy (ES) was applied to reconstruct transaxial slices of SPECT data. Evolutionary algorithms are stochastic global search methods that have been used successfully for many kinds of optimization problems. The newly developed reconstruction algorithm consisting of /spl mu/ parents and /spl lambda/ children uses a random principle to readjust the voxel values, whereas other iterative reconstruction methods use the difference between measured and simulated projection data. The (/spl mu/ + /spl lambda/)-ES was validated against a test image, a heart, and a Jaszczak phantom. The resulting transaxial slices show an improvement in image quality, in comparison to both the filtered back projection method and a standard iterative reconstruction algorithm.