Evolutionary Optimisation of Mechanical Structures: Towards an Integrated Optimisation

This work results from the synthesis of author’s works on the applications of stochastic techniques (genetic algorithms with neural networks) for the optimisation of mechanical structures. The emphasis of this work is on the practical aspects and the feasibility of the aformentioned techniques. The research strategy consists in substituting, for finite element calculations in the optimisation process, an approximate response of a neural network. More precisely, the paper describes the use of backpropagation neural networks in creating function approximations for use in computationally intensive design optimisation based on genetic algorithms. An example of application for space frame optimisation of a helicopter tail boom is given in this paper, for which we can talk of integrated optimisation. This example (including displacement and frequency constraints) show the use of neural networks as a function approximation strategy to limit the computational costs associated with stochastic search methods.     

Evolution strategies for solving discrete optimization problems

A method to solve discrete optimization problems using evolution strategies (ESs) is described. The ESs imitate biological evolution in nature and have two characteristics that differ from other conventional optimization algorithms: (a) ESs use randomized operators instead of the usual deterministic ones; (b) instead of a single design point, the ESs work simultaneously with a population of design points in the space of variables. The important operators of ESs are mutation, selection and recombination. The ESs are commonly applied for continuous optimization problems. For the application to discrete problems, several modifications on the operators mutation and recombination are suggested here. Several examples from the literature are solved with this modified ES and the results compared. The examples show that the modified ES is robust and suitable for discrete optimization problems.     

A heuristic method for learning Bayesian networks using discrete particle swarm optimization

Bayesian networks are a powerful approach for representing and reasoning under conditions of uncertainty. Many researchers aim to find good algorithms for learning Bayesian networks from data. And the heuristic search algorithm is one of the most effective algorithms. Because the number of possible structures grows exponentially with the number of variables, learning the model structure from data by considering all possible structures exhaustively is infeasible. PSO (particle swarm optimization), a powerful optimal heuristic search algorithm, has been applied in various fields. Unfortunately, the classical PSO algorithm only operates in continuous and real-valued space, and the problem of Bayesian networks learning is in discrete space. In this paper, two modifications of updating rules for velocity and position are introduced and a Bayesian networks learning based on binary PSO is proposed. Experimental results show that it is more efficient because only fewer generations are needed to obtain optimal Bayesian networks structures. In the comparison, this method outperforms other heuristic methods such as GA (genetic algorithm) and classical binary PSO.     

一种低复杂度的基于进化策略的自适应运动估计方法

为了进一步提高编码质量并能快速编码,提出了一种新的基于进化策略的自适应运动估计算法。鉴于在进化策略中变异操作与正态分布法则对应,是核心算子,为此将进化策略应用于运动估计,提出了一种新的自适应运动估计算法,并第1次将运动方向信息作为变量引入运动估计算法,同时改进了步长自适应控制机制,以便进一步提高算法的收敛速率,同时采用种群规模的自适应控制,降低了算法的复杂度。试验结果表明,该算法的性能与全搜索算法相近,而复杂度略大于三步法。由于其具有低复杂度和进化算法的内在并行性的特点,故该算法适合硬件实现。     

A hybrid CMA-ES and HDE optimisation algorithm with application to solar energy potential

This paper describes the results of initial experiments to apply computational algorithms to explore a large parameter space containing many variables in the search for an optimal solution for the sustainable design of an urban development using a potentially complicated fitness function. This initial work concentrates on varying the placement of buildings to optimise solar irradiation availability. For this we propose a hybrid of the covariance matrix adaptation evolution strategy (CMA-ES) and hybrid differential evolution (HDE) algorithms coupled with an efficient backwards ray tracing technique. In this paper we concentrate on the formulation of the new hybrid algorithm and its testing using standard benchmarks as well as a solar optimisation problem. The new algorithm outperforms both the standalone CMA-ES and HDE algorithms in benchmark tests and an alternative multi-objective optimisation tool in the case of the solar optimisation problem.     

Hybridization of genetic algorithm with immune system for optimization problems in structural engineering

Optimization is the task of getting the best solution among the feasible solutions. There are many methods available to obtain an optimized solution. Genetic algorithm (GA), which is a heuristic type of optimization method, is discussed in this paper. The focus of the paper is the use of GA for large dimensionality design problems, where computational efficiency is a major concern. The motivation of this paper is to hybridize GA with an immune system mechanism by avoiding the implementation of penalty constants, which are highly sensitive to the choice of algorithm parameters. The principal advantage of the immune system is in its seamless integration with GA-based search for optimal design. It is being hybridized with the immune system mechanism. The hybrid GA and immune system is applied for the design of the optimal mix of high-performance concrete (HPC), which is still based on trial mix and for which no rigorous mathematical approach is available. As such, to infer the values of strength and slump, a wavelet back propagation neural network or wavelet neural network is used for any HPC mix. It is necessary to minimize the cost of HPC/unit weight of HPC subjected to strength and slump constraints. The interwoven algorithm is also applied to obtain optimal sectional areas for minimum weight of space trusses subjected to static loading. Formian programming language is used for the generation of the space trusses, and Feast package is used for the static analysis of the trusses. In addition to the induction of immune system in the GA for constraint handling, it is being applied in this particular application for improving the search of GA in obtaining the best optimal solution. For obtaining the optimal sections of space trusses subjected to earthquake loading, SAP 90 package is used, and reliable results are obtained.     

Evolution-based design of neural fuzzy networks using self-adaptinggenetic parameters

In this paper, an evolution-based approach to design of neural fuzzy networks is presented. The proposed strategy optimizes the whole fuzzy system with minimum rule number according to given specifications, while training the network parameters. The approach relies on an optimization tool, which combines evolution strategies and simulated annealing algorithms in finding the global optimum solution. The optimization variables include membership function parameters and rule numbers which are combined with genetic parameters to create diversity in the search space due to self-adaptation. The optimization technique is independent of the topology under consideration and capable of handling any type of membership function. The algorithmic details of the optimization methodology are discussed in detail, and the generality of the approach is illustrated by different examples     

Layout optimization of looped networks by constrained ant colony optimisation algorithm

A constrained version of ant colony optimisation algorithm (ACOA) is proposed in this paper for layout optimization of looped water distribution networks. A novel formulation is used to represent the layout optimization problem of pipe networks in the proper form required for the application of the ant algorithm. The proposed formulation is based on the engineering concept of reliability in which the number of independent paths from the source node to each of the network nodes is considered as a measure of reliability. In the proposed formulation, the ants are constrained to choose from the options provided by a constraining procedure so that only looped layouts are constructed by the ant leading to huge reduction of search space size compared to the original search space. Three different constraining procedures are used leading to three different algorithms. The proposed methods are used to find the optimal layout of three benchmark examples from the literature and the results are presented and compared to the results of the conventional ant colony optimization algorithm. The results show the efficiency and effectiveness of the proposed method for optimal layout determination of looped networks.     

Improving neural network models of physical systems through dimensional analysis

The paper presents a technique for generating concise neural network models of physical systems. The neural network models are generated through a two-stage process. The first stage uses information embedded in the dimensions or units in which the data is represented. Dimensional analysis techniques are used initially to make this information explicit, and a limited search in the neural network architecture space is then conducted to determine dimensionless representations of variables/parameters that perform well for a given model complexity. The second stage uses information available in the numerical values of the data to search for high-level dimensionless variables/parameters, generated from simple combinations of dimensionless quantities generated in the first stage and which result in concise neural network models with improved performance characteristics. The search for these high-level dimensionless variables/parameters is conducted in an enhanced representation space using functional link networks with flat or near flat architectures. The use and effectiveness of the technique is demonstrated for three applications. The first is the design and analysis of reinforced concrete beams, which is representative of the class of problems associated with the design and analysis of composites. The second is the classical elastica problem, for predicting non-linear post-buckled behaviour of columns and the third, the analysis of a bent bar under a specified combination of loads.     

Bayes网络学习的MCMC方法

基于Bayes统计理论, 提出了一种从数据样本中学习Bayes网络的Markov链Monte Carlo(MCMC)方法. 首先通过先验概率和数据样本的结合得到未归一化的后验概率, 然后使用此后验概率指导随机搜索算法寻找“好”的网络结构模型. 通过对Alarm网络的学习表明了本算法具有较好的性能.     

Neural networks applications in concrete structures

This paper presents and discusses the applications of neural networks in concrete structures. It aims at introducing neural networks applications in structural design. The paper covers two applications of neural networks in concrete structures. Backpropagation networks are chosen for the proposed network, which is written using the programming package MAT-LAB. The overall results are compared and observed for the performance of the networks. Based on the applications it was found that neural networks are comparatively effective for a number of reasons, which include the amount of CPU memory consumed by neural networks is less than that consumed by conventional methods and their ease of use and implementation, neural networks provide both the users and the developers more flexibility to cope with different kinds of problems.     

Developing Complex Systems Using Evolved Pattern Generators

Self-organization of connection patterns within brain areas of animals begins prenatally, and has been shown to depend on internally generated patterns of neural activity. The neural structures continue to develop postnatally through externally driven patterns, when the sensory systems are exposed to stimuli from the environment. The internally generated patterns have been proposed to give the neural system an appropriate bias so that it can learn reliably from complex environmental stimuli. This paper evaluates the hypothesis that complex artificial learning systems can benefit from a similar approach, consisting of initial training with patterns from an evolved pattern generator, followed by training with the actual training set. To test this hypothesis, competitive learning networks were trained for recognizing handwritten digits. The results demonstrate how the approach can improve learning performance by discovering the appropriate initial weight biases, thereby compensating for weaknesses of the learning algorithm. Due to the smaller evolutionary search space, this approach was also found to require much fewer generations than direct evolution of network weights. Since discovering the right biases efficiently is critical for solving large-scale problems with learning, these results suggest that internal training pattern generation is an effective method for constructing complex systems     

Automated evolutionary synthesis matching

This paper discusses the subject of automatic evolutionary sound matching: systems in which evolutionary algorithms are used to automatically derive the parameters of a synthesiser to produce a sound that matches a specified target sound. The paper describes prior work and identifies the principal causes of match inaccuracy, which are often due to optimiser limitations as a result of search space problem difficulty. The components of evolutionary matching systems contributing to problem difficulty are discussed and suggestions as to how improvements can be made through problem simplification or optimiser sophistication are considered. Subsequently, a novel clustering evolution strategy is presented which enables the concurrent optimisation of multiple distinct search space solutions, intended for the purposes of sound matching with standard frequency modulation (FM) synthesisers. The algorithm is shown to outperform standard multi-membered and multi-start (1?+?1) evolution strategies in application to different FM synthesis models for static and dynamic sounds. The comparative study makes use of a contrived matching method, which ensures that results are not affected by the limitations of the matching synthesiser.     

Effects of phenotypic redundancy in structure optimization

Concepts from graph theory and molecular evolution are proposed for analyzing the redundancy in the genotype-phenotype mapping in structure optimization stemming from graph isomorphism. Evolutionary topology optimization of neural networks serves as an example. By means of analytical and random-walk methods, it is shown that rare and frequent structures influence the search process: operators that are unbiased in genotype space may have a remarkable bias in phenotype space. In particular, if the desired structures are rare, the probability that an evolutionary algorithm evolves them may decrease. This is verified experimentally by comparing evolutionary structure optimization algorithms with and without search operators that take the redundancy of phenotypes into account. Further, it is shown how different encodings and restrictions on the search space lead to qualitatively different distributions of rare and frequent structures     

Optimisation of algorithm control parameters in cultural differential evolution applied to molecular crystallography

Evolutionary search and optimisation algorithms have been used successfully in many areas of materials science and chemistry. In recent years, these techniques have been applied to, and revolutionised the study of crystal structures from powder diffraction data. In this paper we present the application of a hybrid global optimisation technique, cultural differential evolution (CDE), to crystal structure determination from powder diffraction data. The combination of the principles of social evolution and biological evolution, through the pruning of the parameter search space shows significant improvement in the efficiency of the calculations over traditional dictates of biological evolution alone. Results are presented in which a range of algorithm control parameters, i.e., population size, mutation and recombination rates, extent of culture-based pruning are used to assess the performance of this hybrid technique. The effects of these control parameters on the speed and efficiency of the optimisation calculations are discussed, and the potential advantages of the CDE approach demonstrated through an average 40% improvement in terms of speed of convergence of the calculations presented, and a maximum gain of 68% with larger population size.     

Efficient multi-objective optimal design of water distribution networks on a budget of simulations using hybrid algorithms

The design of water distribution networks is a large-scale combinatorial, non-linear optimisation problem, involving many complex implicit constraint sets, such as nodal mass balance and energy conservation, which are commonly satisfied through the use of hydraulic network solvers. These problem properties have motivated several prior studies to use stochastic search optimisation, because these derivative-free global search algorithms have been shown to obtain higher quality solutions for large network design problems. Global stochastic search methods, however, require many iterations to be performed in order to achieve a satisfactory solution, and each iteration may involve running computationally expensive simulations. Recently, this problem has been compounded by the evident need to embrace more than a single measure of performance into the design process, since by nature multi-objective optimisation methods require even more iterations. The use of metamodels as surrogates for the expensive simulation functions has been investigated as a possible remedy to this problem. However, the identification of reliable surrogates is not always a viable alternative. Under these circumstances, methods that are capable of achieving a satisfactory level of performance with a limited number of function evaluations represent a valuable alternative. This paper represents a first step towards filling this gap. Two recently introduced multi-objective, hybrid algorithms, ParEGO and LEMMO, are tested on the design problem of a real medium-size network in Southern Italy, and a real large-size network in the UK under a scenario of a severely restricted number of function evaluations. The results obtained suggest that the use of both algorithms, in particular LEMMO, could be successfully extended to the efficient design of large-scale water distribution networks.     

Structural topology and element sizing design optimisation of tall steel frameworks using a hybrid OC–GA method

This paper presents a hybrid optimisation method in which a local search operator based on a rigorously derived optimality criteria (OC) technique is embedded in the framework of a genetic algorithm (GA). The GA framework is particularly useful in the global exploration for optimal topologies, while the OC technique serves as a local search operator for efficient element sizing optimisation of given topologies. The hybrid OC–GA method was developed to strike a balance between the exploration of global search algorithms and the exploitation of efficient local search methods so as to make the hybrid method suitable for optimising tall building structures involving a large number of structural elements. The applicability and efficiency of the hybrid OC–GA method were tested with two 40-storey steel frameworks. The results show that the hybrid method can generate superior designs to pure GA while exhibiting rapid and smooth convergence, suggesting its great potential for optimising both structural form and element size of practical tall building structures.     

A multilevel parallel processing approach to scene labelingproblems

A parallel tree search procedure and multilevel array architectures are presented for scene labeling problems. In the scene labeling problem, when the number of variables is not large, e.g. less than 100, its solutions and search space are not expected to greatly increase with problem size. The multilevel arrays only use a polynomial number of processors at each level and a bus or mesh connection for interlevel communication. This approach is very efficient compared to the binary-tree machine if the number of nodes in the search tree of labeling problem increases polynomially with the number of variables. Because of its regular structure, the multilevel array is particularly suitable for VLSI implementation     

深度神经网络结构搜索综述

深度神经网络在图像识别、语言识别和机器翻译等人工智能任务中取得了巨大进展,很大程度上归功于优秀的神经网络结构设计。神经网络大都由手工设计,需要专业的机器学习知识以及大量的试错。为此,自动化的神经网络结构搜索成为研究热点。神经网络结构搜索（neural architecture search,NAS）主要由搜索空间、搜索策略与性能评估方法3部分组成。在搜索空间设计上,出于计算量的考虑,通常不会搜索整个网络结构,而是先将网络分成几块,然后搜索块中的结构。根据实际情况的不同,可以共享不同块中的结构,也可以对每个块单独搜索不同的结构。在搜索策略上,主流的优化方法包含强化学习、进化算法、贝叶斯优化和基于梯度的优化等。在性能评估上,为了节省计算时间,通常不会将每一个网络都充分训练到收敛,而是通过权值共享、早停等方法尽可能减小单个网络的训练时间。与手工设计的网络相比,神经网络结构搜索得到的深度神经网络具有更好的性能。在ImageNet分类任务上,与手工设计的MobileNetV2相比,通过神经网络结构搜索得到的MobileNetV3减少了近30%的计算量,并且top-1分类精度提升了3.2%;在Cityscapes语义分割任务上,与手工设计的DeepLabv3+相比,通过神经网络结构搜索得到的Auto-DeepLab-L可以在没有ImageNet预训练的情况下,达到比DeepLabv3+更高的平均交并比（mean intersection over union,mIOU）,同时减小一半以上的计算量。神经网络结构搜索得到的深度神经网络通常比手工设计的神经网络有着更好的表现,是未来神经网络设计的发展趋势。