Accelerating floating-point fitness functions in evolutionary algorithms: a FPGA-CPU-GPU performance comparison

Many large combinatorial optimization problems tackled with evolutionary algorithms often require very high computational times, usually due to the fitness evaluation. This fact forces programmers to use clusters of computers, a computational solution very useful for running applications of intensive calculus but having a high acquisition price and operation cost, mainly due to the Central Processing Unit (CPU) power consumption and refrigeration devices. A low-cost and high-performance alternative comes from reconfigurable computing, a hardware technology based on Field Programmable Gate Array devices (FPGAs). The main objective of the work presented in this paper is to compare implementations on FPGAs and CPUs of different fitness functions in evolutionary algorithms in order to study the performance of the floating-point arithmetic in FPGAs and CPUs that is often present in the optimization problems tackled by these algorithms. We have taken advantage of the parallelism at chip-level of FPGAs pursuing the acceleration of the fitness functions (and consequently, of the evolutionary algorithms) and showing the parallel scalability to reach low cost, low power and high performance computational solutions based on FPGA. Finally, the recent popularity of GPUs as computational units has moved us to introduce these devices in our performance comparisons. We analyze performance in terms of computation times and economic cost.     

On the solution of reservoir optimal control problems by means of vector computers

The experience on the use of vector computers for the solution of reservoir optimal control problems is described. The performance (in terms of CPU time) of four different vector computers is analyzed, along with the performance of other commonly used conventional (scalar) computers. It turns out that the solution of complex real-world problems is computationally feasible only when a very fast (vector) computer is available.     

Probabilistic analysis of combinatorial algorithms: A bibliography with selected annotations

Probabilistic methods in evaluation of performance efficiency of combinatorial optimization algorithms are of continuously growing interest, and rapidly increasing effort of researchers in this field is observed. The present paper is a bibliography which contains 70 references dealing with probabilistic evaluation of time complexity and performance accuracy of deterministic algorithms for combinatorial decision and optimization problems. Some entries of the bibliography, mainly those having appreared in journals and nonperiodical issues of limited distribution are shortly annotated (18 references). Basic notions and definitions facilitating better understanding and plain presentation of different results are given.     

Fast combinatorial optimization with parallel digital computers

This paper presents an algorithm which realizes fast search for the solutions of combinatorial optimization problems with parallel digital computers. With the standard weight matrices designed for combinatorial optimization, many iterations are required before convergence to a quasioptimal solution even when many digital processors can be used in parallel. By removing the components of the eigenvectors with eminent negative eigenvalues of the weight matrix, the proposed algorithm avoids oscillation and realizes energy reduction under synchronous discrete dynamics, which enables parallel digital computers to obtain quasi-optimal solutions with much less time than the conventional algorithm.     

Parallel genetic simulated annealing: a massively parallel SIMDalgorithm

Many significant engineering and scientific problems involve optimization of some criteria over a combinatorial configuration space. The two methods most often used to solve these problems effectively-simulated annealing (SA) and genetic algorithms (GA)-do not easily lend themselves to massive parallel implementations. Simulated annealing is a naturally serial algorithm, while GA involves a selection process that requires global coordination. This paper introduces a new hybrid algorithm that inherits those aspects of GA that lend themselves to parallelization, and avoids serial bottle-necks of GA approaches by incorporating elements of SA to provide a completely parallel, easily scalable hybrid GA/SA method. This new method, called Genetic Simulated Annealing, does not require parallelization of any problem specific portions of a serial implementation-existing serial implementations can be incorporated as is. Results of a study on two difficult combinatorial optimization problems, a 100 city traveling salesperson problem and a 24 word, 12 bit error correcting code design problem, performed on a 16 K PE MasPar MP-1, indicate advantages over previous parallel GA and SA approaches. One of the key results is that the performance of the algorithm scales up linearly with the increase of processing elements, a feature not demonstrated by any previous parallel GA or SA approaches, which enables the new algorithm to utilize massive parallel architecture with maximum effectiveness. Additionally, the algorithm does not require careful choice of control parameters, a significant advantage over SA and GA     

飞腾处理器上向量三角函数的设计实现与优化

得益于单指令多数据(single instruction multiple data, SIMD)向量化技术,处理器浮点计算能力获得了成倍的提升,然而当前SIMD向量部件和指令集仅支持加、减、乘、除、逻辑运算等基本操作,对浮点超越函数没有提供直接的支持.作为浮点计算中最耗时的一类函数,如何提高其性能成为底层数学库优化工作的一个重点.面向超越函数中的三角函数,提出一种利用SIMD向量部件设计、实现与优化向量三角函数的方法.该方法结合标量数学库分段计算与向量数学库向量化实现的优势,增加和优化了向量三角函数中的分支处理,既减少了函数实现中的冗余计算,又提高了分支情况下向量部件的利用率.在飞腾处理器上的实验表明:所提优化方法既保证了向量三角函数的精度,同时有效提高了函数性能,与原始向量三角函数相比平均性能加速比为2.04倍.     

A Two-Step Method for Solving Vector Optimization Problems on Permutation Configuration

Cybernetics and Systems Analysis - A class of problems of vector Euclidean combinatorial optimization is considered as problems of discrete optimization on the set of combinatorial configurations...     

提高链式Lin-Kernighan算法性能的策略

Lin-Kernighan算法作为一种高效的组合优化问题优化算法,普遍应用于各种求解组合优化难题的算法中,尤其是旅行商问题的求解。通过对该类问题的可化简性论述,分析并建立了该类问题初始边集的概率化简模型,经实验分析方式确定了模型中的先验性概率值,并建立旅行商化简初始边集的随机算法。将该算法建立的边集作为链式Lin-Kernighan算法的参照优化边集,大幅度提高了链式Lin-Kernighan算法的求解性能,在与多种智能算法结合中取得了较好的收敛效果。     

Efficient benchmarking of algorithm configurators via model-based surrogates

The optimization of algorithm (hyper-)parameters is crucial for achieving peak performance across a wide range of domains, ranging from deep neural networks to solvers for hard combinatorial problems. However, the proper evaluation of new algorithm configuration (AC) procedures (or configurators) is hindered by two key hurdles. First, AC scenarios are hard to set up, including the target algorithm to be optimized and the problem instances to be solved. Second, and even more significantly, they are computationally expensive: a single configurator run involves many costly runs of the target algorithm. Here, we propose a benchmarking approach that uses surrogate scenarios, which are computationally cheap while remaining close to the original AC scenarios. These surrogate scenarios approximate the response surface corresponding to true target algorithm performance using a regression model. In our experiments, we construct and evaluate surrogate scenarios for hyperparameter optimization as well as for AC problems that involve performance optimization of solvers for hard combinatorial problems. We generalize previous work by building surrogates for AC scenarios with multiple problem instances, stochastic target algorithms and censored running time observations. We show that our surrogate scenarios capture overall important characteristics of the original AC scenarios from which they were derived, while being much easier to use and orders of magnitude cheaper to evaluate.     

Generalized deterministic annealing

We develop a general formalism for computing high quality, low-cost solutions to nonconvex combinatorial optimization problems expressible as distributed interacting local constraints. For problems of this type, generalized deterministic annealing (GDA) avoids the performance-related sacrifices of current techniques. GDA exploits the localized structure of such problems by assigning K-state neurons to each optimization variable. The neuron values correspond to the probability densities of K-state local Markov chains and may be updated serially or in parallel; the Markov model is derived from the Markov model of simulated annealing (SA), although it is greatly simplified. Theorems are presented that firmly establish the convergence properties of GDA, as well as supplying practical guidelines for selecting the initial and final temperatures in the annealing process. A benchmark image enhancement application is provided where the performance of GDA is compared to other optimization methods. The empirical data taken in conjunction with the formal analytical results suggest that GDA enjoys significant performance advantages relative to current methods for combinatorial optimization.     

The `neural' phonetic typewriter

The factors that make speech recognition difficult are examined, and the potential of neural computers for this purpose is discussed. A speaker-adaptive system that transcribes dictation using an unlimited vocabulary is presented that is based on a neural network processor for the recognition of phonetic units of speech. The acoustic preprocessing, vector quantization, neural network model, and shortcut learning algorithm used are described. The utilization of phonotopic maps and of postprocessing in symbolic forms are discussed. Hardware implementations and performance of the neural networks are considered     

量子计算机:量子算法与物理实现

量子算法与物理实现是量子计算机研究中的两个基本问题。本文首先总结了相关领域的主要进展,并讨论了有代表性的量子算法,特别介绍了用于求解线性方程组的量子算法,分析了影响新量子算法提出的因素。然后,探讨了物理实现的迪文森佐判据,并介绍了典型的实现方案及性能比较。同时,也关注了对量子计算机研究持有异议的观点。最后,对量子计算机的新研究方向作了探讨。     

蚁群算法及其在求解旅行商问题中的应用

蚁群算法是一种求解组合优化问题较好的方法。在蚁群算法的基本原理基础上,以旅行商问题为例,介绍了该算法求解TSP的数学模型及具体步骤,并通过仿真实验与粒子群优化算法等方法比较分析,表明了该算法在求解组合优化问题方面具有良好的性能。     

Implementation issues in optimization algorithms: do they matter?

Two factors that have a major impact on the performance of an optimization method are (1) formal algorithm specifications and (2) practical implementations. The impact of the latter is typically ignored, although it defines the results measured in experiments. We present an in-depth study of algorithm implementation issues and ask questions such as Does optimizing the implementation of an optimization algorithm pay off? Do bugs matter? and Is using more complicated but also more efficient data structures worth the effort? The intuitive answer to all of these questions is yes, but there is little published evidence. To bridge this gap, we use one of the most studied combinatorial optimization problems – the Traveling Salesman Problem – as a test bed and implement two state-of-the-art approaches for solving it – the Lin-Kernighan Heuristic and an Ejection Chain Method. We investigate implementation effort and performance gain, in order to provide further insights to the above questions.     

An SMO algorithm for the potential support vector machine

We describe a fast sequential minimal optimization (SMO) procedure for solving the dual optimization problem of the recently proposed potential support vector machine (P-SVM). The new SMO consists of a sequence of iteration steps in which the Lagrangian is optimized with respect to either one (single SMO) or two (dual SMO) of the Lagrange multipliers while keeping the other variables fixed. An efficient selection procedure for Lagrange multipliers is given, and two heuristics for improving the SMO procedure are described: block optimization and annealing of the regularization parameter epsilon. A comparison of the variants shows that the dual SMO, including block optimization and annealing, performs efficiently in terms of computation time. In contrast to standard support vector machines (SVMs), the P-SVM is applicable to arbitrary dyadic data sets, but benchmarks are provided against libSVM's epsilon-SVR and C-SVC implementations for problems that are also solvable by standard SVM methods. For those problems, computation time of the P-SVM is comparable to or somewhat higher than the standard SVM. The number of support vectors found by the P-SVM is usually much smaller for the same generalization performance.     

Performance Analysis of Power-Aware Task Scheduling Algorithms on Multiprocessor Computers with Dynamic Voltage and Speed

Task scheduling on multiprocessor computers with dynamically variable voltage and speed is investigated as combinatorial optimization problems, namely, the problem of minimizing schedule length with energy consumption constraint and the problem of minimizing energy consumption with schedule length constraint. The first problem has applications in general multiprocessor computing systems where energy consumption is an important concern and in mobile computers where energy conservation is a main concern. The second problem has applications in real-time multiprocessing systems where timing constraint is a major requirement. These problems emphasize the tradeoff between power and performance and are defined such that the power-performance product is optimized by fixing one factor and minimizing the other. It is found that both problems are equivalent to the sum of powers problem and can be decomposed into two subproblems, namely, scheduling tasks and determining power supplies. Such decomposition makes design and analysis of heuristic algorithms tractable. We analyze the performance of list scheduling algorithms and equal-speed algorithms and prove that these algorithms are asymptotically optimal. Our extensive simulation data validate our analytical results and provide deeper insight into the performance of our heuristic algorithms.     

Vector pipelining, chaining, and speed on the IBM 3090 and CrayX-MP

Vector pipelining and chaining are clarified through the use of timing and pipeline diagrams of the instruction execution process. The technique for evaluating the performance of the concurrent vector operations of vector processors is evaluated by testing two of the most widely used computers with vector facilities: the IBM 3090 and Cray X-MP. On the basis of the testing results analyzed at the assembler level, suggestions are given for machine users and designers about vectorization on these two machines. The ideas presented can be applied to other vector processors. The actual implementations, however, may differ, depending on individual machine architecture     

支持向量学习的多参数同时调节

模型选择是支持向量学习的关键问题.已有模型选择方法采用嵌套的双层优化框架,内层执行支持向量学习,外层通过最小化泛化误差的估计进行模型选择.该框架过程复杂,计算效率低.简化传统的双层优化框架,提出一个支持向量学习的多参数同时调节方法,在同一优化过程中实现模型选择和学习器训练.首先,将支持向量学习中的参数和超参数合并为一个参数向量,利用序贯无约束极小化技术(sequential unconstrained minimization technique,简称SUMT)分别改写支持向量分类和回归的有约束优化问题,得到多参数同时调节模型的多元无约束形式定义;然后,证明多参数同时调节模型目标函数的局部Lipschitz连续性及水平集有界性.在此基础上,应用变尺度方法(variable metric method,简称VMM)设计并实现了多参数同时调节算法.进一步地,基于多参数同时调节模型的性质,证明了算法收敛性,对比分析了算法复杂性.最后,实验验证同时调节算法的收敛性,并实验对比同时调节算法的有效性.理论证明和实验分析表明,同时调节方法是一种坚实、高效的支持向量模型选择方法.     

量子退火算法研究进展

在数学和应用领域,量子退火算法是一类新的量子优化算法.不同于经典模拟退火算法利用热波动来搜寻问题的最优解,量子退火算法利用量子波动产生的量子隧穿效应来使算法摆脱局部最优,而实现全局优化.在已有的研究中,量子退火算法在某些问题上展现出良好的优化效果.系统地综述了量子退火算法的基本原理和近年来的主要研究进展,较为详细地介绍了几个主要的量子退火算法,对量子退火算法的优点和可能的不足进行了分析评述,并对今后的研究方向进行了展望.     

A genetic algorithm-based solution methodology for modular design

Combinatorial optimization problems usually have a finite number of feasible solutions. However, the process of solving these types of problems can be a very long and tedious task. Moreover, the cost and time for getting accurate and acceptable results is usually quite large. As the complexity and size of these problems grow, the current methods for solving problems such as the scheduling problem or the classification problem have become obsolete, and the need for an efficient method that will ensure good solutions for these complicated problems has increased. This paper presents a genetic algorithm (GA)-based method used in the solution of a set of combinatorial optimization problems. A definition of a combinatorial optimization problem is first given. The definition is followed by an introduction to genetic algorithms and an explanation of their role in solving combinatorial optimization problems such as the traveling salesman problem. A heuristic GA is then developed and used as a tool for solving various combinatorial optimization problems such as the modular design problem. A modularity case study is used to test and measure the performance of the developed algorithm.