An adaptive single-point algorithm for global numerical optimization

This paper describes a novel algorithm for numerical optimization, called Simple Adaptive Climbing (SAC). SAC is a simple efficient single-point approach that does not require a careful fine-tunning of its two parameters. SAC algorithm shares many similarities with local optimization heuristics, such as random walk, gradient descent, and hill-climbing. SAC has a restarting mechanism, and a powerful adaptive mutation process that resembles the one used in Differential Evolution. The algorithms SAC is capable of performing global unconstrained optimization efficiently in high dimensional test functions. This paper shows results on 15 well-known unconstrained problems. Test results confirm that SAC is competitive against state-of-the-art approaches such as micro-Particle Swarm Optimization, CMA-ES or Simple Adaptive Differential Evolution.     

一个松弛非单调线搜索法

对于无约束优化问题,我们用松弛技术改进了一般非单调线搜索准则,建立了相应的求解算法,并证明了算法的整体收敛性.部分数值实验结果表明,这个松弛非单调算法有效.     

Parameter sensitivity study of the Nelder-Mead Simplex Method

This paper presents a parameter sensitivity study of the Nelder-Mead Simplex Method for unconstrained optimization. Nelder-Mead Simplex Method is very easy to implement in practice, because it does not require gradient computation; however, it is very sensitive to the choice of initial points selected. Fan-Zahara conducted a sensitivity study using a select set of test cases and suggested the best values for the parameters based on the highest percentage rate of successful minimization. Begambre-Laier used a strategy to control the Particle Swarm Optimization parameters based on the Nelder Mead Simplex Method in identifying structural damage. The main purpose of the paper is to extend their parameter sensitivity study to better understand the parameter’s behavior. The comprehensive parameter sensitivity study was conducted on seven test functions: B2, Beale, Booth, Wood, Rastrigin, Rosenbrock and Sphere Functions to search for common patterns and relationships each parameter has in producing the optimum solution. The results show important relations of the Nelder-Mead Simplex parameters: reflection, expansion, contraction, and Simplex size and how they impact the optimum solutions. This study is crucial, because better understanding of the parameters behavior can motivate current and future research using Nelder-Mead Simplex in creating an intelligent algorithm, which can be more effective, efficient, and save computational time.     

A poly-hybrid PSO optimization method with intelligent parameter adjustment

This paper presents results on a new hybrid optimization method which combines the best features of four traditional optimization methods together with an intelligent adjustment algorithm to speed convergence on unconstrained and constrained optimization problems. It is believed that this is the first time that such a broad array of methods has been employed to facilitate synergistic enhancement of convergence. Particle swarm optimization is based on swarm intelligence inspired by the social behavior and movement dynamics of bird flocking, fish schooling, and swarming theory. This method has been applied for structural damage identification, neural network training, and reactive power optimization. It is also believed that this is the first time an intelligent parameter adjustment algorithm has been applied to maximize the effectiveness of individual component algorithms within the hybrid method. A comprehensive sensitivity analysis of the traditional optimization methods within the hybrid group is used to demonstrate how the relationship among the design variables in a given problem can be used to adjust algorithm parameters. The new method is benchmarked using 11 classical test functions and the results show that the new method outperforms eight of the most recently published search methodologies.     

Improved conjugate gradient implementation for least squares support vector machines

As a promising method for pattern recognition and function estimation, least squares support vector machines (LS-SVM) express the training in terms of solving a linear system instead of a quadratic programming problem as for conventional support vector machines (SVM). In this paper, by using the information provided by the equality constraint, we transform the minimization problem with a single equality constraint in LS-SVM into an unconstrained minimization problem, then propose reduced formulations for LS-SVM. By introducing this transformation, the times of using conjugate gradient (CG) method, which is a greatly time-consuming step in obtaining the numerical solution, are reduced to one instead of two as proposed by Suykens et al. (1999). The comparison on computational speed of our method with the CG method proposed by Suykens et al. and the first order and second order SMO methods on several benchmark data sets shows a reduction of training time by up to 44%.     

The orbital minimization method for electronic structure calculations with finite-range atomic basis sets

The implementation of the orbital minimization method (OMM) for solving the self-consistent Kohn–Sham (KS) problem for electronic structure calculations in a basis of non-orthogonal numerical atomic orbitals of finite-range is reported. We explore the possibilities for using the OMM as an exact cubic-scaling solver for the KS problem, and compare its performance with that of explicit diagonalization in realistic systems. We analyze the efficiency of the method depending on the choice of line search algorithm and on two free parameters, the scale of the kinetic energy preconditioning and the eigenspectrum shift. The results of several timing tests are then discussed, showing that the OMM can achieve a noticeable speedup with respect to diagonalization even for minimal basis sets for which the number of occupied eigenstates represents a significant fraction of the total basis size (>15%). We investigate the hard and soft parallel scaling of the method on multiple cores, finding a performance equal to or better than diagonalization depending on the details of the OMM implementation. Finally, we discuss the possibility of making use of the natural sparsity of the operator matrices for this type of basis, leading to a method that scales linearly with basis size.     

基于树结构分层随机森林在非约束环境下的头部姿态估计

头部姿态估计是人类行为和注意力的关键,受到光照、噪声、身份、遮挡等许多因素的影响。为了提高非约束环境下的估计准确率和鲁棒性,该论文提出了树结构分层随机森林在非约束环境下的多类头部姿态估计。首先,为了消除不同环境的噪声影响,提取人脸区域的组合纹理特征,对人脸区域进行积极人脸子区域的分类,分类结果作为树结构分层随机森林的先验知识输入;其次,提出了一种树结构分层随机森林算法,分层估计多自由度下的头部姿态;再次,为了增强算法的分类能力,使用自适应高斯混合模型作为多层次子森林叶子节点的投票模型。在多个公共数据集上的多种非约束实验环境下进行头部姿态估计,最终实验结果表明所提算法在不同质量的图像上都有很好的估计准确率和鲁棒性。     

Evolutionary training of hardware realizable multilayer perceptrons

The use of multilayer perceptrons (MLP) with threshold functions (binary step function activations) greatly reduces the complexity of the hardware implementation of neural networks, provides tolerance to noise and improves the interpretation of the internal representations. In certain case, such as in learning stationary tasks, it may be sufficient to find appropriate weights for an MLP with threshold activation functions by software simulation and, then, transfer the weight values to the hardware implementation. Efficient training of these networks is a subject of considerable ongoing research. Methods available in the literature mainly focus on two-state (threshold) nodes and try to train the networks by approximating the gradient of the error function and modifying appropriately the gradient descent, or by progressively altering the shape of the activation functions. In this paper, we propose an evolution-motivated approach, which is eminently suitable for networks with threshold functions and compare its performance with four other methods. The proposed evolutionary strategy does not need gradient related information, it is applicable to a situation where threshold activations are used from the beginning of the training, as in “on-chip” training, and is able to train networks with integer weights.     

The IAM-database: an English sentence database for offline handwriting recognition

In this paper we describe a database that consists of handwritten English sentences. It is based on the Lancaster-Oslo/Bergen (LOB) corpus. This corpus is a collection of texts that comprise about one million word instances. The database includes 1,066 forms produced by approximately 400 different writers. A total of 82,227 word instances out of a vocabulary of 10,841 words occur in the collection. The database consists of full English sentences. It can serve as a basis for a variety of handwriting recognition tasks. However, it is expected that the database would be particularly useful for recognition tasks where linguistic knowledge beyond the lexicon level is used, because this knowledge can be automatically derived from the underlying corpus. The database also includes a few image-processing procedures for extracting the handwritten text from the forms and the segmentation of the text into lines and words. Received September 28, 2001 / Revised October 10, 2001     

一类非单调信赖域算法

给出无约束最优化的一类非单调信赖域算法.该算法是对赵英良提出的算法的改进和推广.为了提高这类算法的迭代速度,本文中在rk<0时放大了fl(k)的取值范围.这样可以更快的迭代到rk>0.从而放宽了算法的整体约束条件.另外,赵英良提出的算法在‖δ(k)‖≤σ‖g(k)‖成立的条件下证明了算法的全局收敛性和超线性收敛速度.本文试图去掉此约束条件仍得到算法的全局收敛性及其超线性收敛速度,从而推广了非单调信赖域方法的应用范围.