A nonmonotone ODE-based method for unconstrained optimization

This paper presents a new hybrid algorithm for unconstrained optimization problems, which combines the idea of the IMPBOT algorithm with the nonmonotone line search technique. A feature of the proposed method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified limited-memory BFGS two loop recursion that requires only matrix–vector products, thus reducing the computations and storage. Furthermore, when the trial step is not accepted, the proposed method performs a line search along it using a modified nonmonotone scheme, thus a larger stepsize can be yielded in each line search procedure. Under some reasonable assumptions, the convergence properties of the proposed algorithm are analysed. Numerical results are also reported to show the efficiency of this proposed method.     

A nonmonotone scaled conjugate gradient algorithm for large-scale unconstrained optimization

This paper proposes a nonmonotone scaled conjugate gradient algorithm for solving large-scale unconstrained optimization problems, which combines the idea of scaled memoryless Broyden–Fletcher–Goldfarb–Shanno preconditioned conjugate gradient method with the nonmonotone technique. An attractive property of the proposed method is that the search direction always provides sufficient descent step at each iteration. This property is independent of the line search used. Under appropriate assumptions, the method is proven to possess global convergence for nonconvex smooth functions, and R-linear convergence for strongly convex functions. Preliminary numerical results and related comparisons show the efficiency of the proposed method in practical computation.     

A nonmonotone SQP-filter method for equality constrained optimization

In this paper, we propose a nonmonotone sequential quadratic programming-filter method for solving nonlinear equality constrained optimization. This new method has more flexibility for the acceptance of the trial step and requires less computational costs compared with the monotone methods. Under reasonable conditions, we give the global convergence properties. Further, the second-order correction step and nonmonotone reduction conditions are used to overcome Maratos effect so that quadratic local convergence is achieved. The numerical experiments are reported to show the effectiveness of the proposed algorithm.     

A new cubic convergent method for solving a system of nonlinear equations

We present a new cubic convergent method for solving a system of nonlinear equations. The new method can be viewed as a modified Chebyshev's method in which the difference of Jacobian matrixes replaces three order tensor. Therefore, the new method reduces the storage and computational cost. The new method possesses the local cubic convergence as well as Chebyshev's method. A rule is deduced to ensure the descent property of the search direction, and a nonmonotone line search technique is used to guarantee the global convergence. Numerical results indicate that the new method is competitive and efficient for some classical test problems.     

A trust-region algorithm combining line search filter technique for nonlinear constrained optimization

In this paper, we propose a trust-region algorithm in association with line search filter technique for solving nonlinear equality constrained programming. At current iteration, a trial step is formed as the sum of a normal step and a tangential step which is generated by trust-region subproblem and the step size is decided by interior backtracking line search together with filter methods. Then, the next iteration is determined. This is different from general trust-region methods in which the next iteration is determined by the ratio of the actual reduction to the predicted reduction. The global convergence analysis for this algorithm is presented under some reasonable assumptions and the preliminary numerical results are reported.     

A scaled conjugate gradient method for nonlinear unconstrained optimization

We propose a new optimization problem which combines the good features of the classical conjugate gradient method using some penalty parameter, and then, solve it to introduce a new scaled conjugate gradient method for solving unconstrained problems. The method reduces to the classical conjugate gradient algorithm under common assumptions, and inherits its good properties. We prove the global convergence of the method using suitable conditions. Numerical results show that the new method is efficient and robust.     

A derivative-free affine scaling trust region methods based on probabilistic models with new nonmonotone line search technique for linear inequality constrained minimization without strict complementarity

ABSTRACT

In this paper, a derivative-free trust region methods based on probabilistic models with new nonmonotone line search technique is considered for nonlinear programming with linear inequality constraints. The proposed algorithm is designed to build probabilistic polynomial interpolation models for the objective function. We build the affine scaling trust region methods which use probabilistic or random models within a classical trust region framework. The new backtracking linear search technique guarantee the descent of the objective function, and new iterative points are in the feasible region. In order to overcome the strict complementarity hypothesis, under some reasonable conditions which are weaker than strong second order sufficient condition, we give the new and more simple identification function to structure the affine matrix. The global and local fast convergence of the algorithm are shown and the results of numerical experiments are reported to show the effectiveness of the proposed algorithm.     

A second order method for unconstrained optimization

An algorithm using second derivatives for solving unconstrained optimization problems is presented. In this brief note the descent direction of the algorithm is based on a modification of the Newton direction, while the Armijo rule for choosing the stepsize is used. The rate of convergence of the algorithm is shown to be superlinear. Our computational experience shows that the method performs quite well and our numerical results are presented in Section 4.     

On the worst-case complexity of nonlinear stepsize control algorithms for convex unconstrained optimization

A Nonlinear Stepsize Control (NSC) framework has been proposed by Toint [Nonlinear stepsize control, trust regions and regularizations for unconstrained optimization, Optim.Methods Softw. 28 (2013), pp. 82–95] for unconstrained optimization, generalizing many trust-region and regularization algorithms. More recently, worst-case complexity bounds for the generic NSC framework were proved by Grapiglia et al. [On the convergence and worst-case complexity of trust-region and regularization methods for unconstrained optimization, Math. Program. 152 (2015), pp. 491–520] in the context of non-convex problems. In this paper, improved complexity bounds are obtained for convex and strongly convex objectives.     

A new filled function method for unconstrained global optimization

Many real world problems can be modelled as optimization problems. However, the traditional algorithms for these problems often encounter the problem of being trapped in local minima. The filled function method is an effective approach to tackle this kind of problems. However the existing filled functions have the disadvantages of discontinuity, non-differentiability or sensitivity to parameters which limit their efficiency. In this paper, we proposed a new filled function which is continuous and differentiable without any parameter to tune. Compared to discontinuous or non-differentiable filled functions, the continuous and differentiable filled function mainly has three advantages: firstly, it is not easier to produce extra local minima, secondly, more efficient local search algorithms using gradient information can be applied and thirdly, a continuous and differentiable filled function can be optimized more easily. Based on the new proposed filled function, a new algorithm was designed for unconstrained global optimization problems. Numerical experiments were conducted and the results show the proposed algorithm was more efficient.     

A restarted and modified simplex search for unconstrained optimization

In this paper we propose a simple but efficient modification of the well-known Nelder–Mead (NM) simplex search method for unconstrained optimization. Instead of moving all n simplex vertices at once in the direction of the best vertex, our “shrink” step moves them in the same direction but one by one until an improvement is obtained. In addition, for solving non-convex problems, we simply restart the so-modified NM (MNM) method by constructing an initial simplex around the solution obtained in the previous phase. We repeat restarts until there is no improvement in the objective function value. Thus, our restarted modified NM (RMNM) is a descent and deterministic method and may be seen as an extended local search for continuous optimization. In order to improve computational complexity and efficiency, we use the heap data structure for storing and updating simplex vertices. Extensive empirical analysis shows that: our modified method outperforms in average the original version as well as some other recent successful modifications; in solving global optimization problems, it is comparable with the state-of-the-art heuristics.     

A new spectral conjugate gradient method for large-scale unconstrained optimization

The spectral conjugate gradient methods, with simple construction and nice numerical performance, are a kind of effective methods for solving large-scale unconstrained optimization problems. In this paper, based on quasi-Newton direction and quasi-Newton condition, and motivated by the idea of spectral conjugate gradient method as well as Dai-Kou's selecting technique for conjugate parameter [SIAM J. Optim. 23 (2013), pp. 296–320], a new approach for generating spectral parameters is presented, where a new double-truncating technique, which can ensure both the sufficient descent property of the search directions and the bounded property of the sequence of spectral parameters, is introduced. Then a new associated spectral conjugate gradient method for large-scale unconstrained optimization is proposed. Under either the strong Wolfe line search or the generalized Wolfe line search, the proposed method is always globally convergent. Finally, a large number of comparison numerical experiments on large-scale instances from one thousand to two million variables are reported. The numerical results show that the proposed method is more promising.     

非单调信赖域方法求解无约束非光滑优化问题

提出了非单调信赖域算法求解无约束非光滑优化问题,并和经典的信赖域方法作比较分析。同时,设定了一些条件,在这些假设条件下证明了该算法是整体收敛的。数值实验结果表明,非单调策略对无约束非光滑优化问题的求解是行之有效的,拓展了非单调信赖域算法的应用领域。     

一种求解无约束优化问题的非单调信赖域算法

提出了非单调信赖域算法求解基于锥模型的无约束优化问题,该算法在求解信赖域子问题时充分利用了当前迭代点的一阶梯度信息。提出了一个新的信赖域半径的选取机制,并和经典的信赖域方法作比较分析。设定了一些条件,在这些假设条件下证明了算法是整体收敛的。数值实验结果表明,该算法对基于锥模型的无约束优化问题的求解是行之有效的,拓展了非单调信赖域算法的应用领域。     

Sufficient descent conjugate gradient methods for large-scale optimization problems

Sufficient descent condition is very crucial in establishing the global convergence of nonlinear conjugate gradient method. In this paper, we modified two conjugate gradient methods such that both methods satisfy this property. Under suitable conditions, we prove the global convergence of the proposed methods. Numerical results show that the proposed methods are efficient for the given test problems.     

A new trust-region method with line search for solving symmetric nonlinear equations

A new trust-region method is proposed for symmetric nonlinear equations. In this given algorithm, if the trial step is unsuccessful, one line search will be used instead of repeatedly solving the subproblem of the normal trust-region method. Moreover, the global convergence is established under mild conditions by a new way. The quadratic convergence of the presented method is also proved. Numerical results show that the method is interesting for the given problems.     

解大规模无约束优化的自适应过滤信赖域法

提出一种解大规模无约束优化问题的自适应过滤信赖域法。用目标函数的梯度及迭代点的信息来构造目标函数海赛矩阵的近似数量矩阵,引进了过滤技术和自适应技术,大大提高了计算效率。从理论上证明了新算法的全局收敛性,数值试验结果也表明了新算法的有效性。     

高效的线搜索寻优方法

在涉及计算机寻优等许多工程领域,都需要使用多元函数的最优化。线搜索是多元函数的最优化中已知搜索方向求最优步长的关键技术。为了提出一种高效的线搜索算法,对线搜索进行详细研究,提出一种新的线搜索寻优方法--类康托法。主要方法是去除了Fibonacci法中两个试探点必须保留一个的限制,每次把搜索区间三等分,根据试探点的导数值,来决定去除哪两个子区间。通过理论和实例的证明,结果发现类康托法比0.618法和Fibonacci法更高效,计算速度更快。其中最重要的结论是类康托法为这两种方法收敛速度的高阶无穷小。特别是在精度要求很高的时候,类康托法比这两种算法具有更明显的优势。此外,该方法具有较强的适用性,不但能用于凸函数,也能用于凹函数。     

A spectral conjugate gradient method for solving large-scale unconstrained optimization

This paper establishes a spectral conjugate gradient method for solving unconstrained optimization problems, where the conjugate parameter and the spectral parameter satisfy a restrictive relationship. The search direction is sufficient descent without restarts in per-iteration. Moreover, this feature is independent of any line searches. Under the standard Wolfe line searches, the global convergence of the proposed method is proved when $\left|{\beta }_k\right|\le {\beta }_k^{FR}$ holds. The preliminary numerical results are presented to show effectiveness of the proposed method.