A method for non-differentiable optimization problems

We introduce a new method for solving a class of non-smooth unconstrained optimization problems. The method constructs a sequence<texlscub>x _k </texlscub>in ? ⁿ , and at the kth iteration, a search direction h _k is considered, where h _k is a solution to a variational inequality problem. A convergence theorem for our algorithm model and its discrete version can be easily proved. Furthermore, preliminary computational results show that the new method performs quite well and can compete with other methods.     

A variational method for unconstrained optimization problems

We introduce a new method for solving an unconstrained optimization problem. The method is based on the solution of a variational inequality problem and may be considered a modified trust region algorithm. We prove the convergence of the method and present some numerical results.     

A second order method for unconstrained optimization

An algorithm using second derivatives for solving unconstrained optimization problems is presented. In this brief note the descent direction of the algorithm is based on a modification of the Newton direction, while the Armijo rule for choosing the stepsize is used. The rate of convergence of the algorithm is shown to be superlinear. Our computational experience shows that the method performs quite well and our numerical results are presented in Section 4.     

A nonmonotone ODE-based method for unconstrained optimization

This paper presents a new hybrid algorithm for unconstrained optimization problems, which combines the idea of the IMPBOT algorithm with the nonmonotone line search technique. A feature of the proposed method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified limited-memory BFGS two loop recursion that requires only matrix–vector products, thus reducing the computations and storage. Furthermore, when the trial step is not accepted, the proposed method performs a line search along it using a modified nonmonotone scheme, thus a larger stepsize can be yielded in each line search procedure. Under some reasonable assumptions, the convergence properties of the proposed algorithm are analysed. Numerical results are also reported to show the efficiency of this proposed method.     

A scaled conjugate gradient method for nonlinear unconstrained optimization

We propose a new optimization problem which combines the good features of the classical conjugate gradient method using some penalty parameter, and then, solve it to introduce a new scaled conjugate gradient method for solving unconstrained problems. The method reduces to the classical conjugate gradient algorithm under common assumptions, and inherits its good properties. We prove the global convergence of the method using suitable conditions. Numerical results show that the new method is efficient and robust.     

A limited-memory quasi-Newton algorithm for bound-constrained non-smooth optimization

We consider the problem of minimizing a continuous function that may be non-smooth and non-convex, subject to bound constraints. We propose an algorithm that uses the L-BFGS quasi-Newton approximation of the problem's curvature together with a variant of the weak Wolfe line search. The key ingredient of the method is an active-set selection strategy that defines the subspace in which search directions are computed. To overcome the inherent shortsightedness of the gradient for a non-smooth function, we propose two strategies. The first relies on an approximation of the ε-minimum norm subgradient, and the second uses an iterative corrective loop that augments the active set based on the resulting search directions. While theoretical convergence guarantees have been elusive even for the unconstrained case, we present numerical results on a set of standard test problems to illustrate the efficacy of our approach, using an open-source Python implementation of the proposed algorithm.     

A new modified scaled conjugate gradient method for large-scale unconstrained optimization with non-convex objective function

In this paper, according to the fifth-order Taylor expansion of the objective function and the modified secant equation suggested by Li and Fukushima, a new modified secant equation is presented. Also, a new modification of the scaled memoryless BFGS preconditioned conjugate gradient algorithm is suggested which is the idea to compute the scaling parameter based on a two-point approximation of our new modified secant equation. A remarkable feature of the proposed method is that it possesses a globally convergent even without convexity assumption on the objective function. Numerical results show that the proposed new modification of scaled conjugate gradient is efficient.     

A new filled function method for unconstrained global optimization

Many real world problems can be modelled as optimization problems. However, the traditional algorithms for these problems often encounter the problem of being trapped in local minima. The filled function method is an effective approach to tackle this kind of problems. However the existing filled functions have the disadvantages of discontinuity, non-differentiability or sensitivity to parameters which limit their efficiency. In this paper, we proposed a new filled function which is continuous and differentiable without any parameter to tune. Compared to discontinuous or non-differentiable filled functions, the continuous and differentiable filled function mainly has three advantages: firstly, it is not easier to produce extra local minima, secondly, more efficient local search algorithms using gradient information can be applied and thirdly, a continuous and differentiable filled function can be optimized more easily. Based on the new proposed filled function, a new algorithm was designed for unconstrained global optimization problems. Numerical experiments were conducted and the results show the proposed algorithm was more efficient.     

A new spectral conjugate gradient method for large-scale unconstrained optimization

The spectral conjugate gradient methods, with simple construction and nice numerical performance, are a kind of effective methods for solving large-scale unconstrained optimization problems. In this paper, based on quasi-Newton direction and quasi-Newton condition, and motivated by the idea of spectral conjugate gradient method as well as Dai-Kou's selecting technique for conjugate parameter [SIAM J. Optim. 23 (2013), pp. 296–320], a new approach for generating spectral parameters is presented, where a new double-truncating technique, which can ensure both the sufficient descent property of the search directions and the bounded property of the sequence of spectral parameters, is introduced. Then a new associated spectral conjugate gradient method for large-scale unconstrained optimization is proposed. Under either the strong Wolfe line search or the generalized Wolfe line search, the proposed method is always globally convergent. Finally, a large number of comparison numerical experiments on large-scale instances from one thousand to two million variables are reported. The numerical results show that the proposed method is more promising.     

非单调信赖域方法求解无约束非光滑优化问题

提出了非单调信赖域算法求解无约束非光滑优化问题,并和经典的信赖域方法作比较分析。同时,设定了一些条件,在这些假设条件下证明了该算法是整体收敛的。数值实验结果表明,非单调策略对无约束非光滑优化问题的求解是行之有效的,拓展了非单调信赖域算法的应用领域。     

无约束非线性lp问题的区间极大熵方法

针对信号处理、系统识别等领域中涉及到的无约束非线性lp问题,为减小由于二进制编码的舍入误差对该问题计算结果的影响,对求解该问题的极大熵方法进行了区间扩张.证明了区间扩张后的极大熵函数至少具有二阶收敛性,并设计了具有多项式时间复杂度的区间算法进行求解,举例进行了数值计算.数值计算结果显示,该区间算法可靠,计算结果与区间扩张前相比,结果更加精确.     

A hybrid splitting method for variational inequality problems with separable structure

Alternating direction method (ADM), which decomposes a large-scale original variational inequality (VI) problem into a series of smaller scale subproblems, is very attractive for solving a class of VI problems with a separable structure. This type of method can greatly improve the efficiency, but cannot avoid solving VI subproblems. In this paper, we propose a hybrid splitting method with variable parameters for separable VI problems. Specifically, the proposed method solves only one strongly monotone VI subproblem and a well-posed system of nonlinear equations in each iteration. The global convergence of the new method is established under some standard assumptions as those in classical ADMs. Finally, some preliminary numerical results show that the proposed method performs favourably in practice.     

一种求解无约束优化问题的非单调信赖域算法

提出了非单调信赖域算法求解基于锥模型的无约束优化问题,该算法在求解信赖域子问题时充分利用了当前迭代点的一阶梯度信息。提出了一个新的信赖域半径的选取机制,并和经典的信赖域方法作比较分析。设定了一些条件,在这些假设条件下证明了算法是整体收敛的。数值实验结果表明,该算法对基于锥模型的无约束优化问题的求解是行之有效的,拓展了非单调信赖域算法的应用领域。     

A quasi-Newton method with sparse triple factorization for unconstrained minimization

A new quasi-Newton method for unconstrained minimization is presented. It uses sparse triple factorization of an approximation to the sparse Hessian matrix. At each step a new column and a corresponding row of the approximation to the Hessian is determined and its triple factorization is updated. Our method deals with the same updating problem as in J. Bräuninger's paper [2]. However, we make use of a rank-two instead of a rank-one updating scheme. Our method saves over half the number of operations required in J. Bräuninger's method. Moreover, our method utilizes the sparsity and, therefore, only the nonzero entries of the factors need to be stored. The positive definiteness can be preserved easily by taking suitable precautions. Under reasonable conditions our method is globally convergent and locally superlinearly evenn-step ρ-order convergent.     

求无约束优化问题的无参数填充函数法

填充函数作为求解优化问题的有效方法之一,以填充函数的基本思想为基础,构造了新的无参数填充函数,该函数形式简单,便于计算。分析了该函数的相关性质并设计了相应的算法,最后通过数值实验,结果表明提出的算法是可行的、有效的。     

Finding global minima with a novel filled function for non-smooth unconstrained optimisation

This article presents a novel filled function approach for finding a better minimiser of both non-smooth and smooth unconstrained global optimisations. Unlike any traditional filled function, in theory, the filled function proposed in this article does not assume that the objective function is coercive. Moreover, the proposed function contains only one parameter, which can be easily adjusted. In the algorithm section, an algorithm based on the proposed filled function is presented for unconstrained global optimisation in which the objective function is assumed to be coercive. A numerical test is also provided. The preliminary computational results show that the proposed filled function approach is promising.     

A parallel quasi-Newton algorithm for unconstrained optimization

In this paper, we present an asynchronous parallel quasi-Newton method. We assume that we havep+q processors, which are divided into two groups, the two groups execute in an asynchronous parallel fashion. If we assume that the objective function is twice continuously differentiable and uniformly convex, we discuss the global and superlinear convergence of the parallel BFGS method. Finally we show numerical results of this algorithm.     

Structured symmetric rank-one method for unconstrained optimization

In this paper, we investigate a symmetric rank-one (SR1) quasi-Newton (QN) formula in which the Hessian of the objective function has some special structure. Instead of approximating the whole Hessian via the SR1 formula, we consider an approach which only approximates part of the Hessian matrix that is not easily acquired. Although the SR1 update possesses desirable features, it is unstable in the sense that, it may not retain positive definiteness and may become undefined. Therefore, we describe some safeguards to overcome these difficulties. Since the structured SR1 method provides a more accurate Hessian approximation, therefore the proposed method reduces significantly the computational efforts needed in solving a problem. The results of a series of experiments on a typical set of standard unconstrained optimization problems are reported, which show that the structured SR1 method exhibits a clear improvement in numerical performance over some existing QN algorithms.     

Optimization for signal setting problems using non-smooth techniques

Consider a road network with severe traffic congestion. A set of efficient strategies can be addressed to simultaneously minimize road users’ travel times and maximize possible increase in travel demands. Due to the non-differentiability of the perturbed solutions in equilibrium constraints, non-smooth optimization models for delay-minimizing and capacity-maximizing signal setting problems are established. In this paper, we propose a new bundle subgradient projection approach to solve the signal setting problems with global convergence. Various control policies are numerically tested one against another. Numerical results disclose that the proposed approach has successfully solved the delay-minimizing and capacity maximizing signal setting problems and achieved significant performance in cost reduction when compared to other alternatives.     

解大规模无约束优化的自适应过滤信赖域法

提出一种解大规模无约束优化问题的自适应过滤信赖域法。用目标函数的梯度及迭代点的信息来构造目标函数海赛矩阵的近似数量矩阵,引进了过滤技术和自适应技术,大大提高了计算效率。从理论上证明了新算法的全局收敛性,数值试验结果也表明了新算法的有效性。