A nonmonotone adaptive trust region method and its convergence

In this paper, we combine the new trust region subproblem proposed in [1] with the nonmonotone technique to propose a new algorithm for unconstrained optimization—the nonmonotone adaptive trust region method. The local and global convergence properties of the nonmonotone adaptive trust region method are proved. Its efficiency is tested by numerical results.     

A derivative-free affine scaling trust region methods based on probabilistic models with new nonmonotone line search technique for linear inequality constrained minimization without strict complementarity

ABSTRACT

In this paper, a derivative-free trust region methods based on probabilistic models with new nonmonotone line search technique is considered for nonlinear programming with linear inequality constraints. The proposed algorithm is designed to build probabilistic polynomial interpolation models for the objective function. We build the affine scaling trust region methods which use probabilistic or random models within a classical trust region framework. The new backtracking linear search technique guarantee the descent of the objective function, and new iterative points are in the feasible region. In order to overcome the strict complementarity hypothesis, under some reasonable conditions which are weaker than strong second order sufficient condition, we give the new and more simple identification function to structure the affine matrix. The global and local fast convergence of the algorithm are shown and the results of numerical experiments are reported to show the effectiveness of the proposed algorithm.     

解大规模无约束优化的自适应过滤信赖域法

提出一种解大规模无约束优化问题的自适应过滤信赖域法。用目标函数的梯度及迭代点的信息来构造目标函数海赛矩阵的近似数量矩阵,引进了过滤技术和自适应技术,大大提高了计算效率。从理论上证明了新算法的全局收敛性,数值试验结果也表明了新算法的有效性。     

A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques

In recent years, cubic regularization algorithms for unconstrained optimization have been defined as alternatives to trust-region and line search schemes. These regularization techniques are based on the strategy of computing an (approximate) global minimizer of a cubic overestimator of the objective function. In this work we focus on the adaptive regularization algorithm using cubics (ARC) proposed in Cartis et al. [Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results, Mathematical Programming A 127 (2011), pp. 245–295]. Our purpose is to design a modified version of ARC in order to improve the computational efficiency preserving global convergence properties. The basic idea is to suitably combine a Goldstein-type line search and a nonmonotone accepting criterion with the aim of advantageously exploiting the possible good descent properties of the trial step computed as (approximate) minimizer of the cubic model. Global convergence properties of the proposed nonmonotone ARC algorithm are proved. Numerical experiments are performed and the obtained results clearly show satisfactory performance of the new algorithm when compared to the basic ARC algorithm.     

An efficient adaptive trust-region method for systems of nonlinear equations

We present an adaptive trust-region algorithm to solve systems of nonlinear equations. Using the nonmonotone technique of Grippo, Lampariello and Lucidi, we introduce a new adaptive radius to decrease the total number of iterations and function evaluations. In contrast with the pervious methods, the new adaptive radius ensures that the size of radius is not too large or too small. We show that the sequence generated by the proposed adaptive radius is decreasing, so it prevents the production of too large radius as possible. Furthermore, it is shown that this sequence is reduced slowly, so it prevents the production of the intensely small radius. The global and quadratic convergence of the proposed approach are proved. Preliminary numerical results of our algorithm are also reported which indicate the promising behaviour of the new procedure to solve systems of nonlinear equations.     

一种求解无约束优化问题的非单调信赖域算法

提出了非单调信赖域算法求解基于锥模型的无约束优化问题,该算法在求解信赖域子问题时充分利用了当前迭代点的一阶梯度信息。提出了一个新的信赖域半径的选取机制,并和经典的信赖域方法作比较分析。设定了一些条件,在这些假设条件下证明了算法是整体收敛的。数值实验结果表明,该算法对基于锥模型的无约束优化问题的求解是行之有效的,拓展了非单调信赖域算法的应用领域。     

Nonmonotone trust region methods with curvilinear path in unconstrained optimization

A general nonmonotone trust region method with curvilinear path for unconstrained optimization problem is presented. Although this method allows the sequence of the objective function values to be nonmonotone, convergence properties similar to those for the usual trust region methods with curvilinear path are proved under certain conditions. Some numerical results are reported which show the superiority of the nonmonotone trust region method with respect to the numbers of gradient evaluations and function evaluations.     

一种新的信赖域粒子群算法

针对信赖域方法求解多峰值优化不能收敛到全局最优的问题,本文提出了一种新的信赖域粒子群算法。该算法将信赖域方法和粒子群算法进行有机结合,利用了粒子群搜索性能良好和信赖域方法总体收敛性良好的优点。新算法能够克服信赖域方法的缺点,同时又能有效求解一类欺骗性问题。数值实验说明了算法的有效性和鲁棒性。     

A nonmonotone ODE-based method for unconstrained optimization

This paper presents a new hybrid algorithm for unconstrained optimization problems, which combines the idea of the IMPBOT algorithm with the nonmonotone line search technique. A feature of the proposed method is that at each iteration, a system of linear equations is solved only once to obtain a trial step, via a modified limited-memory BFGS two loop recursion that requires only matrix–vector products, thus reducing the computations and storage. Furthermore, when the trial step is not accepted, the proposed method performs a line search along it using a modified nonmonotone scheme, thus a larger stepsize can be yielded in each line search procedure. Under some reasonable assumptions, the convergence properties of the proposed algorithm are analysed. Numerical results are also reported to show the efficiency of this proposed method.     

解非线性等式约束优化问题的新锥模型信赖域算法术

应用新锥模型信赖域子问题解非线性等式约束问题,提出了一个解此问题的新锥模型信赖域算法,证明了新算法的全局收敛性,并进行了数值比较实验．理论与数值结果表明这个算法是一个值得关注的有效算法．     

A nonmonotone accelerated Levenberg–Marquardt method for the -eigenvalues of symmetric tensors

The eigenvalues of tensors become more and more important in the numerical multilinear algebra. In this paper, based on the nonmonotone technique, an accelerated Levenberg–Marquardt (LM) algorithm is presented for computing the -eigenvalues of symmetric tensors, in which an LM step and an accelerated LM step are computed at each iteration. We establish the global convergence of the proposed algorithm using properties of symmetric tensors and norms. Under the local error-bound condition, the cubic convergence of the nonmonotone accelerated LM algorithm is derived. Numerical results show that this method is efficient.     

Adaptive augmented Lagrangian methods: algorithms and practical numerical experience

In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis et al. [An adaptive augmented Lagrangian method for large-scale constrained optimization, Math. Program. 152 (2015), pp. 201–245.]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our numerical experience suggests that the adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. As with traditional AL algorithms, the adaptive methods are matrix-free and thus represent a viable option for solving large-scale problems.     

线性约束优化的一类非单调信赖域算法

§1.引 言 考虑线性约束优化问题:min.f(x)s.t. aiTx=bi,i∈E,(1.1)aiTx≥bi,i∈I,其中f(x)是可行域X={x∈Rn|aiTx=bi,i∈E;aiTx≥bi,i∈I}上的连续可微函数. 多年来,问题(1.1)一直受到许多研究人员的广泛注意,相继提出了有效集方法、投影梯度法[1,2]等.特别是近几年来,信赖域方法因具有强适性、强收敛性受到更多的重视[3,8,11,12],这些方法都具有一个共同的性质:下降性,即要求在迭代点,目标函数值严格单调下降,放     

全变差图像恢复的自适应步长梯度投影算法

针对图像去噪问题,本文基于全变差对偶公式提出一个新的梯度投影算法.算法采用改进的非单调线搜索和自适应BB（Barzilai-Borwein）步长,有效地改善了Chambolle梯度投影算法收敛慢的缺点.数值结果表明新算法优于一些已有的梯度投影算法.     

Gradient methods exploiting spectral properties

We propose a new stepsize for the gradient method. It is shown that this new stepsize will converge to the reciprocal of the largest eigenvalue of the Hessian, when Dai-Yang's asymptotic optimal gradient method (Computational Optimization and Applications, 2006, 33(1): 73–88) is applied for minimizing quadratic objective functions. Based on this spectral property, we develop a monotone gradient method that takes a certain number of steps using the asymptotically optimal stepsize by Dai and Yang, and then follows by some short steps associated with this new stepsize. By employing one step retard of the asymptotic optimal stepsize, a nonmonotone variant of this method is also proposed. Under mild conditions, R-linear convergence of the proposed methods is established for minimizing quadratic functions. In addition, by combining gradient projection techniques and adaptive nonmonotone line search, we further extend those methods for general bound constrained optimization. Two variants of gradient projection methods combining with the Barzilai-Borwein stepsizes are also proposed. Our numerical experiments on both quadratic and bound constrained optimization indicate that the new proposed strategies and methods are very effective.     

A nonmonotone SQP-filter method for equality constrained optimization

In this paper, we propose a nonmonotone sequential quadratic programming-filter method for solving nonlinear equality constrained optimization. This new method has more flexibility for the acceptance of the trial step and requires less computational costs compared with the monotone methods. Under reasonable conditions, we give the global convergence properties. Further, the second-order correction step and nonmonotone reduction conditions are used to overcome Maratos effect so that quadratic local convergence is achieved. The numerical experiments are reported to show the effectiveness of the proposed algorithm.     

Optimal control of viscous Burgers equation via an adaptive nonmonotone Barzilai–Borwein gradient method

An adaptive nonmonotone spectral gradient method for the solution of distributed optimal control problem (OCP) for the viscous Burgers equation is presented in a black-box framework. Regarding the implicit function theorem, the OCP is transformed into an unconstrained nonlinear optimization problem (UNOP). For solving UNOP, an adaptive nonmonotone Barzilai–Borwein gradient method is proposed in which to make a globalization strategy, first an adaptive nonmonotone strategy which properly controls the degree of nonmonotonicity is presented and then is incorporated into an inexact line search approach to construct a more relaxed line search procedure. Also an adjoint technique is used to effectively evaluate the gradient. The low memory requirement and the guaranteed convergence property make the proposed method quite useful for large-scale OCPs. The efficiency of the presented method is supported by numerical experiments.     

一类新的带记忆模型的非单调信赖域算法

本文就无约束优化问题提出了一个带记忆模型的非单调信赖域算法。与传统的非单调信赖域算法不同,文中的信赖域子问题的逼近模型为记忆模型,该模型使我们可以从更全面的角度来求得信赖域试探步,从而避免了传统非单调信赖域方法中试探步的求取完全依赖于当前点的信息而过于局部化的困难。文中提出了一个带记忆模型的非单调信赖域
域算法,并证明了其全局收敛性。     

物联网中基于信任抗On-off攻击的自适应安全机制

为了降低静态安全机制中不必要的数据源认证开销和防御信任阈值机制中存在的On-off攻击,在物联网（IoT）环境下提出了一种基于信任的自适应安全机制。首先,根据节点在信息交互中的行为表现建立节点间的信任评估模型,进而给出节点总体信任值的度量方法;然后,对于总体信任值高于信任阈值的节点,采用基于信任的自适应检测算法实时地检测这些节点总体信任值的变化情况;最终,中继节点根据自适应检测的结果决定是否验证接收到的消息。仿真实验结果和分析表明,该机制降低了中继节点的能量开销,同时对物联网中的On-off攻击起到较好的防御作用。     

A new nonmonotone filter Barzilai–Borwein method for solving unconstrained optimization problems

In this paper, a finite filter is used in the structure of the Barzilai–Browein (BB) gradient method in order to propose a new modified BB algorithm for solving large-scale unconstrained optimization problems. Our algorithm is equipped with a relaxed nonmonotone line search technique which allows the algorithm to enjoy the nonmonotonicity properties from scratch. Under some suitable conditions, the global convergence property of the new proposed algorithm is established. Numerical results on some test problems in CUTEr library show the efficiency and effectiveness of the new algorithm in practice too.