An affine scaling interior trust-region method combining with nonmonotone line search filter technique for linear inequality constrained minimization

This paper proposes an affine scaling interior trust-region method in association with nonmonotone line search filter technique for solving nonlinear optimization problems subject to linear inequality constraints. Based on a Newton step which is derived from the complementarity conditions of linear inequality constrained optimization, a trust-region subproblem subject only to an ellipsoidal constraint is defined by minimizing a quadratic model with an appropriate quadratic function and scaling matrix. The nonmonotone schemes combining with trust-region strategy and line search filter technique can bring about speeding up the convergence progress in the case of high nonlinear. A new backtracking relevance condition is given which assures global convergence without using the switching condition used in the traditional line search filter technique. The fast local convergence rate of the proposed algorithm is achieved which is not depending on any external restoration procedure. The preliminary numerical experiments are reported to show effectiveness of the proposed algorithm.     

A trust-region algorithm combining line search filter technique for nonlinear constrained optimization

In this paper, we propose a trust-region algorithm in association with line search filter technique for solving nonlinear equality constrained programming. At current iteration, a trial step is formed as the sum of a normal step and a tangential step which is generated by trust-region subproblem and the step size is decided by interior backtracking line search together with filter methods. Then, the next iteration is determined. This is different from general trust-region methods in which the next iteration is determined by the ratio of the actual reduction to the predicted reduction. The global convergence analysis for this algorithm is presented under some reasonable assumptions and the preliminary numerical results are reported.     

A new trust-region method with line search for solving symmetric nonlinear equations

A new trust-region method is proposed for symmetric nonlinear equations. In this given algorithm, if the trial step is unsuccessful, one line search will be used instead of repeatedly solving the subproblem of the normal trust-region method. Moreover, the global convergence is established under mild conditions by a new way. The quadratic convergence of the presented method is also proved. Numerical results show that the method is interesting for the given problems.     

A derivative-free affine scaling trust region methods based on probabilistic models with new nonmonotone line search technique for linear inequality constrained minimization without strict complementarity

ABSTRACT

In this paper, a derivative-free trust region methods based on probabilistic models with new nonmonotone line search technique is considered for nonlinear programming with linear inequality constraints. The proposed algorithm is designed to build probabilistic polynomial interpolation models for the objective function. We build the affine scaling trust region methods which use probabilistic or random models within a classical trust region framework. The new backtracking linear search technique guarantee the descent of the objective function, and new iterative points are in the feasible region. In order to overcome the strict complementarity hypothesis, under some reasonable conditions which are weaker than strong second order sufficient condition, we give the new and more simple identification function to structure the affine matrix. The global and local fast convergence of the algorithm are shown and the results of numerical experiments are reported to show the effectiveness of the proposed algorithm.     

An effective trust-region-based approach for symmetric nonlinear systems

This paper presents a new trust-region procedure for solving symmetric nonlinear systems of equations having several variables. The proposed approach takes advantage of the combination of both an effective adaptive trust-region radius and a non-monotone strategy. It is believed that the selection of an appropriate adaptive radius and the application of a suitable non-monotone strategy can improve the efficiency and robustness of the trust-region framework as well as decrease the computational costs of the algorithm by decreasing the required number of subproblems to be solved. The global convergence and the quadratic convergence of the proposed approach are proved without the non-degeneracy assumption of the exact Jacobian. The preliminary numerical results of the proposed algorithm indicating the promising behaviour of the new procedure for solving nonlinear systems are also reported.     

A Non-monotone Line Search Algorithm for?Unconstrained Optimization

The monotone line search schemes have been extensively used in the iterative methods for solving various optimization problems. It is well known that the non-monotone line search technique can improve the likelihood of finding a global optimal solution and the numerical performance of the methods, especially for some difficult nonlinear problems. The traditional non-monotone line search approach requires that a maximum of recent function values decreases. In this paper, we propose a new line search scheme which requires that a convex combination of recent function values decreases. We apply the new line search technique to solve unconstrained optimization problems, and show the proposed algorithm possesses global convergence and R-linear convergence under suitable assumptions. We also report the numerical results of the proposed algorithm for solving almost all the unconstrained testing problems given in CUTEr, and give numerical comparisons of the proposed algorithm with two famous non-monotone methods.     

An efficient adaptive trust-region method for systems of nonlinear equations

We present an adaptive trust-region algorithm to solve systems of nonlinear equations. Using the nonmonotone technique of Grippo, Lampariello and Lucidi, we introduce a new adaptive radius to decrease the total number of iterations and function evaluations. In contrast with the pervious methods, the new adaptive radius ensures that the size of radius is not too large or too small. We show that the sequence generated by the proposed adaptive radius is decreasing, so it prevents the production of too large radius as possible. Furthermore, it is shown that this sequence is reduced slowly, so it prevents the production of the intensely small radius. The global and quadratic convergence of the proposed approach are proved. Preliminary numerical results of our algorithm are also reported which indicate the promising behaviour of the new procedure to solve systems of nonlinear equations.     

A new class of nonmonotone adaptive trust-region methods for nonlinear equations with box constraints

A nonmonotone trust-region method for the solution of nonlinear systems of equations with box constraints is considered. The method differs from existing trust-region methods both in using a new nonmonotonicity strategy in order to accept the current step and a new updating technique for the trust-region-radius. The overall method is shown to be globally convergent. Moreover, when combined with suitable Newton-type search directions, the method preserves the local fast convergence. Numerical results indicate that the new approach is more effective than existing trust-region algorithms.     

Randomized line search techniques in combined GA for discrete sizing optimization of truss structures

This paper presents two randomized line search techniques, each combined with a genetic algorithm (GA), to improve the convergence and the accuracy ratio for discrete sizing optimization of truss structures. The first technique is a simple one-dimensional line search in which design variable axes are selected randomly as search directions. The second is a line search technique whose search direction is determined randomly by fitness function values and differences in the genotypes of individuals. To apply the above-mentioned line search techniques without difficulty, real coding is adopted for discrete problems. The line search techniques are applied to discrete optimization problems of minimum-weight truss structures subjected to stress and displacement constraints. The proposed techniques provide convergence to better solutions than a conventional GA.     

A cubic regularization algorithm for unconstrained optimization using line search and nonmonotone techniques

In recent years, cubic regularization algorithms for unconstrained optimization have been defined as alternatives to trust-region and line search schemes. These regularization techniques are based on the strategy of computing an (approximate) global minimizer of a cubic overestimator of the objective function. In this work we focus on the adaptive regularization algorithm using cubics (ARC) proposed in Cartis et al. [Adaptive cubic regularisation methods for unconstrained optimization. Part I: motivation, convergence and numerical results, Mathematical Programming A 127 (2011), pp. 245–295]. Our purpose is to design a modified version of ARC in order to improve the computational efficiency preserving global convergence properties. The basic idea is to suitably combine a Goldstein-type line search and a nonmonotone accepting criterion with the aim of advantageously exploiting the possible good descent properties of the trial step computed as (approximate) minimizer of the cubic model. Global convergence properties of the proposed nonmonotone ARC algorithm are proved. Numerical experiments are performed and the obtained results clearly show satisfactory performance of the new algorithm when compared to the basic ARC algorithm.     

Implementable L ∞ penalty-function method for semi-infinite optimization

This paper considers general non-linear semi-infinite programming problems and presents an implementable method which employs an exact L _∞ penalty function. Since the L _∞ penalty function is continuous even if the number of representative constraints changes, trust-region techniques may effectively be adopted to obtain global convergence. Numerical results are given to show the efficiency of the proposed algorithm.     

A non-monotone pattern search approach for systems of nonlinear equations

In this paper, a new pattern search is proposed to solve the systems of nonlinear equations. We introduce a new non-monotone strategy which includes a convex combination of the maximum function of some preceding successful iterates and the current function. First, we produce a stronger non-monotone strategy in relation to the generated strategy by Gasparo et al. [Nonmonotone algorithms for pattern search methods, Numer. Algorithms 28 (2001), pp. 171–186] whenever iterates are far away from the optimizer. Second, when iterates are near the optimizer, we produce a weaker non-monotone strategy with respect to the generated strategy by Ahookhosh and Amini [An efficient nonmonotone trust-region method for unconstrained optimization, Numer. Algorithms 59 (2012), pp. 523–540]. Third, whenever iterates are neither near the optimizer nor far away from it, we produce a medium non-monotone strategy which will be laid between the generated strategy by Gasparo et al. [Nonmonotone algorithms for pattern search methods, Numer. Algorithms 28 (2001), pp. 171–186] and Ahookhosh and Amini [An efficient nonmonotone trust-region method for unconstrained optimization, Numer. Algorithms 59 (2012), pp. 523–540]. Reported are numerical results of the proposed algorithm for which the global convergence is established.     

Successive Chebyshev pseudospectral convex optimization method for nonlinear optimal control problems

This article aims at proposing a successive Chebyshev pseudospectral convex optimization method for solving general nonlinear optimal control problems (OCPs). First, Chebyshev pseudospectral discrete scheme is used to discretize a general nonlinear OCP. At the same time, a convex subproblem is formulated by using the first-order Taylor expansion to convexify the discretized nonlinear dynamic constraints. Second, a trust-region penalty term is added to the performance index of the subproblem, and a successive convex optimization algorithm is proposed to solve the subproblem iteratively. Noted that the trust-region penalty parameters can be adjusted according to the linearization error in iterative process, which improves convergence rate. Third, the Karush–Kuhn–Tucker conditions of the subproblem are derived, and furthermore, a proof is given to show that the algorithm will iteratively converge to the subproblem. Additionally, the global convergence of the algorithm is analyzed and proved, which is based on three key lemmas. Finally, the orbit transfer problem of spacecraft is used to test the performance of the proposed method. The simulation results demonstrate the optimal control is bang-bang form, which is consistent with the result of theoretical proof. Also, the algorithm is of efficiency, fast convergence rate, and high accuracy. Therefore, the proposed method provides a new approach for solving nonlinear OCPs online and has great potential in engineering practice.     

Adaptive augmented Lagrangian methods: algorithms and practical numerical experience

In this paper, we consider augmented Lagrangian (AL) algorithms for solving large-scale nonlinear optimization problems that execute adaptive strategies for updating the penalty parameter. Our work is motivated by the recently proposed adaptive AL trust region method by Curtis et al. [An adaptive augmented Lagrangian method for large-scale constrained optimization, Math. Program. 152 (2015), pp. 201–245.]. The first focal point of this paper is a new variant of the approach that employs a line search rather than a trust region strategy, where a critical algorithmic feature for the line search strategy is the use of convexified piecewise quadratic models of the AL function for computing the search directions. We prove global convergence guarantees for our line search algorithm that are on par with those for the previously proposed trust region method. A second focal point of this paper is the practical performance of the line search and trust region algorithm variants in Matlab software, as well as that of an adaptive penalty parameter updating strategy incorporated into the Lancelot software. We test these methods on problems from the CUTEst and COPS collections, as well as on challenging test problems related to optimal power flow. Our numerical experience suggests that the adaptive algorithms outperform traditional AL methods in terms of efficiency and reliability. As with traditional AL algorithms, the adaptive methods are matrix-free and thus represent a viable option for solving large-scale problems.     

An augmented Lagrangian method for a class of LMI-constrained problems in robust control theory

We present a new approach to a class of non-convex LMI-constrained problems in robust control theory. The problems we consider may be recast as the minimization of a linear objective subject to linear matrix inequality (LMI) constraints in tandem with non-convex constraints related to rank deficiency conditions. We solve these problems using an extension of the augmented Lagrangian technique. The Lagrangian function combines a multiplier term and a penalty term governing the non-convex constraints. The LMI constraints, due to their special structure, are retained explicitly and not included in the Lagrangian. Global and fast local convergence of our approach is then obtained either by an LMI-constrained Newton type method including line search or by a trust-region strategy. The method is conveniently implemented with available semi-definite programming (SDP) interior-point solvers. We compare its performance to the wellknown D - K iteration scheme in robust control. Two test problems are investigated and demonstrate the power and efficiency of our approach.     

Trust-region method for box-constrained semismooth equations and its applications to complementary problems

In this paper, we propose a new trust-region algorithm for bound-constrained semismooth systems of equations. Trust-region subproblem is defined by minimizing a quadratic function subject only to a rectangular constraint. By employing a new active set and nonmonotone techniques, solution of the equations can be found effective. Global and local convergence results of the proposed algorithm are established under reasonable conditions. The algorithm is applied and tested on complementary problems and the experiments show that our method is efficient.     

A globally and quadratically convergent algorithm for general nonlinear programming problems

This paper presents an algorithm for the minimization of a nonlinear objective function subject to nonlinear inequality and equality constraints. The proposed method has the two distinguishing properties that, under weak assumptions, it converges to a Kuhn-Tucker point for the problem and under somewhat stronger assumptions, the rate of convergence is quadratic. The method is similar to a recent method proposed by Rosen in that it begins by using a penalty function approach to generate a point in a neighborhood of the optimum and then switches to Robinson's method. The new method has two new features not shared by Rosen's method. First, a correct choice of penalty function parameters is constructed automatically, thus guaranteeing global convergence to a stationary point. Second, the linearly constrained subproblems solved by the Robinson method normally contain linear inequality constraints while for the method presented here, only linear equality constraints are required. That is, in a certain sense, the new method “knows” which of the linear inequality constraints will be active in the subproblems. The subproblems may thus be solved in an especially efficient manner. Preliminary computational results are presented.     

基于改进细菌菌落优化算法的PID参数整定

针对经典智能优化算法在PID参数整定时存在早熟收敛及陷入无效循环的问题,提出一种改进细菌菌落优化算法。在个体位置更新时引入收缩因子和有指导的随机搜索策略,以平衡算法的全局搜索能力和局部搜索能力,在全局最优位置附近进行动态随机搜索,以提高算法的局部收敛精度。选取ITAE指标作为优化目标构建目标函数和约束条件。以时滞非线性湿度PID控制器为例,仿真结果表明,该算法在提高收敛精度的同时具有自我结束的能力,能够有效抑制超调量。     

A BFGS trust-region method for nonlinear equations

In this paper, a new trust-region subproblem combining with the BFGS update is proposed for solving nonlinear equations, where the trust region radius is defined by a new way. The global convergence without the nondegeneracy assumption and the quadratic convergence are obtained under suitable conditions. Numerical results show that this method is more effective than the norm method.