On the computer implementation of feasible direction interior point algorithms for nonlinear optimization

The paper discusses the computer implementation of a class of interior point algorithms for the minimization of nonlinear functions with equality and inequality constraints. These algorithms consist of fixed point iterations to solve KKT firstorder optimality conditions. At each iteration a descent direction is defined by solving a linear system. Then, the linear system is perturbed in such a way as to deflect the descent direction and obtain a feasible descent direction. A line search is finally performed to obtain a new interior point with a lower objective. Newton, quasi-Newton, or first-order versions of the algorithm can be obtained. This paper is mainly concerned with the solution of the internal linear systems, the algorithms that are employed for the constrained line search and also with the quasi-Newton matrix updating. Some numerical results obtained with a quasi Newton algorithm are also presented. A set of test problems were solved very efficiently with the same values of the internal parameters.     

An SQP feasible descent algorithm for nonlinear inequality constrained optimization without strict complementarity

In this paper, a kind of nonlinear optimization problems with nonlinear inequality constraints are discussed, and a new SQP feasible descent algorithm for solving the problems is presented. At each iteration of the new algorithm, a convex quadratic program (QP) which always has feasible solution is solved and a master direction is obtained, then, an improved (feasible descent) direction is yielded by updating the master direction with an explicit formula, and in order to avoid the Maratos effect, a height-order correction direction is computed by another explicit formula of the master direction and the improved direction. The new algorithm is proved to be globally convergent and superlinearly convergent under mild conditions without the strict complementarity. Furthermore, the quadratic convergence rate of the algorithm is obtained when the twice derivatives of the objective function and constrained functions are adopted. Finally, some numerical tests are reported.     

Semidefinite programming duality and linear time-invariant systems

Several important problems in control theory can be reformulated as semidefinite programming problems, i.e., minimization of a linear objective subject to linear matrix inequality (LMI) constraints. From convex optimization duality theory, conditions for infeasibility of the LMIs, as well as dual optimization problems, can be formulated. These can in turn be reinterpreted in control or system theoretic terms, often yielding new results or new proofs for existing results from control theory. We explore such connections for a few problems associated with linear time-invariant systems.     

Free material optimization for stress constraints

Free material design deals with the question of finding the lightest structure subject to one or more given loads when both the distribution of material and the material itself can be freely varied. We additionally consider constraints on local stresses in the optimal structure. We discuss the choice of formulation of the problem and the stress constraints. The chosen formulation leads to a mathematical program with matrix inequality constraints, so-called nonlinear semidefinite program. We present an algorithm that can solve these problems. The algorithm is based on a generalized augmented Lagrangian method. A number of numerical examples demonstrate the effect of stress constraints in free material optimization. Dedicated to Pauli Pedersen on the occasion of his 70th birthday.     

光滑神经网络解决非李普西茨优化问题的研究

为寻求满足约束条件的优化问题的最优解,针对目标函数是非李普西茨函数,可行域由线性不等式或非线性不等式约束函数组成的区域的优化问题,构造了一种光滑神经网络模型。此模型通过引进光滑逼近技术将目标函数由非光滑函数转换成相应的光滑函数以及结合惩罚函数方法所构造而成。通过详细的理论分析证明了不论初始点在可行域内还是在可行域外,光滑神经网络的解都具有一致有界性和全局性,以及光滑神经网络的任意聚点都是原始优化问题的稳定点等结论。最后通过几个简单的仿真实验证明了理论的正确性。     

Control Liapunov function design of neural networks that solve convex optimization and variational inequality problems

This paper presents two neural networks to find the optimal point in convex optimization problems and variational inequality problems, respectively. The domain of the functions that define the problems is a convex set, which is determined by convex inequality constraints and affine equality constraints. The neural networks are based on gradient descent and exact penalization and the convergence analysis is based on a control Liapunov function analysis, since the dynamical system corresponding to each neural network may be viewed as a so-called variable structure closed loop control system.     

Extension of a hybrid Genetic Algorithm for nonlinear programming problems with equality and inequality constraints

As an extension of the hybrid Genetic Algorithm-HGA proposed by Tang et al. (Comput. Math. Appl. 36 (1998) 11), this paper focuses on the critical techniques in the application of the GA to nonlinear programming (NLP) problems with equality and inequality constraints. Taking into account the equality constraints and embedding the information of infeasible points/chromosomes into the evaluation function, an extended fuzzy-based methodology and three new evaluation functions are proposed to formulate and evaluate the infeasible chromosomes. The extended version of concepts of dominated semi-feasible direction (DSFD), feasibility degree (FD₁) of semi-feasible direction, feasibility degree (FD₂) of infeasible points ‘belonging to’ feasible domain are introduced. Combining the new evaluation functions and weighted gradient direction search into the Genetic Algorithm, an extended hybrid Genetic Algorithm (EHGA) is developed to solve nonlinear programming (NLP) problems with equality and inequality constraints. Simulation shows that this new algorithm is efficient.Scope and purposeNon-linear Programming (NLP) problems with equality and inequality constraints is an important type of constrained optimization problems. Genetic Algorithm (GA) is one of the well known evolutionary computation techniques. In the application of GA to NLP problems, chromosomes randomly generated at the beginning and/or generated by genetic operators during the evolutionary process usually violate the constraints, resulting in infeasible chromosomes. Therefore, the handling of system constraints, particularly the nonlinear equation constraints, and the measurement and evaluation of infeasible chromosomes, are major concerns in GA. Penalty strategy in the construction of fitness function is commonly used to evaluate the infeasible chromosomes in some traditional AG methods. However, this approach essentially narrows down the search space by eliminating all infeasible chromosomes from the evolutionary process, and it may reduce the chances of finding better candidates for the global optimization. In particular, it absolutely ignores the information carried by the infeasible chromosomes itself. Therefore, formulating the infeasible chromosomes by embedding the relevant information into the evaluation function are important when applying GA to NLP.As an extension of the Hybrid Genetic Algorithm-HGA proposed by Tang et al. (1998), this paper focuses on the critical techniques in the application of GA to NLP problems with equality and inequality constraints. Taking into account the equality constraints and embedding the information of infeasible chromosomes into the evaluation function, an extended fuzzy-based methodology and three new evaluation functions are designed to formulate and evaluate the infeasible chromosomes. By introducing an extended version of the concepts of dominated semi-feasible direction (DSFD), feasibility degree (FD₁) of semi-feasible direction, feasibility degree (FD₂) of infeasible points ‘belonging to’ feasible domain, an extended hybrid Genetic Algorithm (EHGA) is developed for solving NLP problems with equality and inequality constraints.     

On the existence of affine invariant descent directions

This paper begins with a brief review of affine invariance and its significance for iterative algorithms. It then explores the existence of affine invariant descent directions for unconstrained minimization. While there may exist several affine invariant descent directions for smooth functions at a given point, it is shown that for quadratic functions, there exists exactly one invariant descent direction in the strictly convex case and generally none in the case where the Hessian is singular or indefinite. These results can be generalized to smooth nonlinear functions and have implications regarding the initialization of minimization algorithms. They stand in contrast to recent works on constrained convex and nonconvex optimization for which there may exist an affine invariant ‘frame’ that depends on the feasible set and that can be used to define an affine invariant descent direction.     

A Structure Exploiting Preprocessor for Semidefinite Programs Derived From the Kalman-Yakubovich-Popov Lemma

Semidefinite programs derived from the Kalman-Yakubovich-Popov (KYP) lemma are quite common in control and signal processing applications. The programs are often of high dimension which makes them hard or impossible to solve with general-purpose solvers. Here we present a customized preprocessor, KYPD, that utilizes the inherent structure of this particular optimization problem. The key to an efficient implementation is to transform the optimization problem into an equivalent semidefinite program. This equivalent problem has much fewer variables and the matrices in the linear matrix inequality constraints are of low rank. KYPD can use any primal-dual solver for semidefinite programs as an underlying solver.     

一类修正的DY共轭梯度法及其全局收敛性

本文提出了一类求解无约束优化问题的修正DY共轭梯度法.算法采用新的迭代格式,每步迭代都可自行产生一个充分下降方向.采用Wolfe线搜索时,证明了全局收敛性.数值实验结果验证了算法是有效的.     

On exponential convergence conditions of an extended projection neural network

Recently the extended projection neural network was proposed to solve constrained monotone variational inequality problems and a class of constrained nonmonotontic variational inequality problems. Its exponential convergence was developed under the positive definiteness condition of the Jacobian matrix of the nonlinear mapping. This note proposes new results on the exponential convergence of the output trajectory of the extended projection neural network under the weak conditions that the Jacobian matrix of the nonlinear mapping may be positive semidefinite or not. Therefore, new results further demonstrate that the extended projection neural network has a fast convergence rate when solving a class of constrained monotone variational inequality problems and nonmonotonic variational inequality problems. Illustrative examples show the significance of the obtained results.     

Robust control via concave minimization local and global algorithms

This paper is concerned with the robust control problem of linear fractional representation (LFT) uncertain systems depending on a time-varying parameter uncertainty. Our main result exploits a linear matrix inequality (LMI) characterization involving scalings and Lyapunov variables subject to an additional essentially nonconvex algebraic constraint. The nonconvexity enters the problem in the form of a rank deficiency condition or matrix inverse relation on the scalings only. It is shown that such problems, but also more generally rank inequalities and bilinear constraints, can be formulated as the minimization of a concave functional subject to LMI constraints. First of all, a local Frank and Wolfe (1956) feasible direction algorithm is introduced in this context to tackle this hard optimization problem. Exploiting the attractive concavity structure of the problem, several efficient global concave programming methods are then introduced and combined with the local feasible direction method to secure and certify global optimality of the solutions. Computational experiments indicate the viability of our algorithms, and in the worst case, they require the solution of a few LMI programs     

Piecewise linear quadratic optimal control

The use of piecewise quadratic cost functions is extended from stability analysis of piecewise linear systems to performance analysis and optimal control. Lower bounds on the optimal control cost are obtained by semidefinite programming based on the Bellman inequality. This also gives an approximation to the optimal control law. An upper bound to the optimal cost is obtained by another convex optimization problem using the given control law. A compact matrix notation is introduced to support the calculations and it is proved that the framework of piecewise linear systems can be used to analyze smooth nonlinear dynamics with arbitrary accuracy     

Sequential LMI approach for the design of a BMI‐based robust observer state feedback controller with nonlinear uncertainties

This paper aims at developing a robust observer–based estimated state feedback control design method for an uncertain dynamical system that can be represented as a linear time‐invariant system connected with an integral quadratic constraint–type nonlinear uncertainty. Traditionally, in existing design methodologies, a convex semidefinite constraint is obtained at the cost of conservatism and unrealistic assumptions. This paper avoids such assumptions and formulates, the design of the robust observer state feedback controller as the feasibility problem of a bilinear matrix inequality (BMI) constraint. Unfortunately, the search for a feasible solution of a BMI constraint is an NP‐hard problem in general. The applicability of a linearization method, such as the variable change method and the congruence transformation, depends on the specific structure of the problem at hand and cannot be generalized. This paper transforms the feasibility analysis of the BMI constraint into an eigenvalue problem and applies the convex‐concave–based sequential linear matrix inequality optimization method to search for a feasible solution. Furthermore, an augmentation of the sequential linear matrix inequality algorithm to improve its numerical stability is presented. In the application part, a vehicle lateral control problem is presented to demonstrate the applicability of the proposed algorithm to a real‐world estimated state feedback control design problem and the necessity of the augmentation for numerical stability.     

具有未知参数的非线性系统动态优化

针对具有未知参数和不等式路径约束的非线性系统动态优化问题,提出一种新颖有效的数值求解方法.首先,将未知参数视为一个动态优化问题的决策变量;其次,利用多重打靶法将无限维的含未知参数动态优化问题转化为有限维的非线性规划问题,进而在不等式路径约束违反的时间段内,用有限多个内点约束替代原不等式路径约束;然后,用内点法求解转化后的非线性规划问题,在路径约束违反的一定容许度下,经过有限多次步数迭代后得到未知参数值的同时得到控制策略,并在理论上对所提出算法的收敛性进行相应证明;最后,对两个经典的含未知参数非线性系统的动态优化问题进行数值仿真以验证所提出算法的有效性.     

Relaxation approach to topology optimization of frame structure under frequency constraint

This paper deals with the frame topology optimization under the frequency constraint and proposes an algorithm that solves a sequence of relaxation problems to obtain a local optimal solution with high quality. It is known that an optimal solution of this problem often has multiple eigenvalues and the feasible set is disconnected. Due to these two difficulties, conventional nonlinear programming approaches often converge to a local optimal solution that is unacceptable from a practical point of view. In this paper, we formulate the frequency constraint as a positive semidefinite constraint of a certain symmetric matrix, and then relax this constraint to make the feasible set connected. The proposed algorithm solves a sequence of the relaxation problems with gradually decreasing the relaxation parameter. The positive semidefinite constraint is treated with the logarithmic barrier function and, hence, the algorithm finds no difficulty in multiple eigenvalues of a solution. Numerical experiments show that global optimal solutions, or at least local optimal solutions with high qualities, can be obtained with the proposed algorithm.     

A nonlinear minimization approach to multiobjective and structured controls for discrete‐time systems

In this paper, a nonlinear minimization approach is proposed for multiobjective and structured controls for discrete‐time systems. The problem of finding multiobjective and structured controls for discrete‐time systems is represented as a quadratic matrix inequality problem. It is shown that the problem is reduced to a nonlinear minimization problem that has a concave objective function and linear matrix inequality constraints. An algorithm for the nonlinear minimization problem is proposed, which is easily implemented with existing semidefinite programming algorithms. The validity of the proposed algorithm is illustrated by comparisons with existing methods. In addition, applications of this work are demonstrated via numerical examples. Copyright © 2004 John Wiley & Sons, Ltd.     

基于梯度投影法与随机优化算法的约束优化方法

针对带有线性等式和不等式约束的无确定函数形式的约束优化问题,提出一种利用梯度投影法与遗传算法、同时扰动随机逼近等随机算法相结合的优化方法。该方法利用遗传算法进行全局搜索,利用同时扰动随机逼近算法进行局部搜索,算法在每次进化时根据线性约束计算父个体处的梯度投影方向,以产生新个体,从而能够严格保证新个体满足全部约束条件。将上述约束优化算法应用于典型约束优化问题,其仿真结果表明了所提出算法的可行性和收敛性。     

基于罚函数内点法的泄露积分型回声状态网的参数优化

为了提升泄露积分型回声状态网（Leaky integrator echo state network,Leaky-ESN）的性能,提出利用罚函数内点法优化Leaky-ESN的全局参数,如泄漏率、内部连接权矩阵谱半径、输入比例因子等,这克服了通过反复试验法选取参数值而降低了Leaky-ESN模型的优越性和性能.Leaky-ESN的全局参数必须保障回声状态网满足回声状态特性,因此它们之间存在不等式约束条件.有学者提出利用随机梯度下降法来优化内部连接权矩阵谱半径、输入比例因子、泄露率三个全局参数,一定程度上提高了Leaky-ESN的逼近精度.然而,随机梯度下降法是解决无约束优化问题的基本算法,在利用随机梯度下降法优化参数时,没有考虑参数必须满足回声特性的约束条件（不等式约束条件）,致使得到的参数值不是最优解.由于罚函数内点法可以求解具有不等式约束的最优化问题,应用范围广,收敛速度较快,具有很强的全局寻优能力.因此,本文提出利用罚函数内点法优化Leaky-ESN的全局参数,并以时间序列预测为例,检验优化后的Leaky-ESN的预测性能,仿真结果表明了本文提出方法的有效性.     

Efficient parallel coordinate descent algorithm for convex optimization problems with separable constraints: Application to distributed MPC

In this paper we propose a parallel coordinate descent algorithm for solving smooth convex optimization problems with separable constraints that may arise, e.g. in distributed model predictive control (MPC) for linear network systems. Our algorithm is based on block coordinate descent updates in parallel and has a very simple iteration. We prove (sub)linear rate of convergence for the new algorithm under standard assumptions for smooth convex optimization. Further, our algorithm uses local information and thus is suitable for distributed implementations. Moreover, it has low iteration complexity, which makes it appropriate for embedded control. An MPC scheme based on this new parallel algorithm is derived, for which every subsystem in the network can compute feasible and stabilizing control inputs using distributed and cheap computations. For ensuring stability of the MPC scheme, we use a terminal cost formulation derived from a distributed synthesis. Preliminary numerical tests show better performance for our optimization algorithm than other existing methods.