A high-performance feedback neural network for solving convex nonlinear programming problems

Based on a new idea of successive approximation, this paper proposes a high-performance feedback neural network model for solving convex nonlinear programming problems. Differing from existing neural network optimization models, no dual variables, penalty parameters, or Lagrange multipliers are involved in the proposed network. It has the least number of state variables and is very simple in structure. In particular, the proposed network has better asymptotic stability. For an arbitrarily given initial point, the trajectory of the network converges to an optimal solution of the convex nonlinear programming problem under no more than the standard assumptions. In addition, the network can also solve linear programming and convex quadratic programming problems, and the new idea of a feedback network may be used to solve other optimization problems. Feasibility and efficiency are also substantiated by simulation examples.     

Spectral projected subgradient with a momentum term for the Lagrangean dual approach

The Lagrangean dual problem, with a non-differentiable convex objective function, is usually solved by using the subgradient method, whose convergence is guaranteed if the optimal value of the dual objective function is known. In practice, this optimal value is approximated by a previously computed bound. In this work, we combine the subgradient method with a different choice of steplength, based on the recently developed spectral projected gradient method, that does not require either exact or approximated estimates of the optimal value. We also add a momentum term to the subgradient direction that accelerates the convergence process towards global solutions. To illustrate the behavior of our new algorithm we solve Lagrangean dual problems associated with integer programming problems. In particular, we present and discuss encouraging numerical results for set covering problems and generalized assignment problems.     

Inexact dual averaging method for distributed multi-agent optimization

We consider a distributed convex optimization problem over a network where multiple agents collectively try to minimize a sum of local convex functions of the same variables, each of which is available to one specific agent only. For solving this optimization problem, we present an inexact version of the dual averaging method. This extends recent results of Duchi (2012), which cover the error-free case, to the case where an error is present in calculating the subgradient of the objective function or in computing the projection. We show that when the errors decrease at appropriate rates, our method achieves the same convergence rate as in the error-free case. In particular, the convergence of the method is also established for nonsummable errors. We also provide numerical results to validate the theoretic results.     

基于极大熵差分进化混合算法求解非线性方程组*

针对非线性方程组,给出了一种新的算法——极大熵差分进化混合算法。首先把非线性方程组转换为一个不可微优化问题;然后用一个称之为凝聚函数的光滑函数直接代替不可微的极大值函数,从而可把非线性方程组的求解转换为无约束优化问题,利用差分进化算法对其进行求解。计算结果表明,该算法在求解的准确性和有效性均优于其他算法。     

Necessary and sufficient conditions for distributed constrained optimal consensus under bounded input

In this paper, we study a distributed constrained optimal consensus problem for discrete‐time first‐order integrator systems under bounded input. Each agent is assigned with a local convex cost function, and all agents are required to achieve consensus at the minimum of the aggregate cost over a common convex constraint set, which is only accessible by part of the agents. A 2‐step control protocol is designed to solve the problem under bounded input. Firstly, at each time step, each agent moves a bounded step of the subgradient descent from an individual cost and another one along the projection direction if it can access the constraint. The second movement is then given by a bounded average of the relative positions to neighbors. Specifically, to coordinate the subgradient step within the network without using global information, we introduce an estimate of the upper bound of all agents' subgradients, which is updated by a unilateral consensus mechanism. Under the given control protocol, we obtain the necessary and sufficient conditions to achieve the constrained optimal consensus for a fixed topology. Under similar conditions, we also solve the problem for switching topologies and conduct a convergence rate analysis for strongly convex costs.     

A recurrent neural network for nonlinear optimization with a continuously differentiable objective function and bound constraints

This paper presents a continuous-time recurrent neural-network model for nonlinear optimization with any continuously differentiable objective function and bound constraints. Quadratic optimization with bound constraints is a special problem which can be solved by the recurrent neural network. The proposed recurrent neural network has the following characteristics. 1) It is regular in the sense that any optimum of the objective function with bound constraints is also an equilibrium point of the neural network. If the objective function to be minimized is convex, then the recurrent neural network is complete in the sense that the set of optima of the function with bound constraints coincides with the set of equilibria of the neural network. 2) The recurrent neural network is primal and quasiconvergent in the sense that its trajectory cannot escape from the feasible region and will converge to the set of equilibria of the neural network for any initial point in the feasible bound region. 3) The recurrent neural network has an attractivity property in the sense that its trajectory will eventually converge to the feasible region for any initial states even at outside of the bounded feasible region. 4) For minimizing any strictly convex quadratic objective function subject to bound constraints, the recurrent neural network is globally exponentially stable for almost any positive network parameters. Simulation results are given to demonstrate the convergence and performance of the proposed recurrent neural network for nonlinear optimization with bound constraints.     

Secure linear system computation in the presence of malicious adversaries

In this paper,we study the system of linear equation problems in the two-party computation setting.Consider that P1 holds an m×m matrix M1 and an m-dimensional column vector B1.Similarly,P2holds M2 and B2.Via executing a secure linear system computation,P1 gets the output x(or⊥)conditioned on(M1+M2)x=(B1+B2),and the rank of matrix M1+M2,while P2 gets nothing.This also can be used to settle other cooperative linear system problems.We firstly design an efficient protocol to solve this problem in the presence of malicious adversaries,then propose a simple way to modify our protocol for having a precise functionality,in which the rank of matrix M1+M2 is not necessary.We note that our protocol is more practical than these existing malicious secure protocols.We also give comparisons with other protocols and extensions to similar functions.     

The incremental subgradient methods on distributed estimations in-network

A unified framework on distributed estimations in-network using incremental subgradient（IS） methods is introduced. The IS methods meet the requirement of the asynchronously processing,which are efficient solvers developed recently focusing on separable non-differentiable convex optimization problems. The main contribution of this paper is to formalize distributed estimators as equivalent separable convex optimization problems,where general skills and several specific cases on signal estimations are presented. Analytical and simulation results show that the IS framework can solve general nonlinear estimation problems in-network,and achieve comparable performances as the centralized estimators.     

求解一类不可微多目标优化问题的社会认知算法*

针对一类不可微多目标优化问题,给出了一个新的算法——极大熵社会认知算法。利用极大熵方法将带有约束的不可微多目标优化问题转化为无约束单目标优化问题,然后利用社会认知算法对其进行求解。该算法是基于社会认知理论,通过一系列的学习代理来模拟人类的社会性和智能性从而完成对目标的优化。利用两个测试算例对其进行测试并与其他算法进行比较,计算结果表明,该算法在求解的准确性和有效性方面均优于其他算法。     

基于自适应罚函数的QPSO算法的代谢通量评估*

针对代谢通量评估问题属于带约束的优化问题,其目标函数是一个非线性、不可微的并且存在多个局部最小点的复杂函数,提出了使用自适应罚函数的量子粒子群优化算法来解决这个问题。通过自适应罚函数的方法解决约束条件,然后使用QPSO算法最小化内部代谢通量。用此算法评估谷氨酸棒杆菌的内部代谢通量并与传统的优化算法来比较,实验结果证明了该算法的可行性和有效性。     

Inexact proximal stochastic second-order methods for nonconvex composite optimization

ABSTRACT

In this paper, we propose a framework of Inexact Proximal Stochastic Second-order (IPSS) method for solving nonconvex optimization problems, whose objective function consists of an average of finitely many, possibly weakly, smooth functions and a convex but possibly nonsmooth function. At each iteration, IPSS inexactly solves a proximal subproblem constructed by using some positive definite matrix which could capture the second-order information of original problem. Proper tolerances are given for the subproblem solution in order to maintain global convergence and the desired overall complexity of the algorithm. Under mild conditions, we analyse the computational complexity related to the evaluations on the component gradient of the smooth function. We also investigate the number of evaluations of subgradient when using an iterative subgradient method to solve the subproblem. In addition, based on IPSS, we propose a linearly convergent algorithm under the proximal Polyak–?ojasiewicz condition. Finally, we extend the analysis to problems with weakly smooth function and obtain the computational complexity accordingly.     

A new gradient-based neural network for solving linear andquadratic programming problems

A new gradient-based neural network is constructed on the basis of the duality theory, optimization theory, convex analysis theory, Lyapunov stability theory, and LaSalle invariance principle to solve linear and quadratic programming problems. In particular, a new function F(x, y) is introduced into the energy function E(x, y) such that the function E(x, y) is convex and differentiable, and the resulting network is more efficient. This network involves all the relevant necessary and sufficient optimality conditions for convex quadratic programming problems. For linear programming and quadratic programming (QP) problems with unique and infinite number of solutions, we have proven strictly that for any initial point, every trajectory of the neural network converges to an optimal solution of the QP and its dual problem. The proposed network is different from the existing networks which use the penalty method or Lagrange method, and the inequality constraints are properly handled. The simulation results show that the proposed neural network is feasible and efficient.     

A novel neural network for nonlinear convex programming

In this paper, we present a neural network for solving the nonlinear convex programming problem in real time by means of the projection method. The main idea is to convert the convex programming problem into a variational inequality problem. Then a dynamical system and a convex energy function are constructed for resulting variational inequality problem. It is shown that the proposed neural network is stable in the sense of Lyapunov and can converge to an exact optimal solution of the original problem. Compared with the existing neural networks for solving the nonlinear convex programming problem, the proposed neural network has no Lipschitz condition, no adjustable parameter, and its structure is simple. The validity and transient behavior of the proposed neural network are demonstrated by some simulation results.     

An inexact continuation accelerated proximal gradient algorithm for low n-rank tensor recovery

The low n-rank tensor recovery problem is an interesting extension of the compressed sensing. This problem consists of finding a tensor of minimum n-rank subject to linear equality constraints and has been proposed in many areas such as data mining, machine learning and computer vision. In this paper, operator splitting technique and convex relaxation technique are adapted to transform the low n-rank tensor recovery problem into a convex, unconstrained optimization problem, in which the objective function is the sum of a convex smooth function with Lipschitz continuous gradient and a convex function on a set of matrices. Furthermore, in order to solve the unconstrained nonsmooth convex optimization problem, an accelerated proximal gradient algorithm is proposed. Then, some computational techniques are used to improve the algorithm. At the end of this paper, some preliminary numerical results demonstrate the potential value and application of the tensor as well as the efficiency of the proposed algorithm.     

模糊时间窗VRP的动态规划和禁忌搜索混合算法

为优化具有模糊时间窗的车辆路径问题,以物流配送成本和顾客平均满意度为目标,建立了多目标数学规划模型。基于Pareto占优的理论给出了求解多目标优化问题的并行多目标禁忌搜索算法,算法中嵌入同时优化顾客满意度的动态规划方法,运用阶段划分,把原问题分解为关于紧路径的优化子问题。对模糊时间窗为线性分段函数形式和非线性凹函数形式的隶属度函数,分别提出了次梯度有限迭代算法和次梯度中值迭代算法来优化顾客的最优开始服务时间。通过Solomon的标准算例,与次梯度投影算法的比较验证了动态规划方法优化服务水平的有效性,与主流的NSGA-II算法的对比实验表明了该研究提出的多目标禁忌搜索算法的优越性。     

An improved stepsize of the subgradient algorithm for solving the lagrangian relaxation problem

The Lagrangian relaxation approach has been successfully applied to many large-scale mathematical programming problems. The Lagrangian relaxation problem itself is a non-differentiable optimization problem. One of the methods for solving such problem is the subgradient algorithm. In this paper, we propose an improved stepsize of the subgradient algorithm for solving the Lagrangian relaxation problem. Our version of the algorithm may significantly improve the rate of convergence of subgradient algorithm when applied to the solution of Lagrangian relaxation problem. An illustrative numerical example is also given.     

离散时间Hopfield网络的动力系统分析

离散时间的Hopfield网络模型是一个非线性动力系统．对网络的状态变量引入新的能量函数，利用凸函数次梯度性质可以得到网络状态能量单调减少的条件．对于神经元的连接权值且激活函数单调非减(不一定严格单调增加)的Hopfield网络，若神经元激活函数的增益大于权值矩阵的最小特征值，则全并行时渐进收敛；而当网络串行时，只要网络中每个神经元激活函数的增益与该神经元的自反馈连接权值的和大于零即可．同时，若神经元激活函数单调，网络连接权值对称，利用凸函数次梯度的性质，证明了离散时间的Hopfield网络模型全并行时收敛到周期不大于2的极限环．     

Solving general convex nonlinear optimization problems by an efficient neurodynamic model

In this paper, a neural network model is constructed on the basis of the duality theory, optimization theory, convex analysis theory, Lyapunov stability theory and LaSalle invariance principle to solve general convex nonlinear programming (GCNLP) problems. Based on the Saddle point theorem, the equilibrium point of the proposed neural network is proved to be equivalent to the optimal solution of the GCNLP problem. By employing Lyapunov function approach, it is also shown that the proposed neural network model is stable in the sense of Lyapunov and it is globally convergent to an exact optimal solution of the original problem. The simulation results also show that the proposed neural network is feasible and efficient.     

Solving Pseudomonotone Variational Inequalities and Pseudoconvex Optimization Problems Using the Projection Neural Network

In recent years, a recurrent neural network called projection neural network was proposed for solving monotone variational inequalities and related convex optimization problems. In this paper, we show that the projection neural network can also be used to solve pseudomonotone variational inequalities and related pseudoconvex optimization problems. Under various pseudomonotonicity conditions and other conditions, the projection neural network is proved to be stable in the sense of Lyapunov and globally convergent, globally asymptotically stable, and globally exponentially stable. Since monotonicity is a special case of pseudomononicity, the projection neural network can be applied to solve a broader class of constrained optimization problems related to variational inequalities. Moreover, a new concept, called componentwise pseudomononicity, different from pseudomononicity in general, is introduced. Under this new concept, two stability results of the projection neural network for solving variational inequalities are also obtained. Finally, numerical examples show the effectiveness and performance of the projection neural network     

Utility-based scheduling in wireless multi-hop networks over non-deterministic fading channels

In this paper, an innovative scheduling scheme is proposed for interference-limited wireless multi-hop networks with non-deterministic fading channels. The scheduling problem is considered as a network utility maximization (NUM) problem subject to link rate constraints. By jointly taking into account of the link scheduling and the statistical variations of signal and interference power, the convex sets for the NUM are derived. Two types of non-deterministic fading channels (i.e., Rayleigh fading channel and Ricean fading channel) are characterized into our NUM models as examples. To solve the convex optimization problem, the subgradient projection method based on dual decomposition is employed. Then, a heuristic algorithm is designed for the TDM mode wireless multi-hop networks by minimizing the discrepancy between the expected network cost and the optimal one in each timeslot. At last, the source–destination session rate and network utility are evaluated in a dedicated wireless multi-hop network scenario. The numerical results demonstrate that the session rates convergence and the network utility is improved by our proposed scheme.