The dual of the dual as an approximation of the primal

A formulation and some properties of the (Lagrangian) dual of the dual of a mathematical programming problem were presented in an article by Unger and Hurter (1972). Here, that formulation is simplified, so that the resulting form is more closely related to the primal. In particular, the dual of the dual is shown to be merely the primal, except that the objective function and each constraint are replaced by their first-order Taylor series approximation.

This formulation is used to obtain new bounds for the primal, and necessary conditions for the convexity of the dual problem, and for the absence of a duality gap. The relationship of the dual of the dual to condensation in geometric programming is described. The dual of the dual is related to several existing mathematical programming algorithms.     

Iteration complexity analysis of dual first-order methods for conic convex programming

In this paper we provide a detailed analysis of the iteration complexity of dual first-order methods for solving conic convex problems. When it is difficult to project on the primal feasible set described by conic and convex constraints, we use the Lagrangian relaxation to handle the conic constraints and then, we apply dual first-order algorithms for solving the corresponding dual problem. We give convergence analysis for dual first-order algorithms (dual gradient and fast gradient algorithms): we provide sublinear or linear estimates on the primal suboptimality and feasibility violation of the generated approximate primal solutions. Our analysis relies on the Lipschitz property of the gradient of the dual function or an error bound property of the dual. Furthermore, the iteration complexity analysis is based on two types of approximate primal solutions: the last primal iterate or an average primal sequence.     

Training a support vector machine in the primal

Most literature on support vector machines (SVMs) concentrates on the dual optimization problem. In this letter, we point out that the primal problem can also be solved efficiently for both linear and nonlinear SVMs and that there is no reason for ignoring this possibility. On the contrary, from the primal point of view, new families of algorithms for large-scale SVM training can be investigated.     

A primal perspective for indefinite kernel SVM problem

Indefinite kernel support vector machine(IKSVM)has recently attracted increasing attentions in machine learning.Since IKSVM essentially is a non-convex problem,existing algorithms either change the spectrum of indefinite kernel directly but risking losing some valuable information or solve the dual form of IKSVM whereas suffering from a dual gap problem.In this paper,we propose a primal perspective for solving the problem.That is,we directly focus on the primal form of IKSVM and present a novel algorithm termed as IKSVM-DC for binary and multi-class classification.Concretely,according to the characteristics of the spectrum for the indefinite kernel matrix,IKSVM-DC decomposes the primal function into the subtraction of two convex functions as a difference of convex functions(DC)programming.To accelerate convergence rate,IKSVM-DC combines the classical DC algorithm with a line search step along the descent direction at each iteration.Furthermore,we construct a multi-class IKSVM model which can classify multiple classes in a unified form.A theoretical analysis is then presented to validate that IKSVM-DC can converge to a local minimum.Finally,we conduct experiments on both binary and multi-class datasets and the experimental results show that IKSVM-DC is superior to other state-of-the-art IKSVM algorithms.     

Implementation and analysis of alternative algorithms for generalized shortest path problems

This paper addresses a special class of deterministic dynamic programming problems which can be formulated as a generalized network problem. Because of the similarities between this class of dynamic programming problems and shortest path problems, we are naming it the Generalized Shortest Path problem. A new primal extreme point algorithm and a new special dual algorithm are proposed here. While researchers have presented a variety of algorithms to solve this class of problems, there has been no comuptational analysis of these algorithms. An in-depth computational analysis is performed to determine the most efficient set of rules for implementing the algorithms of this paper.     

Robust linear and support vector regression

The robust Huber M-estimator, a differentiable cost function that is quadratic for small errors and linear otherwise, is modeled exactly, in the original primal space of the problem, by an easily solvable simple convex quadratic program for both linear and nonlinear support vector estimators. Previous models were significantly more complex or formulated in the dual space and most involved specialized numerical algorithms for solving the robust Huber linear estimator. Numerical test comparisons with these algorithms indicate the computational effectiveness of the new quadratic programming model for both linear and nonlinear support vector problems. Results are shown on problems with as many as 20000 data points, with considerably faster running times on larger problems     

On concave constraint functions and duality in predominantly black-and-white topology optimization

We study the ‘classical’ discrete, solid-void or black-and-white topology optimization problem, in which minimum compliance is sought, subject to constraints on the available material resource. We assume that this problem is solved using methods that relax the discreteness requirements during intermediate steps, and that the associated programming problems are solved using sequential approximate optimization (SAO) algorithms based on duality. More specifically, we assume that the advantages of the well-known Falk dual are exploited. Such algorithms represent the state-of-the-art in (large-scale) topology optimization when multiple constraints are present; an important example being the method of moving asymptotes (MMA).We depart by noting that the aforementioned SAO algorithms are invariably formulated using strictly convex subproblems. We then numerically illustrate that strictly concave constraint functions, like those present in volumetric penalization, as recently proposed by Bruns and co-workers, may increase the difficulty of the topology optimization problem when strictly convex approximations are used in the SAO algorithm. In turn, volumetric penalization methods are of notable importance, since they seem to hold much promise for generating predominantly solid-void or discrete designs.We then argue that the nonconvex problems we study may in some instances efficiently be solved using dual SAO methods based on nonconvex (strictly concave) approximations which exhibit monotonicity with respect to the design variables.Indeed, for the topology problem resulting from SIMP-like volumetric penalization, we show explicitly that convex approximations are not necessary. Even though the volumetric penalization constraint is strictly concave, the maximum of the resulting dual subproblem still corresponds to the optimum of the original primal approximate subproblem.     

Prox-regularization of the dual method of centers for generalized fractional programs

We present in this paper a prox-dual regularization algorithm for solving generalized fractional programming problems. The algorithm combines the dual method of centres for generalized fractional programs and the proximal point algorithm and can handle nondifferentiable convex problems with possibly unbounded feasible constraints set. The proposed procedure generates two sequences of dual and primal values that approximate the optimal value of the considered problem respectively from below and from above at each step. It also generates a sequence of dual solutions that converges to a solution of the dual problem, and a sequence of primal solutions whose every accumulation point is a solution of the primal problem. For a class of problems, including linear fractional programs, the algorithm converges linearly.     

Degree-constrained minimum spanning tree

In this paper the problem of a degree-constrained minimum spanning tree (DCMST) is defined. The problem is formulated as a linear 0–1 integer programming problem. A primal and a dual heuristic (construction) procedure and a branch-and-bound algorithm are proposed to construct a DCMST. These procedures are illustrated with a simple example. Some computational experience with these algorithms is also reported.     

基于高阶马尔科夫随机场的图像去噪声研究

在图像去噪声处理中,高阶马尔科夫随机场通过最小化能量函数达到最优的去噪声结果。为了提高能量函数的优化性能,本文在马尔科夫随机场子模性的基础上对原始问题和对偶问题进行了分析,提出了一种基于原始-对偶方法的子模块之和方法。首先,描述了马尔科夫随机场的线性规划及其对偶问题,并介绍了子模块之和流方法。接下来,通过对子模块之和流方法的原始问题和对偶问题进行分析,提出了同时满足派系松弛和一元松弛条件的近似解计算方法。实验表明,本文提出的方法与四种典型的图像去噪声方法相比具有更好的效果和更短的运行时间。     

Embedded MPC Controller Based on Interior‐Point Method with Convergence Depth Control

To allow the implementation of model predictive control on the chip, we first propose a primal–dual interior point method with convergence depth control to solve the quadratic programming problem of model predictive control. Compared with algorithms based on traditional termination criterion, the proposed method can significantly reduce the computation cost while obtaining an approximate solution of the quadratic programming problem with acceptable optimality and precision. Thereafter, an embedded model predictive controller based on the quadratic programming solver is designed and implemented on a digital signal processor chip and a prototype system is built on a TMDSEVM6678LE digital signal processor chip. The controller is verified on two models by using the hardware in loop frame to mimic real applications. The comparison shows that the whole design is competitive in real‐time applications. The typical computation time for quadratic programming problems with 5 decision variables and 110 constraints can be reduced to less than 2 ms on an embedded platform.     

The second-order cone eigenvalue complementarity problem

The eigenvalue complementarity problem (EiCP) differs from the traditional eigenvalue problem in that the primal and dual variables belong to a closed and convex cone K and its dual, respectively, and satisfy a complementarity condition. In this paper we investigate the solution of the second-order cone EiCP (SOCEiCP) where K is the Lorentz cone. We first show that the SOCEiCP reduces to a special Variational Inequality Problem on a compact set defined by K and a normalization constraint. This guarantees that SOCEiCP has at least one solution, and a new enumerative algorithm is introduced for finding a solution to this problem. The method is based on finding a global minimum of an appropriate nonlinear programming (NLP) formulation of the SOCEiCP using a special branching scheme along with a local nonlinear optimizer that computes stationary points on subsets of the feasible region of NLP associated with the nodes generated by the algorithm. A semi-smooth Newton's method is combined with this enumerative algorithm to enhance its numerical performance. Our computational experience illustrates the efficacy of the proposed techniques in practice.     

多下层分式双层规划问题的改进遗传算法

针对一类具有多个线性下层问题的分式双层规划, 提出一种基于新编码方式的遗传算法。 首先,利用对偶理论,将问题化为单层非线性规划;接着,利用下层对偶问题的可行基编码,针对任意编码个体,解出对偶变量值,使得单层规划变为线性分式规划;最后,求解产生的线性分式规划,其目标值作为个体的适应度值。 这种编码方式及适应度的计算有效提高了遗传算法的效率。 通过对4个算例的计算,验证了算法的有效性。     

A Neural Network for Solving Nonlinear Programming Problems

A neural network for solving convex nonlinear programming problems is proposed in this paper. The distinguishing features of the proposed network are that the primal and dual problems can be solved simultaneously, all necessary and sufficient optimality conditions are incorporated, and no penalty parameter is involved. Based on Lyapunov, LaSalle and set stability theories, we prove strictly an important theoretical result that, for an arbitrary initial point, the trajectory of the proposed network does converge to the set of its equilibrium points, regardless of whether a convex nonlinear programming problem has unique or infinitely many optimal solutions. Numerical simulation results also show that the proposed network is feasible and efficient. In addition, a general method for transforming non-linear programming problems into unconstrained problems is also proposed. ID="A1" Correspondence and offprint requests to: Dr Z Chen, Department of Electronic Engineering, Brunel University, Uxbridge, Middle-sex, UK     

A Super-Resolution Framework for High-Accuracy Multiview Reconstruction

We present a variational framework to estimate super-resolved texture maps on a 3D geometry model of a surface from multiple images. Given the calibrated images and the reconstructed geometry, the proposed functional is convex in the super-resolution texture. Using a conformal atlas of the surface, we transform the model from the curved geometry to the flat charts and solve it using state-of-the-art and provably convergent primal–dual algorithms. In order to improve image alignment and quality of the texture, we extend the functional to also optimize for a normal displacement map on the surface as well as the camera calibration parameters. Since the sub-problems for displacement and camera parameters are non-convex, we revert to relaxation schemes in order to robustly estimate a minimizer via sequential convex programming. Experimental results confirm that the proposed super-resolution framework allows to recover textured models with significantly higher level-of-detail than the individual input images.     

A dual maximum principle for discrete-time linear systems with economic applications

A discrete-time linear optimal control problem with given initial and terminal times, state control constraints, and variable end points is set forth. Corresponding to this primal control problem, or maximization problem, is a dual linear control problem, or minimization problem. A dual maximum principle is proved with the aid of the duality theory of linear programming, where the dual of the Hamiltonian of the primal control problem is the Hamiltonian of the dual control problem. A discrete-time analog of the Hamilton-Jacobi equation is derived; and economic applications are discussed.     

Scheduling network maintenance jobs with release dates and deadlines to maximize total flow over time: Bounds and solution strategies

We consider a problem that marries network flows and scheduling, motivated by the need to schedule maintenance activities in infrastructure networks, such as rail or general logistics networks. Network elements must undergo regular preventive maintenance, shutting down the arc for the duration of the activity. Careful coordination of these arc maintenance jobs can dramatically reduce the impact of such shutdown jobs on the flow carried by the network. Scheduling such jobs between given release dates and deadlines so as to maximize the total flow over time presents an intriguing case to study the role of time discretization. Here we prove that if the problem data is integer, and no flow can be stored at nodes, we can restrict attention to integer job start times. However if flow can be stored, fractional start times may be needed. This makes traditional strong integer programming scheduling models difficult to apply. Here we formulate an exact integer programming model for the continuous time problem, as well as integer programming models based on time discretization that can provide dual bounds, and that can – with minor modifications – also yield primal bounds. The resulting bounds are demonstrated to have small gaps on test instances, and offer a good trade-off for bound quality against computing time.     

Modeling and solving a capacitated stochastic location-allocation problem using sub-sources

We study a capacitated multi-facility location-allocation problem in which the customers have stochastic demands based on Bernoulli distribution function. We consider the capacitated sub-sources of facilities to satisfy demands of customers. In the discrete stochastic problem, the goal is to find optimal locations of facilities among candidate locations and optimal allocations of existing customers to operating facilities so that the total sum of fixed costs of operating facilities, allocation cost of the customers, expected values of servicing and outsourcing costs is minimized. The model is formulated as a mixed-integer nonlinear programming problem. Since finding an optimal solution may require an excessive amount of time depending on nonlinear constraints, we transform the nonlinear constraints of the problem to linear ones to arrive at a simple formulation of the model. Numerical results show that the LINGO 9.0 software is capable of solving small size problems. For medium and large-size problems, we propose two meta-heuristic algorithms, namely a genetic algorithm and a discrete version of colonial competitive algorithm. Computational results show that the proposed algorithms efficiently obtain effective solutions.     

An advanced start algorithm for all-integer programming

In this paper, we present an all-integer cutting plane technique called the Advanced Start Algorithm (ASA), for solving the all-integer (otherwise linear) programming problem (IP). We develop a good advanced primal-infeasible start based on the optimal solution to the LP relaxation, and use a two-stage dual/primal algorithm to obtain the optimal solution to (IP). We illustrate the operation of the ASA on three small problems, and exhibit computational results on a set of standard test problems.     

Adaptive coordinate sampling for stochastic primal–dual optimization

We consider the regularized empirical risk minimization (ERM) of linear predictors, which arises in a variety of problems in machine learning and statistics. After reformulating the original ERM as a bilinear saddle-point problem, we can apply stochastic primal–dual methods to solve it. Sampling the primal or dual coordinates with a fixed nonuniform distribution is usually employed to accelerate the convergence of the algorithm, but such a strategy only exploits the global information of the objective function. To capture its local structures, we propose an adaptive importance sampling strategy that chooses the coordinates based on delicately designed nonuniform and nonstationary distributions. When our adaptive coordinate sampling strategy is applied to the Stochastic Primal-Dual Coordinate (SPDC), we prove that the resulting algorithm enjoys linear convergence. Moreover, we show that the ideal form of our adaptive sampling exhibits strictly sharper convergence rate under certain conditions compared with the vanilla SPDC. We also extend our sampling strategy to other algorithms including Doubly Stochastic Primal-Dual Coordinate (DSPDC) and Stochastic Primal-Dual with O(1) per-iteration cost and Variance Reduction (SPD1-VR), where both primal and dual coordinates are randomly sampled. Our experiment results show that the proposed strategy significantly improves the convergence performance of the methods when compared with existing sampling strategies.