Distributed estimation of Laplacian eigenvalues via constrained consensus optimization problems

From the recent literature, we know that some consecutive measurements of the consensus protocol can be used to compute the exact average of the initial condition. In this paper, we show that these measurements can also be used for estimating the Laplacian eigenvalues of the graph representing the network. As recently shown in the literature, by solving the factorization of the averaging matrix, the Laplacian eigenvalues can be inferred. Herein, the problem is posed as a constrained consensus problem formulated two-fold. The first formulation (direct approach) yields a non-convex optimization problem solved in a distributed way by means of the method of Lagrange multipliers. The second formulation (indirect approach) is obtained after an adequate re-parameterization. The problem is then convex and is solved by using the distributed subgradient algorithm and the alternating direction method of multipliers (ADMM). The proposed algorithms allow estimating the actual Laplacian eigenvalues with high accuracy.     

A hybrid method for hydroelectric generation scheduling using an artificial neural network

This paper presents a decomposition method for finding an optimal operating policy of interconnected hydroelectric power plants using an artificial neural network. The coupling constraints on reservoir storage at the end of the planning horizon are relaxed using coordinating multipliers that result in interval wise decomposition of the overall problem. Resulting subproblems are solved sequentially, which reduces the complexity of the problem. Each subproblem is solved using a two-phase neural network approach. An efficient heuristic algorithm is developed to find the feasible solution. A case study considering scheduling of the Bhakra-Beas reservoir system is also presented in this paper. The new method demonstrates the potential of achieving an improved performance.     

Large-scale convex optimal control problems: time decomposition,incentive coordination, and parallel algorithm

A parallel algorithm based on time decomposition and incentive coordination is developed for long-horizon optimal control problems. This is done by first decomposing the original problem into subproblems with shorter time horizon, and then using the incentive coordination scheme to coordinate the interaction of subproblems. For strictly convex problems it is proved that the decomposed problem with linear incentive coordination is equivalent to the original problem, in the sense that each optimal solution of the decomposed problem produces one global optimal solution of the original problem and vice versa. In other words, linear incentive terms are sufficient in this case and impose no additional computation burden on the subproblems. The high-level parameter optimization problem is shown to be nonconvex, despite the uniqueness of the optimal solution and the convexity of the original problem. Nevertheless, the high-level problem has no local minimum, even though it is nonconvex. A parallel algorithm based on a prediction method is developed, and a numerical example is used to demonstrate the feasibility of the approach     

Explicit/multi-parametric model predictive control (MPC) of linear discrete-time systems by dynamic and multi-parametric programming

This work presents a new algorithm for solving the explicit/multi-parametric model predictive control (or mp-MPC) problem for linear, time-invariant discrete-time systems, based on dynamic programming and multi-parametric programming techniques. The algorithm features two key steps: (i) a dynamic programming step, in which the mp-MPC problem is decomposed into a set of smaller subproblems in which only the current control, state variables, and constraints are considered, and (ii) a multi-parametric programming step, in which each subproblem is solved as a convex multi-parametric programming problem, to derive the control variables as an explicit function of the states. The key feature of the proposed method is that it overcomes potential limitations of previous methods for solving multi-parametric programming problems with dynamic programming, such as the need for global optimization for each subproblem of the dynamic programming step.     

CONLIN: An efficient dual optimizer based on convex approximation concepts

The Convex Linearization method (CONLIN) exhibits many interesting features and it is applicable to a broad class of structural optimization problems. The method employs mixed design variables (either direct or reciprocal) in order to get first order, conservative approximations to the objective function and to the constraints. The primary optimization problem is therefore replaced with a sequence of explicit approximate problems having a simple algebraic structure. The explicit subproblems are convex and separable, and they can be solved efficiently by using a dual method approach.In this paper, a special purpose dual optimizer is proposed to solve the explicit subproblem generated by the CONLIN strategy. The maximum of the dual function is sought in a sequence of dual subspaces of variable dimensionality. The primary dual problem is itself replaced with a sequence of approximate quadratic subproblems with non-negativity constraints on the dual variables. Because each quadratic subproblem is restricted to the current subspace of non zero dual variables, its dimensionality is usually reasonably small. Clearly, the Hessian matrix does not need to be inverted (it can in fact be singular), and no line search process is necessary.An important advantage of the proposed maximization method lies in the fact that most of the computational effort in the iterative process is performed with reduced sets of primal variables and dual variables. Furthermore, an appropriate active set strategy has been devised, that yields a highly reliable dual optimizer.     

Hydrothermal scheduling problems with pumped-storage hydro plants

In this paper, we developed an algorithm in the calculation of hydrothermal generation schedules with pumped-storage plants. In our algorithm, the system is decomposed into a hydro subproblem and a thermal subproblem by using Lagrange relaxation. The hydro subproblem is solved using Mixed Coordination of [TAN91a], and the thermal subproblem is solved analytically. A method is developed based on the Lagrange multipliers (incremental cost rates) in deciding the operation of the pumped-storage plants. Numerical tests show that by properly operating the pumped-storage hydro plants, savings on fuel-cost can be achieved.     

Modified augmented Lagrangian coordination and alternating direction method of multipliers with parallelization in non-hierarchical analytical target cascading

Analytical Target Cascading (ATC) is a decomposition-based optimization methodology that partitions a system into subsystems and then coordinates targets and responses among subsystems. Augmented Lagrangian with Alternating Direction method of multipliers (AL-AD), one of efficient ATC coordination methods, has been widely used in both hierarchical and non-hierarchical ATC and theoretically guarantees convergence under the assumption that all subsystem problems are convex and continuous. One of the main advantages of distributed coordination which consists of several non-hierarchical subproblems is that it can solve subsystem problems in parallel and thus reduce computational time. Therefore, previous studies have proposed an augmented Lagrangian coordination strategy for parallelization by eliminating interactions among subproblems. The parallelization is achieved by introducing a master problem and support variables or by approximating a quadratic penalty function to make subproblems separable. However, conventional AL-AD does not guarantee convergence in the case of parallel solving. Our study shows that, in parallel solving using targets and responses of the current iteration, conventional AL-AD causes mismatch of information in updating the Lagrange multiplier. Therefore, the Lagrange multiplier may not reach the optimal point, and as a result, increasing penalty weight causes numerical difficulty in the augmented Lagrangian coordination approach. To solve this problem, we propose a modified AL-AD with parallelization in non-hierarchical ATC. The proposed algorithm uses the subgradient method with adaptive step size in updating the Lagrange multiplier and also maintains penalty weight at an appropriate level not to cause oscillation. Without approximation or introduction of an artificial master problem, the modified AL-AD with parallelization can achieve similar accuracy and convergence with much less computational cost compared with conventional AL-AD with sequential solving.     

On concave constraint functions and duality in predominantly black-and-white topology optimization

We study the ‘classical’ discrete, solid-void or black-and-white topology optimization problem, in which minimum compliance is sought, subject to constraints on the available material resource. We assume that this problem is solved using methods that relax the discreteness requirements during intermediate steps, and that the associated programming problems are solved using sequential approximate optimization (SAO) algorithms based on duality. More specifically, we assume that the advantages of the well-known Falk dual are exploited. Such algorithms represent the state-of-the-art in (large-scale) topology optimization when multiple constraints are present; an important example being the method of moving asymptotes (MMA).We depart by noting that the aforementioned SAO algorithms are invariably formulated using strictly convex subproblems. We then numerically illustrate that strictly concave constraint functions, like those present in volumetric penalization, as recently proposed by Bruns and co-workers, may increase the difficulty of the topology optimization problem when strictly convex approximations are used in the SAO algorithm. In turn, volumetric penalization methods are of notable importance, since they seem to hold much promise for generating predominantly solid-void or discrete designs.We then argue that the nonconvex problems we study may in some instances efficiently be solved using dual SAO methods based on nonconvex (strictly concave) approximations which exhibit monotonicity with respect to the design variables.Indeed, for the topology problem resulting from SIMP-like volumetric penalization, we show explicitly that convex approximations are not necessary. Even though the volumetric penalization constraint is strictly concave, the maximum of the resulting dual subproblem still corresponds to the optimum of the original primal approximate subproblem.     

Exponential penalty function formulation for multilevel optimization using the analytical target cascading framework

An exponential penalty function (EPF) formulation based on method of multipliers is presented for solving multilevel optimization problems within the framework of analytical target cascading. The original all-at-once constrained optimization problem is decomposed into a hierarchical system with consistency constraints enforcing the target-response coupling in the connected elements. The objective function is combined with the consistency constraints in each element to formulate an augmented Lagrangian with EPF. The EPF formulation is implemented using double-loop (EPF I) and single-loop (EPF II) coordination strategies and two penalty-parameter-updating schemes. Four benchmark problems representing nonlinear convex and non-convex optimization problems with different number of design variables and design constraints are used to evaluate the computational characteristics of the proposed approaches. The same problems are also solved using four other approaches suggested in the literature, and the overall computational efficiency characteristics are compared and discussed.     

二阶总广义变分小波修复模型

目的 针对全变分小波修复模型易导致阶梯效应的缺陷,提出一种加权的二阶总广义变分小波修复模型。方法 不同于全变分小波修复模型,假设的新模型引入二阶导数项且能够自动地调解一阶和二阶导数项。另外,为有效地利用图像的局部结构信息,新模型引入了权函数,它既能保护图像的边缘又增强光滑区域的去噪能力。 为有效地计算新模型,利用交替方向法将该模型变为两个子模型, 然后对两个子模型分别给出相应的理论和算法推导。结果 相比最近基于全变分正则小波修复模型(平均信噪比,平均绝对误差及平均结构相似性指标分别为21.884 4,6.857 8,0.827 2),新模型得到更好的修复效果(平均信噪比,平均绝对误差及平均结构相似性指标分别为22.313 8,6.626 1,0.831 8)。结论 与全变分正则相比,二阶总广义变分正则更好地减轻阶梯效应。目前, 国内外学者对该问题的研究取得一些结果。由于原始-对偶算法需要较小的参数,所以运算的速度较慢,因此更快速的算法理论有待进一步研究。另外,该正则能应用于图像去噪、分割、放大等方面。     

Minimum weight design of membrane structures

Algorithms to find the minimum weight design of a 3-dimensional membrane—rod structure are presented. Constraints are on strength and displacements. Variables are ply thicknesses, cross section areas, angles of orthotropy and node point coordinates in a FE-approximation.

To solve the optimization problem a sequence of strictly convex subproblems is created. Each subproblem is solved by using the duality theory for convex programming.

Results are presented for a delta-wing, a tower and a bridge structure.     

An operational optimization procedure for production scheduling

The methodology takes a sufficiently long time horizon and breaks the problem into two subproblems. The first subproblem is the long range planning model and the second the short run production scheduling model. The long range model is essentially a resource constrained model and has a linear programming formulation with a profit maximization objective function. The long range plan fixes the discretionary marketing variables, such as the selection of product line, and the timing and extent of promotional sales. It estimates manpower requirements and establishes the raw material procurement plans. Lagrange multipliers obtained in the long range model are then used in the short run production scheduling model. The scheduling algorithm, having a Lagrangian function for an objective, is the solution to an unconstrained maximization problem. This then reduces to one of sequential allocation of production facilities to products. The algorithm is being applied on a problem with five production lines, 126 products, 26 time periods and 32 raw material constraints.     

An augmented Lagrangian relaxation for analytical target cascading using the alternating direction method of multipliers

Analytical target cascading is a method for design optimization of hierarchical, multilevel systems. A quadratic penalty relaxation of the system consistency constraints is used to ensure subproblem feasibility. A typical nested solution strategy consists of inner and outer loops. In the inner loop, the coupled subproblems are solved iteratively with fixed penalty weights. After convergence of the inner loop, the outer loop updates the penalty weights. The article presents an augmented Lagrangian relaxation that reduces the computational cost associated with ill-conditioning of subproblems in the inner loop. The alternating direction method of multipliers is used to update penalty parameters after a single inner loop iteration, so that subproblems need to be solved only once. Experiments with four examples show that computational costs are decreased by orders of magnitude ranging between 10 and 1000.     

A high-performance feedback neural network for solving convex nonlinear programming problems

Based on a new idea of successive approximation, this paper proposes a high-performance feedback neural network model for solving convex nonlinear programming problems. Differing from existing neural network optimization models, no dual variables, penalty parameters, or Lagrange multipliers are involved in the proposed network. It has the least number of state variables and is very simple in structure. In particular, the proposed network has better asymptotic stability. For an arbitrarily given initial point, the trajectory of the network converges to an optimal solution of the convex nonlinear programming problem under no more than the standard assumptions. In addition, the network can also solve linear programming and convex quadratic programming problems, and the new idea of a feedback network may be used to solve other optimization problems. Feasibility and efficiency are also substantiated by simulation examples.     

Dantzig-Wolfe decomposition and branch-and-price solving in G12

The G12 project is developing a software environment for stating and solving combinatorial problems by mapping a high-level model of the problem to an efficient combination of solving methods. Model annotations are used to control this process. In this paper we explain the mapping to branch-and-price solving. Dantzig-Wolfe decomposition is automatically performed using the additional information given by the model annotations. The models obtained can then be solved using column generation and branch-and-price. G12 supports the selection of specialised subproblem solvers, the aggregation of identical subproblems to reduce symmetries, automatic disaggregation when required by branch-and-bound, the use of specialised subproblem constraint-branching rules, and different master problem solvers including a hybrid solver based on the volume algorithm. We demonstrate the benefits of the G12 framework on three examples: a trucking problem, cutting stock, and two-dimensional bin packing.     

An LMI approach for robust iterative learning control with quadratic performance criterion

This paper presents the design of iterative learning control based on quadratic performance criterion (Q-ILC) for linear systems subject to additive uncertainty. The robust Q-ILC design can be cast as a min–max problem. We propose a novel approach which employs an upper bound of the worst-case performance, then formulates a non-convex quadratic minimization problem to get the update of iterative control inputs. Applying Lagrange duality, the Lagrange dual function of the non-convex quadratic problem is equivalent to a convex optimization over linear matrix inequalities (LMIs). An LMI algorithm with convergence properties is then given for the robust Q-ILC design. Finally, we provide a numerical example to illustrate the effectiveness of the proposed method.     

Multi-modality in augmented Lagrangian coordination for distributed optimal design

This paper presents an empirical study of the convergence characteristics of augmented Lagrangian coordination (ALC) for solving multi-modal optimization problems in a distributed fashion. A number of test problems that do not satisfy all assumptions of the convergence proof for ALC are selected to demonstrate the convergence characteristics of ALC algorithms. When only a local search is employed at the subproblems, local solutions to the original problem are often attained. When a global search is performed at subproblems, global solutions to the original, non-decomposed problem are found for many of the examples. Although these findings are promising, ALC with a global subproblem search may yield only local solutions in the case of non-convex coupling functions or disconnected feasible domains. Results indicate that for these examples both the starting point and the sequence in which subproblems are solved determines which solution is obtained. We illustrate that the main cause for this behavior lies in the alternating minimization inner loop, which is inherently of a local nature.     

基于集结投影次梯度的机组组合算法研究

针对大规模电力系统机组组合问题,提出了基于集结投影次梯度方法的分解协调算法.首先在上层通过拉格朗日松弛方法将原问题分解为多个子问题,从而减小了求解问题的复杂度,避免了维数灾问题,同时显著降低了计算时间,使得原问题可以在多项式时间内求解,随后下层子问题采用动态规划方法很容易求最优解.算例仿真结果表明,所采用的集结投影次梯度方法调整拉格朗日乘子,避免了传统次梯度方法振荡现象严重的缺点,同时加快了收敛速度,得到了令人满意的机组组合方案.     

Primal explicit max margin feature selection for nonlinear support vector machines

Embedding feature selection in nonlinear support vector machines (SVMs) leads to a challenging non-convex minimization problem, which can be prone to suboptimal solutions. This paper develops an effective algorithm to directly solve the embedded feature selection primal problem. We use a trust-region method, which is better suited for non-convex optimization compared to line-search methods, and guarantees convergence to a minimizer. We devise an alternating optimization approach to tackle the problem efficiently, breaking it down into a convex subproblem, corresponding to standard SVM optimization, and a non-convex subproblem for feature selection. Importantly, we show that a straightforward alternating optimization approach can be susceptible to saddle point solutions. We propose a novel technique, which shares an explicit margin variable to overcome saddle point convergence and improve solution quality. Experiment results show our method outperforms the state-of-the-art embedded SVM feature selection method, as well as other leading filter and wrapper approaches.     

An augmented Lagrangian approach to non-convex SAO using diagonal quadratic approximations

Successful gradient-based sequential approximate optimization (SAO) algorithms in simulation-based optimization typically use convex separable approximations. Convex approximations may however not be very efficient if the true objective function and/or the constraints are concave. Using diagonal quadratic approximations, we show that non-convex approximations may indeed require significantly fewer iterations than their convex counterparts. The nonconvex subproblems are solved using an augmented Lagrangian (AL) strategy, rather than the Falk-dual, which is the norm in SAO based on convex subproblems. The results suggest that transformation of large-scale optimization problems with only a few constraints to a dual form via convexification need sometimes not be required, since this may equally well be done using an AL formulation.