Projected Tikhonov Regularization of Large-Scale Discrete Ill-Posed Problems

The solution of linear discrete ill-posed problems is very sensitive to perturbations in the data. Confidence intervals for solution coordinates provide insight into the sensitivity. This paper presents an efficient method for computing confidence intervals for large-scale linear discrete ill-posed problems. The method is based on approximating the matrix in these problems by a partial singular value decomposition of low rank. We investigate how to choose the rank. Our analysis also yields novel approaches to the solution of linear discrete ill-posed problems with solution norm or residual norm constraints.     

Numerical solutions of 3D Cauchy problems of elliptic operators in cylindrical domain using local weak equations and radial basis functions

This paper is concerned with the numerical solutions of 3D Cauchy problems of elliptic differential operators in the cylindrical domain. We assume that the measurements are only available on the outer boundary while the interior boundary is inaccessible and the solution should be obtained from the measurements from the outer layer. The proposed discretization approach uses the local weak equations and radial basis functions. Since the Cauchy problem is known to be ill-posed, the Thikhonov regularization strategy is employed to solve effectively the discrete ill-posed resultant linear system of equations. Numerical results of a different kind of test problems reveal that the method is very effective.     

The levenberg-marquardt method for approximation of solutions of irregular operator equations

An ill-posed problem is considered in the form of a nonlinear operator equation with a discontinuous inverse operator. It is known that in investigating a high convergence of the methods of the type of Levenberg-Marquardt (LM) method, one is forced to impose very severe constraints on the problem operator. In the suggested article the LM method convergence is set up not for the initial problem, but for the Tikhonov-regularized equation. This makes it possible to construct a stable Fejer algorithm for approximation of the solution of the initial irregular problem at the conventional, comparatively nonburdensome conditions on the operator. The developed method is tested on the solution of an inverse problem of geophysics.     

Exact solutions to some minimum time problems with inequality state constraints

A heuristic method is developed for generating exact solutions to certain minimum time problems, with inequality state and control constraints. The control equation is linear and autonomous, with scalar-valued control. The state constraints are also linear inequalities. Assuming knowledge of a finite sequence, in which state and/or control constraints become active along an optimal path, the maximum principle is reduced to a set of equations and inequalities in a finite number of unknowns. A solution to the equations and inequalities determines both the solution path and a proof of its optimality. Certain types of constraint sequences lead to overdetermined equation systems, and this fact is interpreted in terms of the qualitative behavior of solutions to these problems. Two path-planning problems are solved, as illustrations of the solution technique.     

A constrained regularization method for inverting data represented by linear algebraic or integral equations

CONTIN is a portable Fortran IV package for inverting noisy linear operator equations. These problems occur in the analysis of data from a wide variety experiments. They are generally ill-posed problems, which means that errors in an unregularized inversion are unbounded. Instead, CONTIN seeks the optimal solution by incorporating parsimony and any statistical prior knowledge into the regularizor and absolute prior knowledge into equallity and inequality constraints. This can be greatly increase the resolution and accuracyh of the solution. CONTIN is very flexible, consisting of a core of about 50 subprograms plus 13 small “USER” subprograms, which the user can easily modify to specify special-purpose constraints, regularizors, operator equations, simulations, statistical weighting, etc. Specjial collections of USER subprograms are available for photon correlation spectroscopy, multicomponent spectra, and Fourier-Bessel, Fourier and Laplace transforms. Numerically stable algorithms are used throughout CONTIN. A fairly precise definition of information content in terms of degrees of freedom is given. The regularization parameter can be automatically chosen on the basis of an F-test and confidence region. The interpretation of the latter and of error estimates based on the covariance matrix of the constrained regularized solution are discussed. The strategies, methods and options in CONTIN are outlined. The program itself is described in the following paper.     

Fast algorithm for a constrained infinite horizon LQ problem

This paper presents an efficient algorithmic solution to the infinite horizon linear quadratic optimal control problem for a discrete-time SISO plant subject to bound constraints on a scalar variable. The solution to the corresponding quadratic programming problem is based on the active set method and on dynamic programming. It is shown that the optimal solution can be updated after inclusion or removal of an active constraint by a simple procedure requiring in the order of kn operations, n being the system order and k the time at which the constraint is included or removed.     

Improved form of the conjugate gradient method

The conjugate gradient method, which is used for solving systems of linear algebraic equations, is studied. A notation for the method has been found, which proved to be the simplest and the most resistant to rounding errors. A criterion is constructed for the end of iterations based on the prevalence of rounding errors. Numerical calculations are made illustrating the specifics of the method convergence for well- and ill-posed problems. The generalization of this form is written for problems with preconditioning.     

A two-phase optimization procedure for integer programming problems

This paper presents a simple two-phase method for optimizing integer programming problems with a linear or nonlinear objective function subject to multiple linear or nonlinear constraints. The primary phase is based on a variation of the method of steepest descent in the feasible region, and a hem-stitching approach when a constraint is violated. The secondary phase zeros on the optimum solution by exploring the neighborhood of the suboptimum found in the first phase of the optimization process. The effectiveness of this method is illustrated through the optimization of several examples. The results from the proposed optimization approach are compared to those from methods developed specially for dealing with integer problems. The proposed method is simple, easy to implement yet very effective in dealing with a wide class of integer problems such as spare allocation, reliability optimization, and transportation problems.     

The scenario approach to robust control design

This paper proposes a new probabilistic solution framework for robust control analysis and synthesis problems that can be expressed in the form of minimization of a linear objective subject to convex constraints parameterized by uncertainty terms. This includes the wide class of NP-hard control problems representable by means of parameter-dependent linear matrix inequalities (LMIs). It is shown in this paper that by appropriate sampling of the constraints one obtains a standard convex optimization problem (the scenario problem) whose solution is approximately feasible for the original (usually infinite) set of constraints, i.e., the measure of the set of original constraints that are violated by the scenario solution rapidly decreases to zero as the number of samples is increased. We provide an explicit and efficient bound on the number of samples required to attain a-priori specified levels of probabilistic guarantee of robustness. A rich family of control problems which are in general hard to solve in a deterministically robust sense is therefore amenable to polynomial-time solution, if robustness is intended in the proposed risk-adjusted sense.     

A System to Generate Test Data and Symbolically Execute Programs

This paper describes a system that attempts to generate test data for programs written in ANSI Fortran. Given a path, the system symbolically executes the path and creates a set of constraints on the program's input variables. If the set of constraints is linear, linear programming techniques are employed to obtain a solution. A solution to the set of constraints is test data that will drive execution down the given path. If it can be determined that the set of constraints is inconsistent, then the given path is shown to be nonexecutable. To increase the chance of detecting some of the more common programming errors, artificial constraints are temporarily created that simulate error conditions and then an attempt is made to solve each augmented set of constraints. A symbolic representation of the program's output variables in terms of the program's input variables is also created. The symbolic representation is in a human readable form that facilitates error detection as well as being a possible aid in assertion generation and automatic program documentation.     

Computation of stereo disparity using regularization

The computation of stereo disparity is a mathematically ill-posed problem. However, using regularization theory it may be transformed into a well-posed problem. Standard regularization can be employed to solve ill-posed problems by using stabilizing functionals that impose global smoothness constraints on acceptable solutions. However, the presence of depth discontinuities causes serious difficulties in standard regularization, since smoothness assumptions do not hold across discontinuities. This paper presents a regularization approach to stereopsis based on controlled-continuity stabilizing functionals. These functionals provides a spatial control over smoothness, allowing the introduction of discontinuities into the solution. An iterative method for the computation of stereo disparity is derived, and the result of a computer simulation with a synthetic stereo pair of images is shown.     

Decomposition's Dantzig–Wolfe applied to fuzzy multicommodity flow problems

We present, in this paper, a method for solving linear programming problems with fuzzy costs based on the classical method of decomposition's Dantzig–Wolfe. Methods using decomposition techniques address problems that have a special structure in the set of constraints. An example of such a problem that has this structure is the fuzzy multicommodity flow problem. This problem can be modeled by a graph whose nodes represent points of supply, demand and passage of commodities, which travel on the arcs of the network. The objective is to determine the flow of each commodity on the arcs, in order to meet demand at minimal cost while respecting the capacity constraints of the arcs and the flow conservation constraints of the nodes. Using the theory of fuzzy sets, the proposed method aims to find the optimal solution, working with the problem in the fuzzy form during the resolution procedure.     

Local self-optimizing control of constrained processes

The available methods for selection of controlled variables (CVs) using the concept of self-optimizing control have been developed under the restrictive assumption that the set of active constraints remains unchanged for all the allowable disturbances and implementation errors. To track the changes in active constraints, the use of split-range controllers and parametric programming has been suggested in the literature. An alternate heuristic approach to maintain the variables within their allowable bounds involves the use of cascade controllers. In this work, we propose a different strategy, where CVs are selected as linear combinations of measurements to minimize the local average loss, while ensuring that all the constraints are satisfied over the allowable set of disturbances and implementation errors. This result is extended to select a subset of the available measurements, whose combinations can be used as CVs. In comparison with the available methods, the proposed approach offers simpler implementation of operational policy for processes with tight constraints. We use the case study of forced-circulation evaporator to illustrate the usefulness of the proposed method.     

Iterative constrained weighted least squares estimator for TDOA and FDOA positioning of multiple disjoint sources in the presence of sensor position and velocity uncertainties

Sensor position and velocity uncertainties are known to be able to degrade the source localization accuracy significantly. This paper focuses on the problem of locating multiple disjoint sources using time differences of arrival (TDOAs) and frequency differences of arrival (FDOAs) in the presence of sensor position and velocity errors. First, the explicit Cramér–Rao bound (CRB) expression for joint estimation of source and sensor positions and velocities is derived under the Gaussian noise assumption. Subsequently, we compare the localization accuracy when multiple-source positions and velocities are determined jointly and individually based on the obtained CRB results. The performance gain resulted from multiple-target cooperative positioning is also quantified using the orthogonal projection matrix. Next, the paper proposes a new estimator that formulates the localization problem as a quadratic programming with some indefinite quadratic equality constraints. Due to the non-convex nature of the optimization problem, an iterative constrained weighted least squares (ICWLS) method is developed based on matrix QR decomposition, which can be achieved through some simple and efficient numerical algorithms. The newly proposed iterative method uses a set of linear equality constraints instead of the quadratic constraints to produce a closed-form solution in each iteration. Theoretical analysis demonstrates that the proposed method, if converges, can provide the optimal solution of the formulated non-convex minimization problem. Moreover, its estimation mean-square-error (MSE) is able to reach the corresponding CRB under moderate noise level. Simulations are included to corroborate and support the theoretical development in this paper.     

Evolutionary programming techniques for constrained optimizationproblems

Two evolutionary programming (EP) methods are proposed for handling nonlinear constrained optimization problems. The first, a hybrid EP, is useful when addressing heavily constrained optimization problems both in terms of computational efficiency and solution accuracy. But this method offers an exact solution only if both the mathematical form of the objective function to be minimized/maximized and its gradient are known. The second method, a two-phase EP (TPEP) removes these restrictions. The first phase uses the standard EP, while an EP formulation of the augmented Lagrangian method is employed in the second phase. Through the use of Lagrange multipliers and by gradually placing emphasis on violated constraints in the objective function whenever the best solution does not fulfill the constraints, the trial solutions are driven to the optimal point where all constraints are satisfied. Simulations indicate that the TPEP achieves an exact global solution without gradient information, with less computation time than the other optimization methods studied here, for general constrained optimization problems     

Constrained Optimal Hybrid Control of a Flow Shop System

We consider an optimal control problem for the hybrid model of a deterministic flow shop system, in which the jobs are processed in the order they arrive at the system. The problem is decomposed into a higher-level discrete-event system control problem of determining the optimal service times, and a set of lower-level classical control problems of determining the optimal control inputs for given service times. We focus on the higher-level problem which is nonconvex and nondifferentiable. The arrival times are known and the decision variables are the service times that are controllable within constraints. We present an equivalent convex optimization problem with linear constraints. Under some cost assumptions, we show that no waiting is observed on the optimal sample path. This property allows us to simplify the convex optimization problem by eliminating variables and constraints. We also prove, under an additional strict convexity assumption, the uniqueness of the optimal solution and propose two algorithms to decompose the simplified convex optimization problem into a set of smaller convex optimization problems. The effects of the simplification and the decomposition on the solution times are shown on an example problem.     

Multiple Level Sets for Piecewise Constant Surface Reconstruction in Highly Ill-Posed Problems

This paper considers highly ill-posed surface recovery inverse problems, where the sought surface in 2D or 3D is piecewise constant with several possible level values. These levels may further be potentially unknown. Multiple level set functions are used when there are more than two such levels, and we extend the methods and theory of our previous works to handle such more complex situations. A rather efficient method is developed. Inverse potential problems in two and three space dimensions are solved numerically, demonstrating the method’s capabilities for several both known and unknown level values.     

Direct cut-off method for combinatorial optimization problems with additional constraints

A direct cut-off method to solve combinatorial optimization problems on polyarrangements with additional constraints is proposed and justified. The method allows obtaining a feasible solution at each stage without constructing the linear hull of the set of polyarrangements.     

Active set solver for min‐max robust control with state and input constraints

This paper proposes an online active set strategy for computing the dynamic programming solution to a min‐max robust optimal control problem with quadratic stage cost for linear systems with linear state and input constraints in the presence of bounded disturbances. The solver determines the optimal active constraint set for a given plant state using an iterative procedure that computes the optimal sequence of feedback laws for a candidate active set and updates the active set by performing a line search in state space. The computational complexity of each iteration depends linearly on the length of the prediction horizon. The main contribution of the paper is its treatment of degeneracy caused by linearly dependent state and input constraints, and its efficient handling is a crucial step in formulating the active set algorithm. The proposed approach ensures the continuity of optimal control laws along the line‐of‐search, thus enabling an efficient solution method based on homotopy. Conditions for global optimality are given, and the convergence of the active set solver is established using the geometric properties of an associated multiparametric programming problem. A receding horizon control strategy is proposed, which ensures a specified l₂‐gain from the disturbance input to the state and control inputs in the presence of linearly dependent constraints. Copyright © 2016 John Wiley & Sons, Ltd.     

A Probabilistic Method for Point Matching in the Presence of Noise and Degeneracy

The Bayesian method is widely used in image processing and computer vision to solve ill-posed problems. This is commonly achieved by introducing a prior which, together with the data constraints, determines a unique and hopefully stable solution. Choosing a “correct” prior is however a well-known obstacle. This paper demonstrates that in a certain class of motion estimation problems, the Bayesian technique of integrating out the “nuisance parameters” yields stable solutions even if a flat prior on the motion parameters is used. The advantage of the suggested method is more noticeable when the domain points approach a degenerate configuration, and/or when the noise is relatively large with respect to the size of the point configuration.