An interval arithmetic method for global optimization

An interval arithmetic method is described for finding the global maxima or minima of multivariable functions. The original domain of variables is divided successively, and the lower and the upper bounds of the interval expression of the function are estimated on each subregion. By discarding subregions where the global solution can not exist, one can always find the solution with rigorous error bounds. The convergence can be made fast by Newton's method after subregions are grouped. Further, constrained optimization can be treated using a special transformation or the Lagrange-multiplier technique.     

Explicit A-Priori error bounds and Adaptive error control for approximation of nonlinear initial value differential systems

This paper develops and demonstrates a guaranteed a-priori error bound for the Taylor polynomial approximation of any degree to the solution of initial value ordinary differential equations. The error bound is explicit and does not require upper bounds on the potentially complicated and intrinsically unknown right-hand side nor on any of its higher-order derivatives as with existing bounds, and thus it provides a valuable tool for the numerous applications in which initial value ode problems arise and for which approximate solutions must be sought.     

Linear-least-squares initialization of multilayer perceptrons through backpropagation of the desired response

Training multilayer neural networks is typically carried out using descent techniques such as the gradient-based backpropagation (BP) of error or the quasi-Newton approaches including the Levenberg-Marquardt algorithm. This is basically due to the fact that there are no analytical methods to find the optimal weights, so iterative local or global optimization techniques are necessary. The success of iterative optimization procedures is strictly dependent on the initial conditions, therefore, in this paper, we devise a principled novel method of backpropagating the desired response through the layers of a multilayer perceptron (MLP), which enables us to accurately initialize these neural networks in the minimum mean-square-error sense, using the analytic linear least squares solution. The generated solution can be used as an initial condition to standard iterative optimization algorithms. However, simulations demonstrate that in most cases, the performance achieved through the proposed initialization scheme leaves little room for further improvement in the mean-square-error (MSE) over the training set. In addition, the performance of the network optimized with the proposed approach also generalizes well to testing data. A rigorous derivation of the initialization algorithm is presented and its high performance is verified with a number of benchmark training problems including chaotic time-series prediction, classification, and nonlinear system identification with MLPs.     

On the a-posteriori error bounds for the solution of ordinary nonlinear differential equations

This paper advances a procedure to compute the a-posteriori error bounds for the solution of a wide class of ordinary differential equations with given initial conditions. The method is based on an iterative scheme which yields upper and lower bounds which approach each other. The convergence of these iterations is proved analytically. Application of the proposed method is demonstrated by several examples. The method is independent of the integration scheme used. Also, separate bounds for each component of the solution are specifically available as a function of time compared to the bound on the norm yielded by conventional methods.     

Optimal redundancy resolution of a kinematically redundant manipulator for a cyclic task

This article proposes a method for the global optimization of redundancy over the whole task period in a kinematically redundant manipulator. The necessary conditions based on the calculus of variations for integral-type criteria result in a second-order differential equation. For a cyclic task, the boundary conditions for conservative joint motions are discussed. Then, we reformulate a two-point boundary value problem to an initial value adjustment problem and suggest a numerical search method based on the iterative optimization for providing a globally optimal solution using the gradient projection method. Since the initial joint velocity is parameterized with the number of redundancy, we only search parameter values in the parameterized space using the configuration error between the initial and final time. We show through numerical examples that multiple nonhomotopic extremal solutions satisfying periodic boundary conditions exist according to initial joint velocities for the same initial configuration. Finally, we discuss an algorithm for topological liftings of the paths and demonstrate the generality of the proposed method by considering the dynamics of a manipulator.     

Error estimation in the integration of ordinary differential equations

Linear initial value problems, particularly involving first order differential equations, can be transformed into systems of higher order and treated as boundary value problems. Finite difference analogues considered for obtaining approximate solutions of these boundary value problems are proved to be fourth order convergent processes, by deriving considerable sharper bounds for the discretization error. Numerical examples are given to demonstrate the usefulness of our error bounds.     

A robust error estimator and a residual-free error indicator for reduced basis methods

The Reduced Basis Method (RBM) is a rigorous model reduction approach for solving parameterizedpartial differential equations. It identifies a low-dimensional subspace for approximation of the parametric solution manifold that is embedded in high-dimensional space. A reduced order model is subsequently constructed in this subspace. RBM relies on residual-based error indicators or a posteriori error bounds to guide construction of the reduced solution subspace, to serve as a stopping criteria, and to certify the resulting surrogate solutions. Unfortunately, it is well-known that the standard algorithm for residual norm computation suffers from premature stagnation at the level of the square root of machine precision.In this paper, we develop two alternatives to the standard offline phase of reduced basis algorithms. First, we design a robust strategy for computation of residual error indicators that allows RBM algorithms to enrich the solution subspace with accuracy beyond root machine precision. Secondly, we propose a new error indicator based on the Lebesgue function in interpolation theory. This error indicator does not require computation of residual norms, and instead only requires the ability to compute the RBM solution. This residual-free indicator is rigorous in that it bounds the error committed by the RBM approximation, but up to an uncomputable multiplicative constant. Because of this, the residual-free indicator is effective in choosing snapshots during the offline RBM phase, but cannot currently be used to certify error that the approximation commits. However, it circumvents the need for a posteriori analysis of numerical methods, and therefore can be effective on problems where such a rigorous estimate is hard to derive.     

Reduced-basis output bounds for approximately parametrized elliptic coercive partial differential equations

We present a technique for the rapid and reliable prediction of linear-functional outputs of elliptic coercive partial differential equations with (approximately) affine parameter dependence. The essential components are (i) (provably) rapidly convergent global reduced-basis approximations – Galerkin projection onto a space W_N spanned by solutions of the governing partial differential equation at N selected points in parameter space; (ii) a posteriori error estimation – relaxations of the error-residual equation that provide inexpensive yet sharp bounds for the error in the outputs of interest; and (iii) off-line/on-line computational procedures – methods which decouple the generation and projection stages of the approximation process. The operation count for the on-line stage – in which, given a new parameter value, we calculate the output of interest and associated error bound – depends only on N, typically very small, and the (approximate) parametric complexity of the problem; the method is thus ideally suited for the repeated and rapid evaluations required in the context of parameter estimation, design, optimization, and real-time control.In our earlier work, we develop a rigorous a posteriori error bound framework for the case in which the parametrization of the partial differential equation is exact; in this paper, we address the situation in which our mathematical model is not complete. In particular, we permit error in the data that define our partial differential operator: this error may be introduced, for example, by imperfect specification, measurement, calculation, or parametric expansion of a coefficient function. We develop both accurate predictions for the outputs of interest and associated rigorous a posteriori error bounds; and the latter incorporate both numerical discretization and model truncation effects. Numerical results are presented for a particular instantiation in which the model error originates in the (approximately) prescribed velocity field associated with a three-dimensional convection-diffusion problem.     

Constrained state estimation with applications in water distribution network monitoring

This paper investigates the effects of imposing bounds on the measurements used in weighted least-squares (WLS) state estimation. The active limits for such bounds are derived and algorithms based on linear and quadratic programming kernels are presented. Using the lower limit for the bounds, the constrained WLS scheme becomes an adaptive maximally constrained scheme: M-WLS. For some networks, the poor prior knowledge of the global error characteristic results in some measurements having less influence than would be expected from the local error characteristics of their transducers. By using M-WLS estimation, the influence of such measurements on state estimation may be improved. Analysis of the adaptive bounding of the scheme can also lead to identification of critical measurement discrepancies. For the purpose of illustration, results are presented using simulated measurements; the head measurements (pressures) are consistent with nominal demands (nodal flows) and the demand measurements are generated by superimposing random errors of 2.5ls^-1 rms on the nominal demands.     

Approximation accuracy of some neuro-fuzzy approaches

Many methods have been proposed in the literature for designing fuzzy systems from input-output data (the so-called neuro-fuzzy methods), but very little was done to analyze the performance of the methods from a rigorous mathematical point of view. In this paper, we establish approximation bounds for two of these methods - the table lookup scheme proposed by Wang et al. (1992) and the clustering method studied by Wang (1993, 1997). We derive detailed formulas of the error bounds between the nonlinear function to be approximated and the fuzzy systems designed using the methods based on input-output data. These error bounds show explicitly how the parameters in the two methods influence their approximation capability. We also propose modified versions for the two methods such that the designed fuzzy systems are well-defined over the whole input domain     

On the error structure of the implicit Euler scheme applied to stiff systems of differential equations

In this paper we investigate the structure of the global discretization error of the implicit Euler scheme applied to systems to stiff differential equations, extending earlier work on this subject (cf. [1], [9]). We restrain our considerations to the linear, self-adjoint, constant coefficient case but—in contrast to [1], [9]—we make no assumptions about the nature of the stiff spectrum; the stiff eigenvalues may be arbitrarily distributed on the negative real axis. Our main result says that the global error of the implicit Euler scheme admits an asymptotic expansion in powers of the stepsize τ which is not asymptotically correct in the conventional sense: Near the initial pointt=0 the expansion is spoiled at theO(τ²) by ‘irregular’ error components which are, however, (algebraically) damped, such that away fromt=0 the ‘pure’ asymptotic expansion reappears. We present numerical experiments confirming this result. Our considerations should be particularly helpful for a rigorous, quantitative analysis of the structure of the full (space & time) discretization error in the PDE (method of lines) context, and thus for a sound theoretical justification of extrapolation techniques for this important class of stiff problems.     

Neuro-heuristic computational intelligence for solving nonlinear pantograph systems

We present a neuro-heuristic computing platform for finding the solution for initial value problems (IVPs) of nonlinear pantograph systems based on functional differential equations (P-FDEs) of different orders. In this scheme, the strengths of feed-forward artificial neural networks (ANNs), the evolutionary computing technique mainly based on genetic algorithms (GAs), and the interior-point technique (IPT) are exploited. Two types of mathematical models of the systems are constructed with the help of ANNs by defining an unsupervised error with and without exactly satisfying the initial conditions. The design parameters of ANN models are optimized with a hybrid approach GA–IPT, where GA is used as a tool for effective global search, and IPT is incorporated for rapid local convergence. The proposed scheme is tested on three different types of IVPs of P-FDE with orders 1–3. The correctness of the scheme is established by comparison with the existing exact solutions. The accuracy and convergence of the proposed scheme are further validated through a large number of numerical experiments by taking different numbers of neurons in ANN models.     

Robust resilient control for parametric strict feedback systems with prescribed output and virtual tracking errors

This paper investigates the robust resilient control problem for a class of parametric strict feedback nonlinear systems with prescribed output and virtual tracking errors performance. The resilience is governed by a continuously nonlinear control gains function, which endows the virtual and actual controllers with self‐adjusting abilities with respect to transformed error surfaces. The proposed control scheme is adaptation, estimation, and approximation‐free in the presence of unknown parameters and nonlinearities, and only a number of control gains, which is equal to the relative degree of the considered plants, need to be selected in applications. It is proved by rigorous analysis that the output tracking errors are confined in predefined prescribed performance functions under some nonrestrictive initial conditions, and the bounds of states are obtained characterized as control gains and systemic function‐associated constants. Finally, comparative illustrative examples are given to demonstrate the effectiveness of the proposed control scheme.     

Global optimization of linear hybrid systems with explicit transitions

The global optimization of hybrid systems described by linear time-varying ordinary differential equations is examined. A method to construct convex relaxations of general, nonlinear Bolza-type objective functions or constraints subject to an embedded hybrid system with explicit transitions is presented. The optimization problem can be solved using gradient-based algorithms in a branch and bound framework that is shown to be infinitely convergent when the implied state bounds are employed.     

PAC-Bayesian Compression Bounds on the Prediction Error of Learning Algorithms for Classification

We consider bounds on the prediction error of classification algorithms based on sample compression. We refine the notion of a compression scheme to distinguish permutation and repetition invariant and non-permutation and repetition invariant compression schemes leading to different prediction error bounds. Also, we extend known results on compression to the case of non-zero empirical risk.We provide bounds on the prediction error of classifiers returned by mistake-driven online learning algorithms by interpreting mistake bounds as bounds on the size of the respective compression scheme of the algorithm. This leads to a bound on the prediction error of perceptron solutions that depends on the margin a support vector machine would achieve on the same training sample.Furthermore, using the property of compression we derive bounds on the average prediction error of kernel classifiers in the PAC-Bayesian framework. These bounds assume a prior measure over the expansion coefficients in the data-dependent kernel expansion and bound the average prediction error uniformly over subsets of the space of expansion coefficients.Editor Shai Ben-David     

Simultaneous parametric material and topology optimization with constrained material grading

We consider the problem of parametric material and simultaneous topology optimization of an elastic continuum. To ensure existence of solutions to the proposed optimization problem and to enable the imposition of a deliberate maximal material grading, two approaches are adopted and combined. The first imposes pointwise bounds on design variable gradients, whilst the second applies a filtering technique based on a convolution product. For the topology optimization, the parametrized material is multiplied with a penalized continuous density variable. We suggest a finite element discretization of the problem and provide a proof of convergence for the finite element solutions to solutions of the continuous problem. The convergence proof also implies the absence of checkerboards. The concepts are demonstrated by means of numerical examples using a number of different material parametrizations and comparing the results to global lower bounds.     

Metaheuristic algorithms for approximate solution to ordinary differential equations of longitudinal fins having various profiles

Differential equations play a noticeable role in engineering, physics, economics, and other disciplines. Approximate approaches have been utilized when obtaining analytical (exact) solutions requires substantial computational effort and often is not an attainable task. Hence, the importance of approximation methods, particularly, metaheuristic algorithms are understood. In this paper, a novel approach is suggested for solving engineering ordinary differential equations (ODEs). With the aid of certain fundamental concepts of mathematics, Fourier series expansion, and metaheuristic methods, ODEs can be represented as an optimization problem. The target is to minimize the weighted residual function (error function) of the ODEs. The boundary and initial values of ODEs are considered as constraints for the optimization model. Generational distance and inverted generational distance metrics are used for evaluation and assessment of the approximate solutions versus the exact (numerical) solutions. Longitudinal fins having rectangular, trapezoidal, and concave parabolic profiles are considered as studied ODEs. The optimization task is carried out using three different optimizers, including the genetic algorithm, the particle swarm optimization, and the harmony search. The approximate solutions obtained are compared with the differential transformation method (DTM) and exact (numerical) solutions. The optimization results obtained show that the suggested approach can be successfully applied for approximate solving of engineering ODEs. Providing acceptable accuracy of the proposed technique is considered as its important advantage against other approximate methods and may be an alternative approach for approximate solving of ODEs.     

Error bounds for Loop subdivision surfaces

In this paper, we obtain the error bounds on the distance between a Loop subdivision surface and its control mesh. Both local and global bounds are derived by means of computing and analysing the control meshes with two rounds of refinement directly. The bounds can be expressed with the maximum edge length of all triangles in the initial control mesh. Our results can be used as posterior estimates and also can be used to predict the subdivision depth for any given tolerance.     

Computable explicit bounds for the discretization error of variable stepsize multistep methods

This paper deals with the achievement of explicit computable bounds for the global discretization error of variable stepsize multistep methods which are perturbation of strongly stable fixed stepsize methods. The approach is based on the study of the growth of solutions of certain variable coefficient difference equations satisfied by the global discretization error.     

循环混沌优势进化算法

详细描述了新提出的能够快速收敛的优势进化算法以及基于混沌映射为初始群体的循环优势进化算法。对该策略利用模拟函数做了寻优测试并做了详细的讨论，测试表明在具有3^15个极小值的空间中循环优势进化算法能够在极短的时间内快速寻找到全局唯一的最小点。该全局最优算法策略具有很大的潜力运用于化学问题的复杂QSAR研究的寻优当中。