Numerical nonlinear observers using pseudo‐Newton‐type solvers

In constructing a globally convergent numerical nonlinear observer of Newton‐type for a continuous‐time nonlinear system, a globally convergent nonlinear equation solver with a guaranteed rate of convergence is necessary. In particular, the solver should be Jacobian free, because an analytic form of the state transition map of the nonlinear system is generally unavailable. In this paper, two Jacobian‐free nonlinear equation solvers of pseudo‐Newton type that fulfill these requirements are proposed. One of them is based on the finite difference approximation of the Jacobian with variable step size together with the line search. The other uses a similar idea, but the estimate of the Jacobian is mostly updated through a BFGS‐type law. Then, by using these solvers, globally stable numerical nonlinear observers are constructed. Numerical results are included to illustrate the effectiveness of the proposed methods. Copyright © 2007 John Wiley & Sons, Ltd.     

Incomplete Jacobian Newton method for nonlinear equations

This paper presents a new modified Newton method for nonlinear equations. This method uses a part of elements of the Jacobian matrix to obtain the next iteration point and is refereed to as the incomplete Jacobian Newton (IJN) method. The IJN method may be fit for solving large scale nonlinear equations with dense Jacobian. The conditions of linear, superlinear and quadratic convergence of the IJN method are given and the local convergence results are analyzed and proved. Some special IJN algorithms are designed and numerical experiments are given. The results show that the IJN method is promising.     

On the iterated forms of Kalman filters using statistical linearization

The extended Kalman filter (EKF) is a suboptimal estimator of the conditional mean and covariance for nonlinear state estimation. It is based on first order Taylor series approximation of nonlinear state functions. The unscented Kalman filter (UKF) and the ensemble Kalman filter (EnKF) are suboptimal estimators that are termed as Jacobian free because they do not require the existence of the Jacobian of the nonlinearity. The iterated form of EKF is an estimator of the conditional mode that employs an approximate Newton–Raphson iterative scheme to solve the maximization of the conditional probability density function. In this paper, the iterated forms of UKF and EnKF are presented that perform Newton–Raphson iteration without explicitly differentiating the nonlinear functions. The use of statistical linearization in iterated UKF and EnKF is a nondifferentiable optimization method when the measurement function is nonsmooth or discontinuous. All three iterated forms can be shown to be conditional mean estimators after the first iteration. A simple numerical example involving continuous and discontinuous measurment functions is included to evaluate the performance of the algorithms for the estimation of conditional mean, covariance and mode. A batch reactor simulation is shown for estimating both the states and unknown parameters.     

Approximation of Jacobian Inverse Kinematics Algorithms: Differential Geometric vs. Variational Approach

This paper addresses the approximation problem of Jacobian inverse kinematics algorithms for redundant robotic manipulators. Specifically, we focus on the approximation of the Jacobian pseudo inverse by the extended Jacobian algorithm. The algorithms are defined as certain dynamic systems driven by the task space error, and identified with vector field distributions. The distribution corresponding to the Jacobian pseudo inverse is non-integrable, while that associated with the extended Jacobian is integrable. Two methods of devising the approximating extended Jacobian algorithm are examined. The first method is referred to as differential geometric, and relies on the approximation of a non-integrable distribution (in fact: a codistribution) by an integrable one. As an alternative, the approximation problem has been formulated as the minimization of an approximation error functional, and solved using the methods of the calculus of variations. Performance of the obtained extended Jacobian inverse kinematics algorithms has been compared by means of computer simulations involving the kinematics model of the 7 dof industrial manipulator POLYCRANK. It is concluded that the differential geometric method offers a rapid, while the variational method a systematic tool for solving inverse kinematic problems.     

Nonlinear contractivity of a class of semi-implicit multistep methods

The stability and contractivity of generalized linear multistep methods are studied for a large class of nonlinear stiff initial value problems. These methods are characterized by the fact that the coefficients of the integration formulas are matrices depending on the Jacobian or on an approximation to the Jacobian. Conditions for the parameters of such a multistep method are given which ensure that the method gives contractive numerical solutions over a large class of nonlinear dissipative systems for sufficiently small stepsizesh, where the restriction onh is not due to the stiffness of the problem. Stability and contractive properties of special methods of this class are reported.     

Numerical experience with newton-like methods for nonlinear algebraic systems

In this paper we present an extensive computational experience with several Newton-like methods, namely Newton’s method, the ABS Huang method, the ABS row update method and six Quasi-Newton methods. The methods are first tested on 31 families of problems with dimensionsn=10, 50, 100 and two starting points. Newton’s method appears to be the best in terms of number of solved problems, followed closely by the ABS Huang method. Broyden’s “bad” method and Greenstadt’s second method show a very poor performance. The other four Quasi-Newton methods perform similarly, strongly suggesting that Greenstadt’s first method and Martínez’ column update method are locally and superlinearly convergent, a result that has yet to be proven theoretically. Thomas’ method appears to be marginally more robust and fast and provides moreover a better approximation to the Jacobian. An interesting and somewhat unexpected observation is that the number of iterations for satisfying the convergence test increases very little with the dimension of the problem. In a second set of experiments we look at the structure of the regions of convergence/nonconvergence by starting the methods from all nodes of a regular grid and assigning to each node a number according to the outcome of the iteration. The obtained regions have clearly a fractal type structure, which, on the two tested problems, is much simpler for Newton’s method than for the other methods. Newton’s method also is the one with the smallest nonconvergence region. Among the Quasi-Newton methods Thomas’ method shows a definitely smaller nonconvergence region.     

Finite element analysis of nonlocal coupled parabolic problem using Newton’s method

In this article, we propose finite element method to approximate the solution of a coupled nonlocal parabolic system. An important issue in the numerical solution of nonlocal problems while using the Newton’s method is related to its structure. Indeed, unlike the local case the Jacobian matrix is sparse and banded, the nonlocal term makes the Jacobian matrix dense. As a consequence computations consume more time and space in contrast to local problems. To overcome this difficulty we reformulate the discrete problem and then apply the Newton’s method. We discuss the well-posedness of the weak formulation at continuous as well as at discrete levels. We derive a priori error estimates for both semi-discrete and fully-discrete formulations. Results based on usual finite element method are provided to confirm the theoretical estimates.     

Kinematic analysis and error modeling of TAU parallel robot

The TAU robot presents a new configuration of parallel robots with three degrees of freedom. This robotic configuration is well adapted to perform with a high precision and high stiffness within a large working range compared with a serial robot. It has the advantages of both parallel robots and serial robots. In this paper, the kinematic modeling and error modeling are established with all errors considered using Jacobian matrix method for the robot. Meanwhile, a very effective Jacobian approximation method is introduced to calculate the forward kinematic problem instead of Newton–Raphson method. It denotes that a closed form solution can be obtained instead of a numerical solution. A full size Jacobian matrix is used in carrying out error analysis, error budget, and model parameter estimation and identification. Simulation results indicate that both Jacobian matrix and Jacobian approximation method are correct and with a level of accuracy of micron meters. ADAMS's simulation results are used in verifying the established models.     

放射形配网潮流计算的一种新的牛顿法

前推回推法是放射形配网潮流计算最基本的算法.通过对前推回推法求解过程的数学演化,导出一种新的牛顿类型的算法及其雅可比矩阵直接分解公式.利用比较原理,间接证明该算法是一种具有超线性收敛性的近似牛顿法.与经典牛顿法相比,该算法无须计算雅可比矩阵、无须三角因子分解等过程,直接由前代/回代或回代/前代过程就能完成;与前推回推法相比,该算法无须特定的节点和支路编号过程.文中以一个实际的中等规模配电系统为例,分析、比较前推回推法、导出的近似牛顿法、经典牛顿法等的收敛性和计算速度,证实上述研究结论.     

Generalized newton multi-step iterative methods GMNp,m for solving system of nonlinear equations

A generalization of the Newton multi-step iterative method is presented, in the form of distinct families of methods depending on proper parameters. The proposed generalization of the Newton multi-step consists of two parts, namely the base method and the multi-step part. The multi-step part requires a single evaluation of function per step. During the multi-step phase, we have to solve systems of linear equations whose coefficient matrix is the Jacobian evaluated at the initial guess. The direct inversion of the Jacobian it is an expensive operation, and hence, for moderately large systems, the lower-upper triangular factorization (LU) is a reasonable choice. Once we have the LU factors of the Jacobian, starting from the base method, we only solve systems of lower and upper triangular matrices that are in fact computationally economical. The developed families involve unknown parameters, and we are interested in setting them with the goal of maximizing the convergence order of the global method. Few families are investigated in some detail. The validity and numerical accuracy of the solution of the system of nonlinear equations are presented via numerical simulations, also involving examples coming from standard approximations of ordinary differential and partial differential nonlinear equations. The obtained results show the efficiency of constructed iterative methods, under the assumption of smoothness of the nonlinear function.     

A multilevel hybrid Newton–Krylov–Schwarz method for the Bidomain model of electrocardiology

A multilevel hybrid Newton–Krylov–Schwarz (NKS) method is constructed and studied numerically for implicit time discretizations of the Bidomain reaction–diffusion system in three dimensions. This model describes the bioelectrical activity of the heart by coupling two degenerate parabolic equations with a stiff system of ordinary differential equations. The NKS Bidomain solver employs an outer inexact Newton iteration to solve the nonlinear finite element system originating at each time step of the implicit discretization. The Jacobian update during the Newton iteration is solved by a Krylov method employing a multilevel hybrid overlapping Schwarz preconditioner, additive within the levels and multiplicative among the levels. Several parallel tests on Linux clusters are performed, showing that the convergence of the method is independent of the number of subdomains (scalability), the discretization parameters and the number of levels (optimality).     

A matrix-free quasi-Newton method for solving large-scale nonlinear systems

One of the widely used methods for solving a nonlinear system of equations is the quasi-Newton method. The basic idea underlining this type of method is to approximate the solution of Newton’s equation by means of approximating the Jacobian matrix via quasi-Newton update. Application of quasi-Newton methods for large scale problems requires, in principle, vast computational resource to form and store an approximation to the Jacobian matrix of the underlying problem. Hence, this paper proposes an approximation for Newton-step based on the update of approximation requiring a computational effort similar to that of matrix-free settings. It is made possible by approximating the Jacobian into a diagonal matrix using the least-change secant updating strategy, commonly employed in the development of quasi-Newton methods. Under suitable assumptions, local convergence of the proposed method is proved for nonsingular systems. Numerical experiments on popular test problems confirm the effectiveness of the approach in comparison with Newton’s, Chord Newton’s and Broyden’s methods.     

Efficient methods for enclosing solutions of systems of nonlinear equations

We consider modifications of the interval Newton method which combine two ideas: Reusing the same evaluation of the Jacobian several (says) times and approximately solving the Newton equation by some ‘linear’ iterative process. We show in particular that theR-order of these methods may becomes+1. We illustrate our results by a numerical example.     

A generalized Newton method for non-Hermitian positive definite linear complementarity problem

In this paper, based on the implicit fixed-point equation of the linear complementarity problem (LCP), a generalized Newton method is presented to solve the non-Hermitian positive definite linear complementarity problem. Some convergence properties of the proposed generalized Newton method are discussed. Numerical experiments are presented to illustrate the efficiency of the proposed method.     

Efficient solution of nonlinear elliptic problems using hierarchical matrices with Broyden updates

Summary The numerical solution of nonlinear problems is usually connected with Newton’s method. Due to its computational cost, variants (so-called inexact and quasi–Newton methods) have been developed in which the arising inverse of the Jacobian is replaced by an approximation. In this article we present a new approach which is based on Broyden updates. This method does not require to store the update history since the updates are explicitly added to the matrix. In addition to updating the inverse we introduce a method which constructs updates of the LU decomposition. To this end, we present an algorithm for the efficient multiplication of hierarchical and semi-separable matrices. Since an approximate LU decomposition of finite element stiffness matrices can be efficiently computed in the set of hierarchical matrices, the complexity of the proposed method scales almost linearly. Numerical examples demonstrate the effectiveness of this new approach. This work was supported by the DFG priority program SPP 1146 “Modellierung inkrementeller Umformverfahren”.     

Embedded Additive Runge-Kutta Methods

Additive Runge-Kutta methods for systems of I.V.Ps. x^{\prime}=f(t, x) , x(t_{0})=x_{0} have proved useful in many applications. A method of this type is characterized by a pair of methods ( A , B ), where the method A is semi-implicit and A-stable and the method B is explicit. For stiff systems, these methods may be used with a sequence of decompositions f=J^{(m)}x+g^{(m)}(t, x) , which is established by taking J ( m ) as approximation to the Jacobian of f at t m and then setting g^{(m)}(t, x)= f(t, x)-J^{(m)}x . An additive method with equal non-zero diagonal elements in A gives a computational advantage over many implementation schemes for SIRKs and DIRKs methods, for which the modified Newton or any other iteration method is used. A direct generalization of the algebraic stability is used to obtain some embedded additive R-K methods of order p h 4 with improved stability properties.     

Bi-directional algebraic graphic statics

A pre-existing algebraic graphic statics method is extended to allow for interactive manipulations of the force diagram, from which an updated form diagram is determined. Newton’s method is used to solve a set of non-linear equations, and the required Jacobian matrix is derived. Additional geometric constraints on the form diagram are introduced, and methods for improving the robustness of the method are presented. We discuss the implementation of the method as a back-end to an interactive application, and demonstrate the usability of the method in several examples where the qualities of directly manipulating the force diagram are emphasized.     

一类非线性方程组奇异解的计算方法及其应用

针对一类特殊的非线性方程组雅克比矩阵奇异的问题,提出了一种基于对偶空间的牛顿迭代方法。给出了一个显式的计算对偶空间的公式,在此基础上利用对偶空间作用于原方程组构造新的方程,使扩充后的方程组在近似值点的雅可比矩阵满秩,从而恢复牛顿迭代算法的二次收敛性。实验结果表明,改进后的算法一般迭代3次计算精度就可以达到10^(-15)。所提算法丰富了代数几何中关于理想的对偶空间理论,也为工程应用中的数值计算提供了一种新方法。     

Reliability-based design optimization using SORM and SQP

In this work a second order approach for reliability-based design optimization (RBDO) with mixtures of uncorrelated non-Gaussian variables is derived by applying second order reliability methods (SORM) and sequential quadratic programming (SQP). The derivation is performed by introducing intermediate variables defined by the incremental iso-probabilistic transformation at the most probable point (MPP). By using these variables in the Taylor expansions of the constraints, a corresponding general first order reliability method (FORM) based quadratic programming (QP) problem is formulated and solved in the standard normal space. The MPP is found in the physical space in the metric of Hasofer-Lind by using a Newton algorithm, where the efficiency of the Newton method is obtained by introducing an inexact Jacobian and a line-search of Armijo type. The FORM-based SQP approach is then corrected by applying four SORM approaches: Breitung, Hohenbichler, Tvedt and a recent suggested formula. The proposed SORM-based SQP approach for RBDO is accurate, efficient and robust. This is demonstrated by solving several established benchmarks, with values on the target of reliability that are considerable higher than what is commonly used, for mixtures of five different distributions (normal, lognormal, Gumbel, gamma and Weibull). Established benchmarks are also generalized in order to study problems with large number of variables and several constraints. For instance, it is shown that the proposed approach efficiently solves a problem with 300 variables and 240 constraints within less than 20 CPU minutes on a laptop. Finally, a most well-know deterministic benchmark of a welded beam is treated as a RBDO problem using the proposed SORM-based SQP approach.     

Efficient unsteady high Reynolds number flow computations on unstructured grids

Despite the advances in computer power and numerical algorithms over the last decades, solutions to unsteady flow problems remain computing time intensive. Especially for high Reynolds number flows, nonlinear multigrid, which is commonly used to solve the nonlinear systems of equations, converges slowly. The stiffness induced by the high-aspect ratio cells and turbulence is not tackled well by this solution method.In this paper, it is investigated if a Jacobian-free Newton-Krylov (jfnk) solution method can speed up unsteady flow computations at high Reynolds numbers. Preconditioning of the linear systems that arise after Newton linearization is commonly performed with matrix-free preconditioners or approximate factorizations based on crude approximations of the Jacobian. Approximate factorizations based on a Jacobian that matches the target residual operator are unpopular because these preconditioners consume a large amount of memory and can suffer from robustness issues. However, these preconditioners remain appealing because they closely resemble A^-1.In this paper, it is shown that a jfnk solution method with an approximate factorization preconditioner based on a Jacobian that approximately matches the target residual operator enables a speed up of a factor 2.5-12 over nonlinear multigrid for two-dimensional high Reynolds number flows. The solution method performs equally well as nonlinear multigrid for three-dimensional laminar problems. A modest memory consumption is achieved with partly lumping the Jacobian before constructing the approximate factorization preconditioner, whereas robustness is ensured with enhanced diagonal dominance.