Preconditioned variational methods on rotated finite difference discretisation

Due to their rapid convergence properties, recent focus on iterative methods in the solution of linear system has seen a flourish on the use of gradient techniques which are primarily based on global minimisation of the residual vectors. In this paper, we conduct an experimental study to investigate the performance of several preconditioned gradient or variational techniques to solve a system arising from the so-called rotated (skewed) finite difference discretisation in the solution of elliptic partial differential equations (PDEs). The preconditioned iterative methods consist of variational accelerators, namely the steepest descent and conjugate gradient methods, applied to a special matrix ‘splitting’ preconditioned system. Several numerical results are presented and discussed.     

Overlapping Schwarz and Spectral Element Methods for Linear Elasticity and Elastic Waves

The classical overlapping Schwarz algorithm is here extended to the spectral element discretization of linear elastic problems, for both homogeneous and heterogeneous compressible materials. The algorithm solves iteratively the resulting preconditioned system of linear equations by the conjugate gradient or GMRES methods. The overlapping Schwarz preconditioned technique is then applied to the numerical approximation of elastic waves with spectral elements methods in space and implicit Newmark time advancing schemes. The results of several numerical experiments, for both elastostatic and elastodynamic problems, show that the convergence rate of the proposed preconditioning algorithm is independent of the number of spectral elements (scalability), is independent of the spectral degree in case of generous overlap, otherwise it depends inversely on the overlap size. Some results on the convergence properties of the spectral element approximation combined with Newmark schemes for elastic waves are also presented.     

Numerical simulation of three-dimensional thermal convection on the array processor DAP 510

In this paper we deal with the numerical simulation of time dependent three-dimensional thermal convection on the array processor DAP 510. Applying finite differences in combination with a pressure correction method to the underlying non-linear system of partial differential equations, we reduce the numerical solution of the problem to the solution of a sequence of sparse linear systems. Using polynomial preconditioned conjugate gradient methods for the solution of these systems results in a highly parallel algorithm for the simulation of the considered flows on the DAP 510. Using this parallel algorithm, data can be mapped in different ways onto the processor array. Depending on the number of grid points, several methods are shown. Numerical experiments illustrate the capabilities of the proposed algorithm.     

An Extrapolation Cascadic Multigrid Method Combined with a Fourth-Order Compact Scheme for 3D Poisson Equation

Extrapolation cascadic multigrid (EXCMG) method is an efficient multigrid method which has mainly been used for solving the two-dimensional elliptic boundary value problems with linear finite element discretization in the existing literature. In this paper, we develop an EXCMG method to solve the three-dimensional Poisson equation on rectangular domains by using the compact finite difference (FD) method with unequal meshsizes in different coordinate directions. The resulting linear system from compact FD discretization is solved by the conjugate gradient (CG) method with a relative residual stopping criterion. By combining the Richardson extrapolation and tri-quartic Lagrange interpolation for the numerical solutions from two-level of grids (current and previous grids), we are able to produce an extremely accurate approximation of the actual numerical solution on the next finer grid, which can greatly reduce the number of relaxation sweeps needed. Additionally, a simple method based on the midpoint extrapolation formula is used for the fourth-order FD solutions on two-level of grids to achieve sixth-order accuracy on the entire fine grid cheaply and directly. The gradient of the numerical solution can also be easily obtained through solving a series of tridiagonal linear systems resulting from the fourth-order compact FD discretizations. Numerical results show that our EXCMG method is much more efficient than the classical V-cycle and W-cycle multigrid methods. Moreover, only few CG iterations are required on the finest grid to achieve full fourth-order accuracy in both the \(L^2\)-norm and \(L^{\infty }\)-norm for the solution and its gradient when the exact solution belongs to \(C^6\). Finally, numerical result shows that our EXCMG method is still effective when the exact solution has a lower regularity, which widens the scope of applicability of our EXCMG method.     

Implementation and solution of the diffusion–reaction equation using high-order methods

The orthogonal collocation, Galerkin, tau and least-squares methods are applied to solve a diffusion–reaction problem. In general, the least-squares method suffers from lower accuracy than the other weighted residual methods. The least-squares method holds the most complex linear algebra theory and is thus associated with the most complex implementation. On the other hand, an advantage of the least-squares method is that it always produces a symmetric and positive definite system matrix which can be solved with an efficient iterative technique such as the conjugate gradient method or its preconditioned version. For the present problem, neither the Galerkin, tau and orthogonal collocation techniques produce symmetric and positive definite system matrices, hence the conjugate gradient method is not applicable for these numerical techniques.     

Explicit approximate inverse preconditioning techniques

Summary  The numerical treatment and the production of related software for solving large sparse linear systems of algebraic equations, derived mainly from the discretization of partial differential equation, by preconditioning techniques has attracted the attention of many researchers. In this paper we give an overview of explicit approximate inverse matrix techniques for computing explicitly various families of approximate inverses based on Choleski and LU—type approximate factorization procedures for solving sparse linear systems, which are derived from the finite difference, finite element and the domain decomposition discretization of elliptic and parabolic partial differential equations. Composite iterative schemes, using inner-outer schemes in conjunction with Picard and Newton method, based on approximate inverse matrix techniques for solving non-linear boundary value problems, are presented. Additionally, isomorphic iterative methods are introduced for the efficient solution of non-linear systems. Explicit preconditioned conjugate gradient—type schemes in conjunction with approximate inverse matrix techniques are presented for the efficient solution of linear and non-linear system of algebraic equations. Theoretical estimates on the rate of convergence and computational complexity of the explicit preconditioned conjugate gradient method are also presented. Applications of the proposed methods on characteristic linear and non-linear problems are discussed and numerical results are given.     

A numerical approach for solving the high-order linear singular differential-difference equations

In this paper, a numerical method which produces an approximate polynomial solution is presented for solving the high-order linear singular differential-difference equations. With the aid of Bessel polynomials and collocation points, this method converts the singular differential-difference equations into the matrix equation. The matrix equation corresponds to a system of linear equations with the unknown Bessel coefficients. This method gives the analytic solutions when the exact solutions are polynomials. Finally, some experiments and their numerical solutions are given; by comparing the numerical results obtained from the other methods, we show the high accuracy and efficiency of the proposed method. All of the numerical computations have been performed on a PC using some programs written in MATLAB v7.6.0 (R2008a).     

Stabilized starting algorithms for collocation Runge-Kutta methods

In this paper, we propose a technique to stabilize some starting algorithms often used in the Newton-type iterations appearing when collocation Runge-Kutta methods are applied to solve stiff initial value problems. By following the ideas given in [1], we analyze the order (classical and stiff) of the new starting algorithms and pay special attention to their error amplifying functions. From the computational point of view, the new algorithms require the solution of an additional linear system per integration step, but as shown in the numerical experiments, this extra cost is compensated in most of the problems by their better stability properties.     

工业过程的子空间模型辨识

子空间模型辨识方法(SMI)是一类新兴的直接估计线性状态空间模型的黑箱建模方法,近年来获得了广泛关注.和传统的线性建模方法相比,SMI的优势不仅在于算法本身的简单可靠,也在于它的状态空间表达.本文首先简要介绍了SMI的基本思想以及3种基本算法(N4SID,MOESP,CVA).然后将这类方法应用于一个实际的工业过程建模,同时对3种SMI基本算法和一种传统辨识算法—预测误差方法(PEM)进行了研究对比.     

Numerical solution of nonlinear elliptic partial differential equations by a generalized conjugate gradient method

We have studied previously a generalized conjugate gradient method for solving sparse positive-definite systems of linear equations arising from the discretization of elliptic partial-differential boundary-value problems. Here, extensions to the nonlinear case are considered. We split the original discretized operator into the sum of two operators, one of which corresponds to a more easily solvable system of equations, and accelerate the associated iteration based on this splitting by (nonlinear) conjugate gradients. The behavior of the method is illustrated for the minimal surface equation with splittings corresponding to nonlinear SSOR, to approximate factorization of the Jacobian matrix, and to elliptic operators suitable for use with fast direct methods. The results of numerical experiments are given as well for a mildy nonlinear example, for which, in the corresponding linear case, the finite termination property of the conjugate gradient algorithm is crucial.     

Some three-term conjugate gradient methods with the inexact line search condition

The three-term conjugate gradient methods solving large-scale optimization problems are favored by many researchers because of their nice descent and convergent properties. In this paper, we extend some new conjugate gradient methods, and construct some three-term conjugate gradient methods. An remarkable property of the proposed methods is that the search direction always satisfies the sufficient descent condition without any line search. Under the standard Wolfe line search, the global convergence properties of the proposed methods are proved merely by assuming that the objective function is Lipschitz continuous. Preliminary numerical results and comparisons show that the proposed methods are efficient and promising.     

基于GPU-CUDA的共轭斜量法实现及性能对比

偏微分方程数值解法(包括有限差分法、有限元法)以及大量的数学物理方程数值解法最终都会演变成求解大型线性方程组。因此,探讨快速、稳定、精确的大型线性方程组解法一直是数值计算领域不断深入研究的课题且具有特别重要的意义。在迭代法中,共轭斜量法(又称共轭梯度法)被公认为最好的方法之一。但是,该方法最大缺点是仅适用于线性方程组系数矩阵为对称正定矩阵的情况,而且常规的CPU算法实现非常耗时。为此,通过将线性方程组系数矩阵作转换成对称矩阵后实施基于GPU-CUDA的快速共轭斜量法来解决一般性大型线性方程组的求解问题。试验结果表明:在求解效率方面,基于GPU-CUDA的共轭斜量法运行效率高,当线性方程组阶数超过3000时,其加速比将超过14;在解的精确性与求解过程的稳定性方面,与高斯列主元消去法相当。基于GPU-CUDA的快速共轭斜量法是求解一般性大型线性方程组快速而非常有效的方法。     

Numerical studies on Boussinesq-type equations via a split-step Fourier method

Boussinesq-type nonlinear wave equations with dispersive terms are solved via split-step Fourier methods. We decompose the equations into linear and nonlinear parts, then solve them orderly. The linear part can be projected into phase space by a Fourier transformation, and resulting in a variable separable ordinary differential system, which can be integrated exactly. Next, by an invert Fourier transformation, the classical explicit fourth-order Runge–Kutta method is adopted to solve the nonlinear subproblem. To examine the numerical accuracy and efficiency of the method, we compare the numerical solutions with exact solitary wave solutions. Additionally, various initial-value problems for all the listed Boussinesq-type system are studied numerically. In the study, we can observe that sech ²-type waves for KdV-BBM system will split into several solitons, which is a very interesting physical phenomenon. The interaction between solitons, including overtaking and head-on collisions, is also simulated.     

A projection method for solving nonsymmetric linear systems on multiprocessors

We consider the iterative solution of large sparse linear systems of equations arising from elliptic and parabolic partial differential equations in two or three space dimensions. Specifically, we focus our attention on nonsymmetric systems of equations whose eigenvalues lie on both sides of the imaginary axis, or whose symmetric part is not positive definite. This system of equation is solved using a block Kaczmarz projection method with conjugate gradient acceleration. The algorithm has been designed with special emphasis on its suitability for multiprocessors. In the first part of the paper, we study the numerical properties of the algorithm and compare its performance with other algorithms such as the conjugate gradient method on the normal equations, and conjugate gradient-like schemes such as ORTHOMIN(k), GCR(k) and GMRES(k). We also study the effect of using various preconditioners with these methods. In the second part of the paper, we describe the implementation of our algorithm on the CRAY X-MP/48 multiprocessor, and study its behavior as the number of processors is increased.     

A generalized single product and single‐period inventory problem with non‐linear parameters

In this paper, we extend the study of the classical single‐period newsboy inventory problem by considering costs that are non‐linear functions of the decision variable. We assume that the demand probability density function is known to the decision maker. We prove that, under some much more relaxed conditions, the total expected profit function remains concave and classical optimization methods can thus be used to obtain the global optimal solution. After that, we provide numerical examples for illustrative purpose.     

Runge–Kutta Residual Distribution Schemes

We are concerned with the solution of time-dependent non-linear hyperbolic partial differential equations. We investigate the combination of residual distribution methods with a consistent mass matrix (discretisation in space) and a Runge–Kutta-type time-stepping (discretisation in time). The introduced non-linear blending procedure allows us to retain the explicit character of the time-stepping procedure. The resulting methods are second order accurate provided that both spatial and temporal approximations are. The proposed approach results in a global linear system that has to be solved at each time-step. An efficient way of solving this system is also proposed. To test and validate this new framework, we perform extensive numerical experiments on a wide variety of classical problems. An extensive numerical comparison of our approach with other multi-stage residual distribution schemes is also given.     

Reliable and Efficient Computation of Optical Flow

In this paper, we present two very efficient and accurate algorithms for computing optical flow. The first is a modified gradient-based regularization method, and the other is an SSD-based regularization method. For the gradient-based method, to amend the errors in the discrete image flow equation caused by numerical differentiation as well as temporal and spatial aliasing in the brightness function, we propose to selectively combine the image flow constraint and a contour-based flow constraint into the data constraint by using a reliability measure. Each data constraint is appropriately normalized to obtain an approximate minimum distance (of the data point to the linear flow equation) constraint instead of the conventional linear flow constraint. These modifications lead to robust and accurate optical flow estimation. We propose an incomplete Cholesky preconditioned conjugate gradient algorithm to solve the resulting large and sparse linear system efficiently. Our SSD-based regularization method uses a normalized SSD measure (based on a similar reasoning as in the gradient-based scheme) as the data constraint in a regularization framework. The nonlinear conjugate gradient algorithm in conjunction with an incomplete Cholesky preconditioning is developed to solve the resulting nonlinear minimization problem. Experimental results on synthetic and real image sequences for these two algorithms are given to demonstrate their performance in comparison with competing methods reported in literature.     

Exploring and comparing the best “direct methods” forthe efficient training of MLP-networks

It is well known that the main difficulties of the algorithms based on backpropagation are the susceptibility to local minima and the slow adaptivity to the patterns during the training. In this paper, we present a class of algorithms, which overcome the above difficulties by utilizing some "direct" numerical methods for the computation of the matrices of weights. In particular, we investigate the performances of the FBFBK-LSB (least-squares backpropagation) algorithms and iterative conjugate gradient singular-value decomposition (ICGSVD), respectively, introduced by Barmann and Biegler-Konig (1993) and by the authors. Numerical results on several benchmark problems show a major reliability and/or efficiency of our algorithm ICGSVD.     

Projections and preconditioning for inconsistent least-squares problems

In this paper we describe two iterative algorithms for the numerical solution of linear least-squares problems. They are based on a combination between an extension of the classical Kaczmarzs projections method (Popa [5]) and an approximate orthogonalization technique due to Kovarik. We prove that both new algorithms converge to any solution of an inconsistent and rank-defficient least-squares problem (with respect to the choice of the initial approxi-mation), the convergence being much faster than for the classical Kaczmarz - like methods. Some numerical experiments on a first kind integral equation are described.     

A derivative-free affine scaling trust region methods based on probabilistic models with new nonmonotone line search technique for linear inequality constrained minimization without strict complementarity

ABSTRACT

In this paper, a derivative-free trust region methods based on probabilistic models with new nonmonotone line search technique is considered for nonlinear programming with linear inequality constraints. The proposed algorithm is designed to build probabilistic polynomial interpolation models for the objective function. We build the affine scaling trust region methods which use probabilistic or random models within a classical trust region framework. The new backtracking linear search technique guarantee the descent of the objective function, and new iterative points are in the feasible region. In order to overcome the strict complementarity hypothesis, under some reasonable conditions which are weaker than strong second order sufficient condition, we give the new and more simple identification function to structure the affine matrix. The global and local fast convergence of the algorithm are shown and the results of numerical experiments are reported to show the effectiveness of the proposed algorithm.