Restarted GMRES augmented with harmonic Ritz vectors for shifted linear systems

Frommer and Glassner [Frommer, A. and Glassner, U., 1998, Restarted GMRES for shifted linear systems, SIAM Journal on Scientific Computing, 19, 15–26.] develop a variant of the restarted GMRES method for shifted linear systems at the expense of only one matrix–vector multiplication per iteration. However, restarting slows down the convergence, even stagnation. We present a variant of the restarted GMRES augmented with some approximate eigenvectors for the shifted systems. The convergence can be much faster at little extra expense. Numerical experiments show its efficiency.     

一种GMRES与GMERR的混和算法

近年来Krylov子空间类算法得到了很大的发展，其中GMRES算法已成为求解大型稀疏非对称线性系统的一种成熟并且很有效的解法，但该算法有时会出现停滞，并且它是以残量来判断收敛，并不能很好地衡量近似解的精确程度，而GMERR算法是最近几年出现的另一种Krylov子空间类算法，它和GMRES算法相比是各有千秋，文章结合两种算法的优点，提出了一种组合算法，它对求解大型稀疏非对称线性系统相当有效。     

Parallelizable restarted iterative methods for nonsymmetric linear systems. part I: Theory

Large sparse nonsymmetric problems of the form A u = b are frequently solved using restarted conjugate gradient-type algorithms such as the popular GCR and GMRES algorithms. In this study we define a new class of algorithms which generate the same iterates as the standard GMRES algorithm but require as little as half of the computational expense. This performance improvement is obtained by using short economical three-term recurrences to replace the long recurrence used by GMRES. The new algorithms are shown to have good numerical properties in typical cases, and the new algorithms may be easily modified to be as numerically safe as standard GMRES. Numerical experiments with these algorithms are given in Part II, in which we demonstrate the improved performance of the new schemes on different computer architectures.     

Simpler GMRES with deflated restarting

In this paper we consider the simpler GMRES method augmented by approximate eigenvectors for solving nonsymmetric linear systems. We modify the augmented restarted simpler GMRES proposed by Boojhawon and Bhuruth to obtain a simpler GMRES with deflated restarting. Moreover, we also propose a residual-based simpler GMRES with deflated restarting, which is numerically more stable. The main advantage over the augmented version is that the simpler GMRES with deflated restarting requires less matrix-vector products per restart cycle. Some details of implementation are also considered. Numerical experiments show that the residual-based simpler GMRES with deflated restarting is effective.     

Accelerate weighted GMRES by augmenting error approximations

By augmenting error approximations at every restart cycle, this paper presents an accelerating strategy for restarted weighted generalized minimum residual (GMRES) method. We show that the procedure can effectively correct the occurrence of small skip D-angles, which indicates a slow convergent phase. Numerical results show that the new method converges much regular and faster than the weighted GMRES method. Finally, comparisons are made between the new and the recently proposed LGMRES methods.     

A block variant of the GMRES method on massively parallel processors

This paper presents a block variant of the GMRES method for solving general unsymmetric linear systems. This algorithm generates a transformed Hessenberg matrix by solely using block matrix operations and block data communications. It is shown that this algorithm with block size s, denoted by BVGMRES(s, m), is theoretically equivalent to the GMRES(s, m) method. The numerical results demonstrate that this algorithm can be more efficient than the standard GMRES method on a cache based single CPU computer with optimized BLAS kernels. Furthermore, the gain in efficiency is more significant on MPPs due to both efficient block operations and efficient block data communications. Preliminary numerical results on some real-world problems also show that this algorithm may be stable up to some reasonable block size.     

免停滞的GMRES(m)算法研究

1.引 言 解大型线性方程组仍是当今数值计算中的一个重要问题[1—8],GMRES(m)算法是解大型非对称线性方程组的常用方法[1],其中A∈Rn×n为大型稀疏非奇异矩阵,x,b∈Rn.然而,当A为非正实阵时,GMRES(m)解问题(1.1)可能会停滞.为此我们在第二节将先给出GMRES(m)停     

基于Deflation技术的预调制Restarted GMRES算法

电学层析成像的图像重建需要对逆问题进行求解,而求解过程中存在着非线性、欠定性以及病态性严重等难题,使得图像重建可能不收敛,或者致使收敛,但获得的图像分辨率较低。针对现有的一些图像重建算法,提出基于Deflation技术的预调制Restarted GMRES算法,在原有full GMRES算法基础上,提高了收敛速度以及图像成像分辨率,并通过仿真实验证明。     

Large sparse linear systems arising from mimetic discretization

In this work we perform an experimental study of iterative methods for solving large sparse linear systems arising from a second-order 2D mimetic discretization. The model problem is the 2D Poisson equation with different boundary conditions. We use GMRES with the restarted parameter and BiCGstab as iterative methods. We also use various preconditioning techniques including the robust preconditioner ILUt. The numerical experiments consist of large sparse linear systems with up to 643 200 degrees of freedom.     

SCALABILITY ANALYSIS OF PARALLEL GMRES IMPLEMENTATIONS

Abstract

Applications involving large sparse nonsymmetric linear systems encourage parallel implementations of robust iterative solution methods, such as GMRES(k). Two parallel versions of GMRES(k) based on different data distributions and using Householder reflections in the orthogonalization phase are analyzed with respect to scalability (their ability to maintain fixed efficiency with an increase in problem size and number of processors). A theoretical algorithm-machine model for scalability of GMRES(k) with fixed k is derived and validated by experiments on three parallel computers, each with different machine characteristics. The analysis for an adaptive version of GMRES(k), in which the restart value k is adapted to the problem, is also presented and scalability results for this case are briefly discussed.     

Some preconditioners for elliptic PDE-constrained optimization problems

For the structured systems of linear equations arising from the Galerkin finite element discretizations of elliptic PDE-constrained optimization problems, some preconditioners are proposed to accelerate the convergence rate of Krylov subspace methods such as GMRES for both cases of the Tikhonov parameter $\beta$ not very small (equal or greater than 1e?6) and sufficiently small (less than 1e?6), respectively. We derive the explicit expressions for the eigenvalues and eigenvectors of the corresponding preconditioned matrices. Numerical results show that the corresponding preconditioned GMRES methods perform and match well with the theoretical results.     

On the convergence of subproper (multi)-splitting methods for solving rectangular linear systems

Abstract We give a convergence criterion for stationary iterative schemes based on subproper splittings for solving rectangular systems and show that, for special splittings, convergence and quotient convergence are equivalent. We also analyze the convergence of multisplitting algorithms for the solution of rectangular systems when the coefficient matrices have special properties and the linear systems are consistent. Keywords: Rectangular linear system, iterative method, proper splitting, subproper splitting, regularity, Hermitian positive semi-definite matrix, multi-splitting, quotient convergence AMS Subject Classification: 65F10, 65F15     

GMRES implementations and residual smoothing techniques for solving ill-posed linear systems

There are verities of useful Krylov subspace methods to solve nonsymmetric linear system of equations. GMRES is one of the best Krylov solvers with several different variants to solve large sparse linear systems. Any GMRES implementation has some advantages. As the solution of ill-posed problems are important. In this paper, some GMRES variants are discussed and applied to solve these kinds of problems. Residual smoothing techniques are efficient ways to accelerate the convergence speed of some iterative methods like CG variants. At the end of this paper, some residual smoothing techniques are applied for different GMRES methods to test the influence of these techniques on GMRES implementations.     

A hybrid GMRES/LS-arnoldi method to accelerate the parallel solution of linear systems

We present a parallel hybrid asynchronous method to solve large sparse linear systems by the use of a large parallel machine. This method combines a parallel GMRES(m) algorithm with the least squares method that needs some eigenvalues obtained from a parallel Arnoldi algorithm. All of the algorithms run on different processors of an IBM SP3 or IBM SP4 computer simultaneously. This implementation of this hybrid method allows us to take advantage of the parallelism available and to accelerate the convergence by decreasing considerably the number of iterations.     

A numerical method for determining monotonicity and convergence rate in iterative learning control

In iterative learning control (ILC), a lifted system representation is often used for design and analysis to determine the convergence rate of the learning algorithm. Computation of the convergence rate in the lifted setting requires construction of large N×N matrices, where N is the number of data points in an iteration. The convergence rate computation is O(N²) and is typically limited to short iteration lengths because of computational memory constraints. As an alternative approach, the implicitly restarted Arnoldi/Lanczos method (IRLM) can be used to calculate the ILC convergence rate with calculations of O(N). In this article, we show that the convergence rate calculation using IRLM can be performed using dynamic simulations rather than matrices, thereby eliminating the need for large matrix construction. In addition to faster computation, IRLM enables the calculation of the ILC convergence rate for long iteration lengths. To illustrate generality, this method is presented for multi-input multi-output, linear time-varying discrete-time systems.     

An efficient method for constructing an ILU preconditioner for solving large sparse nonsymmetric linear systems by the GMRES method

The main idea of this paper is in determination of the pattern of nonzero elements of the LU factors of a given matrix A. The idea is based on taking the powers of the Boolean matrix derived from A. This powers of a Boolean matrix strategy (PBS) is an efficient, effective, and inexpensive approach. Construction of an ILU preconditioner using PBS is described and used in solving large nonsymmetric sparse linear systems. Effectiveness of the proposed ILU preconditioner in solving large nonsymmetric sparse linear systems by the GMRES method is also shown. Numerical experiments are performed which show that it is possible to considerably reduce the number of GMRES iterations when the ILU preconditioner constructed here is used. In numerical examples, the influence of k, the dimension of the Krylov subspace, on the performance of the GMRES method using an ILU preconditioner is tested. For all the tests carried out, the best value for k is found to be 10.     

The Influence of Interface Conditions on Convergence of Krylov-Schwarz Domain Decomposition for the Advection-Diffusion Equation

Several variants of Schwarz domain decomposition, which differ in the choice of interface conditions, are studied in a finite volume context. Krylov subspace acceleration, GMRES in this paper, is used to accelerate convergence. Using a detailed investigation of the systems involved, we can minimize the memory requirements of GMRES acceleration. It is shown how Krylov subspace acceleration can be easily built on top of an already implemented Schwarz domain decomposition iteration, which makes Krylov-Schwarz algorithms easy to use in practice. The convergence rate is investigated both theoretically and experimentally. It is observed that the Krylov subspace accelerated algorithm is quite insensitive to the type of interface conditions employed.     

BiCGStab(ℓ) for Families of Shifted Linear Systems

  We consider a seed system Ax = b together with a shifted linear system of the form

A Nested FGMRES Method for Parallel Calculation of Nuclear Reactor Transients

A semi-iterative method based on a nested application of Flexible Generalized Minimum Residual)FGMRES) was developed to solve the linear systems resulting from the application of the discretized two-phase hydrodynamics equations to nuclear reactor transient problems. The complex three-dimensional reactor problem is decomposed into simpler, more manageable problems which are then recombined sequentially by GMRES algorithms. Mathematically, the method consists of using an inner level GMRES to solve the preconditioner equation for an outer level GMRES. Applications were performed on practical, three-dimensional models of operating Pressurized Water Reactors (PWR). Serial and parallel applications were performed for a reactor model with two different details in the core representation. When appropriately tight convergence was enforced at each GMRES level, the results of the semi-iterative solver were in agreement with existing direct solution methods. For the larger model tested, the serial performance of GMRES was about a factor of 3 better than the direct solver and the parallel speedups were about 4 using 13 processors of the INTEL Paragon. Thus, for the larger problem over an order of magnitude reduction in the execution time was achieved indicating that the use of semi-iterative solvers and parallel computing can considerably reduce the computational load for practical PWR transient calculations.     

Convergence of the generalized-α scheme for constrained mechanical systems

A variant of the generalized-α scheme is proposed for constrained mechanical systems represented by index-3 DAEs. Based on the analogy with linear multistep methods, an elegant convergence analysis is developed for this algorithm. Second-order convergence is demonstrated both for the generalized coordinates and the Lagrange multipliers, and those theoretical results are illustrated by numerical tests.