Randomized Householder-Cholesky QR Factorization with Multisketching
Explore the source record for details and available documents.
Engineering topics
Publications and source records attributed to Szyld, Daniel B..
Explore the source record for details and available documents.
In this paper, we extend to the block case the a posteriori bound showing superlinear convergence of the conjugate gradient method developed by van der Vorst and Vuik in [J. Comput. Applied Math., 48 (1993), pp. 327–341]. That is, we obtain similar bounds but now for the block conjugate gradient method. We also present a series of computational experiments, illustrating the validity of the bound developed here as well as the bound by Simoncini and Szyld from [SIAM Review, 47 (2005), pp. 247–272] using angles between subspaces. Using these bounds, we make some observations on the onset of superlinearity and how this onset depends on the eigenvalue distribution and the block size.
The block version of GMRES (BGMRES) is most advantageous over the single right hand side (RHS) counterpart when the cost of communication is high while the cost of floating point operations is not. This is the particular case on modern graphics processing units (GPUs), while it is generally not the case on traditional central processing units (CPUs). Here, in this paper, experiments on both GPUs and CPUs are shown that compare the performance of BGMRES against GMRES as the number of RHS increases, with a particular focus on GPU performance. The experiments indicate that there are many cases in which BGMRES is slower than GMRES on CPUs, but faster on GPUs. Furthermore, when varying the number of RHS on the GPU, there is an optimal number of RHS where BGMRES is clearly most advantageous over GMRES. A computational model for the GPU is developed using hardware specific parameters, providing insight towards how the qualitative behavior of BGMRES changes as the number of RHS increase, and this model also helps explain the phenomena observed in the experiments.
One- and two-level parallel asynchronous methods for the numerical solution of nonlinear systems of equations, especially those arising from (nonlinear) partial differential equations, are studied. The proposed methods are based on domain decomposition techniques. Local convergence theorems are presented in several cases, with appropriate hypotheses. Computational results on a shared memory multiprocessor machine for various problems exhibiting nonlinearities are reported, illustrating the potential of these asynchronous methods, especially for heterogeneous clusters.
Abstract not provided.
We consider the numerical solution of large scale singular (continuous-time) Lyapunov equations of the form AX + XA T + BB T = 0, where A is semistable, that is, its spectrum is contained in the left half plane, with the exception of a few semisimple eigenvalues at zero. We also consider the case of a few semisimple eigenvalues on the imaginary axis. We assume that we know these few eigenvalues (zero or imaginary), and that we have or can compute the corresponding invariant subspaces. We use this information to build an appropriate newly proposed subspace on which to project the Lyapunov equations, and then compute a low-rank approximation to the least squares solution. Selected illustrative numerical examples are provided.
We discuss how parallel implementations of linear iterative solvers generally alternate between phases of data exchange and phases of local computation. Increasingly large problem sizes and more heterogeneous compute architectures make load balancing and the design of low latency network interconnects that are able to satisfy the communication requirements of linear solvers very challenging tasks. In particular, global communication patterns such as inner products become increasingly limiting at scale. We explore the use of asynchronous communication based on one-sided Message Passing Interface primitives in the context of domain decomposition solvers. In particular, a scalable asynchronous two-level Schwarz method is presented. We discuss practical issues encountered in the development of a scalable solver and show experimental results obtained on a state-of-the-art supercomputer system that illustrate the benefits of asynchronous solvers in load balanced as well as load imbalanced scenarios. Using the novel method, we can observe speedups of up to four times over its classical synchronous equivalent.
We consider asynchronous versions of the first- and second-order Richardson methods for solving linear systems of equations. These methods depend on parameters whose values are chosen a priori. We explore the parameter values that can be proven to give convergence of the asynchronous methods. This is the first such analysis for asynchronous second-order methods. We find that for the first-order method, the optimal parameter value for the synchronous case also gives an asynchronously convergent method. For the second-order method, the parameter ranges for which we can prove asynchronous convergence do not contain the optimal parameter values for the synchronous iteration. In practice, however, the asynchronous second-order iterations may still converge using the optimal parameter values, or parameter values close to the optimal ones, despite this result. We explore this behavior with a multithreaded parallel implementation of the asynchronous methods.
We analyze an expansion of the generalized block Krylov subspace framework of [Electron. Trans. Numer. Anal., 47 (2017), pp. 100--126]. This expansion allows the use of low-rank modifications of the matrix projected onto the block Krylov subspace and contains, as special cases, the block GMRES method and the new block Radau--Arnoldi method. Within this general setting, we present results that extend the interpolation property from the nonblock case to a matrix polynomial interpolation property for the block case, and we relate the eigenvalues of the projected matrix to the latent roots of these matrix polynomials. Some error bounds for these modified block FOM methods for solving linear systems are presented. We then show how cospatial residuals can be preserved in the case of families of shifted linear block systems. This result is used to derive computationally practical restarted algorithms for block Krylov approximations that compute the action of a matrix function on a set of several vectors simultaneously. Finally, we prove some error bounds and present numerical results showing that two modifications of FOM, the block harmonic and the block Radau--Arnoldi methods for matrix functions, can significantly improve the convergence behavior.
Convergence of both synchronous and asynchronous optimized Schwarz algorithms for the shifted Laplacian operator on a bounded rectangular domain, in a one-way subdivision of the computational domain, with overlap, is shown. Convergence results are obtained under very mild conditions on the size of the subdomains and on the amount of overlap. A couple of results are also given, relating the convergence rate of the asynchronous method to changes in the size of the domain. Numerical experiments illustrate the theoretical results.
Several numerical methods were produced and analyzed. The main thrust of the work relates to inexact Krylov subspace methods for the solution of linear systems of equations arising from the discretization of partial differential equations. These are iterative methods, i.e., where an approximation is obtained and at each step. Usually, a matrix-vector product is needed at each iteration. In the inexact methods, this product (or the application of a preconditioner) can be done inexactly. Schwarz methods, based on domain decompositions, are excellent preconditioners for these systems. We contributed towards their understanding from an algebraic point of view, developed new ones, and studied their performance in the inexact setting. We also worked on combinatorial problems to help define the algebraic partition of the domains, with the needed overlap, as well as PDE-constraint optimization using the above-mentioned inexact Krylov subspace methods.