Randomized numerical linear algebra: why you should care and how you can use it
Explore the source record for details and available documents.
Murray, Riley John [Sandia National Laboratories (↗
SEARCH · Search NASA
Results for “numerical linear algebra”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Yet Another NLA Library: T-LAPACK
In recent years, Randomized numerical linear algebra (RandNLA) proved to be more than a theoretical novelty: projects like RandLAPACK demonstrate its practical value across architectures, and projects like RandBLAS build trust in randomization as a tool for high-performance NLA. This BoF considers two main questions. First, what are the pressing issues in software standards and implementation that need to be resolved for RandNLA to become a core component of HPC? Second, how can we mobilize a community effort to make progress on these issues? The BoF will engage the audience to discuss the idea of growing the role of RandNLA in high-performance computing and what it would take to scale from niche prototypes to robust, production-quality software libraries.
97 MATHEMATICS AND COMPUTING↗
A Contextually-Aware Sensitivity Analysis to Guide the Design of Randomized Least Squares Solvers in Applications
Our work on the DOE-sponsored project “A Contextually-Aware Sensitivity Analysis to Guide the Design of Randomized Least Squares Solvers in Applications,” was an effort to address critical challenges in nu merical computing and its applications to optimization. The increasing demand for robust and scalable solutions to large-scale linear algebra problems has highlighted the limitations of traditional approaches, particularly in heterogeneous and extreme-scale computing environments. Randomized Numerical Linear Algebra (RandNLA) offers a promising framework to address these challenges, and this proposal builds on this foundation by introducing innovations in sensitivity analysis and computational adaptability.
97 MATHEMATICS AND COMPUTING↗
Developing ML/AI Methods for High-Throughput Characterization of Multiple-Sensor Streams of Tokamak Dynamics for High-Speed Control (Final Report)
This project evaluated and developed new mathematical and algorithmic techniques capable of handling (in real-time) the growing amounts of data generated by modern fusion research. While existing numerical linear algebra (NLA) methods provide the backbone to classical data analysis and algorithms, these methods fundamentally do not port to distributed architectures nor do they allow low-latency data reduction for control. Motivated by the needs for modern fusion reactors, this project explored and implemented new numerical methods to characterize plasma dynamics, respond in real-time to discharge evolution, and to process massive-scale data accurately and rapidly more fully. This project links expertise in multiple-sensor diagnostics of tokamak plasma dynamics from Columbia University’s Plasma Physics Laboratory with expertise in massive-scale data reduction and extreme data control algorithms at Columbia University’s Data Science Institute. This interdisciplinary project (i) applied machine learning methods, (ii) implemented a properly-trained neural-network for very fast processing of high-speed plasma videography, and (ii) developed the applied mathematical methods, based on randomized-NLA (rNLA) routines, for data analysis, reduction, and real-time control. The Columbia University High Beta Tokamak-Extended Pulse (HBT-EP) facility provided data to test new algorithms and partnership with Columbia University's Data Sciences Institute evaluated the broader use of new algorithms for many challenging control applications.
70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
Randomized Algorithms for Low-Rank Matrix and Tensor Decompositions
This paper surveys randomized algorithms in numerical linear algebra for low-rank decompositions of matrices and tensors. The survey begins with a review of classical matrix algorithms that can be accelerated by randomized dimensionality reduction, such as the singular value decomposition (SVD) or interpolative (ID) and CUR decompositions. Recent advances in randomized dimensionality reduction are discussed, including new methods of fast matrix sketching and sampling techniques, which are incorporated into classical matrix algorithms for fast low-rank matrix approximations. The extension of randomized matrix algorithms to tensors is then explored for several low-rank tensor decompositions in the CP and Tucker formats, including the higher-order SVD, ID, and CUR decomposition.
Pearce, Katherine J. [The University of Texas at A↗
Iterative methods in GPU-resident linear solvers for nonlinear constrained optimization
Linear solvers are major computational bottlenecks in a wide range of decision support and optimization computations. The challenges become even more pronounced on heterogeneous hardware, where traditional sparse numerical linear algebra methods are often inefficient. For example, methods for solving ill-conditioned linear systems have relied on conditional branching, which degrades performance on hardware accelerators such as graphical processing units (GPUs). To improve the efficiency of solving ill-conditioned systems, our computational strategy separates computations that are efficient on GPUs from those that need to run on traditional central processing units (CPUs). Our strategy maximizes the reuse of expensive CPU computations. Iterative methods, which thus far have not been broadly used for ill-conditioned linear systems, play an important role in our approach. In particular, we extend ideas from Arioli et al., (2007) to implement iterative refinement using inexact LU factors and flexible generalized minimal residual (FGMRES), with the aim of efficient performance on GPUs. In conclusion, we focus on solutions that are effective within broader application contexts, and discuss how early performance tests could be improved to be more predictive of the performance in a realistic environment.
97 MATHEMATICS AND COMPUTING↗
Surrogate-Based Autotuning for Randomized Sketching Algorithms in Regression Problems
Algorithms from Randomized Numerical Linear Algebra (RandNLA) are known to be effective in handling high-dimensional computational problems, providing high-quality empirical performance as well as strong probabilistic guarantees. However, their practical application is complicated by the fact that the user needs to set various algorithm-specific tuning parameters which are different from those used in traditional NLA. This paper demonstrates how a surrogate-based autotuning approach can be used to address fundamental problems of parameter selection in RandNLA algorithms. In particular, we provide a detailed investigation of surrogate-based autotuning for sketch-and-precondition (SAP)-based randomized least squares methods, which have been one of the great success stories in modern RandNLA. Empirical results show that our surrogate-based autotuning approach can achieve near-optimal performance with much less tuning cost than a random search (up to about 7.6x fewer trials of different parameter configurations). Moreover, while our experiments focus on least squares, our results demonstrate a general-purpose autotuning pipeline applicable to any kind of RandNLA algorithm.
Cho, Younghyun↗
Ensemble Simulation Techniques and Fast Randomized Algorithms
The major goals of the project were to develop and analyze new ensemble simulation techniques, including trajectory stratification and preconditioned MCMC techniques, as well as develop fast numerical linear algebra techniques closely related to ensemble simulation ideas. The trajectory stratification techniques involve simulating in parallel short trajectory fragments of a Markov process confined to a specific region of space‐time and then patching together the statistics gathered to assemble estimates of very general dynamical properties. We have also developed this approach for rare event simulation and extended the techniques to applications requiring a more general framework (such as electronic structure calculations). The preconditioned MCMC techniques involve simulating multiple Markov chains in parallel and then using information from the ensemble to speed the mixing of each individual chain. The fast randomized linear algebra methods are motivated by the diffusion Monte Carlo technique, but are applicable to finding the dominant eigenvalue of (almost) general matrices. For most non‐negative matrices, the schemes result in an error (compared to the power method) that is constant in the dimension of the problem. For more general matrices, we see a very clear sublinear cost trend in computational tests.
73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
Randomized Algorithms for Linear Solvers
Recently, randomized algorithms in numerical linear algebra, specifically those centered around random sketching, have gained traction in primarily theoretical research due to their potential to significantly reduce problem dimensionality at the cost of an O(1) multiplicative distortion factor. It has been assumed that this sketching can be done efficiently, but thorough investigation into how precisely to do it has been neglected. Moreover, the theory-based community has argued for sketching’s ability to reduce computational cost via complexity analysis, but has not researched how it affects the stability of the algorithms. At Sandia, efficient linear solvers that scale well on modern HPC architectures while maintaining stability are imperative for practical applications. In this LDRD, we developed a random sketching strategy that is substantially faster than existing ones, and demonstrate its superior performance in practice on a NVIDIA H100 GPU. Moreover, we show how this can be used to significantly outperform existing linear least squares solvers while improving the solver’s stability as well. Additionally, we demonstrate how this sketching strategy can be used to make a fast, stable QR factorization that can subsequently be used in s-step and block Krylov solvers. Finally, we incorporate a sketching-based block orthogonalization scheme into s-step GMRES, which is stable and faster than existing approaches on the Perlmutter supercomputer.
97 MATHEMATICS AND COMPUTING↗
Numerical stability in problems of linear algebra.
Mathematical problems are introduced as mappings from the space of input data to that of the desired output information. Then a numerical process is defined as a prescribed recurrence of elementary operations creating the mapping of the underlying mathematical problem. The ratio of the error committed by executing the operations of the numerical process (the roundoff errors) to the error introduced by perturbations of the input data (initial error) gives rise to the concept of lambda-stability. As examples, several processes are analyzed from this point of view, including, especially, old and new processes for solving systems of linear algebraic equations with tridiagonal matrices. In particular, it is shown how such a priori information can be utilized as, for instance, a knowledge of the row sums of the matrix. Information of this type is frequently available where the system arises in connection with the numerical solution of differential equations.
Babuska, I.↗
BoBa
BoBa is a C++ software library for working with large matrices, tensors, and tensor decompositions. The library provides tools for dense matrix and tensor operations, tensor decompositions, and tensor decomposition methods that support modern CPU and GPU architectures. It includes portable abstractions for linear algebra, tensor algebra, and multidimensional computation. BoBa is intended for scientific computing applications that involve large multidimensional data sets or high dimensional mathematical models. Its capabilities support tasks such as data compression, linear algebra, efficient numerical computation, and the development of scalable algorithms for heterogeneous hardware. Tutorials, tests, and example applications are included to help users learn and apply the library.
Yao, Jin [Lawrence Livermore National Laboratory (↗
Evaluation of data driven low-rank matrix factorization for accelerated solutions of the Vlasov equation
Low-rank methods have shown success in accelerating simulations of a collisionless plasma described by the Vlasov equation, but still rely on computationally costly linear algebra every time step. We propose a data-driven factorization method using artificial neural networks, specifically with convolutional layer architecture, that trains on existing simulation data. At inference time, the model outputs a low-rank decomposition of the distribution field of the charged particles, and we demonstrate that this step is faster than the standard linear algebra technique. Numerical experiments show that the method achieves comparable reconstruction accuracy for interpolation tasks, generalizing to unseen test data in a manner beyond just memorizing training data; patterns in factorization also inherently followed the same numerical trend as those within algebraic methods (e.g., truncated singular-value decomposition). However, when training on the first 70% of a time-series data and testing on the remaining 30%, the method fails to meaningfully extrapolate. Despite this limiting result, the technique may have benefits for simulations in a statistical steady-state or otherwise showing temporal stability. These results suggest that while the model offers a computationally efficient alternative for datasets with temporal stability, its current formulation is best suited for interpolation rather than for predicting future states in time-evolving systems. This study thus lays the groundwork for further refinement of neural network-based approaches to low-rank matrix factorization in high-dimensional plasma simulations.
97 MATHEMATICS AND COMPUTING↗
A finite-element method for lifting surfaces in steady incompressible subsonic flow
The problem of potential steady subsonic flow for lifting surfaces is considered. This problem requires the solution of an integral equation relating the value of the potential discontinuity on the lifting surface and its wake to the values of the normal derivative of the potential which are known from the boundary conditions. The lifting surface is divided into small (quadrilateral hyperboloidal) surface elements, which are described in terms of the Cartesian components of the four corner points. The values of the potential discontinuity and the normal derivative of the potential are assumed to be constant within each element and equal to their values at the centroids of the elements. This yields a set of linear algebraic equations. Numerical results are in good agreement with existing ones.
Morino, L.↗
Quantum many-body linear algebra, Hamiltonian moments, and a coupled-cluster inspired framework
Here, we propose a general strategy to develop quantum many-body approximations of primitives in linear algebra algorithms. As a practical example, we introduce a coupled-cluster inspired framework to produce approximate Hamiltonian moments and demonstrate its application in various linear algebra algorithms for ground state estimation. Through numerical examples, we illustrate the difference between the ground-state energies arising from quantum many-body linear algebra and those from the analogous many-body perturbation theory. Our results support the general idea of designing quantum many-body approximations outside of perturbation theory, providing a route to new algorithms and approximations.
Algorithms and data structure↗
Numerical marching techniques for fluid flows with heat transfer
The finite difference formulation and method of solution is presented for a wide variety of fluid flow problems with associated heat transfer. Only a few direct results from these formulations are given as examples, since the book is intended primarily to serve a discussion of the techniques and as a starting point for further investigations; however, the formulations are sufficiently complete that a workable computer program may be written from them. In the appendixes a number of topics are discussed which are of interest with respect to the finite difference equations presented. These include a very rapid method for solving certain sets of linear algebraic equations, a discussion of numerical stability, the inherent error in flow rate for confined flow problems, and a method for obtaining high accuracy with a relatively small number of mesh points.
Hornbeck, R. W.↗
IBM system/360 assembly language interval arithmetic software
Computer software designed to perform interval arithmetic is described. An interval is defined as the set of all real numbers between two given numbers including or excluding one or both endpoints. Interval arithmetic consists of the various elementary arithmetic operations defined on the set of all intervals, such as interval addition, subtraction, union, etc. One of the main applications of interval arithmetic is in the area of error analysis of computer calculations. For example, it has been used sucessfully to compute bounds on sounding errors in the solution of linear algebraic systems, error bounds in numerical solutions of ordinary differential equations, as well as integral equations and boundary value problems. The described software enables users to implement algorithms of the type described in references efficiently on the IBM 360 system.
Phillips, E. J.↗
A finite element method for potential aerodynamics around complex configurations
A general formulation for steady and oscillatory, subsonic and supersonic, potential linearized aerodynamic flow around complex configurations is presented. A linear integral equation relating the unknown potential on the surface of the body to the known downwash is used. The formulation is applied to the analysis of the flow field around wings and wing-body combinations. The surface is divided into small quadrilateral elements which are approximated with a hyperboloidal surface. The potential is assumed to be constant within each element. This yields a set of linear algebraic equations. The coefficients are evaluated analytically. Numerical results for steady and oscillatory, subsonic and supersonic flows indicate that the method, intrinsically general and flexible, is also fast, accurate and in excellent agreement with existing results.
Chen, L.-T.↗
Juncture stress fields in multicellular shell structures. Volume I - Numerical methods of solving large matrices Final report
Numerical techniques for digital computer solution of algebraic simultaneous linear equations from finite difference approximation of partial differential equations of thin elastic shells
DIGITAL COMPUTER↗