Search NASA⌕ Search

SEARCH · Search NASA

Results for “matrix-vector”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

90 records · Page 5

Optimizing the Performance of Sparse-Matrix Vector Products on Next-Generation Processors

Matrix-vector products are ubiquitous in high-performance scientific applications and have a growing set of occurrences in advanced data analysis activities. Achieving high performance for these kernels is therefore paramount, in part, because these operations can consume vast amounts of application execution time. In this report we document the development of several sparse-matrix vector product kernel implementations using a variety of programming models and approaches. Each kernel is run on a broad set of matrices selected to demonstrate the wide variety of matrix structure and sparsity that is possible with a single, generic kernel. For benchmarking and performance analysis, we utilize leading computing architectures for the NNSA/ASC program including Intel's Knights Landing processor and IBM's POWER8.

97 MATHEMATICS AND COMPUTING↗

A Themodynamically Consistent Fractional Visco-Elasto-Plastic Model with Memory-Dependent Damage for Anomalous Materials

We develop a thermodynamically consistent, fractional visco-elastoplastic model coupled with damage for anomalous materials. The model utilizes Scott-Blair rheological elements for both visco- elastic/plastic parts. The constitutive equations are obtained through Helmholtz free-energy potentials for Scott-Blair elements, together with a memory-dependent fractional yield function and dissipation inequalities. A memory-dependent Lemaitre-type damage is introduced through fractional damage energy release rates. For time-fractional integration of the resulting nonlinear system of equations, we develop a first-order semi-implicit fractional return-mapping algorithm. We also develop a finite-difference discretization for the fractional damage energy release rate, which results into Hankel-type matrix-vector operations for each time-step, allowing us to reduce the computational complexity from O(N 3 ) to O(N 2 ) through the use of Fast Fourier Transforms. Our numerical results demonstrate that the fractional orders for visco-elasto-plasticity play a crucial role in damage evolution, due to the competition between the anomalous plastic slip and bulk damage energy release rates.

36 MATERIALS SCIENCE↗

A New Capability of E4D For 3D Parallel Joint Inversion of DC Resistivity And Traveltime Data on Unstructured Mesh

A major challenge in interpreting geophysical data is how to derive consistent three-dimensional (3D) earth models of different physical properties from spatially and temporally limited measurements. Joint inversion with cross-gradient constraints is an approach to find such models by imposing structural similarities between different physical parameters. We have developed a parallel distributed-memory joint inversion code for direct-current (DC) resistivity and traveltime data using the cross-gradient constraint on unstructured mesh. The code utilizes existing E4D framework for parallel forward simulation, distributed storage and computation of the Jacobian matrix of forward operator, and parallel execution of matrix-vector multiplication during inversion. Besides, the joint inversion is solved by nonlinear conjugate gradient algorithm parallelized for DC resistivity and traveltime data. The joint inversion capability of E4D was tested using synthetic data from cross-borehole DC resistivity and traveltime data. The results indicate that the shape and size of the anomalies from the joint inversion are more reliable than those from separate inversions.

58 GEOSCIENCES↗

Preliminary Theoretical Analysis of Mixed Precision Krylov Subspace Methods (Q3 Report)

The third quarter of the project was spent performing theoretical finite precision analysis of Krylov subspace method variants that use mixed precision. Our focus here is on the Conjugate Gradient (CG) method and the Lanczos method. We have performed an analysis of maximum attainable accuracy for the classical CG method in which 3 precisions are used: a working precision ε, a precision ε IP for the inner product computations, and a precision ε MV for the matrix-vector products. Our results show that performing inner product computations in lower precision does not affect the attainable accuracy. Further, we have performed a complete error analysis of the s-step Lanczos algorithm. In this case, we show that the numerical behavior of the algorithm can be significantly improved by using extra precision in a small part of the computation. We summarize the main theorems in the remainder of the document. Other activities include attending biweekly xSDK meetings. The subsequent quarter will be spent finalizing these results into technical reports and/or manuscripts for submission to journals, as well as identifying opportunities for future work.

97 MATHEMATICS AND COMPUTING↗

Understanding HPC Benchmark Performance on Intel Broadwell and Cascade Lake Processors

Hardware platforms in high performance computing are constantly getting more complex to handle even when considering multicore CPUs alone. Numerous features and configuration options in the hardware and the software environment that are relevant for performance are not even known to most application users or developers. Microbenchmarks, i.e., simple codes that fathom a particular aspect of the hardware, can help to shed light on such issues, but only if they are well understood and if the results can be reconciled with known facts or performance models. The insight gained from microbenchmarks may then be applied to real applications for performance analysis or optimization. In this paper we investigate two modern Intel x86 server CPU architectures in depth: Broadwell EP and Cascade Lake SP. We highlight relevant hardware configuration settings that can have a decisive impact on code performance and show how to properly measure on-chip and off-chip data transfer bandwidths. The new victim L3 cache of Cascade Lake and its advanced replacement policy receive due attention. Finally we use DGEMM, sparse matrix-vector multiplication, and the HPCG benchmark to make a connection to relevant application scenarios.

97 MATHEMATICS AND COMPUTING↗

CrossSim Inference Manual v2.0

Neural networks are largely based on matrix computations. During forward inference, the most heavily used compute kernel is the matrix-vector multiplication (MVM): $W \vec{x} $. Inference is a first frontier for the deployment of next-generation hardware for neural network applications, as it is more readily deployed in edge devices, such as mobile devices or embedded processors with size, weight, and power constraints. Inference is also easier to implement in analog systems than training, which has more stringent device requirements. The main processing kernel used during inference is the MVM.

97 MATHEMATICS AND COMPUTING↗

A Low-Rank QTT-based Finite Element Method for Elasticity Problems

We present an efficient and robust numerical algorithm for solving the linear elasticity problem that combines the Quantized Tensor Train format and a domain partitioning strategy. This approach makes it possible to solve the linear elasticity problem on a computational domain that is more general than a square. By integrating Z-ordering and subdomain concatenation, our method substantially decreases memory usage and achieves a notable reduction in rank compared to established Finite Element implementations like the FEniCS platform. This efficiency is maintained while still guaranteeing exponential convergence with respect to the number of degrees of freedom. This performance gain, however, requires a fundamental rethinking of how core finite element operations are implemented. This includes changes to mesh discretization, node and degree of freedom ordering, stiffness matrix and internal nodal force assembly, and the execution of algebraic matrix-vector operations. In this work, we discuss all these aspects in detail and assess the method’s performance in the numerical approximation of three representative test cases.

97 MATHEMATICS AND COMPUTING↗

Lecture Notes on Multigrid Methods

The Lecture Notes are primarily based on a sequence of lectures given by the author while been a Fulbright scholar at 'St. Kliment Ohridski' University of Sofia, Sofia, Bulgaria during the winter semester of 2009-2010 academic year. The notes are somewhat expanded version of the actual one semester class he taught there. The material covered is slightly modified and adapted version of similar topics covered in the author's monograph 'Multilevel Block-Factorization Preconditioners' published in 2008 by Springer. The author tried to keep the notes as self-contained as possible. That is why the lecture notes begin with some basic introductory matrix-vector linear algebra, numerical PDEs (finite element) facts emphasizing the relations between functions in finite dimensional spaces and their coefficient vectors and respective norms. Then, some additional facts on the implementation of finite elements based on relation tables using the popular compressed sparse row (CSR) format are given. Also, typical condition number estimates of stiffness and mass matrices, the global matrix assembly from local element matrices are given as well. Finally, some basic introductory facts about stationary iterative methods, such as Gauss-Seidel and its symmetrized version are presented. The introductory material ends up with the smoothing property of the classical iterative methods and the main definition of two-grid iterative methods. From here on, the second part of the notes begins which deals with the various aspects of the principal TG and the numerous versions of the MG cycles. At the end, in part III, we briefly introduce algebraic versions of MG referred to as AMG, focusing on classes of AMG specialized for finite element matrices.

99 GENERAL AND MISCELLANEOUS↗

Highly Parallel Solver for Multi-scale Parquet Quantum Modeling of Strongly Correlated Materials

The parquet formalism to calculate the two-particle Green s functions of large systems requires the solution of a large, sparse, complex system of quadratic equations. If Nf Matsubara frequencies are used for a system of size Nc , and Newton s method is used to solve the nonlinear system, the Jacobian system has O(8Nt^3 ) variables and O(40Nt^4 ) complex entries where Nt = Nc Nf . For Nt = 1024, the nonlinear system has over 8.5 billion degrees of freedom and the sparse Jacobian will require over 351 TBytes of memory. The Jacobian is very expensive to store but the matrix-vector products can be computed directly. We are developing a highly scalable parallel solver that uses both OpenMP and MPI to exploit the multicore nodes. We present initial scalability results on the Cray XT5 that suggests the code can be scaled to solve larger problems with Nt 1024.

97 MATHEMATICS AND COMPUTING↗

STRUMPACK -- STRUctured Matrices PACKage

STRUMPACK - STRUctured Matrices PACKage - is a package for computations with sparse and dense structured matrix, i.e., matrices that exhibit some kind of low-rank property, in particular Hierarchically Semi Separable structure (HSS). Such matrices appear in many applications, e.g., FEM, BEM, Integral equations. etc. Exploiting this structure using certain compression algorithms allow for fast solution of linear systems and/or fast computation of matrix-vector products, which are the two main building blocks of matrix computations. STRUMPACK has presently two main components: a distributed-memory dense matrix computations package and a shared-memory sparse direct solver.

Source record↗

OpenSHMEM Implementation of HPCG Benchmark

We describe the effort to implement the HPCG benchmark using OpenSHMEM and MPI one-sided communication. Unlike the High Performance LINPACK (HPL) benchmark that places em- phasis on large dense matrix computations, the HPCG benchmark is dominated by sparse operations such as sparse matrix-vector product, sparse matrix triangular solve, and long vector operations. The MPI one-sided implementation is developed using the one-sided OpenSHMEM implementation. Pre- liminary results comparing the original MPI, OpenSHMEM, and MPI one-sided implementations on an SGI cluster, Cray XK7 and Cray XC30 are presented. The results suggest the MPI, OpenSHMEM, and MPI one-sided implementations all obtain similar overall performance but the MPI one-sided im- plementation seems to slightly increase the run time for multigrid preconditioning in HPCG on the Cray XK7 and Cray XC30.

OpenSHMEM HPCG↗

HFEM3D

Software solves the three-dimensional Poisson equation div(k(grad(u)) = f, by the finite element method for the case when material properties, k, are distributed over hierarchy of edges, facets and tetrahedra in the finite element mesh. Method is described in Weiss, CJ, Finite element analysis for model parameters distributed on a hierarchy of geometric simplices, Geophysics, v82, E155-167, doi:10.1190/GEO2017-0058.1 (2017). A standard finite element method for solving Poisson’s equation is augmented by including in the 3D stiffness matrix additional 2D and 1D stiffness matrices representing the contributions from material properties associated with mesh faces and edges, respectively. The resulting linear system is solved iteratively using the conjugate gradient method with Jacobi preconditioning. To minimize computer storage for program execution, the linear solver computes matrix-vector contractions element-by-element over the mesh, without explicit storage of the global stiffness matrix. Program output vtk compliant for visualization and rendering by 3rd party software. Program uses dynamic memory allocation and as such there are no hard limits on problem size outside of those imposed by the operating system and configuration on which the software is run. Dimension, N, of the finite element solution vector is constrained by the the addressable space in 32-vs-64 bit operating systems. Total storage requirements for the problem. Total working space required for the program is approximately 13*N double precision words.

Weiss, Chester J↗

A Flexible Power Method for Solving Infinite Dimensional Tensor Eigenvalue Problems

We propose a flexible power method for computing the leftmost, i.e., algebraically smallest, eigenvalue of an infinite dimensional tensor eigenvalue problem, $H x = \lambda x$, where the infinite dimensional symmetric matrix $H$ exhibits a translational invariant structure. We assume the smallest eigenvalue of $H$ is simple and apply a power iteration of $e^{-H}$ with the eigenvector represented in a compact way as a translational invariant infinite Tensor Ring (iTR). Hence, the infinite dimensional eigenvector can be represented by a finite number of iTR cores of finite rank. In order to implement this power iteration, we use a small parameter $t$ so that the infinite matrix-vector operation $e^{-Ht}x$ can efficiently be approximated by the Lie product formula, also known as Suzuki--Trotter splitting, and we employ a low rank approximation through a truncated singular value decomposition on the iTR cores in order to keep the cost of subsequent power iterations bounded. We also use an efficient way for computing the iTR Rayleigh quotient and introduce a finite size iTR residual which is used to monitor the convergence of the Rayleigh quotient and to modify the timestep $t$. In this paper, we discuss 2 different implementations of the flexible power algorithm and illustrate the automatic timestep adaption approach for several numerical examples.

Beeumen, Roel Van↗

Optimizing a Multiple Right-Hand Side Dslash Kernel for Intel Knights Corner

Lattice Quantum Chromo-dynamics (LQCD) stands unique as the only computationally tractable, non-perturbative, and model-independent quantum field theory of the strong nuclear force. LQCD achieves tractability by discretizing our familiar 4-dimensional space-time onto a hypercubic lattice. The core component of the linear solver used in LQCD is the Wilson-Dirac operator whose finite difference component is called the Dslash operator. Dslash is a nearest neighbor stencil operator summing matrix-vector multiplications over lattice points. The arithmetic intensity of a regular Dslash operator acting on a single vector in single precision is between 0.88-2.29 (depending on the tricks employed) making it memory bandwidth bound on most current architectures, including Xeon Phi. Maximizing the performance of Dslash on Xeon Phi requires deliberate exploitation of its architectural features. One key feature of Xeon Phi is its powerful vector processing unit with lengthy vector registers. There is a class of LQCD computations which aims to solve the Dirac equation for the same lattice gauge fields multiple times. In this case one can gain additional reuse of the gauge field, and potentially simplified vectorization, through applying Dslash to several vectors at the same time; often referred to as Multiple Right Hand Side (MRHS) Dslash. Previous work in this area focused on applying this approach to the Highly Improved Staggered operator, and in cases to the multi-Grid algorithm for Wilson clover fermions. Here, we present two approaches to MRHS Dslash: a vector register blocking approach and one using the software package QPhiX with a custom Code Generator for low-level intrinsics. We explore the Knights Corner instruction set, memory hierarchy, and threading capabilities. For standard lattice sizes used in testing, we observed significant speedups, with sustained performance of over 700 GFLOPS (single precision) in one instance. We achieved up to 27% of theoretical peak performance compared to a maximum of 12% obtained by the previous single RHS method in QPhiX.

Joó, Bálint↗

A nested dissection approach to sparse matrix partitioning for parallel computations.

We consider how to distribute sparse matrices among processes to reduce communication costs in parallel sparse matrix computations, specifically, sparse matrix-vector multiplication. Our main contributions are: (i) an exact graph model for communication with general (two-dimensional) matrix distribution, and (ii) a recursive partitioning algorithm based on nested dissection (substructuring). We show that the communication volume is closely linked to vertex separators. We have implemented our algorithm using hypergraph partitioning software to enable a fair comparison with existing methods. We present numerical results for sparse matrices from several application areas, with up to 9 million nonzeros. The results show that our new approach is superior to traditional 1d partitioning and comparable to a current leading partitioning method, the finegrain hypergraph method, in terms of communication volume. Our nested dissection method has two advantages over the fine-grain method: it is faster to compute, and the resulting distribution requires fewer communication messages.

97 MATHEMATICS AND COMPUTING↗

Improving Memory Subsystem Performance Using ViVA: Virtual Vector Architecture

The disparity between microprocessor clock frequencies and memory latency is a primary reason why many demanding applications run well below peak achievable performance. Software controlled scratchpad memories, such as the Cell local store, attempt to ameliorate this discrepancy by enabling precise control over memory movement; however, scratchpad technology confronts the programmer and compiler with an unfamiliar and difficult programming model. In this work, we present the Virtual Vector Architecture (ViVA), which combines the memory semantics of vector computers with a software-controlled scratchpad memory in order to provide a more effective and practical approach to latency hiding. ViVA requires minimal changes to the core design and could thus be easily integrated with conventional processor cores. To validate our approach, we implemented ViVA on the Mambo cycle-accurate full system simulator, which was carefully calibrated to match the performance on our underlying PowerPC Apple G5 architecture. Results show that ViVA is able to deliver significant performance benefits over scalar techniques for a variety of memory access patterns as well as two important memory-bound compact kernels, corner turn and sparse matrix-vector multiplication -- achieving 2x-13x improvement compared the scalar version. Overall, our preliminary ViVA exploration points to a promising approach for improving application performance on leading microprocessors with minimal design and complexity costs, in a power efficient manner.

97 MATHEMATICS AND COMPUTING↗

Improved parallel data partitioning by nested dissection with applications to information retrieval.

The computational work in many information retrieval and analysis algorithms is based on sparse linear algebra. Sparse matrix-vector multiplication is a common kernel in many of these computations. Thus, an important related combinatorial problem in parallel computing is how to distribute the matrix and the vectors among processors so as to minimize the communication cost. We focus on minimizing the total communication volume while keeping the computation balanced across processes. In [1], the first two authors presented a new 2D partitioning method, the nested dissection partitioning algorithm. In this paper, we improve on that algorithm and show that it is a good option for data partitioning in information retrieval. We also show partitioning time can be substantially reduced by using the SCOTCH software, and quality improves in some cases, too.

99 GENERAL AND MISCELLANEOUS↗

Reduce Operations: Send Volume Balancing While Minimizing Latency

Communication hypergraph model was introduced in a two-phase setting for encapsulating multiple communication cost metrics (bandwidth and latency), which are proven to be important in parallelizing irregular applications. In the first phase, computational-task-to-processor assignment is performed with the objective of minimizing total volume while maintaining computational load balance. In the second phase, communication-task-to-processor assignment is performed with the objective of minimizing total number of messages while maintaining communication-volume balance. The reduce-communication hypergraph model suffers from failing to correctly encapsulate send-volume balancing. We propose a novel vertex weighting scheme that enables part weights to correctly encode send-volume loads of processors for send-volume balancing. The model also suffers from increasing the total communication volume during partitioning. To decrease this increase, we propose a method that utilizes the recursive bipartitioning framework and refines each bipartition by vertex swaps. For performance evaluation, we consider column-parallel SpMV, which is one of the most widely known applications in which the reduce-task assignment problem arises. Extensive experiments on 313 matrices show that, compared to the existing model, the proposed models achieve considerable improvements in all communication cost metrics. Furthermore, these improvements lead to an average decrease of 30% in parallel SpMV time on 512 processors for 70 matrices with high irregularity.

97 MATHEMATICS AND COMPUTING↗