Degrees of freedom and modular structure in matrix multiplication
Matrix multiplication algorithm based on degrees of freedom analysis for various transformations and spectral analysis
Andrews, H. C.↗
SEARCH · Search NASA
Results for “Matrix Multiplication”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Parallel matrix multiplication on the Connection Machine
Matrix multiplication is a computation and communication intensive problem. Six parallel algorithms for matrix multiplication on the Connection Machine are presented and compared with respect to their performance and processor usage. For n by n matrices, the algorithms have theoretical running times of O(n to the 2nd power log n), O(n log n), O(n), and O(log n), and require n, n to the 2nd power, n to the 2nd power, and n to the 3rd power processors, respectively. With careful attention to communication patterns, the theoretically predicted runtimes can indeed be achieved in practice. The parallel algorithms illustrate the tradeoffs between performance, communication cost, and processor usage.
Tichy, Walter F.↗
E-beam generated holographic masks for optical vector-matrix multiplication
An optical vector matrix multiplication scheme that encodes the matrix elements as a holographic mask consisting of linear diffraction gratings is proposed. The binary, chrome on glass masks are fabricated by e-beam lithography. This approach results in a fairly simple optical system that promises both large numerical range and high accuracy. A partitioned computer generated hologram mask was fabricated and tested. This hologram was diagonally separated outputs, compact facets and symmetry about the axis. The resultant diffraction pattern at the output plane is shown. Since the grating fringes are written at 45 deg relative to the facet boundaries, the many on-axis sidelobes from each output are seen to be diagonally separated from the adjacent output signals.
Arnold, S. M.↗
A family of new efficient arrays for matrix multiplication
The authors present a regular iterative algorithm for matrix multiplication and show that several well-known matrix multiplication arrays are directly obtained from it, differing only in the choice of iteration vector. They then present a regular iterative algorithm for matrix multiplication using the method of Winograd (1968) and show in detail how to derive one array from this algorithmic description. Other arrays in the same family can similarly be obtained for different choices of the iteration space. The new arrays compute the product of two matrices faster than available conventional arrays and use a smaller number of processor cells.
Jagadish, H. V.↗
Integrated-optical approaches to matrix multiplication
The solution of matrix equations is essential to carrying out a large variety of control algorithms and to reducing certain types of data such as the output of a multispectral sensor array. Optical techniques and, in particular, integrated-optical circuits (IOC's) can provide compact, low-power devices for performing the mitrix multiplications necessary for the solution of these problems. A specific IOC for performing vector-matrix multiplication and several approaches to the design of IOC's for matrix-matrix multiplication will be discussed.
Verber, C. M.↗
Software for Fault-Tolerant Matrix Multiplication
Formal Linear Algebra Recovery Environment is a computer program for high-performance, fault-tolerant matrix multiplication. The program is based on an extension of the prior theory and practice of fault-tolerant matrix matrix multiplication of the form C = AB. This extension provides low-overhead methods for detecting errors, not only in C, but also in A and/or B. These methods enable the detection of all errors as long as, in a given case, only one entry in A, B, or C is corrupted. The program also provides for following a low-overhead rollback approach to correct errors once detected. Results of computational experiments have demonstrated that the methods implemented in this program work well in practice while imposing an acceptably low level of overhead, relative to high-performance matrix-multiplication methods that do not afford fault tolerance.
Katz, Daniel↗
The use of the Winograd matrix multiplication algorithm in digital multispectral processing
The Winograd procedure for matrix multiplication provides a method whereby general matrix products may be computed more efficiently than the normal method. The algorithm and the time savings that can be effected are described. A FORTRAN program is provided which performs a general matrix multiply according to this algorithm. A variation of this procedure that may be used to calculate Gaussian probability density functions is also described. It is shown how a time savings can be effected in this calculation. The extension of this method to other similar calculations should yield similar savings.
Vanrooy, D. L.↗
Optical matrix-matrix multiplication method demonstrated by the use of a multifocus hololens
A method of optical matrix-matrix multiplication is presented. The feasibility of the method is also experimentally demonstrated by the use of a dichromated-gelatin multifocus holographic lens (hololens). With the specific values of matrices chosen, the average percentage error between the theoretical and experimental data of the elements of the output matrix of the multiplication of some specific pairs of 3 x 3 matrices is 0.4 percent, which corresponds to an 8-bit accuracy.
Liu, H. K.↗
autoGEMM: Pushing the Limits of Irregular Matrix Multiplication on Arm Architectures
This paper presents an open-source library that pushes the limits of performance portability for irregular General Matrix Multiplication (GEMM) on the widely-used Arm architectures. Our library, autoGEMM, is designed to support a wide range of Arm processors: from edge devices to HPC-grade CPUs. autoGEMM generates optimized kernels for various hardware configurations by auto-combining fragments of autogenerated micro-kernels that employ hand-written optimizations to maximize computational efficiency. We optimize the kernel pipeline by tuning the register reuse and the data load/store overlapping. In addition, we use a dynamic tiling scheme to generate balanced tile shapes. Finally, we position autoGEMM on top of the TVM framework where our dynamic tiling scheme prunes the search space for TVM to identify the optimal combination of parameters for code optimization. Evaluations on five different classes of Arm chips demonstrate the advantages of autoGEMM. For small matrices, autoGEMM achieves 98% of peak and up to 2.0x speedup over state-of-the-art libraries such as LIBXSMM and LibShalom. For irregular matrices (i.e. tall skinny and long rectangles), autoGEMM is 1.3-2.0x faster than widely-used libraries such as OpenBLAS and Eigen. autoGEMM is publicly available at: https://github.com/wudu98/autoGEMM.
Wu, Du↗
Architecture-Aware Models of AI Engines for High-Performance Matrix Matrix Multiplication
The AI Engine (AIE) architecture, available in systems from mobile SoCs to server-class FPGAs, aims to efficiently execute AI/ML tasks through a two-dimensional array of compute tiles. Previous work on AIEs has explored different approaches to mapping computation across spatial arrays, but the compute kernel running on each tile has not been the focus. Additionally, the AIE-ML architecture introduces memory tiles and omits programmable logic, requiring new approaches to staging and moving data throughout the array. In this work we update analytical models developed for CPUs to produce the design of high performance kernels while introducing new model considerations such as memory structure, throughput, and latency as required by the AIE hardware. We evaluate our models by developing AIE-ML kernels for matrix multiplication in low-precision data types showing performance up to 95% of compute peak for the kernel when data resides in local memory and above 90% of compute peak when data resides in main memory.
Binder, Elliott D. [Carnegie Mellon University, Pi↗
An efficient sparse matrix multiplication scheme for the CYBER 205 computer
This paper describes the development of an efficient algorithm for computing the product of a matrix and vector on a CYBER 205 vector computer. The desire to provide software which allows the user to choose between the often conflicting goals of minimizing central processing unit (CPU) time or storage requirements has led to a diagonal-based algorithm in which one of four types of storage is selected for each diagonal. The candidate storage types employed were chosen to be efficient on the CYBER 205 for diagonals which have nonzero structure which is dense, moderately sparse, very sparse and short, or very sparse and long; however, for many densities, no diagonal type is most efficient with respect to both resource requirements, and a trade-off must be made. For each diagonal, an initialization subroutine estimates the CPU time and storage required for each storage type based on results from previously performed numerical experimentation. These requirements are adjusted by weights provided by the user which reflect the relative importance the user places on the two resources. The adjusted resource requirements are then compared to select the most efficient storage and computational scheme.
Lambiotte, Jules J., Jr.↗
Efficient spares matrix multiplication scheme for the CYBER 203
This work has been directed toward the development of an efficient algorithm for performing this computation on the CYBER-203. The desire to provide software which gives the user the choice between the often conflicting goals of minimizing central processing (CPU) time or storage requirements has led to a diagonal-based algorithm in which one of three types of storage is selected for each diagonal. For each storage type, an initialization sub-routine estimates the CPU and storage requirements based upon results from previously performed numerical experimentation. These requirements are adjusted by weights provided by the user which reflect the relative importance the user places on the resources. The three storage types employed were chosen to be efficient on the CYBER-203 for diagonals which are sparse, moderately sparse, or dense; however, for many densities, no diagonal type is most efficient with respect to both resource requirements. The user-supplied weights dictate the choice.
Lambiotte, J. J., Jr.↗
Review of NASTRAN development relative to efficiency of execution
This paper reviews the development of NASTRAN relative to the efficiency of execution, with particular emphasis on those items which have changed significantly since the original release of NASTRAN. Features discussed include main and secondary storage utilization, matrix packing, matrix assembly, matrix multiplication, matrix decomposition and equation solution. Also a brief look into the future discusses the questions of faster arithmetic units and more effective storage utilization. In some cases the improvements in NASTRAN efficiency have resulted from taking advantage of hardware developments, while in other cases increased efficiency has resulted from improvements in the state of the art for data processing or matrix operations. The modular design of NASTRAN has made it possible to improve the efficiency in many parts of NASTRAN without changing the basic design of the program.
Mccormick, C. W.↗
Two-Tower Quantum Matrix Chain Multiplication: Trading Qubits for Depth
Matrix chain multiplication -- computing $\mathcal{W} = M^{(0)}\cdots M^{(K-1)}$ where $M^{(k)} \in \mathbb{R}^{P_k \times P_{k+1}}$-- arises in scientific computing, machine learning, and graph analysis. Despite the importance of this problem, for chains of distinct matrices, the classical number of operations grows linearly with the chain length $K$ and polynomially in the matrix dimensions. We present \emph{Two-Tower Matrix Multiplication}, a quantum subroutine that encodes the product $\mathcal{W}$ of the $K$ matrices into a quantum state in circuit depth $\mathcal{O}(\max_{k} \mathrm{polylog} (P_k P_{k+1}))$, which is independent of~$K$ within the QRAM-based state-preparation model, whereas the qubit count is $\mathcal{O}\bigl(\sum_{k} \log P_k \bigr)$; the total gate count remains linear in $K$, so the gain is in the circuit depth. The construction interleaves state-preparation operators across two layers; within each layer, all operators act on disjoint registers and execute in parallel. This subroutine can be specialized for the chain-vector case, which computes the product of $K-1$ matrices applied to a vector. We prove the correctness of the subroutine for all $K$ and provide two implementations using the Qiskit and QCLAB frameworks. The subroutine is applicable to any downstream quantum algorithm that operates on a matrix encoded in the statevector, including norm estimation, graph-matrix powers, linear system solving, and quantum machine learning kernels.
Antonioli, Giacomo [Pisa U.] (ORCID:00090000668703↗
Matrix-vector multiplication in thin photorefractive GaAs crystals
Optical matrix-vector multiplication using four-wave mixing in a thin photorefractive GaAs crystal is demonstrated. Using a thin wafer of GaAs offers the potential to integrate the encoding spatial light modulators directly on the wave-mixing medium.
Cheng, Li-Jen↗
Quantum Time-Space Tradeoffs for Matrix Problems
We consider the time and space required for quantum computers to solve a wide variety of problems involving matrices, many of which have only been analyzed classically in prior work. Our main results show that for a range of linear algebra problems—including matrix-vector product, matrix inversion, matrix multiplication and powering—existing classical time-space tradeoffs, several of which are tight for every space bound, also apply to quantum algorithms with at most a constant factor loss. For example, for almost all fixed matrices 𝐴, including the discrete Fourier transform matrix, we prove that quantum circuits with at most 𝑇 input queries and 𝑆 qubits of memory require 𝑇 = Ω(𝑛 2 /𝑆) to compute matrix-vector product 𝐴𝑥 for 𝑥 ∈{0,1 𝑛 . We similarly prove that matrix multiplication for 𝑛 ×𝑛 binary matrices requires 𝑇 = Ω(𝑛 3 /$\sqrt{𝑆}$). Because many of our lower bounds are matched by deterministic algorithms with the same time and space complexity, our results show that quantum computers cannot provide any asymptotic advantage for these problems with any space bound. We obtain matching lower bounds for the stronger notion of quantum cumulative memory complexity—the sum of the space per layer of a circuit. We also consider Boolean (i.e., AND-OR) matrix multiplication and matrix-vector products, improving the previous quantum time-space tradeoff lower bounds for 𝑛 × 𝑛 Boolean matrix multiplication to 𝑇 = Ω(𝑛 2.5 /𝑆 1/4 ) from 𝑇 = Ω(𝑛 2.5 /𝑆 1/2 ). Our improved lower bound for Boolean matrix multiplication is based on a new coloring argument that extracts more from the strong direct product theorem that was the basis for prior work. To obtain our tight lower bounds for linear algebra problems, we require much stronger bounds than strong direct product theorems. We obtain these bounds by adding a new bucketing method to the quantum recording-query technique of Zhandry that lets us apply classical arguments to upper bound the success probability of quantum circuits.
lower bounds↗
Fast polar decomposition of an arbitrary matrix
The polar decomposition of an m x n matrix A of full rank, where m is greater than or equal to n, can be computed using a quadratically convergent algorithm. The algorithm is based on a Newton iteration involving a matrix inverse. With the use of a preliminary complete orthogonal decomposition the algorithm can be extended to arbitrary A. How to use the algorithm to compute the positive semi-definite square root of a Hermitian positive semi-definite matrix is described. A hybrid algorithm which adaptively switches from the matrix inversion based iteration to a matrix multiplication based iteration due to Kovarik, and to Bjorck and Bowie is formulated. The decision when to switch is made using a condition estimator. This matrix multiplication rich algorithm is shown to be more efficient on machines for which matrix multiplication can be executed 1.5 times faster than matrix inversion.
Higham, Nicholas J.↗
Parallel Gaussian elimination of a block tridiagonal matrix using multiple microcomputers
The solution of a block tridiagonal matrix using parallel processing is demonstrated. The multiprocessor system on which results were obtained and the software environment used to program that system are described. Theoretical partitioning and resource allocation for the Gaussian elimination method used to solve the matrix are discussed. The results obtained from running 1, 2 and 3 processor versions of the block tridiagonal solver are presented. The PASCAL source code for these solvers is given in the appendix, and may be transportable to other shared memory parallel processors provided that the synchronization outlines are reproduced on the target system.
Blech, Richard A.↗