Fast multiplication of random dense matrices with sparse matrices
This work focuses on accelerating the multiplication of a dense random matrix with a (fixed) sparse matrix, which is frequently used in sketching algorithms.
Demmel, James↗
SEARCH · Search NASA
Results for “matrices”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Symmetry-Based Structured Matrices for Efficient Approximately Equivariant Networks
There has been much recent interest in designing symmetry-aware neural networks (NNs) exhibiting relaxed equivariance. Such NNs aim to interpolate between being exactly equivariant and being fully flexible, affording consistent performance benefits. In a separate line of work, certain structured parameter matrices -- those with displacement structure, characterized by low displacement rank (LDR) -- have been used to design small-footprint NNs. Displacement structure enables fast function and gradient evaluation, but permits accurate approximations via compression primarily to classical convolutional neural networks (CNNs). In this work, we propose a general framework -- based on a novel construction of symmetry-based structured matrices -- to build approximately equivariant NNs with significantly reduced parameter counts. Our framework integrates the two aforementioned lines of work via the use of so-called Group Matrices (GMs), a forgotten precursor to the modern notion of regular representations of finite groups. GMs allow the design of structured matrices -- resembling LDR matrices -- which generalize the linear operations of a classical CNN from cyclic groups to general finite groups and their homogeneous spaces. We show that GMs can be employed to extend all the elementary operations of CNNs to general discrete groups. Further, the theory of structured matrices based on GMs provides a generalization of LDR theory focussed on matrices with cyclic structure, providing a tool for implementing approximate equivariance for discrete groups. We test GM-based architectures on a variety of tasks in the presence of relaxed symmetry. We report that our framework consistently performs competitively compared to approximately equivariant NNs, and other structured matrix-based compression frameworks, sometimes with a one or two orders of magnitude lower parameter count.
Samudre, Ashwin↗
What is the gradient of a scalar function defined on a subspace of square matrices?
We illustrate a technique to calculate the gradient of scalar functions that are defined on any arbitrary matrix subspace. It generalizes our earlier work titled “What is the gradient of a scalar function of a symmetric matrix ?”(Indian Journal of Pure and Applied Mathematics (2022), https://doi.org/10.1007/s13226-022-00313-x), in which we considered the special case of the subspace of symmetric matrices. Extant methods to calculate the gradient in such cases have an inherent flaw which leads to spurious results that populate several publications, as well as respected textbooks and handbooks on matrix calculus. Here, we examine these sources and results in a rigorous and concrete mathematical setting of a finite-dimensional inner-product space and discover the inherent flaw and also a remedy. We demonstrate two ways to calculate the derivative/gradient and second derivative for scalar functions of matrices defined over an arbitrary matrix subspace; the first method is by considering any (differentiable) extension to the space of square matrices and projection of its gradient onto the given subspace. The second method utilizes an ordered basis and computes each component of the gradient through evaluation of the directional derivative. All the ideas presented are illustrated by non-trivial examples, namely, considering the subspace of 3 x 3 circulant and Toeplitz matrices and presenting the results of gradient-descent with both the spurious and correct gradients. Moreover, our bibliography makes it clear that a rigorous approach to matrix calculus is not common in practice, and our presentation of matrix calculus in the language of inner-product spaces will be significant and meaningful for applied mathematicians, engineers and researchers working in inter-disciplinary fields to avoid the conceptual pitfalls that exist.
97 MATHEMATICS AND COMPUTING↗
An Evaluation of Actinide Reactivity with CO 2 , O 2 , and O 2 /He Gases using Inductively Coupled Plasma Tandem Mass Spectrometry: Application to Simultaneous Measurement of 241 Am/ 241 Pu Ratios in Unseparated Complex Matrices
Accurate actinide measurements are critical within the field of nuclear science. Traditional methods for actinide quantification require time-consuming sample processing prior to analysis. There is a need for rapid analytical techniques that still maintain a high degree of accuracy. In this work, actinide reactivity was assessed for multiple oxygen-containing reaction gases using quadrupole inductively coupled plasma tandem mass spectrometry (Q-ICP-MS/MS) to evaluate actinide analysis in complex sample matrices without analyte-matrix separation. A novel method was developed to measure 241 Am/ 241 Pu in complex sample matrices using O 2 /He reaction gas with no matrix removal or analyte pre-concentration. This inline method reduces matrix-derived polyatomic interferences that complicate traditional ICP-MS analyses by mass-shifting to 241 Am 16 O + and 241 Pu 16 O 2 + , allowing Am and Pu to be mass separated for simultaneous analysis. While mass shifting is efficient, a small portion of Am + (<1.3%) and Pu + (<1.4%) react to from AmO 2 + and PuO + , respectively. Therefore, a mass balance approach was used, in combination with reactivity determined from 242 Pu and 243 Am standard solutions, to correct for residual 241 PuO + and 241 AmO 2 + . The method was validated by measuring 241 Am/ 241 Pu in Pu isotope standards CRM-136 and CRM-137 (separated in March/April 1970 and February 2022, respectively) in both neat solutions and complex matrices containing diluted soil (NIST SRM 2711a, >1000 µg·g -1 ). Method detection limits of 15.9 and 9.6 fg·g -1 were determined for 241 Am and 241 Pu, respectively, and 241 Am/ 241 Pu ratios were measured with accuracies within <3.5%. In conclusion, this work presents the first direct analysis of 241 Am/ 241 Pu in unseparated complex matrices, advancing capabilities for rapid actinide measurements.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Determining the Ensemble N -Representability of Reduced Density Matrices
The N-representability problem for reduced density matrices remains a fundamental challenge in electronic structure theory. Following our previous work that employs a unitary-evolution algorithm based on an adaptive derivative-assembled pseudo-Trotter variational quantum algorithm to probe pure-state N-representability of reduced density matrices [J. Chem. Theory Comput. 2024, 20, 9968], in this work we propose a practical framework for determining the ensemble N-representability of a p-body matrix. This is accomplished using a purification strategy that embeds an ensemble state into a pure state defined on an extended Hilbert space, such that the reduced density matrices of the purified state reproduce those of the original ensemble. By iteratively applying variational unitaries to an initial purified state, the proposed algorithm minimizes the Hilbert-Schmidt distance between its p-body reduced density matrix and a specified target p-body matrix, which serves as a measure of the N-representability of the target. This methodology facilitates both error correction of defective ensemble reduced density matrices and quantum-state reconstruction on a quantum computer, offering a route for density-matrix refinement. We validate the algorithm with numerical simulations on systems of two, three, and four electrons in both simple models as well as molecular systems at finite temperature, demonstrating its robustness.
37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
Esterification of 3-quinuclidinol, a marker for the incapacitant BZ, for analysis by EI-GC-MS in OPCW test matrices
The analysis by EI-GC-MS of 3-quinuclidinol (3Q), a marker for the riot control and incapacitating agent quinuclinidyl benzilate (BZ), using several, new acylation strategies is described. After evaluating and optimizing conditions for the acylations, these were tested on their ability to successfully derivatize 3Q in three different matrices featured in the 44th Organisation for the Prohibition of Chemical Weapons (OPCW) proficiency test (PT). As 3Q is a highly polar compound, the work here describes acylation strategies that generate 3Q analogs with superior gas chromatographic profiles relative to the underivatized material. The acyl groups studied in this work included acetyl, benzoyl, pentafluorobenzoyl and the bis(3,5-trifluoromethyl)benzoyl. The acylated 3Q products provide sharp, detectable peaks with significantly different retention times from that of the unmodified 3Q. Thus, the retention times for acetyl-, benzoyl-, pentafluorobenzoyl- and 3,5-bis(trifluoromethyl)benzoyl-3Q were determined to be 15.9, 24.9, 22.8 and 22.1 min respectively, in stark contrast to the unmodified 3Q which provides a sharp peak with a retention time centered at ~ 13.5 min only at high concentrations (> 10 µg/mL or µg/g) while a broad peak (RT ~ 15–15.5 min) at low concentrations (< 5 µg/mL or µg/g). The developed protocol was used to derivatize 3Q in three separate matrices. The first two matrices were liquid samples featured during the 44th OPCW PT at two separate concentrations in each (5 and 50 µg/mL). The last matrix was a soil sample featured in the same 44th OPCW PT wherein the 3Q had been spiked at a 12 µg/g concentration. The approach involves the extraction of 3Q from all matrices followed by its acylation resulting in a second reportable analytical method for an OPCW PT. The presented derivatization strategies should find wide applicability particularly in laboratories involved with the analysis of this chemical weapon agent degradation product and those participating in the yearly OPCW PTs.
3-quinuclidinol↗
Improved Evaluation of Large Network Matrices for Linear Power Flow Within Optimization Problems
This work presents methods for evaluating the Power Transfer Distribution Factor (PTDF) and Line Outage Distribution Factor (LODF) matrices by employing sparse linear algebra for large-scale computing applications. These matrices play a critical role in many power system applications, such as the Unit Commitment Problem (UC), pre- and post-contingency power flow analysis, and transmission expansion. These matrices are typically dense, which means they require a significant amount of time and memory to be computed for large networks. However, by analyzing the structure of the matrices and their computation method, it is possible to use reduced memory methods based on sparse matrix operations. This paper shows that sparse linear algebra algorithms are faster and require less memory and time than traditional dense approaches. Additionally, we explore the effect of matrix sparsification by eliminating trailing digits on power flow calculations.
large scale↗
What is the gradient of a scalar function defined on a subspace of square matrices?
We illustrate a technique to calculate the gradient of scalar functions that are defined on any arbitrary matrix subspace. It generalizes our earlier work titled “What is the gradient of a scalar function of a symmetric matrix ?”(Indian Journal of Pure and Applied Mathematics (2022), doi:10.1007/s13226-022-00313-x), in which we considered the special case of the subspace of symmetric matrices. Extant methods to calculate the gradient in such cases have an inherent flaw which leads to spurious results that populate several publications, as well as respected textbooks and handbooks on matrix calculus. We examine these sources and results in a rigorous and concrete mathematical setting of a finite-dimensional inner-product space and discover the inherent flaw and also a remedy. We demonstrate two ways to calculate the derivative/gradient and second derivative for scalar functions of matrices defined over an arbitrary matrix subspace; the first method is by considering any (differentiable) extension to the space of square matrices and projection of its gradient onto the given subspace. The second method utilizes an ordered basis and computes each component of the gradient through evaluation of the directional derivative. All the ideas presented are illustrated by non-trivial examples, namely, considering the subspace of 3 × 3 circulant and Toeplitz matrices and presenting the results of gradient-descent with both the spurious and correct gradients. Moreover, our bibliography makes it clear that a rigorous approach to matrix calculus is not common in practice, and our presentation of matrix calculus in the language of inner-product spaces will be significant and meaningful for applied mathematicians, engineers and researchers working in inter-disciplinary fields to avoid the conceptual pitfalls that exist.
97 MATHEMATICS AND COMPUTING↗
Improved Evaluation of Large Network Matrices for Linear Power Flow Within Optimization Problems: Preprint
This work discusses methods for evaluating the Power Transfer Distribution Factor (PTDF) and Line Outage Distribution Factor (LODF) matrices by employing sparse linear algebra for large-scale computing applications. These matrices are critical in many power systems applications, such as the Unit Commitment Problem (UC), pre- and post-contingency power flow analysis, and transmission expansion. These matrices are typically dense, which means they require a significant amount of time and memory to be computed for large networks. However, by analyzing the structure of the matrices and their computation method, it is possible to use reduced memory methods based on sparse matrix operations. This paper shows that sparse linear algebra algorithms are faster and require less memory and time than traditional dense approaches. Additionally, we explore the effect of matrix sparsification by eliminating trailing digits on power flow calculations.
ENERGY PLANNING, POLICY, AND ECONOMY↗
Controlling matric potential in microfluidics to examine microbial dynamics in unsaturated porous media
The use of microfluidics for the study of soil microbial ecology is an emerging field. Most microfluidic studies of biological systems, however, have been performed under fully saturated conditions that are not representative of natural soil. Therefore, while microfluidics offer many unique capabilities that other methodologies cannot, they are not currently suited to address the effects of matric potential, an important variable defining the microbial moisture niche. Here, a methodology is presented that allows the user to control the aqueous conditions within microfluidic networks by manipulating matric potential using a hanging water column. The method relies on hydrophilic surface treatment of the microfluidic device using polyvinyl alcohol (PVA) and incorporating a bed of small pores at the network boundaries, which serve as a porous ceramic plate analogue (PPA). The method was validated on a simple capillary bundle and then on a more complex pore network. A water retention curve, exhibiting hysteresis, was generated for the pore network over a narrow matric potential range of 0 to – 5 kPa. Both the drainage and wetting curves were reproducible, as were the spatial configuration and the number of fragmented moisture niches in the pore network, particularly on the drainage curve. In contrast, the wetting curve exhibited greater variability in spatial configuration due to the “ink bottle effect,” where capillarity was interrupted by wider pore bodies. Ultimately, the methodology provides realistic pore-scale moisture conditions that can be easily manipulated and maintained, enabling new opportunities to explore soil biophysics and microbial biogeography in unsaturated porous media. As a brief example, images showing the localization of fluorescently tagged Pantoea sp. YR343 at −4.3 kPa are presented, highlighting bacterial distributions in water films and air-water interfaces.
59 BASIC BIOLOGICAL SCIENCES↗
Universal Polarization Transformations: Spatial Programming of Polarization Scattering Matrices Using a Deep Learning‐Designed Diffractive Polarization Transformer
Abstract Controlled synthesis of optical fields having nonuniform polarization distributions presents a challenging task. Here, a universal polarization transformer is demonstrated that can synthesize a large set of arbitrarily‐selected, complex‐valued polarization scattering matrices between the polarization states at different positions within its input and output field‐of‐views (FOVs). This framework comprises 2D arrays of linear polarizers positioned between isotropic diffractive layers, each containing tens of thousands of diffractive features with optimizable transmission coefficients. After its deep learning‐based training, this diffractive polarization transformer can successfully implement N i N o = 10 000 different spatially‐encoded polarization scattering matrices with negligible error, where N i and N o represent the number of pixels in the input and output FOVs, respectively. This universal polarization transformation framework is experimentally validated in the terahertz spectrum by fabricating wire‐grid polarizers and integrating them with 3D‐printed diffractive layers to form a physical polarization transformer. Through this set‐up, an all‐optical polarization permutation operation of spatially‐varying polarization fields is demonstrated, and distinct spatially‐encoded polarization scattering matrices are simultaneously implemented between the input and output FOVs of a compact diffractive processor. This framework opens up new avenues for developing novel devices for universal polarization control and may find applications in, e.g., remote sensing, medical imaging, security, material inspection, and machine vision.
Optical neural networks↗
Reduced-order model to approximate response matrices for filter stack spectrometers
We present a reduced-order model to calculate response matrices rapidly for filter stack spectrometers (FSSs). The reduced-order model allows response matrices to be built modularly from a set of pre-computed photon and electron transport and scattering calculations through various filter and detector materials. While these modular response matrices are not appropriate for high-fidelity analysis of experimental data, they encode sufficient physics to be used as a forward model in design optimization studies of FSSs, particularly for machine learning approaches that require sampling and testing a large number of FSS designs.
46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
Quantum block encoding for one-pair semiseparable matrices
Quantum block encoding (QBE) is a crucial step in the development of most quantum algorithms, as it provides an embedding of a given matrix into a suitable larger unitary matrix. Historically, the development of efficient techniques for QBE has mostly focused on sparse matrices; less effort has been devoted to data-sparse (e.g., rank-structured) matrices. In this work we examine a particular case of rank structure, namely, one-pair semiseparable matrices. We present a new block encoding approach that relies on a suitable factorization of the given matrix as the product of triangular and diagonal factors. To encode the matrix, the algorithm needs $2\log(N)+7$ ancillary qubits. Assuming that the data input oracles can be implemented with polylogarithmic depth, or that a QRAM input model is available, our proposed method requires $\mathcal{O}({\rm polylog} (N))$ time and has an error of $\mathcal{O}(N^2)$, where $N$ is the matrix size.
Antonioli, Giacomo [Pisa U.; CERN] (ORCID:00090000↗
Carbamoylation as an Effective Tool in the Analysis of the Soman Nerve Agent Marker Pinacolyl Alcohol in Soil Matrices by EI-GC-MS and LC-HRMS
Pinacolyl alcohol (PA) is a Schedule 2 chemical commonly featured in most proficiency tests (PTs) administered by the Organisation for the Prohibition of Chemical Weapons (OPCW) due to its direct as well as forensic link to the nerve agent Soman. Therefore, its detection by Chemical Weapons Convention (CWC) inspection teams during on-site investigations is a strong indicator of the past or latent presence of Soman in the environment. Its small molecular weight (102), early elution time, and poor ionization profile make PA a challenging analyte to detect particularly at low concentrations (∼1–10 μg/g). In this work, 1,1′-carbonyldimidazole (CDI) has been used to effectively modify PA for the first time in two different soil matrices (Virginia type A soil and silt sediment) at two separate concentrations (1 and 10 μg/g) for its subsequent detection by EI-GC-MS and LC-HRMS methods. For the EI-GC-MS analysis, the PA carbamate derivative (PIC) exhibits improved chromatography relative to PA such as improved peak shape, increased molecular weight (196 for PIC and 102 for PA) and increased retention time (16.9 min for PIC and ∼4.1 min for PA). In addition, the derivatization also improves the detection of PA by LC-HRMS as the PIC product possesses protonation sites (i.e., imidazole ring) relative to none exhibited by PA. More importantly, the carbamoylation proceeds under mild conditions (55 °C, no base) and rapidly (3 h), characteristics that make it an appealing protocol for the analysis of PA during OPCW PTs or real case scenarios particularly in instances where it is present at low concentrations. It is anticipated that the protocol can be applied to the forensic analysis of this important Soman marker in various environmental matrices.
Chemistry↗
Cosmic topology. Part IIa. Eigenmodes, correlation matrices, and detectability of orientable Euclidean manifolds
Abstract If the Universe has non-trivial spatial topology, observables depend on both the parameters of the spatial manifold and the position and orientation of the observer. In infinite Euclidean space, most cosmological observables arise from the amplitudes of Fourier modes of primordial scalar curvature perturbations. Topological boundary conditions replace the full set of Fourier modes with specific linear combinations of selected Fourier modes as the eigenmodes of the scalar Laplacian. We present formulas for eigenmodes in orientable Euclidean manifolds with the topologiesE 1 –E 6 ,E 11 ,E 12 ,E 16 , andE 18 that encompass the full range of manifold parameters and observer positions, generalizing previous treatments. Under the assumption that the amplitudes of primordial scalar curvature eigenmodes are independent random variables, for each topology we obtain the correlation matrices of Fourier-mode amplitudes (of scalar fields linearly related to the scalar curvature) and the correlation matrices of spherical-harmonic coefficients of such fields sampled on a sphere, such as the temperature of the cosmic microwave background (CMB). We evaluate the detectability of these correlations given the cosmic variance of the observed CMB sky. We find that topologies where the distance to our nearest clone is less than about 1.2 times the diameter of the last scattering surface of the CMB give a correlation signal that is larger than cosmic variance noise in the CMB. This implies that if cosmic topology is the explanation of large-angle anomalies in the CMB, then the distance to our nearest clone is not much larger than the diameter of the last scattering surface. We argue that the topological information is likely to be better preserved in three-dimensional data, such as will eventually be available from large-scale structure surveys.
Astronomy & Astrophysics↗
Semi-analytical covariance matrices for two-point correlation function for DESI 2024 data
We present an optimized way of producing the fast semi-analytical covariance matrices for the Legendre moments of the two-point correlation function, taking into account survey geometry and mimicking the non-Gaussian effects. We validate the approach on simulated (mock) catalogs for different galaxy types, representative of the Dark Energy Spectroscopic Instrument (DESI) Data Release 1, used in 2024 analyses. We find only a few percent differences between the mock sample covariance matrix and our results, which can be expected given the approximate nature of the mocks, although we do identify discrepancies between the shot-noise properties of the DESI fiber assignment algorithm and the faster approximation (emulator) used in the mocks. Importantly, we find a close agreement (≤ 8% relative differences) in the projected errorbars for distance scale parameters for the baryon acoustic oscillation measurements. This confirms our method as an attractive alternative to simulation-based covariance matrices, especially for non-standard models or galaxy sample selections, making it particularly relevant to the broad current and future analyses of DESI data.
79 ASTRONOMY AND ASTROPHYSICS↗
Generalized Analytical Estimation of Sensitivity Matrices in Unbalanced Distribution Networks
Fast and accurate estimation of sensitivity matrices is significant for the enhancement of distribution system modeling and automation. Analytical estimations have mainly focused on voltage magnitude sensitivity to active/reactive power injections for unbalanced networks with Wye-connected loads and neglecting DERs' smart inverter functionality. Hence, this paper enhances the scope of analytical estimation of sensitivity matrices for unbalanced networks with 1- Φ, 2- Φ, and 3- Φ Delta/Wye-connected loads, DERs with smart inverter functionality, and substation/line step-voltage regulators (SVR). A composite bus model comprising of DER, Delta- and Wye-connected load is proposed to represent a generic distribution bus, which can be simplified to load, PV, or voltage-controlled bus as required. Furthermore, the proposed matrix-based analytical method consolidates voltage magnitude and angle sensitivity to active/reactive power injection and tap-position of all SVRs into a single algorithm. Extensive case studies on IEEE and EPRI networks show the accuracy and wide scope of the proposed algorithm compared to the existing benchmark method.
24 POWER TRANSMISSION AND DISTRIBUTION↗
Explicit Quantum Circuits for Block Encodings of Certain Sparse Matrices
Many standard linear algebra problems can be solved on a quantum computer by using recently developed quantum linear algebra algorithms that make use of block encodings and quantum eigenvalue/singular value transformations. A block encoding embeds a properly scaled matrix of interest A in a larger unitary transformation U that can be decomposed into a product of simpler unitaries and implemented efficiently on a quantum computer. Although quantum algorithms can potentially achieve exponential speedup in solving linear algebra problems compared to the best classical algorithm, such a gain in efficiency ultimately hinges on our ability to construct an efficient quantum circuit for the block encoding of A, which is difficult in general, and not trivial even for well structured sparse matrices. Here, in this paper, we give a few examples on how efficient quantum circuits can be explicitly constructed for some well structured sparse matrices and discuss a few strategies used in these constructions. We also provide implementations of these quantum circuits in MATLAB.
97 MATHEMATICS AND COMPUTING↗