Search NASA⌕ Search

SEARCH · Search NASA

Results for “tensor decomposition”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Near-Efficient and Non-Asymptotic Multiway Inference

We establish non-asymptotic efficiency guarantees for tensor decomposition–based inference in count data models. Under a Poisson framework, we consider two related goals: (i) parametric inference , the estimation of the full distributional parameter tensor, and (ii) multiway analysis , the recovery of its canonical polyadic (CP) decomposition factors. Our main result shows that in the rank-one setting, a rank-constrained maximum-likelihood estimator achieves multiway analysis with variance matching the Cramér–Rao Lower Bound (CRLB) up to absolute constants and logarithmic factors. This provides a general framework for studying “near-efficient” multiway estimators in finite-sample settings. For higher ranks, we illustrate that our multiway estimator may not attain the CRLB; nevertheless, CP-based parametric inference remains nearly minimax optimal, with error bounds that improve on prior work by offering more favorable dependence on the CP rank. Numerical experiments corroborate near-efficiency in the rank-one case and highlight the efficiency gap in higher-rank scenarios.

97 MATHEMATICS AND COMPUTING↗

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗

pyDRESCALk

Modern data scientists are tasked to analyze ever-growing data sets with increasingly complex relationships. Tensor decompositions have come to play a central role in identifying underlying latent structures in higher-order data. The problem of fitting tensor models to different distributions is complicated by the combinations of size, dimensionality, and sparsity present in real world data. The situation demands efficient algorithms designed for shared-memory and distributed systems. This work will present new research that tackles these challenges on several different fronts, leveraging optimizations in numerical algorithms and sparse tensor representations in heterogeneous high performance computing environments.

Bhattarai, Manish↗

Bayesian tensorized neural networks with automatic rank selection

Tensor decomposition is an effective approach to compress over-parameterized neural networks and to enable their deployment on resource-constrained hardware platforms. However, directly applying tensor compression in the training process is a challenging task due to the difficulty of choosing a proper tensor rank. In order to address this challenge, this paper proposes a low-rank Bayesian tensorized neural network. Our Bayesian method performs automatic model compression via an adaptive tensor rank determination. We also present approaches for posterior density calculation and maximum a posteriori (MAP) estimation for the end-to-end training of our tensorized neural network. Here, we provide experimental validation on a two-layer fully connected neural network, a 6-layer CNN and a 110-layer residual neural network where our work produces 7.4x to 137x more compact neural networks directly from the training while achieving high prediction accuracy.

Low-rank tensor↗

Tensor Text-Mining Methods for Malware Identification and Detection, Malware Dynamics Characterization, and Hosts Ranking

Malware is one of the most persistent and costly cyber threats endangering reputation, confidentiality, integrity, and availability for organizations and national security. Consequently, many of the incident detection and prevention systems, and incident responders have begun to utilize machine learning as a helper in the fight against malware and other cyber threats. However, cyber defenders rely on interpretability and generalizability, yet the popular machine learning methods are black-box and often use traditional supervised solutions that do not generalize to novel malware. Therefore, there is a need to improve the existing solutions. At the same time, the majority of the prior research ignored essential evaluation criteria when reporting the results of their methods, which disables the safe reproducibility of the methods in a production environment. Tensor decomposition, on the other hand, enables interpretable unsupervised analysis of the large-scale data for the discovery of hidden patterns. Our findings, performed on real-world and large-scale experiments, show that tensor factorization-based methods yield performance results that surpasses or competes with existing supervised solutions with the added benefit of interpretability and generalizability. With the ability to analyze complex and large-scale data using tensors, we report results that reflect real-world production environments. We propose to develop new game- changing tools for malware identification and characterization that can trace malware evolution, rank the infected or malicious hosts, and streamline the work of incident response teams, malware analysts, and incident detection and prevention systems.

97 MATHEMATICS AND COMPUTING↗

Minkowski and Galilei/Newton Fluid Dynamics: A Geometric 3 + 1 Spacetime Perspective

A kinetic theory of classical particles serves as a unified basis for developing a geometric 3 + 1 spacetime perspective on fluid dynamics capable of embracing both Minkowski and Galilei/Newton spacetimes. Parallel treatment of these cases on as common a footing as possible reveals that the particle four-momentum is better regarded as comprising momentum and inertia rather than momentum and energy; and, consequently, that the object now known as the stress-energy or energy-momentum tensor is more properly understood as a stress-inertia or inertia-momentum tensor. In dealing with both fiducial and comoving frames as fluid dynamics requires, tensor decompositions in terms of the four-velocities of observers associated with these frames render use of coordinate-free geometric notation not only fully viable, but conceptually simplifying. A particle number four-vector, three-momentum ( 1 , 1 ) tensor, and kinetic energy four-vector characterize a simple fluid and satisfy balance equations involving spacetime divergences on both Minkowski and Galilei/Newton spacetimes. Reduced to a fully 3 + 1 form, these equations yield the familiar conservative formulations of special relativistic and non-relativistic fluid dynamics as partial differential equations in inertial coordinates, and in geometric form will provide a useful conceptual bridge to arbitrary-Lagrange–Euler and general relativistic formulations.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Reduced scaling Hilbert space variational Monte Carlo

We show that for both single-Slater-Jastrow and Jastrow geminal power wave functions the formal cost scaling of Hilbert space variational Monte Carlo can be reduced from fifth to fourth order in the system size, thus bringing it in line with the long-standing scaling of its real space counterpart. While traditional quantum chemistry methods can reduce costs related to the two-electron integral tensor through various tensor decomposition methods, we show that such approaches are ineffective in the presence of Hilbert space Jastrow factors. Instead, we develop a simple semi-stochastic approach that can take similar advantage of the near-sparsity of this four-index tensor. Through demonstrations on alkanes of increasing length, we show that accuracy and overall statistical uncertainty are not meaningfully affected and that a total cost crossover is reached as early as 50 electrons when using a minimal basis. Further study will be needed to assess where the crossover occurs in more compact molecular geometries and larger basis sets and to explore how in that context the crossover can be accelerated.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

ExaTN

A challenging scientific problem that will benefit from exascale computing is predictive simulation of two- and three-dimensional quantum many-body Hamiltonians in condensed matter physics, together with the related strong entanglement problems in quantum chemistry and quantum computing. Efficient numerical simulations will require new insights into wavefunction compression algorithms that scale on extremely heterogeneous HPC architectures. To address this need, we propose to develop a scalable math library of parallel multi-linear algebraic numerical primitives for hierarchical tensor compression. Our library will subsequently be used for building an efficient, performance-portable HPC framework for simulating strongly correlated quantum systems. In particular, our will will be the first to deliver a massively parallel implementation of the multiscale entanglement renormalization ansatz (MERA), a hierarchical tensor decomposition quantum scheme capable of expressing local expectation values of strongly entangled higher-dimensional quantum systems efficiently. To address the extreme complexity and high computational cost of such schemes, we will automate their parallel implementation by delivering an integrated software framework, ExaTN (Exascale Tensor Networks), that will process hierarchical tensor representations on exascale HPC systems via an asymchronous task-based programming model. As a specific application, our framework will provide scientists with a capability of simulating strongly entangled quantum systems, thus positioning ORNL as a strategic leader in modeling of strongly correlated materials.

McCaskey, AlexanderJ↗

GPU Accelerated Event Detection Algorithm

Smart grid external require new algorithmic approaches as well as parallel formulations. One of the critical components is the prediction of changes and detection of anomalies within the power grid. The state-of-the-art algorithms are not suited to handle the demands of streaming data analysis. (i) need for events detection algorithms that can scale with the size of data, (ii) need for algorithms that can not only handle multi dimensional nature of the data, but also model both spatial and temporal dependencies in the data, which, for the most part, are highly nonlinear, (iii) need for algorithms that can operate in an online fashion with streaming data. The GAEDA code is a new online anomaly detection techniques that take into account spatial, temporal, multi-dimensional aspects of the data set. The basic idea behind the proposed approach is to (a) to convert a multi-dimensional sequence into a univariate time series that captures the changes between successive windows extracted from the original sequence using singular value decomposition (SVD), and then (b) to apply known anomaly detection techniques for univariate time series. A key challenge for the proposed approach is to make the algorithm scalable to huge datasets by adopting techniques from perturbation theory, incremental SVD analysis. We used recent advances in tensor decomposition techniques which reduce computational complexity to monitor the change between successive windows and detect anomalies in the same manner as described above. Therefore we propose to develop the parallel solutions on many core systems such as GPUs, because these algorithms involve lot of numerical operations and are highly data-parallelizable.

Source record↗

Space–Time Least-Squares Petrov–Galerkin Projection for Nonlinear Model Reduction

This work proposes a space--time least-squares Petrov--Galerkin (ST-LSPG) projection method for model reduction of nonlinear dynamical systems. In contrast to typical nonlinear model-reduction methods that first apply (Petrov--)Galerkin projection in the spatial dimension and subsequently apply time integration to numerically resolve the resulting low-dimensional dynamical system, the proposed method applies projection in space and time simultaneously. To accomplish this, the method first introduces a low-dimensional space--time trial subspace, which can be obtained by computing tensor decompositions of state-snapshot data. The method then computes discrete-optimal approximations in this space--time trial subspace by minimizing the residual arising after time discretization over all space and time in a weighted $\ell^2$-norm. This norm can be defined to enable complexity reduction (i.e., hyper-reduction) in time, which leads to space--time collocation and space--time Gauss--Newton with Approximated Tensors (GNAT) variants of the ST-LSPG method. Advantages of the approach relative to typical spatial-projection-based nonlinear model reduction methods such as Galerkin projection and least-squares Petrov--Galerkin projection include a reduction of both the spatial and temporal dimensions of the dynamical system, and a priori error bounds that bound the solution error by the best space--time approximation error and whose stability constants exhibit slower growth in time. Numerical examples performed on model problems in fluid dynamics demonstrate the ability of the method to generate orders-of-magnitude computational savings relative to spatial-projection-based reduced-order models without sacrificing accuracy for a fixed spatio-temporal discretization.

97 MATHEMATICS AND COMPUTING↗

Communication Lower Bounds and Optimal Algorithms for Multiple Tensor-Times-Matrix Computation

Multiple tensor-times-matrix (Multi-TTM) is a key computation in algorithms for computing and operating with the Tucker tensor decomposition, which is frequently used in multidimensional data analysis. Here, we establish communication lower bounds that determine how much data movement is required (under mild conditions) to perform the Multi-TTM computation in parallel. The crux of the proof relies on analytically solving a constrained, nonlinear optimization problem. We also present a parallel algorithm to perform this computation that organizes the processors into a logical grid with twice as many modes as the input tensor. We show that, with correct choices of grid dimensions, the communication cost of the algorithm attains the lower bounds and is therefore communication optimal. Finally, we show that our algorithm can significantly reduce communication compared to the straightforward approach of expressing the computation as a sequence of tensor-times-matrix operations when the input and output tensors vary greatly in size.

HBL-inequalities↗

Homogeneous binary trees as ground states of quantum critical Hamiltonians

Many-body states whose wave functions admit a representation in terms of a uniform binary-tree tensor decomposition are shown to obey power-law two-body correlation functions. Any such state can be associated with the ground state of a translationally invariant Hamiltonian which, depending on the dimension of the systems sites, involves at most couplings between third-neighboring sites. Under general conditions it is shown that they describe unfrustrated systems which admit an exponentially large degeneracy of the ground state.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Exploration of Electrochemical Reactions at Organic–Inorganic Halide Perovskite Interfaces via Machine Learning in In Situ Time-of-Flight Secondary Ion Mass Spectrometry

The instability of hybrid organic–inorganic perovskite (HOIP) devices is one of the significant challenges preventing commercialization. Exploring these phenomena is severely limited by the complexity of the intrinsic electrochemistry of HOIPs, the presence of multiple volatile and mobile ionic species, and the possible role of environmentally induced reactions at surfaces and triple-phase junctions. Here, in situ studies of the electrochemistry of methylammonium lead bromide perovskite with the Au electrode interface are reported via light- and voltage-dependent time-of-flight secondary ion mass spectrometry (ToF-SIMS) imaging of lateral perovskite heterostructures. While ToF-SIMS allows for the visualization of the chemical composition along the surface and its evolution with light and electrical bias, the interpretation of the multidimensional data obtained is often limited due to strong correlations between chemical signatures and the need to track multiple peaks at once. Here, a machine learning workflow combining the Hough transform and non-negative matrix factorization and no-negative tensor decomposition is developed to avoid this limitation and extract salient features of associated chemical changes and to separate the light- and voltage-dependent dynamics. Finally, combining these in situ characterizations and the machine learning workflow provides comprehensive information on the chemical nature of moving species, ion accumulation, and interfacial electrochemical reactions in HOIP devices.

36 MATERIALS SCIENCE↗

Stress Dependency of Brittle Creep in Granite: Insights into Source Mechanisms and Parameters

Creep in rocks refers to the gradual deformation of rock material over time under the influence of constant stress. Characterizing these deformations is of great importance for engineering design, geotechnical assessment, mining operations, geological studies, and understanding natural hazards. While laboratory experiments and a variety of numerical approaches have offered explanations for microcrack interaction and damage accumulation under the three stages of creep (primary, secondary and tertiary) in conventional creep experiments, the micromechanisms of the fractures produced in brittle creep and its dependency on the applied stress have not been explored in detail. The present study focused on investigating the fracturing mechanisms that occur during creep-induced fracturing at different stress levels and estimation of the source parameters and energy budget components. A series of uniaxial compression creep experiments have been conducted at different stress level ratios (70%, 75%, 80% and 85%), to the unconfined compressive strength (UCS) of double-flawed Barre granite specimen. Creep measurements were complemented with the Acoustic Emission (AE) measurements. The creep-induced fractures were classified into double-couple (DC), compensated linear vector dipole (CLVD) and isotropic (ISO) components using the AE moment tensor decomposition technique. The results show that non-double-couple sources dominated during creep at all the specified stress levels; however, their proportions decreased as the stress level was increased. The source parameters estimation indicated a significant increase in the magnitude of the events and the radiated seismic energy with increasing levels of stress and a slight increase in the evaluated source radius and stress drop with increasing stress levels. Furthermore, this study contributes to the existing knowledge of creep-induced fracturing by providing insights into the fracturing mechanisms and the radiated seismic energy produced, which can be helpful for the development of improved models and strategies for rock engineering and geoscience applications.

58 GEOSCIENCES↗

Gradient flow based phase-field modeling using separable neural networks

Allen–Cahn equation is a reaction–diffusion equation and is widely used for modeling phase separation. Machine learning methods for solving the Allen–Cahn equation in its strong form suffer from inaccuracies in collocation techniques, errors in computing higher-order spatial derivatives, and the large system size required by the space–time approach. To overcome these challenges, we propose solving the gradient flow of the Ginzburg–Landau free energy functional, which is equivalent to the Allen–Cahn equation, thereby avoiding the second-order spatial derivatives associated with the Allen–Cahn equation. A minimizing movement scheme is employed to solve the gradient flow problem, eliminating the complexities of a space–time approach. We utilize a separable neural network that efficiently represents the phase field through low-rank tensor decomposition. As we use the minimizing movement scheme to numerically solve the gradient flow problem, we thus, refer to the proposed method as the Separable Deep Minimizing Movement (SDMM) method. The evaluation of the functional in the minimizing movement scheme using the Gauss quadrature technique bypasses the inaccuracies associated with collocation techniques traditionally used to solve partial differential equations. A hyperbolic tangent transformation is introduced on the phase field prior to the evaluation of the functional to ensure that it remains strictly bounded within the values of the two phases. For this transformation, theoretical guarantee for energy stability of the minimizing movement scheme is established. Our results suggest that this transformation helps to improve the accuracy and efficiency significantly. The proposed method resolves the challenges faced by state-of-the-art machine learning techniques, outperforming them in both accuracy and efficiency. It is also the first machine learning method to achieve an order of magnitude speed improvement over the finite element method. In addition to its formulation and computational implementation, several case studies illustrate the applicability of the proposed method.

42 ENGINEERING↗

Perturbations of nested branes with induced gravity

We study the behaviour of weak gravitational fields in models where a 4D brane is embedded inside a 5D brane equipped with induced gravity, which in turn is embedded in a 6D spacetime. We consider a specific regularization of the branes internal structures where the 5D brane can be considered thin with respect to the 4D one. We find exact solutions corresponding to pure tension source configurations on the thick 4D brane, and study perturbations at first order around these background solutions. To perform the perturbative analysis, we adopt a bulk-based approach and we express the equations in terms of gauge invariant and master variables using a 4D scalar-vector-tensor decomposition. We then propose an ansatz on the behaviour of the perturbation fields when the thickness of the 4D brane goes to zero, which corresponds to configurations where gravity remains finite everywhere in the thin limit of the 4D brane. We study the equations of motion using this ansatz, and show that they give rise to a consistent set of differential equations in the thin limit, from which the details of the internal structure of the 4D brane disappear. We conclude that the thin limit of the ''ribbon'' 4D brane inside the (already thin) 5D brane is well defined (at least when considering first order perturbations around pure tension configurations), and that the gravitational field on the 4D brane remains finite in the thin limit. We comment on the crucial role of the induced gravity term on the 5D brane.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

The critical tension in the Cascading DGP model

We study the behaviour of weak gravitational fields in the 6D Cascading DGP model using a bulk-based approach. To deal with the ambiguity in the thin limit of branes of codimension higher than one, we consider a specific regularization of the internal structure of the branes where the 5D brane can be considered thin with respect to the 4D one. We consider the solutions corresponding to pure tension sources on the 4D brane, and study perturbations at first order around these background solutions. We adopt a 4D scalar-vector-tensor decomposition, and focus on the scalar sector of perturbations. We show that, in a suitable 4D limit, the trace part of the 4D metric perturbations obeys a decoupled equation which suggests that it is a ghost for background tensions smaller than a critical tension, while it is a healthy field otherwise. We give a geometrical interpretation of the existence of the critical tension and of the reason why the relevant field is a ghost or not depending on the background tension. We however find a value of the critical tension which is different from the one already found in the literature. Differently from the results in the literature, our analysis implies that, choosing the background tension suitably, we can construct ghost-free models for any value of the free parameters of the theory. We suggest that the difference lies in the procedure used to evaluate the pillbox integration across the codimension-2 brane. We confirm the validity of our analysis by performing numerically the integration in a particular case where the solution inside the thick cod-2 brane is known exactly. We stress that the singular structure of the perturbation fields in the nested branes set-ups is very subtle, and that great care has to be taken when deriving the codimension-2 junction conditions.

79 ASTRONOMY AND ASTROPHYSICS↗