Search NASA⌕ Search

SEARCH · Search NASA

Results for “Tensor train”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

A tensor train-based isogeometric solver for large-scale 3D poisson problems

We introduce a three-dimensional (3D), fully tensor train (TT) assembled isogeometric analysis (IGA) framework, TT-IGA, for solving partial differential equations (PDEs). Our method reformulates IGA discrete operators into TT format, enabling efficient compression and computation. Geometry evaluations use the original NURBS description at sampling points and TT approximation is applied to geometry-derived coefficient fields and discrete operators. We demonstrate the effectiveness of the proposed TT-IGA framework on the three-dimensional Poisson equation, achieving substantial reductions in memory and computational cost without compromising solution quality.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Space-Time Finite Element Tensor Network Approach for the Time-Dependent Convection–Diffusion–Reaction Equation with Variable Coefficients

In this paper, we present a new space-time Galerkin-like method, where we treat the discretization of spatial and temporal domains simultaneously. This method utilizes a mixed formulation of the tensor-train (TT) and quantized tensor-train (QTT) (please see Section Tensor-Train Decomposition), designed for the finite element discretization (Q1-FEM) of the time-dependent convection–diffusion–reaction (CDR) equation. We reformulate the assembly process of the finite element discretized CDR to enhance its compatibility with tensor operations and introduce a low-rank tensor structure for the finite element operators. Recognizing the banded structure inherent in the finite element framework’s discrete operators, we further exploit the QTT format of the CDR to achieve greater speed and compression. Additionally, we present a comprehensive approach for integrating variable coefficients of CDR into the global discrete operators within the TT/QTT framework. The effectiveness of the proposed method, in terms of memory efficiency and computational complexity, is demonstrated through a series of numerical experiments, including a semi-linear example.

convection–diffusion–reaction equation↗
DOE OSTI2024

Georgia Tech Accelerated, Compressed, and Regularized Compute of Kinetic-based PDEs (Final Report)

This report summarizes the collaborative effort between Lawrence Livermore National Laboratory and Georgia Tech to enhance the BoBa library for tensor train computation in PDE solvers, with a target on kinetic equations and their continuum limits. We aimed to reduce computational cost and memory usage by replacing traditional array-based computations with tensor trains. We examined the compressibility of time-evolving solutions to the Euler equations with discontinuities. We also explored using the first invsicid and linear regularization of the compressible flow equations via the information geometric regularization (IGR). We explored this in a tensor train formulation. To identify that inverse terms in the IGR equations pose problems for tensor train formulations and investigate efficient methods for batched inversion of tensor trains.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Collocation methods for nonlinear differential equations on low-rank manifolds

We introduce new methods for integrating nonlinear differential equations on low-rank manifolds. These methods rely on interpolatory projections onto the tangent space, enabling low-rank time integration of vector fields that can be evaluated entry-wise. A key advantage of our approach is that it does not require the vector field to exhibit low-rank structure, thereby overcoming significant limitations of traditional dynamical low-rank methods based on orthogonal projection. To construct the interpolatory projectors, we develop a sparse tensor sampling algorithm based on the discrete empirical interpolation method (DEIM) that parameterizes tensor train manifolds and their tangent spaces with cross interpolation. Using these projectors, we propose two time integration schemes on low-rank tensor train manifolds. The first scheme integrates the solution at selected interpolation indices and constructs the solution with cross interpolation. The second scheme generalizes the well-known orthogonal projector-splitting integrator to interpolatory projectors. We demonstrate the proposed methods with applications to several tensor differential equations arising from the discretization of partial differential equations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗
DOE OSTI2023

Neutron (and other Particle) Transport at LANL: An Overview [Presentation]

For decades, Los Alamos National Laboratory has been at the forefront of neutron transport methods research and code development. One such code is PARTISN, the LANL parallel time-dependent discrete ordinate neutron transport code. In this presentation, we describe the various research efforts currently underway by the PARTISN and other code teams. Some examples of current research are a block automated mesh refinement scheme, the application of tensor trains to the discretized neutron transport equation, and GPU code porting. The block automated mesh refinement scheme uses cross section information to refine and coarsen the solution mesh to improve time to solution and reduce memory. The tensor train approach expresses discretized transport operators as tensor products of vectors and matrices to compress the size of linear systems being solved by transport codes. Rather than relying on matrix-free methods such as the transport sweep, we have access to an operator that can be inverted, reshaped, or manipulated algebraically. Finally, we describe how PARTISN is used, what problems we are looking to solve, and what the future holds for neutron transport at LANL. In addition to this, we briefly describe the various research efforts in other particle transport teams using both deterministic and Monte Carlo methods. In the presentation, we list possible opportunities for collaboration between the laboratory and faculty and students.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2019

A sequential DNN based Baseline Energy Prediction Framework with Long term Error Mitigation

In this paper,we aim to present a novel sequential framework containing two deep network architectures for baseline energy prediction, applied to a simulated commercial building dataset. We have observed that this sequential framework helps to improve long term prediction accuracy, there by mitigating prediction error accumulation over time. The proposed framework utilizes convolution layers to extract features from the input data space without changing spatial relations between variables. These features are memorized by tensor train based gated recurrent units for an accurate long-term prediction. Although, architecture comprising the amalgamation of convolution layers and memory cell have shown promising results in domains such as social media analysis, language modeling, video frame prediction and image recognition, this paper extends its scope to the context of energy applications. Furthermore, the addition of tensor train based gated recurrent units is motivated by the necessity of computational time reduction during the training process, there by making this framework more suitable for future in-field deployment. Lastly, results show the current framework outperforms other existing machine learning based methods, in both short-term and long-term prediction categories.

Chakraborty, Indrasis↗
DOE OSTI2024

A Low-Rank QTT-based Finite Element Method for Elasticity Problems

We present an efficient and robust numerical algorithm for solving the linear elasticity problem that combines the Quantized Tensor Train format and a domain partitioning strategy. This approach makes it possible to solve the linear elasticity problem on a computational domain that is more general than a square. By integrating Z-ordering and subdomain concatenation, our method substantially decreases memory usage and achieves a notable reduction in rank compared to established Finite Element implementations like the FEniCS platform. This efficiency is maintained while still guaranteeing exponential convergence with respect to the number of degrees of freedom. This performance gain, however, requires a fundamental rethinking of how core finite element operations are implemented. This includes changes to mesh discretization, node and degree of freedom ordering, stiffness matrix and internal nodal force assembly, and the execution of algebraic matrix-vector operations. In this work, we discuss all these aspects in detail and assess the method’s performance in the numerical approximation of three representative test cases.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Quantum annealing algorithms for Boolean tensor networks

Abstract Quantum annealers manufactured by D-Wave Systems, Inc., are computational devices capable of finding high-quality heuristic solutions of NP-hard problems. In this contribution, we explore the potential and effectiveness of such quantum annealers for computing Boolean tensor networks. Tensors offer a natural way to model high-dimensional data commonplace in many scientific fields, and representing a binary tensor as a Boolean tensor network is the task of expressing a tensor containing categorical (i.e., $$\{0, 1\}$$ { 0 , 1 } ) values as a product of low dimensional binary tensors. A Boolean tensor network is computed by Boolean tensor decomposition, and it is usually not exact. The aim of such decomposition is to minimize the given distance measure between the high-dimensional input tensor and the product of lower-dimensional (usually three-dimensional) tensors and matrices representing the tensor network. In this paper, we introduce and analyze three general algorithms for Boolean tensor networks: Tucker, Tensor Train, and Hierarchical Tucker networks. The computation of a Boolean tensor network is reduced to a sequence of Boolean matrix factorizations, which we show can be expressed as a quadratic unconstrained binary optimization problem suitable for solving on a quantum annealer. By using a novel method we introduce called parallel quantum annealing, we demonstrate that Boolean tensor’s with up to millions of elements can be decomposed efficiently using a DWave 2000Q quantum annealer.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Direct interpolative construction of the discrete Fourier transform as a matrix product operator

The quantum Fourier transform (QFT), which can be viewed as a reindexing of the discrete Fourier transform (DFT), has been shown to be compressible as a low-rank matrix product operator (MPO) or quantized tensor train (QTT) operator. However, the original proof of this fact does not furnish a construction of the MPO with a guaranteed error bound. Meanwhile, the existing practical construction of this MPO, based on the compression of a quantum circuit, is not as efficient as possible. We present a simple closed-form construction of the QFT MPO using the interpolative decomposition, with guaranteed near-optimal compression error for a given rank. This construction can speed up the application of the QFT and the DFT, respectively, in quantum circuit simulations and QTT applications. We also connect our interpolative construction to the approximate quantum Fourier transform (AQFT) by demonstrating that the AQFT can be viewed as an MPO constructed using a different interpolation scheme.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Tensor Network Space-Time Spectral Collocation Method for Time-Dependent Convection-Diffusion-Reaction Equations

Emerging tensor network techniques for solutions of partial differential equations (PDEs), known for their ability to break the curse of dimensionality, deliver new mathematical methods for ultra-fast numerical solutions of high-dimensional problems. Here, we introduce a Tensor Train (TT) Chebyshev spectral collocation method, in both space and time, for the solution of the time-dependent convection-diffusion-reaction (CDR) equation with inhomogeneous boundary conditions, in Cartesian geometry. Previous methods for numerical solution of time-dependent PDEs often used finite difference for time, and a spectral scheme for the spatial dimensions, which led to a slow linear convergence. Spectral collocation space-time methods show exponential convergence; however, for realistic problems they need to solve large four-dimensional systems. We overcome this difficulty by using a TT approach, as its complexity only grows linearly with the number of dimensions. We show that our TT space-time Chebyshev spectral collocation method converges exponentially, when the solution of the CDR is smooth, and demonstrate that it leads to a very high compression of linear operators from terabytes to kilobytes in TT-format, and a speedup of tens of thousands of times when compared to a full-grid space-time spectral method. These advantages allow us to obtain the solutions at much higher resolutions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Nonnegative canonical tensor decomposition with linear constraints: nnCANDELINC

Abstract There is an emerging interest for tensor factorization applications in big‐data analytics and machine learning. To speed up the factorization of extra‐large datasets, organized in multidimensional arrays (also known as tensors), easy to compute compression‐based tensor representations, such as, Tucker and tensor train formats, are used to approximate the initial large‐tensor. Further, tensor factorization is used to extract latent features that can facilitate discoveries of new mechanisms and signatures hidden in the data, where the explainability of the latent features is of principal importance. Nonnegative tensor factorization extracts latent features that are naturally sparse and parts of the data, which makes them easily interpretable. However, to take into account available domain knowledge and subject matter expertise, often additional constraints need to be imposed, which lead us to canonical decomposition with linear constraints (CANDELINC), a canonical polyadic decomposition with rank deficient factors. In CANDELINC, Tucker compression is used as a preprocessing step, which lead to a larger residual error but to more explainable latent features. Here, we propose a nonnegative CANDELINC (nnCANDELINC) accomplished via a specific nonnegative Tucker decomposition; we refer to as minimal or canonical nonnegative Tucker. We derive several results required to understand the specificity of nnCANDELINC, focusing on the difficulties of preserving the nonnegative rank of a tensor to its Tucker core and comparing the real valued to nonnegative case. Finally, we demonstrate nnCANDELINC performance on synthetic and real‐world examples.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

A Linear-Complexity Tensor Butterfly Algorithm for Compressing High-Dimensional Oscillatory Integral Operators

This paper presents a multilevel tensor compression algorithm called tensor butterfly algorithm for efficiently representing large-scale and high-dimensional oscillatory integral operators, including Green's functions for wave equations and integral transforms such as Radon transforms and Fourier transforms. The proposed algorithm leverages a tensor extension of the so-called complementary low-rank property of existing matrix butterfly algorithms. The algorithm partitions the discretized integral operator tensor into subtensors of multiple levels and factorizes each subtensor at the middle level as a Tucker-type interpolative decomposition, whose factor matrices are formed in a multilevel fashion. For a d-dimensional (d > 1) integral operator discretized into a 2d-mode tensor with n2d entries, the overall CPU time and memory requirement scale as O(nd), in stark contrast to the O(nd log n) complexity of existing matrix algorithms such as matrix butterfly algorithms and fast Fourier transforms (FFTs), where n is the number of points per direction. When comparing with other tensor algorithms such as quantized tensor train (QTT), the proposed algorithm also shows superior CPU and memory performance for tensor contraction. Remarkably, the tensor butterfly algorithm can efficiently model high-frequency Green's function interactions between two unit cubes, each spanning 512 wavelengths per direction, which represents problems of scale over 512× larger than that existing butterfly algorithms can handle, with the same amount of computation resources. On the other hand, for a problem representing 64 wavelengths per direction, which is the largest size existing algebraic matrix algorithms can handle, our tensor butterfly algorithm exhibits 200x speedups and 30× memory reduction compared with existing ones. Moreover, the tensor butterfly algorithm also permits O(nd)-complexity FFTs and Radon transforms up to d = 6 dimensions.

Kielstra, P Michael↗