SEARCH · Search NASA

Results for “Tensor network methods”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

All sources NASA NTRS DOE OSTI

DOE OSTI2022

Towards Compact Neural Networks via End-to-End Training: A Bayesian Tensor Approach with Automatic Rank Determination

Post-training model compression can reduce the inference costs of deep neural networks, but uncompressed training still consumes enormous hardware resources and energy. To enable low-energy training on edge devices, it is highly desirable to directly train a compact neural network from scratch with a low memory cost. Low-rank tensor decomposition is an effective approach to reduce the memory and computing costs of large neural networks. However, directly training low-rank tensorized neural networks is a very challenging task because it is hard to determine a proper tensor rank a priori, and the tensor rank controls both model complexity and accuracy. Here, this paper presents a novel end-to-end framework for low-rank tensorized training. We first develop a Bayesian model that supports various low-rank tensor formats (e.g., CANDECOMP/PARAFAC, Tucker, tensor-train, and tensor-train matrix) and reduces neural network parameters with automatic rank determination during training. Then we develop a customized Bayesian solver to train large-scale tensorized neural networks. Our training methods shows orders-of-magnitude parameter reduction and little accuracy loss (or even better accuracy) in the experiments. On a very large deep learning recommendation system with over 4.2 ×10 9 model parameters, our method can reduce the parameter number to 1.6 ×10 5 automatically in the training process (i.e., by 2.6 ×10 4 times) while achieving almost the same accuracy. Code is available at https://github.com/colehawkins/bayesian-tensor-rank-determination.

compact neural networks↗

DOE OSTI2023

Real-time evolution of Anderson impurity models via tensor network influence functionals

In this work, we present and analyze two tensor network-based influence functional approaches for simulating the real-time dynamics of quantum impurity models such as the Anderson model. Via comparison with recent numerically exact simulations, we show that such methods accurately capture the long-time nonequilibrium quench dynamics. The two parameters that must be controlled in these tensor network influence functional approaches are a time discretization (Trotter) error and a bond dimension (tensor network truncation) error. We show that the actual numerical uncertainties are controlled by an intricate interplay of these two approximations, which we demonstrate in different regimes. Our work opens the door to using these tensor network influence functional methods as general impurity solvers.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2026

A Tensor Network-Based Quantum Algorithm for the Nonlinear 1D Burgers' Equation

In this work, we implement a tensor network-based quantum algorithm to solve unsteady, nonlinear partial differential equations (PDEs). The challenge lies in how to effectively represent, encode, process, and evolve the nonlinear system of PDEs on quantum computers. We will discuss the new techniques using the compressible 1-dimensional (1D) Burgers' equation as an example, because it represents the fundamental nonlinear feature and yet removes certain complexity in physics, allowing us to focus on the design of quantum algorithms. Previous attempts to solve nonlinear PDEs in quantum computation have often involved storing multiple copies of solutions or employing linearizations. Neither is practical due to exponential scaling with evolution time or insufficient solution accuracy. Our framework is based on matrix product states (MPSs) and matrix product operators (MPOs). For example, the velocity field is represented by MPS, whereas the linear and nonlinear spatial differential terms of the velocity field are processed by MPOs. Our primary focus herein is to verify and validate the various tensor network components of the algorithm using solutions obtained by the classical algorithms on high performance computing (HPC) architectures. We use a classical time marching method to demonstrate the functionality of the tensor network operations to model the PDE and their robustness with the time evolution of the system. Our classical simulation results demonstrate the utility of tensor network-based operations in modeling nonlinear PDEs and highlight the necessity as well as potential advantages of using quantum simulations for these techniques.

Gopalakrishnan Meena, Murali [ORNL] (ORCID:0000000↗

DOE OSTI2024

Random insights into the complexity of two-dimensional tensor network calculations

Projected entangled pair states (PEPS) offer memory-efficient representations of some quantum many-body states that obey an entanglement area law and are the basis for classical simulations of ground states in two-dimensional (2d) condensed matter systems. However, rigorous results show that exactly computing observables from a 2d PEPS state is generically a computationally hard problem. Yet approximation schemes for computing properties of 2d PEPS are regularly used, and empirically seen to succeed, for a large subclass of (“not too entangled”) condensed matter ground states. Adopting the philosophy of random matrix theory, in this work, we analyze the complexity of approximately contracting a 2d random PEPS by exploiting an analytic mapping to an effective replicated statistical mechanics model that permits a controlled analysis at a large bond dimension. Through this statistical-mechanics lens, we argue that (i) although approximately sampling wave-function amplitudes of random PEPS faces a computational-complexity phase transition above a critical bond dimension, and (ii) one can generically efficiently estimate the norm and correlation functions for any finite bond dimension. Furthermore, these results are supported numerically for various bond-dimension regimes. It is an important open question whether the above results for random PEPS apply more generally also to PEPS representing physically relevant ground states.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2024

Inclusive reactions from finite Minkowski spacetime correlation functions

The need to determine scattering amplitudes of few-hadron systems for arbitrary kinematics expands a broad set of subfields of modern-day nuclear and hadronic physics. In this work, we expand upon previous explorations on the use of real-time methods, like quantum computing or tensor networks, to determine few-body scattering amplitudes. Such calculations must be performed in a finite Minkowski spacetime, where scattering amplitudes are not well defined. Our previous work presented a conjecture of a systematically improvable estimator for scattering amplitudes constructed from finite-volume correlation functions. Here we provide further evidence that the prescription works for larger kinematic regions than previously explored as well as a broader class of scattering amplitudes. Finally, we devise a new method for estimating the order of magnitude of the error associated with finite time separations needed for such calculations. In units of the lightest mass of the theory, we find that to constrain amplitudes using real-time methods within O ( 10 % ) , the spacetime volumes must satisfy m L ∼ O ( 10 – 10 2 ) ) and m T ∼ O ( 10 2 – 10 4 ) . Published by the American Physical Society 2024

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

DOE OSTI2024

Spectral gaps of two- and three-dimensional many-body quantum systems in the thermodynamic limit

We present an expression for the spectral gap, opening up new possibilities for performing and accelerating spectral calculations of quantum many-body systems. We develop and demonstrate one such possibility in the context of tensor network simulations. Our approach requires only minor modifications of the widely used simple update method and is computationally lightweight relative to other approaches. We validate it by computing spectral gaps of the 2D and 3D transverse-field Ising models and find strong agreement with previously reported perturbation theory results. Published by the American Physical Society 2024

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2024

Symbolic diagnostics to interpret and analyze neural network models

Embedded machine-learned models (EMLMs) have the promise to improve the predictive accuracy of engineering simulators in environments of national interest. EMLMs often comprise complex input-output maps (e.g., neural networks), which make them unamenable to rigorous analysis and generally difficult to interpret. In the face of decades of theory, this lack of interpretability is a significant barrier to building confidence in these models. This work outlines an approach to interpret EMLMs using sparse polynomial regression for comparison with theoretical understanding. To do so, we build on the concept of Locally Interpretable Model-agnostic Explanations (LIME) using physics-informed clustering, prototype selection, and library construction. While general, we demonstrate our method on tensor-basis neural networks used in Reynolds-Averaged Navier-Stokes simulations of hypersonic fluid flows. Results are presented for a simulated toy model and for direct numerical simulations (DNS) of turbulent flows over a flat plate.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2022

Quantum phase transition dynamics in the two-dimensional transverse-field Ising model

The quantum Kibble-Zurek mechanism (QKZM) predicts universal dynamical behavior near the quantum phase transitions (QPTs). It is now well understood for the one-dimensional quantum matter. Higher-dimensional systems, however, remain a challenge, complicated by the fundamentally different character of the associated QPTs and their underlying conformal field theories. In this work, we take the first steps toward theoretical exploration of the QKZM in two dimensions for interacting quantum matter. We study the dynamical crossing of the QPT in the paradigmatic Ising model by a joint effort of modern state-of-the-art numerical methods, including artificial neural networks and tensor networks. As a central result, we quantify universal QKZM behavior close to the QPT. We also note that, upon traversing further into the ferromagnetic regime, deviations from the QKZM prediction appear. We explain the observed behavior by proposing an extended QKZM taking into account spectral information as well as phase ordering. Our work provides a testing platform for higher-dimensional quantum simulators.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2022

Reverse-mode differentiation in arbitrary tensor network format: with application to supervised learning.

This paper describes an efficient reverse-mode differentiation algorithm for contraction operations of tensor networks that may have arbitrary and unconventional network topologies. The approach leverages the tensor contraction tree of Evenbly and Pfeifer (2014), which provides an instruction set for the contraction sequence of a network. We show that this tree can be efficiently leveraged for differentiation of a full tensor network contraction using a recursive scheme that exploits (1) the bilinear property of contraction and (2) the property that trees have single path from root to leaves. While differentiation of tensor-tensor contraction is already possible in most automatic differentiation packages, we show that exploiting these two additional properties in the specific context of contraction sequences can improve efficiency. Following a description of the algorithm and computational complexity analysis, we investigate its utility for gradient-based supervised learning for low-rank function recovery and for fitting real-world unstructured datasets. We demonstrate improved performance over alternating least-squares optimization approaches and the capability to handle heterogeneous and arbitrary tensor network formats. When compared to alternating minimization algorithms, we find that the gradient-based approach requires a smaller oversampling ratio (number of samples compared to number model parameters) for recovery. This increased efficiency extends to fitting unstructured data of varying dimensionality and when employing a variety of tensor network formats. Here, we show improved learning using the hierarchical Tucker method over the tensor-train in high-dimensional settings on a number of benchmark problems.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2025

Discrete generative diffusion models without stochastic differential equations: A tensor network approach

Diffusion models (DMs) are a class of generative machine learning methods that sample a target distribution by transforming samples of a trivial (often Gaussian) distribution using a learned stochastic differential equation. In standard DMs, this is done by learning a “score function” that reverses the effect of adding diffusive noise to the distribution of interest. Here we consider the generalisation of DMs to lattice systems with discrete degrees of freedom, and where noise is added via Markov chain jump dynamics. We show how to use tensor networks (TNs) to efficiently define and sample such “discrete diffusion models” (DDMs) without explicitly having to solve a stochastic differential equation. We show the following: (i) by parametrising the data and evolution operators as TNs, the denoising dynamics can be represented exactly; (ii) the auto-regressive nature of TNs allows to generate samples efficiently and without bias; (iii) for sampling Boltzmann-like distributions, TNs allow to construct an efficient learning scheme that integrates well with Monte Carlo. We illustrate this approach to study the equilibrium of two models with non-trivial thermodynamics, the d = 1 constrained Fredkin chain and the d = 2 Ising model. Published by the American Physical Society 2025

Causer, Luke (ORCID:0000000194243473)↗

DOE OSTI2021

Multi-Area Model-Free State Estimation via Distributed Tensor Decomposition

This paper proposes a model-free method for distribution system state estimation based on tensor completion using canonical polyadic decomposition. In particular, we consider a setting where the network is divided into multiple areas. The measured physical quantities at buses located in the same area are processed by an area controller. A third-order tensor is constructed to collect these measured quantities. The measurements are analyzed locally to recover the full state information of the network. A closed-form iterative algorithm based on the alternating direction method of multipliers is developed to obtain the low-rank factors of the whole network state tensor where information exchange happens only between neighboring areas. To demonstrate the efficacy of the developed algorithm, numerical simulations are carried out using an IEEE test system.

alternating direction method of multipliers↗

DOE OSTI2021

Effective $\mathbb{Z}_{3}$ model for finite-density QCD with tensor networks

The tensor renormalization group is a promising numerical method used to study lattice statistical field theories. However, this approach is computationally expensive in 2+1 and 3+1 dimensions. Here we use tensor renormalization group methods to study an effective three-dimensional $\mathbb{Z}_{3}$ model for the heavy-quark, high-temperature, strong-coupling limit of single-flavor 3+1 dimensional quantum chromodynamics. Our results are cross-checked using the worm Monte Carlo algorithm. We present the phase diagram of the model through the measurement of the Polyakov loop, the nearest-neighbor Polyakov loop correlator, and their susceptibilities. The tensor renormalization group results are in good agreement with the literature

Bloch, Jacques↗

DOE OSTI2022

Multiarea Distribution System State Estimation via Distributed Tensor Completion

Here, this paper proposes a model-free distribution system state estimation method based on tensor completion using canonical polyadic decomposition. In particular, we consider a setting where the network is divided into multiple areas. The measured physical quantities at buses located in the same area are processed by an area controller. A three-way tensor is constructed to collect these measured quantities. The measurements are analyzed locally to recover the full state information of the network. A distributed closed-form iterative algorithm based on the alternating direction method of multipliers is developed to obtain the low-rank factors of the whole network state tensor where information exchange happens only between neighboring areas. The convergence properties of the distributed algorithm and the sufficient conditions on the number of samples for each smaller network that guarantee the identifiability of the factors of the state tensor are presented. To demonstrate the efficacy of the proposed algorithm and to check the identifiability conditions, numerical simulations are carried out using the IEEE 123-bus system and a large-scale real utility feeder.

24 POWER TRANSMISSION AND DISTRIBUTION↗

DOE OSTI2025

Tree tensor network hierarchical equations of motion based on time-dependent variational principle for efficient open quantum dynamics in structured thermal environments

In this work, we introduce an efficient method, TTN-HEOM, for exactly calculating the open quantum dynamics for driven quantum systems interacting with highly structured bosonic baths by combining the tree tensor network (TTN) decomposition scheme with the bexcitonic generalization of the numerically exact hierarchical equations of motion (HEOM). The method yields a series of quantum master equations for all core tensors in the TTN that efficiently and accurately capture the open quantum dynamics for non-Markovian environments to all orders in the system–bath interaction. These master equations are constructed based on the time-dependent Dirac–Frenkel variational principle, which isolates the optimal dynamics for the core tensors given the TTN ansatz. The dynamics converges to the HEOM when increasing the rank of the core tensors, a limit in which the TTN ansatz becomes exact. We introduce TENSO, tensor equations for non-Markovian structured open systems, as a general-purpose Python code to propagate the TTN-HEOM dynamics. We implement three general propagators for the coupled master equations: two fixed-rank methods that require a constant memory footprint during the dynamics and one adaptive-rank method with a variable memory footprint controlled by the target level of computational error. We exemplify the utility of these methods by simulating a two-level system coupled to a structured bath containing one Drude–Lorentz component and eight Brownian oscillators, which is beyond what can presently be computed using the standard HEOM. Our results show that the TTN-HEOM is capable of simulating both dephasing and relaxation dynamics of driven quantum systems interacting with structured baths, even those of chemical complexity, with an affordable computational cost.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

Fast and converged classical simulations of evidence for the utility of quantum computing before fault tolerance

A recent quantum simulation of observables of the kicked Ising model on 127 qubits implemented circuits that exceed the capabilities of exact classical simulation. We show that several approximate classical methods, based on sparse Pauli dynamics and tensor network algorithms, can simulate these observables orders of magnitude faster than the quantum experiment and can also be systematically converged beyond the experimental accuracy. Our most accurate technique combines a mixed Schrödinger and Heisenberg tensor network representation with the Bethe free entropy relation of belief propagation to compute expectation values with an effective wave function–operator sandwich bond dimension >16,000,000, achieving an absolute accuracy, without extrapolation, in the observables of <0.01, which is converged for many practical purposes. We thereby identify inaccuracies in the experimental extrapolations and suggest how future experiments can be implemented to increase the classical hardness.

Science & Technology - Other Topics↗

DOE OSTI2023

Tensor renormalization group study of 3D principal chiral model

We study the three-dimensional SU(2) principal chiral model using different tensor renormalization group methods based on the triad and anisotropic decomposition of the tensor. The tensor network representation is formulated based on the character expansion of the Boltzmann weight. We compare the average action obtained using these two tensor network algorithms and confirm that the resulting critical coupling and exponent are comparable with the recent estimations from the Monte Carlo methods.

Unmuth-Yockey, Judah↗

DOE OSTI2023

Tensor renormalization group study of 3D principal chiral model

We study the three-dimensional $SU(2)$ principal chiral model (PCM) using different tensor renormalization group methods based on the triad and anisotropic decomposition of the tensor. The tensor network representation is formulated based on the character expansion of the Boltzmann weight. We compare the average action obtained using these two tensor network algorithms and confirm that the resulting critical coupling and exponent are comparable with the recent estimations from the Monte Carlo methods.

Akiyama, Shinichiro↗

DOE OSTI2020

Generalized Canonical Polyadic Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work develops a generalized canonical polyadic (GCP) low-rank tensor decomposition that allows other loss functions besides squared error. For instance, we can use logistic loss or Kullback--Leibler divergence, enabling tensor decomposition for binary or count data. We present a variety of statistically motivated loss functions for various scenarios. We provide a generalized framework for computing gradients and handling missing data that enables the use of standard optimization methods for fitting the model. Furthermore, we demonstrate the flexibility of the GCP decomposition on several real-world examples including interactions in a social network, neural activity in a mouse, and monthly rainfall measurements in India.

97 MATHEMATICS AND COMPUTING↗