Search NASA⌕ Search

SEARCH · Search NASA

Results for “Universal approximation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

RandONets: Shallow networks with random projections for learning linear and nonlinear operators

Deep neural networks have been extensively used for the solution of both the forward and the inverse problem for dynamical systems. However, their implementation necessitates optimizing a high-dimensional space of parameters and hyperparameters. This fact, along with the requirement of substantial computational resources, pose a barrier to achieving high numerical accuracy, but also interpretability. Here, to address the above challenges, we present Random Projection-based Operator Networks (RandONets): shallow networks with random projections and tailor-made numerical analysis methods that learn accurately and fast linear and nonlinear operators. Building on previous works, we prove that RandOnets are universal approximators of linear and nonlinear operators. Due to their simplicity, RandONets provide a one-step transformation of the input space, facilitating interpretability. For the evaluation of their performance, we focus on operators of PDEs. We show, that RandONets outperform by several orders of magnitude, both in terms of numerical approximation accuracy and computational cost, the “vanilla” DeepONets. Hence, we believe that our method will trigger further developments in the field of scientific machine learning, for the development of new ‘’light”schemes that will provide high accuracy while reducing dramatically the computational cost. A MATLAB toolbox for RandONets, including demos, is available on GitHub at https://github.com/GianlucaFabiani/RandONets.

Interpretable machine learning↗
DOE OSTI2023

Toward DMC Accuracy Across Chemical Space with Scalable Δ-QML

In the past decade, quantum diffusion Monte Carlo (DMC) has been demonstrated to successfully predict the energetics and properties of a wide range of molecules and solids by numerically solving the electronic many-body Schrödinger equation. With O(N 3 ) scaling with the number of electrons N, DMC has the potential to be a reference method for larger systems that are not accessible to more traditional methods such as CCSD(T). Assessing the accuracy of DMC for smaller molecules becomes the stepping stone in making the method a reference for larger systems. We show that when coupled with quantum machine learning (QML)-based surrogate methods, the computational burden can be alleviated such that quantum Monte Carlo (QMC) shows clear potential to undergird the formation of high-quality descriptions across chemical space. We discuss three crucial approximations necessary to accomplish this: the fixed-node approximation, universal and accurate references for chemical bond dissociation energies, and scalable minimal amons-set-based QML (AQML) models. Numerical evidence presented includes converged DMC results for over 1000 small organic molecules with up to five heavy atoms used as amons and 50 medium-sized organic molecules with nine heavy atoms to validate the AQML predictions. Finally, numerical evidence collected for Δ-AQML models suggests that already modestly sized QMC training data sets of amons suffice to predict total energies with near chemical accuracy throughout chemical space.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Conditioned quantum-assisted deep generative surrogate for particle-calorimeter interactions

Particle collisions at accelerators like the Large Hadron Collider (LHC), recorded by experiments such as ATLAS and CMS, enable precise standard model measurements and searches for new phenomena. Simulating these collisions significantly influences experiment design and analysis but incurs immense computational costs, projected at millions of CPU-years annually during the high luminosity LHC (HL-LHC) phase. Currently, simulating a single event with Geant4 consumes around 1000 CPU seconds, with calorimeter simulations especially demanding. To address this, we propose a conditioned quantum-assisted generative model, integrating a conditioned variational autoencoder (VAE) and a conditioned restricted Boltzmann machine (RBM). Our RBM architecture is tailored for D-Wave’s Pegasus-structured advantage quantum annealer for sampling, leveraging the flux bias for conditioning. This approach combines classical RBMs as universal approximators for discrete distributions with quantum annealing’s speed and scalability. We also introduce an adaptive method for efficiently estimating effective inverse temperature, and validate our framework on Dataset 2 of CaloChallenge.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Effect of likelihood misspecification in Gaussian process-driven autonomous experimentation

In recent years, several groups have designed Autonomous Experiment (AE) models with the aim of using them as an alternative method for neutron scattering scanning. In an AE, Gaussian processes (GPs) are most frequently used due to their interpretability, their non-parametric nature, their universal approximation, and their closed-form predictive distribution. GPs have two key components, namely, the model for the likelihood of a neutron count knowing the underlying dynamic structure factor and the acquisition function. In this paper, we investigate the impact, on the quality of an AE, of the likelihood and acquisition function choices, in energy scans and (Q, ω) ones, with respect to the signal-over-noise ratio. While we hypothesized that the quality of GP predictions would decrease when the normal to Poisson likelihood approximation breaks down at low count rates, we found that the use of the correct Poisson likelihood does not improve the quality of the data collected, as well as yields very poor results in (Q, ω) scans at low count rates. In fact, the best results are obtained with a combination of normal likelihood, including the observation noise, and the change in variance acquisition function. In addition, we find that the performance, or quality of the predictive distribution, is a misleading measure of efficiency, that is, of the quality of the data collected.

Perryman, David Elliott [Inst. Laue-Langevin (ILL)↗
DOE OSTI2023

Scientific machine learning for modeling and simulating complex fluids

The formulation of rheological constitutive equations—models that relate internal stresses and deformations in complex fluids—is a critical step in the engineering of systems involving soft materials. While data-driven models provide accessible alternatives to expensive first-principles models and less accurate empirical models in many engineering disciplines, the development of similar models for complex fluids has lagged. The diversity of techniques for characterizing non-Newtonian fluid dynamics creates a challenge for classical machine learning approaches, which require uniformly structured training data. Consequently, early machine-learning based constitutive equations have not been portable between different deformation protocols or mechanical observables. Here, we present a data-driven framework that resolves such issues, allowing rheologists to construct learnable models that incorporate essential physical information, while remaining agnostic to details regarding particular experimental protocols or flow kinematics. These scientific machine learning models incorporate a universal approximator within a materially objective tensorial constitutive framework. By construction, these models respect physical constraints, such as frame-invariance and tensor symmetry, required by continuum mechanics. We demonstrate that this framework facilitates the rapid discovery of accurate constitutive equations from limited data and that the learned models may be used to describe more kinematically complex flows. This inherent flexibility admits the application of these “digital fluid twins” to a range of material systems and engineering problems. We illustrate this flexibility by deploying a trained model within a multidimensional computational fluid dynamics simulation—a task that is not achievable using any previously developed data-driven rheological equation of state.

Science & Technology - Other Topics↗
DOE OSTI2020

Reynolds-Averaged Turbulence Modeling Using Deep Learning with Local Flow Features: An Empirical Approach

Reynolds-Averaged Navier-Stoke (RANS) models offer an alternative avenue in predicting flow characteristics when the corresponding experiments are difficult to achieve due to geometry complexity, limited budget, or knowledge. RANS models require the knowledge of subgrid scale physics to solve conservation equations for mass, energy, and momentum. Mechanistic turbulence models, such as k-epsilon, are generally evaluated and calibrated for specific flow conditions with various degrees of uncertainty. These models have limited capability to assimilate a substantial amount of data due to model form constraints. Meanwhile, deep learning (DL) has been proven to be universal approximators with the potential to assimilate available, relevant, and adequately evaluated data. Moreover, deep neural networks (DNNs) can create surrogate models without knowing function forms. Such a data-driven approach can be used in updating fluid models based on observations as opposed to hard-wiring models with precalibrated correlations. The paper presents progress in applying DNNs to model Reynolds stress using two machine learning (ML) frameworks. A novel flow feature coverage mapping is proposed to quantify the physics coverage of DL-based closures. It can be used to examine the sufficiency of training data and input flow features for data-driven turbulence models. The case of a backward-facing step is formulated to demonstrate that not only can DNNs discover underlying correlation behind fluid data but also they can be implemented in RANS to predict flow characteristics without numerical stability issues. Finally, the presented research is a crucial stepping-stone toward the data-driven turbulence modeling, which potentially benefits the design of data-driven experiments that can be used to validate fluid models with ML-based fluid closures.

42 ENGINEERING↗
DOE OSTI2025

Exploring the energy landscape of RBMs: reciprocal space insights into bosons, hierarchical learning and symmetry breaking

Deep generative models have become ubiquitous due to their ability to learn and sample from complex distributions. Despite the proliferation of various frameworks, the relationships among these models remain largely unexplored, a gap that hinders the development of a unified theory of AI learning. In this work, we address two central challenges: clarifying the connections between different deep generative models and deepening our understanding of their learning mechanisms. We focus on Restricted Boltzmann Machines (RBMs), a class of generative models known for their universal approximation capabilities for discrete distributions. By introducing a reciprocal space formulation for RBMs, we reveal a connection between these models, diffusion processes, and systems of coupled bosons. Our analysis shows that at initialization, the RBM operates at a saddle point, where the local curvature is determined by the singular values of the weight matrix, whose distribution follows the Marc̆enko-Pastur law and exhibits rotational symmetry. During training, this rotational symmetry is broken due to hierarchical learning, where different degrees of freedom progressively capture features at multiple levels of abstraction. This leads to a symmetry breaking in the energy landscape, reminiscent of Landau’s theory. This symmetry breaking in the energy landscape is characterized by the singular values and the weight matrix eigenvector matrix. We derive the corresponding free energy in a mean-field approximation. We show that in the limit of infinite size RBM, the reciprocal variables are Gaussian distributed. Our findings indicate that in this regime, there will be some modes for which the diffusion process will not converge to the Boltzmann distribution. To illustrate our results, we trained replicas of RBMs with different hidden layer sizes using the MNIST dataset. Our findings not only bridge the gap between disparate generative frameworks but also shed light on the fundamental processes underpinning learning in deep generative models.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

First measurement of the characteristic depletion radius of dark matter haloes from weak lensing

ABSTRACT We use weak lensing observations to make the first measurement of the characteristic depletion radius, one of the three radii that characterize the region where matter is being depleted by growing haloes. The lenses are taken from the halo catalogue produced by the extended halo-based group/cluster finder applied to DESI Legacy Imaging Surveys DR9, while the sources are extracted from the DECaLS DR8 imaging data with the fourier_quad pipeline. We study halo masses $12 \lt \log (M_{\rm grp} \,[{\rm M_{\odot }}\, h^{-1}]) \le 15.3$ within redshifts 0.2 ≤ z ≤ 0.3. The virial and splashback radii are also measured and used to test the original findings on the depletion region. When binning haloes by mass, we find consistency between most of our measurements and predictions from the cosmicgrowth simulation, with exceptions to the lowest mass bins. The characteristic depletion radius is found to be roughly 2.5 times the virial radius and 1.7–3 times the splashback radius, in line with an approximately universal outer density profile, and the average enclosed density within the characteristic depletion radius is found to be roughly 29 times the mean matter density of the Universe in our sample. When binning haloes by both mass and a proxy for halo concentration, we do not detect a significant variation of the depletion radius with concentration, on which the simulation prediction is also sensitive to the choice of concentration proxy. We also confirm that the measured splashback radius varies with concentration differently from simulation predictions.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

GFINNs: GENERIC formalism informed neural networks for deterministic and stochastic dynamical systems

Here we propose the GENERIC formalism informed neural networks (GFINNs) that obey the symmetric degeneracy conditions of the GENERIC formalism. GFINNs comprise two modules, each of which contains two components. We model each component using a neural network whose architecture is designed to satisfy the required conditions. The component-wise architecture design provides flexible ways of leveraging available physics information into neural networks. We prove theoretically that GFINNs are sufficiently expressive to learn the underlying equations, hence establishing the universal approximation theorem. We demonstrate the performance of GFINNs in three simulation problems: gas containers exchanging heat and volume, thermoelastic double pendulum and the Langevin dynamics. In all the examples, GFINNs outperform existing methods, hence demonstrating good accuracy in predictions for both deterministic and stochastic systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Inference, Prediction, & Entropy-Rate Estimation of Continuous-Time, Discrete-Event Processes

Inferring models, predicting the future, and estimating the entropy rate of discrete-time, discrete-event processes is well-worn ground. However, a much broader class of discrete-event processes operates in continuous-time. Here, we provide new methods for inferring, predicting, and estimating them. The methods rely on an extension of Bayesian structural inference that takes advantage of neural network’s universal approximation power. Based on experiments with complex synthetic data, the methods are competitive with the state-of-the-art for prediction and entropy-rate estimation.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Wormhole calculus, replicas, and entropies

We investigate contributions of spacetime wormholes, describing baby universe emission and absorption, to calculations of entropies and correlation functions, for example those based on the replica method. We find that the rules of the “wormhole calculus”, developed in the 1980s, together with standard quantum mechanical prescriptions for computing entropies and correlators, imply definite rules for limited patterns of connection between replica factors in simple calculations. These results stand in contrast with assumptions that all topologies connecting replicas should be summed over, and call into question the explanation for the latter. In a “free” approximation baby universes introduce probability distributions for coupling constants, and we review and extend arguments that successive experiments in a “parent” universe increasingly precisely fix such couplings, resulting in ultimately pure evolution. Once this has happened, the nontrivial question remains of how topology-changing effects can modify the standard description of black hole information loss.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2025

Parametric matrix models

We present a general class of machine learning algorithms called parametric matrix models. In contrast with most existing machine learning models that imitate the biology of neurons, parametric matrix models use matrix equations that emulate physical systems. Similar to how physics problems are usually solved, parametric matrix models learn the governing equations that lead to the desired outputs. Parametric matrix models can be efficiently trained from empirical data, and the equations may use algebraic, differential, or integral relations. While originally designed for scientific computing, we prove that parametric matrix models are universal function approximators that can be applied to general machine learning problems. After introducing the underlying theory, we apply parametric matrix models to a series of different challenges that show their performance for a wide range of problems. For all the challenges tested here, parametric matrix models produce accurate results within an efficient and interpretable computational framework that allows for input feature extrapolation.

Computational science↗
DOE OSTI2025

Universal control in bosonic systems with weak Kerr nonlinearities

Resonators with weak single-photon self-Kerr nonlinearities can theoretically be used to prepare Fock states in the presence of a loss much larger than their nonlinearities. Two necessary ingredients are large displacements and a two-photon (parametric) drive. Here, in this study, we find that these systems can be controlled to achieve any desired gate operation in a finite-dimensional subspace (whose dimensionality can be chosen at will). Moreover, we show that the two-photon driving requirement can be relaxed and that full controllability is achievable with only one-photon (linear) drives. We make use of both Trotter-Suzuki decompositions and gradient-based optimization to find control pulses for a desired gate, which reduces the computational overhead by using a small blockaded subspace. We also discuss the infidelity arising from input power limitations in realistic settings, as well as from corrections to the rotating-wave approximation. Our universal control protocol opens the possibility for quantum information processing using a wide range of lossy systems with weak nonlinearities.

Yuan, Ming [Univ. of Chicago, IL (United States)] ↗
DOE OSTI2025

Direct NeTS sampling of nuclear graphite $S(α, β, T)$ in Serpent

For advanced reactor applications, Neural Thermal Scattering (NeTS) modules were developed to predict the thermal scattering law (TSL or $S(α, β, T)$) of a nuclear graphite neutron moderator. NeTS are multi-layer, feedforward artificial neural networks, which act as universal function approximators designed for TSL datasets. In this case, a 4-layer neural network with 164 neurons per layer is trained using FLASSH evaluated data in PyTorch and serialized as a torchscript dictionary to predict $S(α, β, T)$ on-the-fly. Relative, absolute and maximum percent deviations of NeTS from File 7 data generated using the FLASSH code are on the order of 0.01%, 0.1% and 1%, respectively, with low inference latencies of 0.000172 s per $S(α, β, T)$ at a given temperature. Capturing the full dimensionality of possible inelastic neutron-lattice interactions, NeTS functionality is embedded in the Serpent Monte Carlo code, where $S(α, β, T)_{NeTS}$ sampling is conducted on-the-fly and compared to ACE look-up-tables for predicting TREAT criticality. k-eff differences between sampling algorithms of 6 pcm are observed and are within the order of Monte Carlo uncertainty. Compared to discrete and continuous-energy ACE files (30 MB and 131 MB per temperature), the NeTS format is on the order of 200–300 kB for a continuous-temperature, interpolation-free representation of $S(α, β, T)$ and cross sections. NeTS-in-Serpent runtimes comparable with ACE look-up tables are achieved by scaling NeTS for high performance computing architectures with hybrid OpenMP + MPI parallelization. This work validates a novel, self-contained reactor physics framework for predictive cross sections, and demonstrates a general methodology for embedding modern machine learning libraries within existing neutronic analysis frameworks.

Nuclear Criticality Safety Program (NCSP)↗
DOE OSTI2026

Nonlinear thermodynamic computing out of equilibrium

We present the design for a thermodynamic computer that can perform arbitrary nonlinear calculations in or out of equilibrium. Simple thermodynamic circuits, fluctuating degrees of freedom in contact with a thermal bath and confined by a quartic potential, display an activity that is a nonlinear function of their input. Such circuits can therefore be regarded as thermodynamic neurons, and can serve as the building blocks of networked structures that act as thermodynamic neural networks, universal function approximators whose operation is powered by thermal fluctuations. We simulate a digital model of a thermodynamic neural network, and show that its parameters can be adjusted by genetic algorithm to perform nonlinear calculations at specified observation times, regardless of whether the system has attained thermal equilibrium. This work expands the field of thermodynamic computing beyond the regime of thermal equilibrium, enabling fully nonlinear computations, analogous to those performed by classical neural networks, at specified observation times.

Whitelam, Stephen [Lawrence Berkeley National Labo↗
DOE OSTI2023

Physics informed deep neural network embedded in a chemical transport model for the Amazon rainforest

Secondary organic aerosols (SOA) are fine particles in the atmosphere, which interact with clouds, radiation and affect the Earth’s energy budget. SOA formation involves chemistry in gas phase, aqueous aerosols, and clouds. Simulating these chemical processes involve solving a stiff set of differential equations, which are computationally expensive steps for three-dimensional chemical transport models. Deep neural networks (DNNs) are universal function approximators that could be used to represent the complex nonlinear changes in aerosol physical and chemical processes; however, key challenges such as generalizability to extended time periods, preservation of mass balance, simulating sparse model outputs, and maintaining physical constraints have limited their use in atmospheric chemistry. Here, we develop an approach of using a physics-informed DNN that overcomes previous such challenges and demonstrates its applicability for the chemical formation processes of isoprene epoxydiol SOA (IEPOX-SOA) over the Amazon rainforest. The DNN is trained with data generated by simulating IEPOX-SOA over the entire atmospheric column, using the Weather Research and Forecasting Model coupled with Chemistry (WRF-Chem). The trained DNN is then embedded within WRF-Chem to replace the computationally expensive default solver of IEPOX-SOA formation. The trained DNN predictions generalizes well with the default model simulation of the IEPOX-SOA mass concentrations and its size distribution (20 size bins) over several days of simulations in both dry and wet seasons. The embedded DNN reduces the computational expense of WRF-Chem by a factor of 2. Our approach shows promise in terms of application to other computationally expensive chemistry solvers in climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Confronting Domain Shift in Trained Neural Networks

Neural networks (NNs) are known as universal function approximators and can interpolate nonlinear functions between observed data points. However, when the target domain for deployment shifts from the training domain and NNs must extrapolate, the results are notoriously poor. Prior work Martinez et al. (2019) has shown that NN uncertainty estimates can be used to correct binary predictions in shifted domains without retraining the model. We hypothesize that this approach can be extended to correct real-valued time series predictions. As an exemplar, we consider two mechanical systems with nonlinear dynamics. The first system consists of a spring-mass system where the stiffness changes abruptly, and the second is a real experimental system with a frictional joint that is an open challenge for structural dynamicists to model efficiently. Our experiments will test whether 1) NN uncertainty estimates can identify when the input domain has shifted from the training domain and 2) whether the information used to calculate uncertainty estimates can be used to correct the NN’s time series predictions. While the method as proposed did not significantly improve predictions, our results did show potential for modifications that could improve models’ predictions and play a role in structural health monitoring systems that directly impact public safety.

97 MATHEMATICS AND COMPUTING↗