Search NASA⌕ Search

SEARCH · Search NASA

Results for “differentiable machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Differentiable vertex fitting for jet flavor tagging

This work explores the use of differentiable programming to integrate domain knowledge, in the form of domain specific software, into neural networks to develop scientific machine learning systems. We propose a differentiable vertex fitting algorithm that estimates the crossing point of multiple curves. In the high energy physics setting, these curves are defined by particle equations of motion and the crossing point represents the origin of particle production. This differentiable vertex fitting algorithm can be seamlessly integrated into neural networks, and we show its utility and efficacy in the high energy physics application of the classification of jets, i.e., collimated streams of particles in particle detectors whose originating parent particle we aim to classify. We demonstrate how differentiable vertex fitting can be integrated into larger transformer-based models for jet flavor tagging and show improvements in heavy flavor jet classification when compared to baseline models. Published by the American Physical Society 2024

Smith, Rachel E. C. (ORCID:0000000335851262)↗
DOE OSTI2025

Progress in end-to-end optimization of fundamental physics experimental apparata with differentiable programming

In this article we examine recent developments in the research area concerning the creation of end-to-end models for the complete optimization of measuring instruments. The models we consider rely on differentiable programming methods and on the specification of a software pipeline including all factors impacting performance — from the data-generating processes to their reconstruction and the inference on the parameters of interest — along with the careful specification of a utility function well aligned with the end goals of the experiment. Building on previous studies originated within the MODE Collaboration, we focus specifically on applications involving instruments for particle physics experimentation, as well as industrial and medical applications that share the detection of radiation as their data-generating mechanism. This report illustrates the most recent advancements in the area, and outlines, for each of the discussed applications as well as for automatic differentiation itself, ongoing and future work.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Machine learning changes the rules for flux limiters

Learning to integrate non-linear equations from highly resolved direct numerical simulations has seen recent interest for reducing the computational load for fluid simulations. Here, we focus on determining a flux-limiter for shock capturing methods. Focusing on flux limiters provides a specific plug-and-play component for existing numerical methods. Since their introduction, an array of flux limiters has been designed. Using the coarse-grained Burgers' equation, we show that flux-limiters may be rank-ordered in terms of their log-error relative to high-resolution data. We then develop a theory to find an optimal flux-limiter and present flux-limiters that outperform others tested for integrating Burgers' equation on lattices with [Formula: see text], and 2x, 3x, 4x, and 8x coarse-grainings. We train a continuous piecewise linear limiter by minimizing the mean-squared misfit to six-grid point segments of high-resolution data, averaged over all segments. While flux limiters are generally designed to have an output of φ(r) = 1 at a flux ratio of r = 1, our limiters are not bound by this rule and yet produce a smaller error than standard limiters. Here we find that our machine learned limiters have distinctive features that may provide new rules-of-thumb for the development of improved limiters. Additionally, we use our theory to learn flux-limiters that outperform standard limiters across a range of values (as opposed to at a specific fixed value) of coarse-graining, number of discretized bins, and diffusion parameter. This demonstrates the ability to produce flux limiters that should be more broadly useful than standard limiters for general applications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Neural network surrogate models for equations of state

Equation of state (EOS) data provide necessary information for accurate multiphysics modeling, which is necessary for fields such as inertial confinement fusion. Here, we suggest a neural network surrogate model of energy and entropy and use thermodynamic relationships to derive other necessary thermodynamic EOS quantities. We incorporate phase information into the model by training a phase classifier and using phase-specific regression models, which improves the modal prediction accuracy. Our model predicts energy values to 1% relative error and entropy to 3.5% relative error in a log-transformed space. Although sound speed predictions require further improvement, the derived pressure values are accurate within 10% relative error. Our results suggest that neural network models can effectively model EOS for inertial confinement fusion simulation applications.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

A differentiable simulation package for performing inference of synchrotron-radiation-based diagnostics

The direction of particle accelerator development is ever-increasing beam quality, currents and repetition rates. This poses a challenge to traditional diagnostics that directly intercept the beam due to the mutual destruction of both the beam and the diagnostic. An alternative approach is to infer beam parameters non-invasively from the synchrotron radiation emitted in bending magnets. However, inferring the beam distribution from a measured radiation pattern is a complex and computationally expensive task. To address this challenge we present SYRIPY ( SYnchrotron Radiation In PYthon ), a software package intended as a tool for performing inference of synchrotron-radiation-based diagnostics. SYRIPY has been developed using PyTorch , which makes it both differentiable and able to leverage the high performance of GPUs, two vital characteristics for performing statistical inference. The package consists of three modules: a particle tracker, Lienard–Wiechert solver and Fourier optics propagator, allowing start-to-end simulation of synchrotron radiation detection to be carried out. SYRIPY has been benchmarked against SRW , the prevalent numerical package in the field, showing good agreement and up to a 50× speed improvement. Finally, we have demonstrated how SYRIPY can be used to perform Bayesian inference of beam parameters using stochastic variational inference.

43 PARTICLE ACCELERATORS↗
NASA NTRS2018

Atmospheric Chemistry Modeling Using a Regression Forest Model

Atmospheric chemistry is central to many environmental issues such as air pollution, climate change, and stratospheric ozone loss. Chemistry Transport Models (CTM) are a central tool for understanding these issues, whether for research or for forecasting. These models split the atmosphere in a large number of grid-boxes and consider the emission of compounds into these boxes and their subsequent transport, deposition, and chemical processing. The chemistry is represented through a series of simultaneous ordinary differential equations, one for each compound. Given the difference in life-times between the chemical compounds (milli-seconds for O1D to years for CH4) these equations are numerically stiff and solving them consists of a significant fraction of the computational burden of a CTM. We have investigated a machine learning approach to solving the differential equations instead of solving them numerically. From an annual simulation of the GEOS-Chem model we have produced a training dataset consisting of the concentration of compounds before and after the differential equations are solved, together with some key physical parameters for every grid-box and time-step. From this dataset we have trained a machine learning algorithm (regression forest) to be able to predict the concentration of the compounds after the integration step based on the concentrations and physical state at the beginning of the time step. We have then included this algorithm back into the GEOS-Chem model, bypassing the need to integrate the chemistry. This machine learning approach shows many of the characteristics of the full simulation and has the potential to be substantially faster. There are a wide range of application for such an approach - generating boundary conditions, for use in air quality forecasts, chemical data assimilation systems, centennial scale climate simulations etc. We discuss our approches' speed and accuracy, and highlight some potential future directions for improving this approach.

Keller, Christoph A.↗
DOE OSTI2021

On the eigenvector bias of Fourier feature networks: From regression to solving multi-scale PDEs with physics-informed neural networks

Physics-informed neural networks (PINNs) are demonstrating remarkable promise in integrating physical models with gappy and noisy observational data, but they still struggle in cases where the target functions to be approximated exhibit high-frequency or multi-scale features. Here in this work we investigate this limitation through the lens of Neural Tangent Kernel (NTK) theory and elucidate how PINNs are biased towards learning functions along the dominant eigen-directions of their limiting NTK. Using this observation, we construct novel architectures that employ spatio-temporal and multi-scale random Fourier features, and justify how such coordinate embedding layers can lead to robust and accurate PINN models. Numerical examples are presented for several challenging cases where conventional PINN models fail, including wave propagation and reaction–diffusion dynamics, illustrating how the proposed methods can be used to effectively tackle both forward and inverse problems involving partial differential equations with multi-scale behavior.

42 ENGINEERING↗
DOE OSTI2023

DPFEHM: a differentiable subsurface physics simulator

The Earth’s subsurface is a key resource that provides energy via fossil fuels and geothermal sources, stores drinking water, and is used in the fight against climate change via carbon sequestration. Simulating the physical processes that occur in the Earth’s subsurface with computers enables better use of this resource. DPFEHM is a Julia package that includes computer models with a focus on the Earth’s subsurface, especially fluid flow, which is critical for the aforementioned applications. DPFEHM is able to solve the groundwater flow equations (single phase flow), Richards equation (air/water), the advection-dispersion equation, and the 2d wave equation. One of the key features of DPFEHM is that it supports automatic differentiation, so it can be integrated into machine learning workflows using frameworks such as Flux or PyTorch. The automatic differentiation capabilities give it the same performance as adjoint methods.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Generative AI models for learning flow maps of stochastic dynamical systems in bounded domains

Simulating stochastic differential equations (SDEs) in bounded domains, presents significant computational challenges due to particle exit phenomena, which requires accurate modeling of interior stochastic dynamics and boundary interactions. Despite the success of machine learning-based methods in learning SDEs, existing learning methods are not applicable to SDEs in bounded domains because they cannot accurately capture the particle exit dynamics. We present a unified hybrid data-driven approach that combines a conditional diffusion model with an exit prediction neural network to capture both interior stochastic dynamics and boundary exit phenomena. Our ML model consists of two major components: a neural network that learns exit probabilities using binary cross-entropy loss with rigorous convergence guarantees, and a training-free diffusion model that generates state transitions for non-exiting particles using closed-form score functions. The two components are integrated through a probabilistic sampling algorithm that determines particle exit at each time step and generates appropriate state transitions. Here, the performance of the proposed approach is demonstrated via three test cases: a one-dimensional simplified problem for theoretical verification, a two-dimensional advection-diffusion problem in a bounded domain, and a three-dimensional problem of interest to magnetically confined fusion plasmas.

Bounded domains↗
DOE OSTI2023

Evaluation of dual-weighted residual and machine learning error estimation for projection-based reduced-order models of steady partial differential equations

Projection-based reduced-order models (pROMs) show great promise as a means to accelerate many-query applications such as forward error propagation, solving inverse problems, and design optimization. In order to deploy pROMs in the context of high-consequence decision making, accurate error estimates are required to determine the region(s) of applicability in the parameter space. The following paper considers the dual-weighted residual (DWR) error estimate for pROMs and compares it to another promising pROM error estimate, machine learned error models (MLEM). Here, we show how DWR can be applied to ROMs and then evaluate DWR on two partial differential equations (PDEs): a two-dimensional linear convection–reaction–diffusion equation, and a three-dimensional static hyper-elastic beam. It is shown that DWR is able to estimate errors for pROMs extrapolating outside of their training set while MLEM is best suited for pROMs used to interpolate within the pROM training set.

42 ENGINEERING↗
DOE OSTI2023

Learning subgrid-scale models with neural ordinary differential equations

We propose a new approach to learning the subgrid-scale model when simulating partial differential equations (PDEs) solved by the method of lines and their representation in chaotic ordinary differential equations, based on neural ordinary differential equations (NODEs). Solving systems with fine temporal and spatial grid scales is an ongoing computational challenge, and closure models are generally difficult to tune. Machine learning approaches have increased the accuracy and efficiency of computational fluid dynamics solvers. In this approach neural networks are used to learn the coarse- to fine-grid map, which can be viewed as subgrid-scale parameterization. We propose a strategy that uses the NODE and partial knowledge to learn the source dynamics at a continuous level. Our method inherits the advantages of NODEs and can be used to parameterize subgrid scales, approximate coupling operators, and improve the efficiency of low-order solvers. Numerical results with the two-scale Lorenz 96 ODE, the convection-diffusion PDE, and the viscous Burgers' PDE are used to illustrate this approach.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Mucin 4 is a cellular biomarker of necrotizing bronchiolitis in influenza A virus infection

Influenza A virus (IAV) in the human and swine host infects epithelial cells lining the respiratory tract causing a necrotizing bronchitis and bronchiolitis. These epithelial surfaces are protected by large glycoproteins called mucins. Mucin 4 (MUC4) is a transmembrane mucin that consists of an alpha subunit responsible for surface protection and intracellular beta subunit involved in signal transduction which repress apoptosis and stimulate epithelial proliferation. This study was designed to determine the expression and potential role of MUC4 during IAV infection. We used immunohistochemistry in combination with machine learning image analysis to quantify differential protein expression of MUC4 subunits in IAV-infected and uninfected lung in a porcine model. MUC4 protein basal expression in control animals varied significantly by litter. MUC4 protein expression was significantly increased in bronchioles with necrotizing bronchiolitis compared to histologically normal bronchioles, likely representing a regenerative response to restore mucosal integrity of conducting airways. Understanding the impact of differential MUC4 expression among healthy individuals and during IAV infection will facilitate control strategies by elucidating mechanisms associated with susceptibility to IAV that can be therapeutically or genetically regulated and may be extended to other respiratory diseases.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2022

Autodifferentiable Ensemble Kalman Filters

Data assimilation is concerned with sequentially estimating a temporally evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Here, our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.

autodifferentiation↗
DOE OSTI2022

Machine Learning Models for Binary Molecular Classification using VUV Absorption Spectra

Machine learning methods were combined with differential absorption spectroscopy measurements in the vacuum-ultraviolet region (5.167 – 9.920 eV) in order to develop predictive capabilities for inferring molecular structure from the spectra. Several types of species were analyzed and, for modeling purposes, were defined using a single classification: (1) alkane, (2) conjugation with oxygen (e.g. diacetyl, ethyl vinyl ether), (3) non-conjugated alkene (e.g. 1-butene, 1,4-cyclohexadiene), (4) oxygen-containing (e.g. 1-butanol, tetrahydrofuran), or (5) cyclic (e.g. cyclopentane, cyclohexanone). The latter molecular classification excluded cyclic ethers. Several modeling methods were employed in the analysis of 102 absorption spectra, 24 of which were measured for the first time. The primary objective was to identify suitable methods that enable accurate predictions of molecular structure classifications with minimized statistical uncertainties. Rather than identifying a single, unifying method to reliably predict molecular structure contributions to VUV absorption spectra, coordination is required among a particular method, the type of molecular structure detail (e.g. conjugation), and absorption region of interest. The latter is accomplished using a binning approach, wherein absorption regions of ~0.5 eV were utilized rather than the entire ~4.8 eV range. Photon energy binning enabled analysis of region-specific predictions of accuracy, precision, and recall. The outcome from the binning approach is that, rather than utilizing the entire spectrum, optimal determination of molecular structure using machine learning methods depends on the absorption region. Furthermore, the present work provides separate machine learning models for each molecular classification, which enables the identification of multi-functional species relevant to atmospheric chemistry and combustion chemistry, where isomer-resolved speciation is critical to understanding complex reaction networks.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Neuromorphic Computing is Turing-Complete

Neuromorphic computing is a non-von Neumann computing paradigm that performs computation by emulating the human brain. Neuromorphic systems are extremely energy-efficient and known to consume thousands of times less power than CPUs and GPUs. They have the potential to drive critical use cases such as autonomous vehicles, edge computing and internet of things in the future. For this reason, they are sought to be an indispensable part of the future computing landscape. Neuromorphic systems are mainly used for spike-based machine learning applications, although there are some non-machine learning applications in graph theory, differential equations, and spike-based simulations. These applications suggest that neuromorphic computing might be capable of general-purpose computing. However, general-purpose computability of neuromorphic computing has not been established yet. In this work, we prove that neuromorphic computing is Turing-complete and therefore capable of general-purpose computing. Specifically, we present a model of neuromorphic computing, with just two neuron parameters (threshold and leak), and two synaptic parameters (weight and delay). We devise neuromorphic circuits for computing all the μ-recursive functions (i.e., constant, successor and projection functions) and all the μ-recursive operators (i.e., composition, primitive recursion and minimization operators). Given that the μ-recursive functions and operators are precisely the ones that can be computed using a Turing machine, this work establishes the Turing-completeness of neuromorphic computing.

Date, Prasanna↗