Search NASA⌕ Search

SEARCH · Search NASA

Results for “Differentiable neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Fourier Neural Networks as Function Approximators and Differential Equation Solvers

We present a Fourier neural network (FNN) that can be mapped directly to the Fourier decomposition. The choice of activation and loss function yields results that replicate a Fourier series expansion closely while preserving a straightforward architecture with a single hidden layer. The simplicity of this network architecture facilitates the integration with any other higher-complexity networks, at a data pre- or postprocessing stage. We validate this FNN on naturally periodic smooth functions and on piecewise continuous periodic functions. We showcase the use of this FNN for modeling or solving partial differential equations with periodic boundary conditions. The main advantages of the current approach are the validity of the solution outside the training region, interpretability of the trained model, and simplicity of use.

Fourier decomposition↗
DOE OSTI2023

Phase Space Reconstruction from Accelerator Beam Measurements Using Neural Networks and Differentiable Simulations

Characterizing the phase space distribution of particle beams in accelerators is a central part of accelerator understanding and performance optimization. However, conventional reconstruction-based techniques either use simplifying assumptions or require specialized diagnostics to infer high-dimensional (> $2D$) beam properties. In this Letter, we introduce a general-purpose algorithm that combines neural networks with differentiable particle tracking to efficiently reconstruct high-dimensional phase space distributions without using specialized beam diagnostics or beam manipulations. Furthermore, we demonstrate that our algorithm accurately reconstructs detailed 4D phase space distributions with corresponding confidence intervals in both simulation and experiment using a single focusing quadrupole and diagnostic screen. This technique allows for the measurement of multiple correlated phase spaces simultaneously, which will enable simplified 6D phase space distribution reconstructions in the future.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Differentiable hybrid neural network approach for enhancing reactor dynamics simulations

Reactor dynamics simulations provide essential insights into the time-dependent behavior of nuclear reactors under various operating conditions. However, high-fidelity simulations can be computationally intensive, requiring significant computational resources. Here, to address this challenge, this study employs a differentiable hybrid model that utilizes neural networks as a corrector to enhance the performance of a low-fidelity simulation, aligning its predictions with those of a high-fidelity simulation. Low-fidelity and high-fidelity simulations were obtained by adjusting the mesh size in the System Dynamics Analysis Tool. The differentiable hybrid model was trained in two approaches: time-step-wise and sequence-wise. It was then applied to simulate various transients in a molten salt reactor. Its performance was evaluated by comparing its responses to transients against those of the high-fidelity simulation. An additional approach was performed using a data-driven model to correct the low-fidelity simulation. In comparison, the differentiable hybrid model showed significant improvements in transient prediction, effectively addressing the limitations of the low-fidelity simulations. The results highlighted the robustness of the differentiable hybrid model in both training approaches. It delivered simulations that were at least 3.8 times faster than high-fidelity models. In the time-step-wise approach, it achieved at least a 39% improvement in accuracy. In the sequence-wise approach, it showed at least an 81% accuracy improvement over the full transient. This approach offers a promising path for improving computational efficiency without compromising accuracy in nuclear reactor simulations, making it suitable for real-time digital twin applications.

42 - ENGINEERING↗
DOE OSTI2021

Recovering the phase and amplitude of X-ray FEL pulses using neural networks and differentiable models

Dynamics experiments are an important use-case for X-ray free-electron lasers (XFELs), but time-domain measurements of the X-ray pulses themselves remain a challenge. Shot-by-shot X-ray diagnostics could enable a new class of simpler and potentially higher-resolution pump-probe experiments. Here, we report training neural networks to combine low-resolution measurements in both the time and frequency domains to recover X-ray pulses at high-resolution. Critically, we also recover the phase, opening the door to coherent-control experiments with XFELs. The model-based generative neural-network architecture can be trained directly on unlabeled experimental data and is fast enough for real-time analysis on the new generation of MHz XFELs.

42 ENGINEERING↗
DOE OSTI2024

Four-dimensional phase-space reconstruction of flat and magnetized beams using neural networks and differentiable simulations

Beams with cross-plane coupling or extreme asymmetries between the two transverse phase spaces are often encountered in particle accelerators. Flat beams with large transverse-emittance ratios are critical for future linear colliders. Similarly, magnetized beams with significant cross-plane coupling are expected to enhance the performance of electron cooling in hadron beams. Preparing these beams requires precise control and characterization of the four-dimensional transverse phase space. In this study, we employ generative phase-space reconstruction techniques to rapidly characterize magnetized and flat-beam phase-space distributions using a conventional quadrupole-scan method. The reconstruction technique is experimentally demonstrated on an electron beam produced at the Argonne Wakefield Accelerator and successfully benchmarked against conventional diagnostics techniques. Specifically, we show that predicted beam parameters from the reconstructed phase-space distributions (e.g., as magnetization and flat-beam emittances) are in excellent agreement with those measured from the conventional diagnostic methods. Published by the American Physical Society 2024

43 PARTICLE ACCELERATORS↗
DOE OSTI2020

Solving differential equations using deep neural networks

Recent work on solving partial differential equations (PDEs) with deep neural networks (DNNs) is presented. The paper reviews and extends some of these methods while carefully analyzing a fundamental feature in numerical PDEs and nonlinear analysis: irregular solutions. First, the Sod shock tube solution to the compressible Euler equations is discussed and analyzed. This analysis includes a comparison of a DNN-based approach with conventional finite element and finite volume methods, and demonstrates that the DNN is competitive in terms of degrees of freedom required for a given accuracy. Further, the DNN-based approach is extended to consider performance improvements and simultaneous parameter space exploration. Next, a shock solution to compressible magnetohydrodynamics (MHD) is solved for, and used in a scenario where experimental data is utilized to enhance a PDE system that is a priori insufficient to validate against the observed/experimental data. This is accomplished by enriching the model PDE system with source terms that are then inferred via supervised training with synthetic experimental data. The resulting DNN framework for PDEs enables straightforward system prototyping and natural integration of large data sets (be they synthetic or experimental), all while simultaneously enabling single-pass exploration of an entire parameter space.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Two-Scale Neural Networks for Partial Differential Equations with Small Parameters

We propose a two-scale neural network method for solving partial differential equations (PDEs) with small parameters using physics-informed neural networks (PINNs). We directly incorporate the small parameters into the architecture of neural networks. The proposed method enables solving PDEs with small parameters in a simple fashion, without adding Fourier features or other computationally taxing searches of truncation parameters. Various numerical examples demonstrate reasonable accuracy in capturing features of large derivatives in the solutions caused by small parameters.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Reduced basis approximations of parameterized dynamical partial differential equations via neural networks

Projection-based reduced order models are effective at approximating parameter-dependent differential equations that are parametrically separable. When parametric separability is not satisfied, which occurs in both linear and nonlinear problems, projection-based methods fail to adequately reduce the computational complexity. Devising alternative reduced order models is crucial for obtaining efficient and accurate approximations to expensive high-fidelity models. In this work, we develop a timestepping procedure for dynamical parameter-dependent problems, in which a neural-network is trained to propagate the coefficients of a reduced basis expansion. This results in an online stage with a computational cost independent of the size of the underlying problem. Here, we demonstrate our method on several parabolic partial differential equations, including a problem that is not parametrically separable.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

HomPINNs: Homotopy physics-informed neural networks for learning multiple solutions of nonlinear elliptic differential equations

Physics-informed neural networks (PINNs) based machine learning is an emerging framework for solving nonlinear differential equations. However, due to the implicit regularity of neural network structure, PINNs can only find the flattest solution in most cases by minimizing the loss functions. In this paper, we combine PINNs with the homotopy continuation method, a classical numerical method to compute isolated roots of polynomial systems, and propose a new deep learning framework, named homotopy physics-informed neural networks (HomPINNs), for solving multiple solutions of nonlinear elliptic differential equations. The implementation of an HomPINN is a homotopy process that is composed of the training of a fully connected neural network, named the starting neural network, and training processes of several PINNs with different tracking parameters. The starting neural network is to approximate a starting function constructed by the trivial solutions, while other PINNs are to minimize the loss functions defined by boundary condition and homotopy functions, varying with different tracking parameters. These training processes are regraded as different steps of a homotopy process, and a PINN is initialized by the well-trained neural network of the previous step, while the first starting neural network is initialized using the default initialization method. Finally, several numerical examples are presented to show the efficiency of our proposed HomPINNs, including reaction-diffusion equations with a heart-shaped domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Approximation of refrigerant thermophysical properties using neural networks to speed up transient thermofluid simulations

Accurate and efficient evaluations of refrigerant thermophysical properties and their partial derivatives are essential for transient simulations of thermofluid systems, where several computations need to be executed at each integration time step. Since the utilization of an Equation of State for retrieving properties based on a pair of independent inputs typically involves numerical iterations in solution procedures, when the input variables differ from the refrigerant state variables employed in dynamic models, a variety of approaches including lookup table interpolation and curve fitting have been developed to explicitly approximate these properties based on the state variables, and consequently eliminate internal iterations. This paper presents an alternative method that exploits derivative-informed neural networks to model refrigerant properties explicitly from inputs of pressure and enthalpy, while ensuring consistent partial derivatives generated by differentiating the neural networks. Computational speed and accuracy of the proposed approach are demonstrated via transient simulations of a discretized heat exchanger model in Modelica, and comparisons against other property evaluation routines. Simulation results indicate that the proposed approach can realize a significant speedup with negligible discrepancies in predicted transients. The method is implemented in an open-source Modelica library.

Ma, Jiacheng↗
DOE OSTI2024

Uncertainty quantification of graph convolution neural network models of evolving processes

The application of neural network models to scientific machine learning tasks has proliferated in recent years. In particular, neural networks have proved to be adept at modeling processes with spatial–temporal complexity. Nevertheless, these highly parameterized models have garnered skepticism in their ability to produce outputs with quantified error bounds over the regimes of interest. Hence there is a need to find uncertainty quantification methods that are suitable for neural networks. In this work we present comparisons of the parametric uncertainty quantification of neural networks modeling complex spatial–temporal processes with Hamiltonian Monte Carlo and Stein variational gradient descent and its projected variant. Specifically we apply these methods to graph convolutional neural network models of evolving systems modeled with recurrent neural network and neural ordinary differential equations architectures. We show that Stein variational inference is a viable alternative to Monte Carlo methods with some clear advantages for complex neural network models. For our exemplars, Stein variational interference gave similar pushed forward uncertainty profiles through time compared to Hamiltonian Monte Carlo, albeit with generally more generous variance. As a result, projected Stein variational gradient descent also produced similar uncertainty profiles to the non-projected counterpart, but large reductions in the active weight space were confounded by the stability of the neural network predictions and the convoluted likelihood landscape.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Physics-informed Karhunen-Loeve and Neural Network Approximations for Solving Inverse Differential Equation Problems

Here we present the PI-CKL-NN method for parameter estimation in differential equation (DE) models given sparse measurements of the parameters and states. In the proposed approach, the space- or time-dependent parameters are approximated by Karhunen-Loeve (KL) expansions that are conditioned on the parameters’ measurements, and the states are approximated by deep neural networks (DNNs). The unknown weights in the KL expansions and DNNs are found my minimizing the cost function that enforces the measurements of the states the DE constraint. Regularization is achieved by adding the l2 norm of the conditional KL coefficients into the loss function. Our approach assumes that the parameter fields are correlated in space or time and enforces the statistical knowledge (the mean and the covariance function) in addition to the DE constraints and measurements as opposed to the physics-informed neural network (PINN) and other similar physics-informed machine learning methods where only DE constraints and data are used for parameter estimation. We use the PI-CKL-NN method for parameter estimation in an ordinary differential equation with an unknown time-dependent parameter and the one- and two-dimensional partial differential diffusion equations with unknown space-dependent diffusion coefficients. We also demonstrate that PI-CKL-NN is more accurate than the PINN method, especially when the observations of the parameters are very sparse

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Solving inverse problems in stochastic models using deep neural networks and adversarial training

Inverse problems associated with stochastic models constitute a significant portion of scientific and engineering applications. In such cases the unknown quantities are distributions. The applicability of traditional methods is limited because of their demanding assumptions or prohibitive computational consumption; for example, maximum likelihood methods require closed-form density functions, and Markov Chain Monte Carlo needs a large number of simulations. We propose a new method that estimates the unknown distribution by matching the statistical properties between observed and simulated random processes. We leverage the expressive power of neural networks to approximate the unknown distribution and use a discriminative neural network for computing the statistical discrepancies between the observed and simulated random processes. Here we demonstrated numerically that the proposed methods can estimate both the model parameters and learn complicated unknown distributions.

42 ENGINEERING↗
DOE OSTI2021

Rapid 3D nanoscale coherent imaging via physics-aware deep learning

Phase retrieval, the problem of recovering lost phase information from measured intensity alone, is an inverse problem that is widely faced in various imaging modalities ranging from astronomy to nanoscale imaging. The current process of phase recovery is iterative in nature. As a result, the image formation is time consuming and computationally expensive, precluding real-time imaging. Here, we use 3D nanoscale X-ray imaging as a representative example to develop a deep learning model to address this phase retrieval problem. We introduce 3D-CDI-NN, a deep convolutional neural network and differential programing framework trained to predict 3D structure and strain, solely from input 3D X-ray coherent scattering data. Our networks are designed to be “physics-aware” in multiple aspects; in that the physics of the X-ray scattering process is explicitly enforced in the training of the network, and the training data are drawn from atomistic simulations that are representative of the physics of the material. We further refine the neural network prediction through a physics-based optimization procedure to enable maximum accuracy at lowest computational cost. 3D-CDI-NN can invert a 3D coherent diffraction pattern to real-space structure and strain hundreds of times faster than traditional iterative phase retrieval methods. Our integrated machine learning and differential programing solution to the phase retrieval problem is broadly applicable across inverse problems in other application areas.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
DOE OSTI2021

Mixed Precision Fermi-Operator Expansion on Tensor Cores from a Machine Learning Perspective

Here we present a second-order recursive Fermi-operator expansion scheme using mixed precision floating point operations to perform electronic structure calculations using tensor core units. A performance of over 100 teraFLOPs is achieved for half-precision floating point operations on Nvidia’s A100 tensor core units. The second-order recursive Fermi-operator scheme is formulated in terms of a generalized, differentiable deep neural network structure, which solves the quantum mechanical electronic structure problem. We demonstrate how this network can be accelerated by optimizing the weight and bias values to substantially reduce the number of layers required for convergence. We also show how this machine learning approach can be used to optimize the coefficients of the recursive Fermi-operator expansion to accurately represent the fractional occupation numbers of the electronic states at finite temperatures.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗