Search NASA⌕ Search

SEARCH · Search NASA

Results for “Automatic Differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11

NSFnets (Navier-Stokes flow nets): Physics-informed neural networks for the incompressible Navier-Stokes equations

In the last 50 years there has been a tremendous progress in solving numerically the Navier-Stokes equations using finite differences, finite elements, spectral, and even meshless methods. Yet, in many real cases, we still cannot incorporate seamlessly (multi-fidelity) data into existing algorithms, and for industrial-complexity applications the mesh generation is time consuming and still an art. Moreover, solving ill-posed problems (e.g., lacking boundary conditions) or inverse problems is often prohibitively expensive and requires different formulations and new computer codes. Here, we employ physics-informed neural networks (PINNs), encoding the governing equations directly into the deep neural network via automatic differentiation, to overcome some of the aforementioned limitations for simulating incompressible laminar and turbulent flows. We develop the Navier-Stokes flow nets (NSFnets) by considering two different mathematical formulations of the Navier-Stokes equations: the velocity-pressure (VP) formulation and the vorticity-velocity (VV) formulation. Since this is a new approach, we first select some standard benchmark problems to assess the accuracy, convergence rate, computational cost and flexibility of NSFnets; analytical solutions and direct numerical simulation (DNS) databases provide proper initial and boundary conditions for the NSFnet simulations. The spatial and temporal coordinates are the inputs of the NSFnets, while the instantaneous velocity and pressure fields are the outputs for the VP-NSFnet, and the instantaneous velocity and vorticity fields are the outputs for the VV-NSFnet. This is unsupervised learning and, hence, no labeled data are required beyond boundary and initial conditions and the fluid properties. The residuals of the VP or VV governing equations, together with the initial and boundary conditions, are embedded into the loss function of the NSFnets. No data is provided for the pressure to the VP-NSFnet, which is a hidden state and is obtained via the incompressibility constraint without extra computational cost. Unlike the traditional numerical methods, NSFnets inherit the properties of neural networks (NNs), hence the total error is composed of the approximation, the optimization, and the generalization errors. Here, we empirically attempt to quantify these errors by varying the sampling (“residual”) points, the iterative solvers, and the size of the NN architecture. For the laminar flow solutions, we show that both the VP and the VV formulations are comparable in accuracy but their best performance corresponds to different NN architectures. The initial convergence rate is fast but the error eventually saturates to a plateau due to the dominance of the optimization error. For the turbulent channel flow, we show that NSFnets can sustain turbulence at , but due to expensive training we only consider part of the channel domain and enforce velocity boundary conditions on the subdomain boundaries provided by the DNS data base. We also perform a systematic study on the weights used in the loss function for balancing the data and physics components, and investigate a new way of computing the weights dynamically to accelerate training and enhance accuracy. In the last part, we demonstrate how NSFnets should be used in practice, namely for ill-posed problems with incomplete or noisy boundary conditions as well as for inverse problems. We obtain reasonably accurate solutions for such cases as well without the need to change the NSFnets and at the same computational cost as in the forward well-posed problems. As a result, we also present a simple example of transfer learning that will aid in accelerating the training of NSFnets for different parameter settings.

97 MATHEMATICS AND COMPUTING↗

Learning functional priors and posteriors from data and physics

In this work, we develop a new Bayesian framework based on deep neural networks to be able to extrapolate in space-time using historical data and to quantify uncertainties arising from both noisy and gappy data in physical problems. Specifically, the proposed approach has two stages: (1) prior learning and (2) posterior estimation. At the first stage, we employ the physics-informed Generative Adversarial Networks (PI-GAN) to learn a functional prior either from a prescribed function distribution, e.g., Gaussian process, or from historical data and physics. At the second stage, we employ the Hamiltonian Monte Carlo (HMC) method to estimate the posterior in the latent space of PI-GANs. In addition, we use two different approaches to encode the physics: (1) automatic differentiation, used in the physicsinformed neural networks (PINNs) for scenarios with explicitly known partial differential equations (PDEs), and (2) operator regression using the deep operator network (DeepONet) for PDE-agnostic scenarios. We then test the proposed method for (1) meta-learning for one-dimensional regression, and forward/inverse PDE problems (combined with PINNs); (2) PDE-agnostic physical problems (combined with DeepONet), e.g., fractional diffusion as well as saturated stochastic (100-dimensional) flows in heterogeneous porous media; and (3) spatial-temporal regression problems, i.e., inference of a marine riser displacement field using experimental data from the Norwegian Deepwater Programme (NDP). The results demonstrate that the proposed approach can provide accurate predictions as well as uncertainty quantification given very limited scattered and noisy data, since historical data could be available to provide informative priors. In summary, the proposed method is capable of learning flexible functional priors, e.g., both Gaussian and non-Gaussian process, and can be readily extended to big data problems by enabling mini-batch training using stochastic HMC or normalizing flows since the latent space is generally characterized as low dimensional.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

NEML2: An efficient and modular multiphysics constitutive modeling library for hybrid computing environments

This paper presents NEML2, an open-source, high-performance library developed for constitutive material modeling, designed to support the flexible and modular development of models for complex material behavior. Building on the foundational structure of its predecessor, NEML, the NEML2 library introduces significant improvements, including enhanced vectorization, automatic differentiation, and seamless integration with PyTorch, facilitating the application of machine learning techniques in material simulations. NEML2 provides a C++ backend with Python bindings, enabling users to create custom material models that can be executed efficiently on both CPU and GPU platforms. The library also supports coupling with Multiphysics simulation frameworks like MOOSE, making it suitable for realistic simulations involving coupled physical processes. Rigorous quality assurance through unit and regression testing ensures the reliability of results, while the extensible, user-friendly design encourages collaboration and reproducibility across the scientific community. This paper provides an overview of NEML2’s architecture, core features, and applications, highlighting its impact on accelerating material qualification and advancing computational methods in materials science.

GPU↗

Scalable algorithms for physics-informed neural and graph networks

Physics-informed machine learning (PIML) has emerged as a promising new approach for simulating complex physical and biological systems that are governed by complex multiscale processes for which some data are also available. In some instances, the objective is to discover part of the hidden physics from the available data, and PIML has been shown to be particularly effective for such problems for which conventional methods may fail. Unlike commercial machine learning where training of deep neural networks requires big data, in PIML big data are not available. Instead, we can train such networks from additional information obtained by employing the physical laws and evaluating them at random points in the space–time domain. Such PIML integrates multimodality and multifidelity data with mathematical models, and implements them using neural networks or graph networks. Here, we review some of the prevailing trends in embedding physics into machine learning, using physics-informed neural networks (PINNs) based primarily on feed-forward neural networks and automatic differentiation. For more complex systems or systems of systems and unstructured data, graph neural networks (GNNs) present some distinct advantages, and here we review how physics-informed learning can be accomplished with GNNs based on graph exterior calculus to construct differential operators; we refer to these architectures as physics-informed graph networks (PIGNs). We present representative examples for both forward and inverse problems and discuss what advances are needed to scale up PINNs, PIGNs and more broadly GNNs for large-scale engineering problems.

42 ENGINEERING↗

The DESC stellarator code suite Part 3: Quasi-symmetry optimization

The DESC stellarator optimization code takes advantage of advanced numerical methods to search the full parameter space much faster than conventional tools. Only a single equilibrium solution is needed at each optimization step thanks to automatic differentiation, which efficiently provides exact derivative information. A Gauss–Newton trust-region optimization method uses second-order derivative information to take large steps in parameter space and converges rapidly. With just-in-time compilation and GPU portability, high-dimensional stellarator optimization runs take orders of magnitude less computation time with DESC compared to other approaches. This paper presents the theory of the DESC fixed-boundary local optimization algorithm along with demonstrations of how to easily implement it in the code. Example quasi-symmetry optimizations are shown and compared to results from conventional tools. Three different forms of quasi-symmetry objectives are available in DESC, and their relative advantages are discussed in detail. In the examples presented, the triple product formulation yields the best optimization results in terms of minimized computation time and particle transport. This paper concludes with an explanation of how the modular code suite can be extended to accommodate other types of optimization problems.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

The DESC stellarator code suite. Part 2. Perturbation and continuation methods

A new perturbation and continuation method is presented for computing and analysing stellarator equilibria. The method is formally derived from a series expansion about the equilibrium condition $\boldsymbol {F} \equiv \boldsymbol {J}\times \boldsymbol {B} - \boldsymbol {\nabla } p = 0$ , and an efficient algorithm for computing solutions to second- and third-order perturbations is developed. The method has been implemented in the DESC stellarator equilibrium code, using automatic differentiation to compute the required derivatives. Examples are shown demonstrating its use for computing complicated equilibria, perturbing a tokamak into a stellarator and performing parameter scans in pressure, rotational transform and boundary shape in a fraction of the time required for a full solution.

Conlin, Rory (ORCID:0000000183662111)↗

A Unified Workflow for Sensitivity-Based Kinetic Analysis in Microkinetic Models

Degrees of rate control (DRC), apparent activation energies, and apparent reaction orders are established local sensitivity diagnostics for interpreting microkinetic models, but applying them routinely to large mechanisms often requires substantial reaction-specific bookkeeping, perturbation design, and postprocessing. Here, in this study, we present a unified derivative-based workflow that evaluates these quantities from a single compiled reaction-network model and target-rate definition. For any user-provided microkinetic model, the workflow compiles the mechanism into stoichiometrically consistent mass-action rate equations, solves the surface dynamics, and uses automatic differentiation to compute sensitivities with respect to rate constants, temperature, and gas partial pressures. By combining their calculations in the same framework, the workflow clearly demonstrates the relationships between different DRCs and the apparent activation energy. Using existing examples of propylene partial oxidation and methane oxidation on Pd(100), we verify expected transient redistribution of rate control, distinguish net Campbell DRCs from one-sided directional sensitivities, and show how apparent activation energy can be reconstructed either from one-sided DRCs or from state-based DRCs while critical mechanistic insights are obtained consistently. In the methane oxidation case, a pathway-subset test further illustrates how a simplified mechanism preserves key kinetic signatures of a full model, showing the potential of our user-friendly tool for model construction beyond kinetic analysis.

36 MATERIALS SCIENCE↗

Recent Advances toward Efficient Calculation of Higher Nuclear Derivatives in Quantum Chemistry

In this article, we provide an overview of state-of-the-art techniques that are being developed for efficient calculation of second and higher nuclear derivatives of quantum mechanical (QM) energy. Calculations of nuclear Hessians and anharmonic terms incur high costs and memory and scale poorly with system size. Three emerging classes of methods—machine learning (ML), automatic differentiation (AD), and matrix completion (MC)—have demonstrated promise in overcoming these challenges. We illustrate studies that employ unsupervised ML methods to reduce the need for multiple Hessian calculations in dynamics simulations and those that utilize supervised ML to construct approximate potential energy surfaces and estimate Hessians and anharmonic terms at reduced cost. By extension, if electronic structure operations could be written in a manner similar to functions underlying ML methods, rapid differentiation or AD routines can be employed to inexpensively calculate higher arbitrary-order derivatives. While ML approaches are typically black-box, we describe methods such as compressed sensing (CS) and MC, which explicitly leverage problem-specific mathematical properties of higher derivatives such as sparsity and low-rank, to complete higher derivative information using only a small, incomplete sample. The three classes of methods facilitate reliable predictions of observables ranging from infrared spectra to thermal conductivity and constitute a promising way forward in accurately capturing otherwise intractable higher-order responses of QM energy to nuclear perturbations.

38 RADIATION CHEMISTRY, RADIOCHEMISTRY, AND NUCLEA↗

Adaptive Data-Driven Deep-Learning Surrogate Model for Frontal Polymerization in Dicyclopentadiene

Frontal polymerization (FP) is a self-sustaining curing process that enables rapid and energy-efficient manufacturing of thermoset polymers and composites. Computational methods conventionally used to simulate the FP process are time-consuming, and repeating simulations are required for sensitivity analysis, uncertainty quantification, or optimization of the manufacturing process. Here, in this work, we develop an adaptive surrogate deep-learning model for FP of dicyclopentadiene (DCPD), which predicts the evolution of temperature and degree of cure orders of magnitude faster than the finite-element method (FEM). The adaptive algorithm provides a strategy to select training samples efficiently and save computational costs by reducing the redundancy of FEM-based training samples. The adaptive algorithm calculates the residual error of the FP governing equations using automatic differentiation of the deep neural network. A probability density function expressed in terms of the residual error is used to select training samples from the Sobol sequence space. The temperature and degree of cure evolution of each training sample are obtained by a 2D FEM simulation. The adaptive method is more efficient and has a better prediction accuracy than the random sampling method. With the well-trained surrogate neural network, the FP characteristics (front speed, shape, and temperature) can be extracted quickly from the predicted temperature and degree-of-cure fields.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

High-Performance Semiempirical Excited-State Molecular Dynamics Powered by Graphics Processing Units

Here, this Letter introduces excited-state molecular dynamics in PYSEQM, a GPU-accelerated semiempirical quantum chemistry engine implemented in PyTorch. The new module enables Born–Oppenheimer molecular dynamics (BOMD) using configuration-interaction singles and random phase approximation for excited states, allowing long trajectories and large statistical ensembles to be simulated efficiently on a single GPU. We also implement an extended Lagrangian excited-state BOMD (XL-ESMD) scheme that propagates auxiliary electronic variables, enabling relaxed ground and excited-state convergence thresholds without compromising energy conservation. The excited-state BOMD implementation scales smoothly from small chromophores to a nearly 900-atom dendrimer (taking 6.5 s per MD step). PYSEQM also supports batched execution, allowing many geometries or trajectories to be evaluated in a single GPU launch, substantially increasing throughput and making ensemble-based protocols routine. As a demonstration, we compute absorption, emission, and infrared spectra from trajectories propagated on the ground and first excited states. The XL-ESMD scheme yields identical spectra at significantly lower computational cost, establishing the role of extended Lagrangian based dynamics for efficient excited-state BOMD simulations. Beyond raw performance, PYSEQM’s PyTorch foundation provides automatic differentiation for forces, efficient GPU batching, and seamless interfacing with machine learning models. These capabilities position PYSEQM as a practical platform for machine learning-augmented excited-state dynamics and lay the foundation for future data-driven nonadiabatic excited-state dynamics modeling of ultrafast spectroscopic probes.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Single-shot x-ray phase-contrast and dark-field imaging based on coded binary phase mask

We introduce a coded-mask-based multi-contrast imaging method for high-resolution phase-contrast and dark-field imaging. The method uses a binary phase mask designed to provide an ultra-high-contrast pattern and reference-free single-shot measurement and an algorithm based on maximum-likelihood optimization and automatic differentiation to perform simultaneous reconstruction of absorption, phase, and dark-field object images. Further, we demonstrate that the method has great potential for real-time quantitative phase imaging and wavefront sensing when combined with deep learning.

Qiao, Zhi (ORCID:0000000286285320)↗

Quantitative phase retrieval and characterization of magnetic nanostructures via Lorentz (scanning) transmission electron microscopy

Magnetic materials phase reconstruction using Lorentz transmission electron microscopy (LTEM) measurements have traditionally been achieved using longstanding methods such as off-axis holography (OAH) fast-Fourier transform technique and the transport-of-intensity equation (TIE). The increase in access to processing power alongside the development of advanced algorithms have allowed for phase retrieval of nanoscale magnetic materials with greater efficacy and resolution. Specifically, reverse-mode automatic differentiation (RMAD) and the extended electron ptychography iterative engine (ePIE) are two recent developments of phase retrieval that can be applied to analyzing micro-to-nano- scale magnetic materials. This work evaluates phase retrieval using TIE, RMAD, and ePIE in simulations of Permalloy (Ni 80 Fe 20 ) nanoscale islands, or nanomagnets. Extending beyond simulations, we demonstrate total phase retrieval and image reconstructions of a NiFe nanowire using OAH and RMAD in LTEM and ePIE in Lorentz-mode-4D scanning transmission electron microscopy experiments and determine the saturation magnetization through corroborations with micromagnetic modeling. Finally, we demonstrate the efficacy of these methods in retrieving the total phase and highlight its use in characterizing and analyzing the proximity effect of the magnetic nanostructures.

Lorentz transmission electron microscopy↗

Neural network reconstruction of the dense matter equation of state from neutron star observables

The Equation of State (EoS) of strongly interacting cold and hot ultra-dense QCD matter remains a major challenge in the field of nuclear astrophysics. With the advancements in measurements of neutron star masses, radii, and tidal deformabilities, from electromagnetic and gravitational wave observations, neutron stars play an important role in constraining the ultra-dense QCD matter EoS. Here, in this work, we present a novel method that exploits deep learning techniques to reconstruct the neutron star EoS from mass-radius (M-R) observations. We employ neural networks (NNs) to represent the EoS in a model-independent way, within the range of ~1-7 times the nuclear saturation density. The unsupervised Automatic Differentiation (AD) framework is implemented to optimize the EoS, so as to yield through TOV equations, an M-R curve that best fits the observations. We demonstrate that this method works by rebuilding the EoS on mock data, i.e., mass-radius pairs derived from a randomly generated polytropic EoS. The reconstructed EoS fits the mock data with reasonable accuracy, using just 11 mock M-R pairs observations, close to the current number of actual observations.

79 ASTRONOMY AND ASTROPHYSICS↗

Electromagnetic coil optimization for reduced Lorentz forces

Abstract The reduction of magnetic forces on electromagnetic coils is an important consideration in the design of high-field devices such as the stellarator or tokamak. Unfortunately, these forces may be too time-consuming to evaluate by conventional finite element modeling within an optimization loop. Although mutual forces can be computed rapidly by approximating large-bore coils as infinitely thin, this approximation does not hold for self-forces as it leads to an unphysical divergence. Recently, a novel reduced model for the self-field, self-force, and self-inductance of electromagnetic coils based on filamentary models was rigorously derived and demonstrated to be highly accurate and numerically efficient to evaluate (Hurwitz et al 2024 IEEE Trans. Magn. 60 7001614). In this paper, we present an implementation of the reduced self-force model employing automatic differentiation within the simsopt stellarator design software and use it in derivative-based coil optimization for a quasi-axisymmetric stellarator. We show that it is possible to significantly reduce point-wise forces throughout the coils, though this comes with trade-offs to fast particle losses and the minimum distance between coils and the plasma surface. The trade-off between magnetic forces and coil-surface distance is mediated by the minimum coil–coil distance for coils near the inboard side of the ‘bean’ cross-section of the plasma. The relationship between forces and fast particle losses is mediated by the normal field error. Coil forces can be lowered to a threshold with minimal deterioration to losses. Importantly, the magnet optimization approach here can be used also for tokamaks, other fusion concepts, and applications outside of fusion.

Hurwitz, Siena (ORCID:0000000166599659)↗

Combinatorial algorithms for computational science and engineering.

We discuss recent activities in the SciDAC Applied Mathematics Institute for Combinatorial Scientific Computing and Petascale Simulations (CSCAPES). The Institute for Combinatorial Scientific Computing and Petascale Simulations (CSCAPES) was established in 2006 to help contribute to the SciDAC mission by developing infrastructural algorithmic and software technologies for supporting high-performance computing. Its scope encompasses four areas: load-balancing and related tasks in parallel processing (such as data migration and task scheduling); automatic differentiation (AD), a technology for transforming computer source code for computing a function into code for computing its derivatives; basic ingredients in linear solver technology; and runtime data and iteration reordering to improve performance in irregular computation.

99 GENERAL AND MISCELLANEOUS↗

Optimal control of large quantum systems: assessing memory and runtime performance of GRAPE

Abstract Gradient Ascent Pulse Engineering (GRAPE) is a popular technique in quantum optimal control, and can be combined with automatic differentiation (AD) to facilitate on-the-fly evaluation of cost-function gradients. We illustrate that the convenience of AD comes at a significant memory cost due to the cumulative storage of a large number of states and propagators. For quantum systems of increasing Hilbert space size, this imposes a significant bottleneck. We revisit the strategy of hard-coding gradients in a scheme that fully avoids propagator storage and significantly reduces memory requirements. Separately, we present improvements to numerical state propagation to enhance runtime performance. We benchmark runtime and memory usage and compare this approach to AD-based implementations, with a focus on pushing towards larger Hilbert space sizes. The results confirm that the AD-free approach facilitates the application of optimal control for large quantum systems which would otherwise be difficult to tackle.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Extending the Takagi-Taupin equations for x-ray nanobeam Bragg coherent diffraction

We present an approach for simulating x-ray nanobeam Bragg coherent diffraction patterns based on the Takagi-Taupin equations. Compared to conventional methods, the current approach can be universally applied to any weakly strained system including semi-infinite crystals that diffract dynamically. It addresses issues such as the curved wave front and redivergence of the focused incident beam. In conclusion, we show excellent agreement with experimental data for a strained La 0.7 ⁢Sr 0.3⁢ MnO 3 thin film on a SrTiO 3 substrate and a path to extracting physical information using automatic differentiation.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Physics-Informed Neural Networks for Heat Transfer Problems

Abstract Physics-informed neural networks (PINNs) have gained popularity across different engineering fields due to their effectiveness in solving realistic problems with noisy data and often partially missing physics. In PINNs, automatic differentiation is leveraged to evaluate differential operators without discretization errors, and a multitask learning problem is defined in order to simultaneously fit observed data while respecting the underlying governing laws of physics. Here, we present applications of PINNs to various prototype heat transfer problems, targeting in particular realistic conditions not readily tackled with traditional computational methods. To this end, we first consider forced and mixed convection with unknown thermal boundary conditions on the heated surfaces and aim to obtain the temperature and velocity fields everywhere in the domain, including the boundaries, given some sparse temperature measurements. We also consider the prototype Stefan problem for two-phase flow, aiming to infer the moving interface, the velocity and temperature fields everywhere as well as the different conductivities of a solid and a liquid phase, given a few temperature measurements inside the domain. Finally, we present some realistic industrial applications related to power electronics to highlight the practicality of PINNs as well as the effective use of neural networks in solving general heat transfer problems of industrial complexity. Taken together, the results presented herein demonstrate that PINNs not only can solve ill-posed problems, which are beyond the reach of traditional computational methods, but they can also bridge the gap between computational and experimental heat transfer.

Engineering↗