Search NASA⌕ Search

SEARCH · Search NASA

Results for “automatic differentiation (AD)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1998

Optimization Issues with Complex Rotorcraft Comprehensive Analysis

This paper investigates the use of the general purpose automatic differentiation (AD) tool called Automatic Differentiation of FORTRAN (ADIFOR) as a means of generating sensitivity derivatives for use in Boeing Helicopter's proprietary comprehensive rotor analysis code (VII). ADIFOR transforms an existing computer program into a new program that performs a sensitivity analysis in addition to the original analysis. In this study both the pros (exact derivatives, no step-size problems) and cons (more CPU, more memory) of ADIFOR are discussed. The size (based on the number of lines) of the VII code after ADIFOR processing increased by 70 percent and resulted in substantial computer memory requirements at execution. The ADIFOR derivatives took about 75 percent longer to compute than the finite-difference derivatives. However, the ADIFOR derivatives are exact and are not functions of step-size. The VII sensitivity derivatives generated by ADIFOR are compared with finite-difference derivatives. The ADIFOR and finite-difference derivatives are used in three optimization schemes to solve a low vibration rotor design problem.

Walsh, Joanne L.↗
NASA NTRS1999

Demonstration of Automatically-Generated Adjoint Code for Use in Aerodynamic Shape Optimization

Gradient-based optimization requires accurate derivatives of the objective function and constraints. These gradients may have previously been obtained by manual differentiation of analysis codes, symbolic manipulators, finite-difference approximations, or existing automatic differentiation (AD) tools such as ADIFOR (Automatic Differentiation in FORTRAN). Each of these methods has certain deficiencies, particularly when applied to complex, coupled analyses with many design variables. Recently, a new AD tool called ADJIFOR (Automatic Adjoint Generation in FORTRAN), based upon ADIFOR, was developed and demonstrated. Whereas ADIFOR implements forward-mode (direct) differentiation throughout an analysis program to obtain exact derivatives via the chain rule of calculus, ADJIFOR implements the reverse-mode counterpart of the chain rule to obtain exact adjoint form derivatives from FORTRAN code. Automatically-generated adjoint versions of the widely-used CFL3D computational fluid dynamics (CFD) code and an algebraic wing grid generation code were obtained with just a few hours processing time using the ADJIFOR tool. The codes were verified for accuracy and were shown to compute the exact gradient of the wing lift-to-drag ratio, with respect to any number of shape parameters, in about the time required for 7 to 20 function evaluations. The codes have now been executed on various computers with typical memory and disk space for problems with up to 129 x 65 x 33 grid points, and for hundreds to thousands of independent variables. These adjoint codes are now used in a gradient-based aerodynamic shape optimization problem for a swept, tapered wing. For each design iteration, the optimization package constructs an approximate, linear optimization problem, based upon the current objective function, constraints, and gradient values. The optimizer subroutines are called within a design loop employing the approximate linear problem until an optimum shape is found, the design loop limit is reached, or no further design improvement is possible due to active design variable bounds and/or constraints. The resulting shape parameters are then used by the grid generation code to define a new wing surface and computational grid. The lift-to-drag ratio and its gradient are computed for the new design by the automatically-generated adjoint codes. Several optimization iterations may be required to find an optimum wing shape. Results from two sample cases will be discussed. The reader should note that this work primarily represents a demonstration of use of automatically- generated adjoint code within an aerodynamic shape optimization. As such, little significance is placed upon the actual optimization results, relative to the method for obtaining the results.

Green, Lawrence↗
DOE OSTI2023

Efficient GPU Implementation of Automatic Differentiation for Computational Fluid Dynamics

Many scientific and engineering applications require repeated calculations of derivatives of output functions with respect to input parameters. Automatic Differentiation (AD) is a method that automates derivative calculations and can significantly speed up code development. In Computational Fluid Dynamics (CFD), derivatives of flux functions with respect to state variables (Jacobian) are needed for efficient solutions of the nonlinear governing equations. AD of flux functions on graphics processing units (GPUs) is challenging as flux computations involve many intermediate variables that create high register pressure and require significant memory traffic because of the need to store the derivatives. This paper presents a forward-mode AD method based on multivariate dual numbers that addresses these challenges and simultaneously reduces the floating-point operation count. The dimension of the multivariate dual numbers is optimized for performance. The flux computations are restructured to minimize the number of temporary variables and reduce register pressure. For effective utilization of memory bandwidth, shared memory is used to store the local flux Jacobian. This AD implementation is compared with several other Jacobian implementations on an NVIDIA V100 GPU (V100). For three-dimensional perfect-gas compressible-flow equations implemented in a practical CFD code, the AD implementation of a flux Jacobian based on multivariate dual numbers of dimension 5 outperforms all other GPU AD implementations on V100. Its performance is comparable with the optimized hand-differentiated version. Finally, the implementation achieves 75% of the peak floating-point throughput and 61 % of the peak global device memory bandwidth usage.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1994

Parallel Calculation of Sensitivity Derivatives for Aircraft Design using Automatic Differentiation

Sensitivity derivative (SD) calculation via automatic differentiation (AD) typical of that required for the aerodynamic design of a transport-type aircraft is considered. Two ways of computing SD via code generated by the ADIFOR automatic differentiation tool are compared for efficiency and applicability to problems involving large numbers of design variables. A vector implementation on a Cray Y-MP computer is compared with a coarse-grained parallel implementation on an IBM SP1 computer, employing a Fortran M wrapper. The SD are computed for a swept transport wing in turbulent, transonic flow; the number of geometric design variables varies from 1 to 60 with coupling between a wing grid generation program and a state-of-the-art, 3-D computational fluid dynamics program, both augmented for derivative computation via AD. For a small number of design variables, the Cray Y-MP implementation is much faster. As the number of design variables grows, however, the IBM SP1 becomes an attractive alternative in terms of compute speed, job turnaround time, and total memory available for solutions with large numbers of design variables. The coarse-grained parallel implementation also can be moved easily to a network of workstations.

Bischof, c. H.↗
DOE OSTI2026

Ground and excited state gradients with end-to-end differentiable semiempirical quantum chemistry

Accurate and efficient gradients of molecular energy with respect to nuclear degrees of freedom are essential for geometry optimization and molecular dynamics, including simulations that go beyond the Born–Oppenheimer regime. A common approach involves deriving analytical formulas for new electronic structure methods, which is often conceptually difficult and requires tedious coding. Here, we implement analytical, semi-numerical, and automatic differentiation (AD)-based gradient pathways for semiempirical Hamiltonian models in the PYSEQM software package, leveraging both graphics processing unit (GPU) and central processing unit (CPU) architectures. We further extend these capabilities to excited states calculated using the configuration interaction singles and time-dependent Hartree–Fock ansätze. We benchmark wall time, peak memory usage, and accuracy across three molecular families of varying chemical complexity, including systems of up to a thousand atoms. For ground-state simulations, analytical and AD gradients achieve near-identical GPU runtimes, while semi-numerical gradients are slower on GPU but remain competitive on CPU. For excited states, both analytical and custom AD approaches using implicit differentiation show similar performance and low memory requirements, whereas gradients with full AD are memory-limited. AD gradients match analytical ones in accuracy across all tested systems, aided by a quaternion-based diatomic frame rotation for two-center quantities that ensures smooth energy surfaces. Overall, automatic differentiation emerges as a practical alternative to analytical gradients in semiempirical quantum chemistry, offering high accuracy while allowing seamless integration in AI-driven workflows and popular packages, such as PyTorch and JAX. Our results provide actionable guidance for selecting optimal gradient strategies in large-scale ground- and excited-state molecular dynamics simulations.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Towards reverse mode automatic differentiation of Kokkos-based codes

Derivative computation is a key component of optimization, sensitivity analysis, uncertainty quantification, and the solving of nonlinear problems. Automatic differentiation (AD) is a powerful technique for evaluating such derivatives, and in recent years, has been integrated into programming environments such as Jax, PyTorch, and TensorFlow to support derivative computations needed for training of machine learning models, facilitating wide-spread use of these technologies. The C++ language has become the de facto standard for scientific computing due to numerous factors, yet language complexity has made the wide-spread adoption of AD technologies for C++ difficult, hampering the incorporation of powerful differentiable programming approaches into C++ scientific simulations. This is exacerbated by the increasing emergence of architectures, such as GPUs, with limited memory capabilities and requiring massive thread-level concurrency. C++ AD tools must effectively use these environments to bring novel scientific simulations to next-generation DOE experimental and observational facilities. In this project, we investigated source transformation-based automatic differentiation using LLVM compiler infrastructure to automatically generate portable and efficient gradient computations of Kokkos-based code. We have demonstrated that our proposed strategy is feasible by investigating the usage of a prototype LLVM-based source transformation tool to generate gradients of simple functions made of sequences of simple Kokkos parallel regions. Speedups of up to 500x compared to Sacado were observed on NVIDIA V100 GPU.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

JuTrack: A Julia package for auto-differentiable accelerator modeling and particle tracking

Efficient accelerator modeling and particle tracking are key for the design and configuration of modern particle accelerators. In this work, we present JuTrack, a nested accelerator modeling package developed in the Julia programming language and enhanced with compiler-level automatic differentiation (AD). With the aid of AD, JuTrack enables rapid derivative calculations in accelerator modeling, facilitating sensitivity analyses and optimization tasks. Here we demonstrate the effectiveness of AD-derived derivatives through several practical applications, including sensitivity analysis of space-charge-induced emittance growth, nonlinear beam dynamics analysis for a synchrotron light source, and lattice parameter tuning of the future Electron-Ion Collider (EIC). Through the incorporation of automatic differentiation, this package opens up new possibilities for accelerator physicists in beam physics studies and accelerator design optimization.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

Qualitative and quantitative enhancement of parameter estimation for model-based diagnostics using automatic differentiation with an application to inertial fusion

Parameter estimation using observables is a fundamental concept in the experimental sciences. Mathematical models that represent the physical processes can enable reconstructions of the experimental observables and greatly assist in parameter estimation by turning it into an optimization problem which can be solved by gradient-free or gradient-based methods. In this work, the recent rise in flexible frameworks for developing differentiable scientific computing programs is leveraged in order to dramatically accelerate data analysis of a common experimental diagnostic relevant to laser–plasma and inertial fusion experiments, Thomson scattering. A differentiable Thomson-scattering data analysis tool is developed that uses reverse-mode automatic differentiation (AD) to calculate gradients. By switching from finite differencing to reverse-mode AD, three distinct outcomes are achieved. First, gradient descent is accelerated dramatically to the extent that it enables near real-time usage in laser–plasma experiments. Second, qualitatively novel quantities which require $\mathcal{O}(10^3)$ parameters can now be included in the analysis of data which enables unprecedented measurements of small-scale laser–plasma phenomena. Third, uncertainty estimation approaches that leverage the value of the Hessian become accurate and efficient because reverse-mode AD can be used for calculating the Hessian.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

ZEUS: An Efficient GPU Optimization Method Integrating PSO, BFGS, and Automatic Differentiation

We introduce a novel, efficient computational method, ZEUS, for numerical optimization, and provide an open-source implementation. It has four key ingredients: (1) particle swarm optimization (PSO), (2) the use of the Broyden-Fletcher-Goldfarb-Shanno (BFGS) method, (3) automatic differentiation (AD), and (4) GPUs. Our approach addresses the computational challenges inherent in high-dimensional, non-convex optimization problems. In the first phase of the algorithm, we get a potentially good set of starting points using PSO. Thereafter, we run BFGS independently in parallel from these starting points. BFGS is one of the best-performing algorithms for numerical optimization. However, it requires the gradient of the function being optimized. ZEUS integrates automatic differentiation into BFGS thus avoiding the need for the user to calculate derivatives explicitly. The use of GPUs allows ZEUS to speed up the calculations substantially. We carry out systematic studies to explore the trade-offs between the number of PSO iterations taken, starting points, and BFGS iteration depth. We show that a handful of iterations of PSO can improve global convergence when combined with BFGS. We also present performance studies using common test functions. The source code can be found at https://github.com/fnal-numerics/global-optimizer-gpu.

Soos, Dominik [Old Dominion U.]↗
DOE OSTI2025

Degrees of rate control and AutoDiff-driven direct sensitivity analysis in heterogeneous catalysis

Despite the wide application and benefits of the degree of rate control (DRC) analysis, several details remain argued, particularly about the conservation of DRCs at transient (TR) and steady-state (SS) conditions, especially for complex reaction networks. This work argues that previous proofs about the conservation properties of DRCs have been incomplete, and we provide new mathematical proofs at TR and SS conditions. In addition, we use both analytical (automatic differentiation) and numerical (finite difference) approaches to compute DRCs for the case study of ethane hydrogenolysis (EH) over Pt(111). This work confirms that at both TR and SS conditions, the sum of all DRCs, i.e., sum of the degrees of kinetic (DKRC) and thermodynamic rate control (DTRC), is conserved at zero. At SS conditions, the sum of DKRC is conserved at 1 while the sum of DTRC is conserved at −1. In corroboration of previous works, we show that the DTRC for any adsorbate at SS is equal to the product of the species coverage and a constant. In contrast, at TR conditions, the individual sums of both DTRC and DKRC are not conserved and can be any real number, with potential implications for the novel field of dynamic catalysis. Finally, we show that the conventional finite difference (FD) approach, only useful at SS, is prone to inaccuracy and very sensitive to the value of the differential change applied. The optimal differential value also varies significantly with system and rate definition. Consequently, we describe and illustrate in this work the application of the automatic differentiation (AD) approach for the more accurate determination of DRCs at both TR and SS conditions.

Automatic differentiation↗
DOE OSTI2026

MapsTorch : automatic differentiation for X-ray fluorescence data analysis

X-ray fluorescence (XRF) is a popular spectroscopy technique for elemental analysis. Spectrum fitting and parameter tuning are at the core of XRF analysis and are conventionally manually intensive, especially for synchrotron experiments involving large amounts of diverse samples. This work introduces the automatic differentiation (AD) technique to XRF and an open-source package called MapsTorch. By transforming an analytical model of the XRF spectrum into a differentiable computation graph with AD, MapsTorch enables robust optimization of parameters and elemental intensities. We evaluate MapsTorch by conducting computational experiments on a large number of historical synchrotron XRF datasets and compare its performance with the currently practiced fitting tool NLopt. The results show that MapsTorch consistently achieves high-quality fits and often leads to better fitting quality than NLopt, particularly in tasks such as initial spectrum fitting and elemental intensity refinement. The robust performance of MapsTorch paves the way for developing automated and high-throughput XRF data analysis workflows to handle the increasing data volumes expected from next-generation synchrotron facilities.

X-ray fluorescence↗
DOE OSTI2026

Sparsity Applications for Gradient‐Based Optimization of Wind Farms

Optimizing wind farms is essential for designing efficient energy systems, especially as farms grow larger and span multiple sites. However, this optimization becomes increasingly challenging due to the rising computational cost associated with more turbines. Gradient‐based optimization methods scale better than gradient‐free approaches for large problems, but the most computationally expensive component remains the calculation of gradients for the objective function and constraint Jacobians. To address this, we propose leveraging sparsity to accelerate gradient evaluations and reduce the size of the constraint Jacobian. Wind farms naturally exhibit sparsity—many turbines do not influence each other under certain wind directions. However, unlike traditional sparse problems with fixed patterns, wind farm sparsity is dynamic, requiring new strategies to handle changing interactions efficiently. This paper presents a study of sparsity in wind farm optimization and introduces several methods to exploit it. These strategies are tested on multiple farms using the analytic Cumulative Curl model, with gradients computed via automatic differentiation (AD). The same sparsity‐aware techniques are also applicable to finite difference (FD) methods, where they can yield even greater speedups due to the high cost of directional evaluations. Results show that sparse methods achieve up to a 10x speedup with less than ± 5% variance in optimized wake losses compared to traditional methods. These findings suggest that sparsity‐aware optimization not only maintains solution quality but also scales efficiently with farm size, enabling more comprehensive design exploration at reduced computational cost.

17 WIND ENERGY↗
DOE OSTI2024

Separable physics-informed DeepONet: Breaking the curse of dimensionality in physics-informed machine learning

The deep operator network (DeepONet) has shown remarkable potential in solving partial differential equations (PDEs) by mapping between infinite-dimensional function spaces using labeled datasets. However, in scenarios lacking labeled data, the physics-informed DeepONet (PI-DeepONet) approach, which utilizes the residual loss of the governing PDE to optimize the network parameters, faces significant computational challenges, particularly due to the curse of dimensionality. This limitation has hindered its application to high-dimensional problems, making even standard 3D spatial with 1D temporal problems computationally prohibitive. Additionally, the computational requirement increases exponentially with the discretization density of the domain. Here, to address these challenges and enhance scalability for high-dimensional PDEs, we introduce the Separable physics-informed DeepONet (Sep-PI-DeepONet). This framework employs a factorization technique, utilizing sub-networks for individual one-dimensional coordinates, thereby reducing the number of forward passes and the size of the Jacobian matrix required for gradient computations. By incorporating forward-mode automatic differentiation (AD), we further optimize computational efficiency, achieving linear scaling of computational cost with discretization density and dimensionality, making our approach highly suitable for high-dimensional PDEs. We demonstrate the effectiveness of Sep-PI-DeepONet through three benchmark PDE models: the viscous Burgers’ equation, Biot’s consolidation theory, and a parameterized heat equation. Our framework maintains accuracy comparable to the conventional PI-DeepONet while reducing training time by two orders of magnitude. Notably, for the heat equation solved as a 4D problem, the conventional PI-DeepONet was computationally infeasible (estimated 289.35 h), while the Sep-PI-DeepONet completed training in just 2.5 h. These results underscore the potential of Sep-PI-DeepONet in efficiently solving complex, high-dimensional PDEs, marking a significant advancement in physics-informed machine learning.

Neural operator↗
DOE OSTI2025

Differentiable lagrangian shock hydrodynamics with application to stable shock acceleration of density interfaces

We develop a gradient based optimization approach for the equations of compressible, Lagrangian hydrodynamics and demonstrate how it can be employed to automatically uncover strategies to control hydrodynamic instabilities arising from shock acceleration of density interfaces. Strategies for controlling the Richtmyer-Meshkov instability (RMI) are of great benefit for inertial confinement fusion (ICF) where shock interactions with many small imperfections in the density interface lead to instabilities which rapidly grow over time. These instabilities lead to mixing which, in the case of laser driven ICF, quenches the runaway fusion process ruining the potential for positive energy return. Here, we demonstrate that control of these instabilities can be achieved by optimization of initial conditions with ( > 100) parameters. Optimizing over a large parameter space like this is not possible with gradient-free optimization strategies. This requires computation of the gradient of the outputs of a numerical solution to the equations of Lagrangian hydrodynamics with respect to the inputs. We show that the efficient computation of these gradients is made possible via a judicious application of (i) adjoint methods, the exact formal representation of sensitivities involving partial differential equations, and (ii) automatic differentiation (AD), the algorithmic calculation of derivatives of functions. Careful regularization of multiple operators including artificial viscosity and timestep control is required. We perform design optimization of > 100 parameter energy field driving the Richtmyer Meshkov instability showing significant suppression while simultaneously enhancing the acceleration of the interface relative to a nominal baseline case.

Hydrophysics↗
DOE OSTI2024

Optimal control of large quantum systems: assessing memory and runtime performance of GRAPE

Abstract Gradient Ascent Pulse Engineering (GRAPE) is a popular technique in quantum optimal control, and can be combined with automatic differentiation (AD) to facilitate on-the-fly evaluation of cost-function gradients. We illustrate that the convenience of AD comes at a significant memory cost due to the cumulative storage of a large number of states and propagators. For quantum systems of increasing Hilbert space size, this imposes a significant bottleneck. We revisit the strategy of hard-coding gradients in a scheme that fully avoids propagator storage and significantly reduces memory requirements. Separately, we present improvements to numerical state propagation to enhance runtime performance. We benchmark runtime and memory usage and compare this approach to AD-based implementations, with a focus on pushing towards larger Hilbert space sizes. The results confirm that the AD-free approach facilitates the application of optimal control for large quantum systems which would otherwise be difficult to tackle.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2024

Scrutinizing Variables for Checkpoint Using Automatic Differentiation

Checkpoint/Restart (C/R) saves the running state of the programs periodically, which consumes considerable time and system resources. We observe that not every piece of data is involved in the computation in typical HPC applications; such unused data should be excluded from checkpointing for better storage and compute efficiency. We propose a systematic approach that leverages automatic differentiation (AD) to scrutinize every element within variables (e.g., arrays) necessary for checkpointing. This allows us to identify critical and uncritical elements and eliminate uncritical elements from checkpointing. Specifically, we inspect every single element within a variable necessary for checkpointing with an AD tool to determine whether the element has an impact on the application output or not. We validate our approach with all benchmarks from the NPB suite. We visualize the distribution of critical and uncritical elements within a variable with respect to its binary impact (yes or no) on the application output.

Huang, Xin [Kobe University]↗
DOE OSTI2024

Learning Constrained Parametric Differentiable Predictive Control Policies With Guarantees

We present differentiable predictive control (DPC), a method for offline learning of constrained neural control policies for nonlinear dynamical systems with performance guarantees. We show that the sensitivities of the parametric optimal control problem can be used to obtain direct policy gradients. Specifically, we employ automatic differentiation (AD) to efficiently compute the sensitivities of the model predictive control (MPC) objective function and constraints penalties. To guarantee safety upon deployment, we derive probabilistic guarantees on closed-loop stability and constraint satisfaction based on indicator functions and Hoeffding’s inequality. We empirically demonstrate that the proposed method can learn neural control policies for various parametric optimal control tasks. In particular, we show that the proposed DPC method can stabilize systems with unstable dynamics, track time-varying references, and satisfy nonlinear state and input constraints. Our DPC method has practical time savings compared to alternative approaches for fast and memory-efficient controller design. Specifically, DPC does not depend on a supervisory controller as opposed to approximate MPC based on imitation learning. We demonstrate that, without losing performance, DPC is scalable with greatly reduced demands on memory and computation compared to implicit and explicit MPC while being more sample efficient than model-free reinforcement learning (RL) algorithms.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Extending JuTrack’s capabilities to the FRIB accelerator to enhance online modeling

JuTrack is a Julia-based accelerator modeling and tracking package that utilizes compiler-level automatic differentiation (AD) to enable fast and accurate derivative calculations. While JuTrack provides a solid foundation for beam dynamics simulations, its capabilities must be extended to support the Facility for Rare Isotopes (FRIB) linac. This includes modeling heavy-ion linac accelerator components such as the liquid-lithium charge stripper, which facilitates efficient acceleration by remove electrons from heavy isotopes, and incorporating multi-charge state acceleration tracking, which allows for charge-dependent beam dynamics. These extensions address challenges such as the beam matching and optimization of multi charge state through various accelerating structures and beam-material interaction modeling while maintaining the auto differentiation capability. This work focuses on adapting JuTrack to incorporate these elements, enhancing its online modeling abilities. We present modifications to JuTrack’s framework and demonstrate their performance in FRIB simulations.

Accelerator Physics↗