Search NASA⌕ Search

SEARCH · Search NASA

Results for “Automatic Differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗

Cheaper Adjoints by Reversing Address Computations

The reverse mode of automatic differentiation is widely used in science and engineering. A severe bottleneck for the performance of the reverse mode, however, is the necessity to recover certain intermediate values of the program in reverse order. Among these values are computed addresses, which traditionally are recovered through forward recomputation and storage in memory. We propose an alternative approach for recovery that uses inverse computation based on dependency information. Address storage constitutes a significant portion of the overall storage requirements. An example illustrates substantial gains that the proposed approach yields, and we show use cases in practical applications.

Hascoët, L.↗

COEK

COEK (C++ Optimization Expression Kernel) is a C++ library that supports the expression and manipulation of algebraic expressions commonly used in optimization applications. COEK. Supports several mechanisms for defining and managing expressions and performing automatic differentiation. COEK also includes low-level interfaces for defining optimization models and interfacing with optimization solvers.SAND2019-14491 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Hart, William↗

POEK

POEK (Python Optimization Expression Kernel) is a Python library that supports the expression and manipulation of algebraic expressions commonly used in optimization applications. POEK interfaces with the COEK library to support fast methods for defining and managing expressions and performing automatic differentiation. POEK also interfaces with COEK to directly interface with optimization solvers (i.e. without relying on file I/O). SAND2019-14492 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Hart, William↗

NEURAL NETWORK FOR COHERENT DIFFRACTION IMAGE INVERSION

A deep neural network model plus automatic differentiation is developed for retrieving phase information from 3D coherent diffraction images. The model is implemented using Tensorflow and the training dataset is generated using physics-based atomistic simulations. Custom codes are written to handle the resampling of diffraction images to oversampling ratios appropriate for the neural network model.

CHAN, HENRY↗

Calibr8 v.1.0

Calibr8 provides an application to rapidly prototype and perform material model calibration for complex plasticity models using advanced adjoint or forward sensitivity analyses for execution on massively parallel machines. These techniques can be orders of magnitude faster than traditional finite difference approaches for material model calibration. The underlying technology used in Calibr8 is automatic differentiation, which allows for the rapid implementation and testing of new plasticity models within its framework. Additionally, Calibr8 can perform adjoint-based error estimation to approximate discretization errors for user-implemented plasticity models. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525. SAND2021-10630 O

Granzow, BrianN.↗

dfdjaxGP

A small python package to fit Gaussian processes using Jax and leveraging the automatic differentiation in Jax to predict arbitrary derivatives from the GP. This package is meant to supplement the scientific community use of Gaussian process prediction with derivatives. The package is designed to smoothly work standalone or be used with the numpyro probabilistic programming language.

Grosskopf, Micheal↗

Isopod (inverse Optimization And Design): Moose-based Application For Performing Inverse Optimization On Multiphysics Modeling And Simulation

ISOPOD (InverSe OPtimizatiOn and Design) will be a MOOSE (Multiphysics Object Oriented Simulation Environment)-based framework which will iteratively estimate the parameters by minimizing the difference between simulated and experimental observables. While inverse optimization has long been used for parameter estimation, the unique feature of ISOPOD is rapid solution of new inverse problems, leading to accelerated Research, Development and Demonstration (RD&D) for complex parameter estimation problems. This will be achieved by extending the software structure of MOOSE to inverse optimization, as well as building on the unique automatic differentiation feature of MOOSE.

Munday, LynnB↗

A set of test problems for nonlinear optimization

The software is a set of test problems for nonlinear optimization algorithms, including subroutines such as linear algebra routines and automatic differentiation algorithms. The test problems come from chemical engineering open literature, and describe optimization tasks related to the design and operation of processes such as carbon capture, Hydrogen production, heat exchange, and distillation.

Parker, Robert↗

HydraGNN v5.0

HydraGNN v5.0 expands the code base into a more portable, scalable, and flexible framework for scientific graph learning, with particular strength in atomistic machine-learning interatomic potentials and large-scale distributed training. The release adds Fully Sharded Data Parallel (FSDP) support alongside existing DDP and DeepSpeed paths, including FSDP-aware checkpointing and optimizer integration, and introduces a configurable multi-precision training workflow supporting FP32, BF16, and FP64 across GPUs and Intel XPUs. For atomistic modeling, HydraGNN v5.0 strengthens its MLIP capabilities through dynamic graph construction at every forward pass, energy-conserving force prediction via automatic differentiation, and per-atom energy loss formulations, while extending EGNN models to properly handle periodic boundary conditions. The release also broadens model expressiveness through graph-level attribute conditioning, adds new multi-task and model-parallel extensions such as MACE support and encoder/decoder branch optimization, and expands application coverage with integrated examples for datasets including OC25, Nabla2-DFT, QCML, Open Polymers 2026, and OPF. In parallel, HydraGNN v5.0 improves production readiness through performance optimizations for large-scale runs, stratified sampling and linear-regression preprocessing utilities, and tested installation scripts for DOE supercomputers including Frontier, Aurora, Perlmutter, and Andes. Overall, the release advances HydraGNN as a robust software platform for scalable graph neural networks across materials science, chemistry, and scientific machine learning workflows

Lupo Pasini, Massimiliano [Oak Ridge National Labo↗

SIRENOpt.jl

SAND2026-22945O SIRENOpt.jl is a Julia software package for prototype hybrid power, storage, and platform dynamics. It integrates solar, wind, wave, hydrokinetic, diesel, generator, converter, battery, hydrogen, desalination, mooring, and floating-platform model interfaces in an automatic-differentiation-friendly simulation framework. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Michelen Strofer, Carlos [Sandia National Lab. (SN↗

Multislice forward modeling of coherent surface scattering imaging on surface and interfacial structures

To study nanostructures on substrates, surface-sensitive reflection-geometry scattering techniques such as grazing incident small angle X-ray scattering are commonly used to yield an averaged statistical structural information of the surface sample. Grazing incidence geometry can probe the absolute three-dimensional structural morphology of the sample if a highly coherent beam is used. Coherent surface scattering imaging (CSSI) is a powerful yet non-invasive technique similar to coherent X-ray diffractive imaging (CDI) but performed at small angles and grazing-incidence reflection geometry. A challenge with CSSI is that conventional CDI reconstruction techniques cannot be directly applied to CSSI because the Fourier-transform-based forward models cannot reproduce the dynamical scattering phenomenon near the critical angle of total external reflection of the substrate-supported samples. To overcome this challenge, we have developed a multislice forward model which can successfully simulate the dynamical or multi-beam scattering generated from surface structures and the underlying substrate. The forward model is also demonstrated to be able to reconstruct an elongated 3D pattern from a single shot scattering image in the CSSI geometry through fast-performing CUDA-assisted PyTorch optimization with automatic differentiation.

77 NANOSCIENCE AND NANOTECHNOLOGY↗

Imaging extended single crystal lattice distortion fields with multi-peak Bragg ptychography

Recent advances in phase-retrieval-based x-ray imaging methods have demonstrated the ability to reconstruct 3D distortion vector fields within a nanocrystal by using coherent diffraction information from multiple crystal Bragg reflections. However, these works do not provide a solution to the challenges encountered in imaging lattice distortions in crystals with significant defect content that result in phase wrapping. Moreover, these methods only apply to isolated crystals smaller than the x-ray illumination, and therefore cannot be used for imaging of distortions in extended crystals. We introduce multi-peak Bragg ptychography which addresses both challenges via an optimization framework that combines stochastic gradient descent and phase unwrapping methods for robust image reconstruction of lattice distortions and defects in extended crystals. Our work uses modern automatic differentiation toolsets so that the method is easy to extend to other settings and easy to implement in high-performance computers. This work is particularly timely given the broad interest in using the increased coherent flux in fourth-generation synchrotrons for innovative material research.

36 MATERIALS SCIENCE↗

Inverse Analysis with Variational Autoencoders: A Comparison of Shallow and Deep Networks

Inverse problems are applied to determine unknown properties by matching observational data with a physical model that often takes many parameters as input. To overcome the underconstrained nature of inverse problems and achieve good performance, an approach is presented involving regularization with a technique known as a variational autoencoder (VAE), which is trained to map a high-dimensional parameter space with a complex structure to a low-dimensional latent space with a simple structure. We apply this approach to unconditioned realizations of the parameters (heterogeneous hydraulic fields) for a hydrogeological inverse problem. Two types of hydraulic conductivity fields are used to evaluate the characterization for the different levels of heterogeneity complexity of the physical inputs. This approach keeps the computational cost of generating the training data low. The reason is unconditioned realizations neither rely on the observational data used to perform the inverse analysis nor require any groundwater flow model forward runs. In addition, this approach applies regularization on a low-dimensional latent space from the VAE and increases optimization efficiency through automatic differentiation. Furthermore, two different neural network (NN) structures are tested for their utility in using the VAE for inverse analysis. The performance of a deep, convolutional neural network strongly depends on the dimensionality of the latent space, which requires tuning. In contrast, a shallow, dense neural network provides consistently accurate characterization without tuning. Furthermore, our approach evaluates the advantages of the shallow, dense neural network over the deep, convolutional one and enables future application to a wide range of inverse problems.

97 MATHEMATICS AND COMPUTING↗

Extending JuTrack’s capabilities to the FRIB accelerator to enhance online modeling

JuTrack is a Julia-based accelerator modeling and tracking package that utilizes compiler-level automatic differentiation (AD) to enable fast and accurate derivative calculations. While JuTrack provides a solid foundation for beam dynamics simulations, its capabilities must be extended to support the Facility for Rare Isotopes (FRIB) linac. This includes modeling heavy-ion linac accelerator components such as the liquid-lithium charge stripper, which facilitates efficient acceleration by remove electrons from heavy isotopes, and incorporating multi-charge state acceleration tracking, which allows for charge-dependent beam dynamics. These extensions address challenges such as the beam matching and optimization of multi charge state through various accelerating structures and beam-material interaction modeling while maintaining the auto differentiation capability. This work focuses on adapting JuTrack to incorporate these elements, enhancing its online modeling abilities. We present modifications to JuTrack’s framework and demonstrate their performance in FRIB simulations.

Accelerator Physics↗

JAXtronomy: A JAX port of lenstronomy

Gravitational lensing is a phenomenon where light bends around massive objects, resulting in distorted images seen by an observer. Studying gravitationally lensed systems provides insights into cosmology and astrophysics, including constraints of the expansion rate of the Universe and the distribution of dark matter. Thus, we introduce JAXtronomy, a re-implementation of the gravitational lensing software package lenstronomy (Birrer, 2021; Birrer & Amara, 2018) using JAX (Bradbury et al., 2018). JAX is a Python library that uses an accelerated linear algebra (XLA) compiler to improve the performance of computing software. Our core design principle of JAXtronomy is to maintain an identical API to that of lenstronomy. The main JAX features utilized in JAXtronomy are just-in-time compilation, which can lead to significant reductions in execution time, and automatic differentiation, which allows for the implementation of gradient-based algorithms that were previously impossible. Additionally, JAX allows code to be run on GPUs or parallelized across CPU cores, further boosting the performance of JAXtronomy.

astronomy↗

Efficient and robust gradient enhanced Kriging emulators.

%E2%80%9CNaive%E2%80%9D or straight-forward Kriging implementations can often perform poorly in practice. The relevant features of the robustly accurate and efficient Kriging and Gradient Enhanced Kriging (GEK) implementations in the DAKOTA software package are detailed herein. The principal contribution is a novel, effective, and efficient approach to handle ill-conditioning of GEK's %E2%80%9Ccorrelation%E2%80%9D matrix, RN%CC%83, based on a pivoted Cholesky factorization of Kriging's (not GEK's) correlation matrix, R, which is a small sub-matrix within GEK's RN%CC%83 matrix. The approach discards sample points/equations that contribute the least %E2%80%9Cnew%E2%80%9D information to RN%CC%83. Since these points contain the least new information, they are the ones which when discarded are both the easiest to predict and provide maximum improvement of RN%CC%83's conditioning. Prior to this work, handling ill-conditioned correlation matrices was a major, perhaps the principal, unsolved challenge necessary for robust and efficient GEK emulators. Numerical results demonstrate that GEK predictions can be significantly more accurate when GEK is allowed to discard points by the presented method. Numerical results also indicate that GEK can be used to break the curse of dimensionality by exploiting inexpensive derivatives (such as those provided by automatic differentiation or adjoint techniques), smoothness in the response being modeled, and adaptive sampling. Development of a suitable adaptive sampling algorithm was beyond the scope of this work; instead adaptive sampling was approximated by omitting the cost of samples discarded by the presented pivoted Cholesky approach.

Dalbey, Keith R.↗

Stochastic Optimization of Complex Systems

This project focused on methodologies for the solution of stochastic optimization problems based on relaxation and penalty methods, Monte Carlo simulation, parallel processing, and inverse optimization. The main results of the project were the development of a convergent method for the solution of models that include expectation constraints as in equilibrium models, improvement of Monte Carlo convergence through the use of a new method of sample batch optimization, the development of new parallel processing methods for stochastic unit commitment models, and the development of improved methods in combination with parallel processing for incorporating automatic differentiation methods into optimization.

97 MATHEMATICS AND COMPUTING↗