Search NASA⌕ Search

SEARCH · Search NASA

Results for “Automatic Differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3

Reconstructing the neutron star equation of state from observational data via automatic differentiation

Neutron star observables like masses, radii, and tidal deformability are direct probes to the dense matter equation of state (EoS). Here, a novel deep learning method that optimizes an EoS in the automatic differentiation framework of solving inverse problems is presented. The trained neural network EoS yields narrow bands for the relationship between the pressure and speed of sound as a function of the mass density. The results are consistent with those obtained from conventional approaches and the observational bound on the tidal deformability inferred from the gravitational wave event, GW170817.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Adorym: a multi-platform generic X-ray image reconstruction framework based on automatic differentiation

We describe and demonstrate an optimization-based X-ray image reconstruction framework called Adorym. Our framework provides a generic forward model, allowing one code framework to be used for a wide range of imaging methods ranging from near-field holography to fly-scan ptychographic tomography. By using automatic differentiation for optimization, Adorym has the flexibility to refine experimental parameters including probe positions, multiple hologram alignment, and object tilts. It is written with strong support for parallel processing, allowing large datasets to be processed on high-performance computing systems. We demonstrate its use on several experimental datasets to show improved image quality through parameter refinement.

36 MATERIALS SCIENCE↗

Scrutinizing Variables for Checkpoint Using Automatic Differentiation

Checkpoint/Restart (C/R) saves the running state of the programs periodically, which consumes considerable time and system resources. We observe that not every piece of data is involved in the computation in typical HPC applications; such unused data should be excluded from checkpointing for better storage and compute efficiency. We propose a systematic approach that leverages automatic differentiation (AD) to scrutinize every element within variables (e.g., arrays) necessary for checkpointing. This allows us to identify critical and uncritical elements and eliminate uncritical elements from checkpointing. Specifically, we inspect every single element within a variable necessary for checkpointing with an AD tool to determine whether the element has an impact on the application output or not. We validate our approach with all benchmarks from the NPB suite. We visualize the distribution of critical and uncritical elements within a variable with respect to its binary impact (yes or no) on the application output.

Huang, Xin [Kobe University]↗

Machine learning of consistent thermodynamic models using automatic differentiation

In this study, we propose a data-driven method to describe consistent equations of state (EOS) for arbitrary systems. Complex EOS are traditionally obtained by fitting suitable analytical expressions to thermophysical data. A key aspect of EOS is that the relationships between state variables are given by derivatives of the system free energy. In this work, we model the free energy with an artificial neural network and utilize automatic differentiation to directly learn the derivatives of the free energy. We demonstrate this approach on two different systems, the analytic van der Waals EOS and published data for the Lennard-Jones fluid, and we show that it is advantageous over direct learning of thermodynamic properties (i.e., not as derivatives of the free energy but as independent properties), in terms of both accuracy and the exact preservation of the Maxwell relations. Furthermore, the method implicitly provides the free energy of a system without explicit integration.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Dose-efficient automatic differentiation for ptychographic reconstruction

Ptychography, as a powerful lensless imaging method, has become a popular member of the coherent diffractive imaging family over decades of development. The ability to utilize low-dose X-rays and/or fast scans offers a big advantage in a ptychographic measurement (for example, when measuring radiation-sensitive samples), but results in low-photon statistics, making the subsequent phase retrieval challenging. Here, we demonstrate a dose-efficient automatic differentiation framework for ptychographic reconstruction (DAP) at low-photon statistics and low overlap ratio. As no reciprocal space constraint is required in this DAP framework, the framework, based on various forward models, shows superior performance under these conditions. It effectively suppresses potential artifacts in the reconstructed images, especially for the inherent periodic artifact in a raster scan. We validate the effectiveness and robustness of this method using both simulated and measured datasets.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Distributed Automatic Differentiation for Ptychography

Synchrotron radiation light source facilities are leading the way to ultrahigh resolution X-ray imaging. High resolution imaging is essential to understanding the fundamental structure and interaction of materials at the smallest length scale possible. Diffraction based methods achieve nanoscale imaging by replacing traditional objective lenses by pixelated area detectors and computational image reconstruction. Among these methods, ptychography is quickly becoming the standard for sub-30 nanometer imaging of extended samples, but at the expense of increasingly high data rates and volumes. This paper presents a new distributed algorithm for solving the ptychographic image reconstruction problem based on automatic differentiation. Input datasets are subdivided between multiple graphics processing units (GPUs); each subset of the problem is then solved either entirely independent of other subsets (asynchronously) or through sharing gradient information with other GPUs (synchronously). The algorithm was evaluated on simulated and real data acquired at the Advanced Photon Source, scaling up to 192 GPUs. The synchronous variant of our method outperformed an existing multi-GPU implementation in terms of accuracy while running at a comparable execution time.

distributed algorithms↗

Determination of Stability and Control Derivatives using Computational Fluid Dynamics and Automatic Differentiation

With the recent interest in novel control effectors there is a need to determine the stability and control derivatives of new aircraft configurations early in the design process. These derivatives are central to most control law design methods and would allow the determination of closed-loop control performance of the vehicle. Early determination of the static and dynamic behavior of an aircraft may permit significant improvement in configuration weight, cost, stealth, and performance through multidisciplinary design. The classical method of determining static stability and control derivatives - constructing and testing wind tunnel models - is expensive and requires a long lead time for the resultant data. Wind tunnel tests are also limited to the preselected control effectors of the model. To overcome these shortcomings, computational fluid dynamics (CFD) solvers are augmented via automatic differentiation, to directly calculate the stability and control derivatives. The CFD forces and moments are differentiated with respect to angle of attack, angle of sideslip, and aircraft shape parameters to form these derivatives. A subset of static stability and control derivatives of a tailless aircraft concept have been computed by two differentiated inviscid CFD codes and verified for accuracy with central finite-difference approximations and favorable comparisons to a simulation database.

Park, Michael A.↗

Three dimensions, two microscopes, one code: Automatic differentiation for x-ray nanotomography beyond the depth of focus limit

Conventional tomographic reconstruction algorithms assume that one has obtained pure projection images, involving no within-specimen diffraction effects nor multiple scattering. Advances in x-ray nanotomography are leading toward the violation of these assumptions, by combining the high penetration power of x-rays, which enables thick specimens to be imaged, with improved spatial resolution that decreases the depth of focus of the imaging system. We describe a reconstruction method where multiple scattering and diffraction effects in thick samples are modeled by multislice propagation and the 3D object function is retrieved through iterative optimization. We show that the same proposed method works for both full-field microscopy and for coherent scanning techniques like ptychography. Our implementation uses the optimization toolbox and the automatic differentiation capability of the open-source deep learning package TensorFlow, demonstrating a straightforward way to solve optimization problems in computational imaging with flexibility and portability.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Particle-image Velocimetry Sensitivity through Automatic Differentiation

A commonly encountered difficulty in modern experimental fluid dynamics experiments is the need for full-flow visualization and measurement; traditional techniques such as hot-wire probes and laser Doppler velocimetry cannot provide such information without difficult scanning procedures. Particle-image velocimetry (PIV) is a measurement technique which can overcome this spatial limitation to provide real-time three-dimensional fluid velocity vectors for an entire plane within the flow, as opposed to a single point that is provided by other measurement techniques. To accomplish this, a PIV system seeds the fluid flow with tracer particles; these must have nearly the same density as that of the fluid and a high drag coefficient to ensure that they follow the fluid flow without deviation. A laser is sent through optics to expand its beam into a sheet which illuminates the plane of interest in the fluid. The particles reflect more light than the fluid, and their image is recorded using a high speed camera. Due to limitations in bandwidth, video cannot be taken fast enough to be useful, and instead pairs of images are captured with a known separation in time. The illuminated particles will move between the two images taken by the camera, and their relative motion can be used to compute the fluid velocity at that point. When a single camera is used, only two-dimensional velocity vectors can be computed, but with the addition of another camera this can be improved to full three-dimensional vectors. The flow's time evolution can be measured by taking series of image pairs and processing each pair to get the velocity field at the time between each image pair. Only the two-dimensional version will be considered in this work. To compute the velocity vectors of the flow, the images are broken up into interrogation regions, usually in a uniform grid. In the most basic PIV algorithm, corresponding regions in the image pairs are cross-correlated to compute the most likely displacement from one image to the next. This is used since the cross correlation will achieve a maximum value when the two interrogation regions line up the best. The location of the peak in the correlation map and scaling information for the optical setup is used to compute the particle displacement from one image to the next. This process is similar to holding two semi-transparent images and moving one until they match as well as possible. Sub-pixel displacements between images can be obtained by fitting a Gaussian function to the correlation peak. Knowing the displacement vector for each set of interrogation regions in the image pair, it is a simple matter to divide by the time gap between the two images to compute the fluid velocity at the point in time and space. Since there are many interrogation regions in each image pair, the velocity field for a sheet is sampled according to the arrangement of the interrogation regions. The sensitivity of the final computed vector to the input images is difficult to compute, and has led most considerations of uncertainty in the PIV field to apply Monte Carlo methods and correlations to quantify the systematic and random errors which result from application of various PIV algorithms. In this work, automatic differentiation (AD) has been used to trace the dependence of velocity vectors computed using the correlation-peak PIV algorithm back to the input values given to an artificial image generator. These initial results form the foundation for continued work in the application of AD to algorithms in PIV. They demonstrate that AD methods can be successfully applied to PIV algorithms without undue complexity. The derivatives of each individual vector with regard to the coefficients of the velocity field used to create the artificial images can be computed and the statistics of them considered collectively, or correlated between each other to quantify the effectiveness of a PIV algorithm. With the initial implementation complete, adding extra algorithms is relatively simple and considerable data can be generated for analysis. It is hoped that application of these methods can lead to improved PIV algorithms which demonstrate better sensitivity to small variations in particle displacement. While these methods can be used as an analysis tool for refinement of new algorithms, computation of actual resultant vector sensitivities to input images' individual pixels is the next logical step of this work. Such information will be analyzed with experimental data to produce a better understanding of how output vectors' uncertainty relates to the actual image pairs, and lead to automatic bad vector removal. (authors)

42 ENGINEERING↗

Rapid multiphase-field model development using a modular free energy based approach with automatic differentiation in MOOSE/MARMOT

In this paper, we present a novel rapid phase-field model development capability using a modular free energy based approach in the finite element based MOOSE framework. The phase-field equations are implemented in a general form as separate code form the model specific data such as the thermodynamic free energy density and mobility functions. This isolates researchers from complicated code and allows them to focus on implementing new material models. In addition, using a native symbolic differentiation algorithm, all required derivatives of the free energy density are automatically determined, removing sources for potential human error, and guaranteeing optimally preconditioned equation systems through fully populated Jacobian matrices. Free energy terms contributing to a phase-field model are abstracted into self contained objects that can be combined at will at simulation run time. Combining multiple chemical and mechanical free energy contributions allows the construction of coupled phase-field, mechanics, and multiphase models. Through just-in-time compilation, we greatly reduce the computational time required to evaluate the differentiated expressions. Lastly, we demonstrate the new capability on a variety of example simulations.

97 MATHEMATICS AND COMPUTING↗

POSTER: Automatic Differentiation of Parallel Loops with Formal Methods

The accompanying poster to this short paper presents a combination of reverse mode AD and formal methods to enable efficient differentiation of (or backpropagation through) shared-memory parallel code. Compared to the state of the art, our approach can more often avoid the need for atomic updates or private data copies during the parallel derivative computation, even in the presence of unstructured or data-dependent data access patterns. This is achieved by gathering information about the memory access patterns from the input program, which is assumed to be correctly parallelized. This information is then used to build a model of assertions in a theorem prover, which can be used to check the safety of shared memory accesses during the parallel derivative computation

Automatic Differentiation↗

MITgcm-AD v2: Open source tangent linear and adjoint modeling framework for the oceans and atmosphere enabled by the Automatic Differentiation tool Tapenade

The Massachusetts Institute of Technology General Circulation Model (MITgcm) is widely used by the climate science community to simulate planetary atmosphere and ocean circulations. A defining feature of the MITgcm is that it has been developed to be compatible with an algorithmic differentiation (AD) tool, TAF, enabling the generation of tangent-linear and adjoint models. These provide gradient information which enables dynamics-based sensitivity and attribution studies, state and parameter estimation, and rigorous uncertainty quantification. Importantly, gradient information is essential for computing comprehensive sensitivities and performing efficient large-scale data assimilation, ensuring that observations collected from satellites and in-situ measuring instruments can be effectively used to optimize a large uncertain control space. As a result, the MITgcm forms the dynamical core of a key data assimilation product employed by the physical oceanography research community: Estimating the Circulation and Climate of the Ocean (ECCO) state estimate. Although MITgcm and ECCO are used extensively within the research community, the AD tool TAF is proprietary and hence inaccessible to a large proportion of these users. The new version 2 (MITgcm-AD v2) framework introduced here is based on the source-to-source AD tool Tapenade, which has recently been open-sourced. Another feature of Tapenade is that it stores required variables by default (instead of recomputing them) which simplifies the implementation of efficient, AD-compatible code. The framework has been integrated with the MITgcm model’s main branch and is now freely available.

Adjoints↗

Integrated Design of Downwind Land-Based Wind Turbines using Analytic Gradients

Wind turbines are complex systems where component-level changes can have significant system-level effects. Effective wind turbine optimization generally requires an integrated analysis approach with a large number of design variables. Optimizing across large variable sets is orders of magnitude more efficient with gradient-based methods as compared with gradient-free method, particularly when using exact gradients. We have developed a wind turbine analysis set of over 100 components where 90% of the models provide numerically exact gradients through symbolic differentiation, automatic differentiation, and adjoint methods. This framework is applied to a specific design study focused on downwind land-based wind turbines. Downwind machines are of potential interest for large wind turbines where the blades are often constrained by the stiffness required to prevent a tower strike. The mass of these rotor blades may be reduced by utilizing a downwind configuration where the constraints on tower strike are less restrictive. The large turbines of this study range in power rating from 5-7MW and in diameter from 105m to 175m. The changes in blade mass and power production have important effects on the rest of the system, and thus the nacelle and tower systems are also optimized. For high-speed wind sites, downwind configurations do not appear advantageous. The decrease in blade mass (10%) is offset by increases in tower mass caused by the bending moment from the rotor-nacelle-assembly. For low-wind speed sites, the decrease in blade mass is more significant (25-30%) and shows potential for modest decreases in overall cost of energy (around 1-2%).

17 WIND ENERGY↗