Search NASA⌕ Search

SEARCH · Search NASA

Results for “algorithmic differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 433 records · Page 24
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Optimal sensor network design for multi-scale, time-varying differential algebraic equation systems: Application to an entrained-flow gasifier refractory brick

An algorithm for optimal sensor network design for multi-scale, time-varying differential algebraic equation systems with non-separable dynamics is presented. As the process is time-varying, an integral normalized posterior error covariance of a multi-scale filter is minimized to obtain the optimal sensor locations. For reducing the computational cost, an adaptive sampling rate approach is considered for the slowly-varying variables. The algorithm is applied to a smart refractory brick with embedded sensors as part of an entrained-flow gasifier. Thermistors and interdigital capacitors are considered as candidate measurement technologies for estimating temperature and slag penetration profile along the gasifier wall. Finally, when the optimal set of sensors obtained from the algorithm is used for estimating temperature and slag penetration profiles in a multi-scale Kalman filter framework, satisfactory estimates are obtained despite high measurement noise and model mismatch.

42 ENGINEERING↗
DOE OSTI2012

The solution of the point kinetics equations via converged accelerated Taylor series (CATS)

This paper deals with finding accurate solutions of the point kinetics equations including non-linear feedback, in a fast, efficient and straightforward way. A truncated Taylor series is coupled to continuous analytical continuation to provide the recurrence relations to solve the ordinary differential equations of point kinetics. Non-linear (Wynn-epsilon) and linear (Romberg) convergence accelerations are employed to provide highly accurate results for the evaluation of Taylor series expansions and extrapolated values of neutron and precursor densities at desired edits. The proposed Converged Accelerated Taylor Series, or CATS, algorithm automatically performs successive mesh refinements until the desired accuracy is obtained, making use of the intermediate results for converged initial values at each interval. Numerical performance is evaluated using case studies available from the literature. Nearly perfect agreement is found with the literature results generally considered most accurate. Benchmark quality results are reported for several cases of interest including step, ramp, zigzag and sinusoidal prescribed insertions and insertions with adiabatic Doppler feedback. A larger than usual (9) number of digits is included to encourage honest benchmarking. The benchmark is then applied to the enhanced piecewise constant algorithm (EPCA) currently being developed by the second author. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

End-to-End Differentiable Modeling and Management of the Environment

Focal Area: (2) Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization. We emphasize the importance of leveraging optimization techniques from AI/machine learning (ML) to solve challenging problems in Earth system modeling. Science Challenge: Automatic differentiation has had a transformative effect on ML by allowing the calculation of gradients of arbitrary functions in an incredibly large class of models. We can potentially realize similar improvements in parameter estimation and control for Earth system models (ESMs) by reimplementing them in computational frameworks from ML. Practitioners working with large (>10 7 parameters) models in ML and AI can obtain good predictive performance in a range of spatiotemporal tasks by making use of optimization via stochastic gradient descent and incorporating prior knowledge at multiple levels. We propose writing ESMs in open-source computational frameworks such as Torch, Tensorflow, and JAX to greatly expand the scope of environmental forecasting and management challenges, which can be addressed by leveraging automatic differentiation and gradient descent-like algorithms. We do not call for a wholesale replacement of physical models with data-driven surrogates, but rather advocate for interleaving physical and empirical equations in a manner that is most faithful to the extent of our scientific knowledge and observational data. Central to this topic is the merging of differentiable physical simulations with differentiable optimization layers, which are now both beginning to come to the forefront.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2015

Synergies from using higher order symplectic decompositions both for ordinary differential equations and quantum Monte Carlo methods

Decomposition methods of the Suzuki-Trotter type of various orders have been derived in different fields. Applying them both to classical ordinary differential equations (ODEs) and quantum systems allows to judge their effectiveness and gives new insights for many body quantum mechanics where reference data are scarce. Further, based on data for 6 × 6 system we conclude that sampling with sign (minus-sign problem) is probably detrimental to the accuracy of fermionic simulations with determinant algorithms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2011

FBP and BPF reconstruction methods for circular X-ray tomography with off-center detector

Purpose: Circular scanning with an off-center planar detector is an acquisition scheme that allows to save detector area while keeping a large field of view (FOV). Several filtered back-projection (FBP) algorithms have been proposed earlier. The purpose of this work is to present two newly developed back-projection filtration (BPF) variants and evaluate the image quality of these methods compared to the existing state-of-the-art FBP methods. Methods: The first new BPF algorithm applies redundancy weighting of overlapping opposite projections before differentiation in a single projection. The second one uses the Katsevich-type differentiation involving two neighboring projections followed by redundancy weighting and back-projection. An averaging scheme is presented to mitigate streak artifacts inherent to circular BPF algorithms along the Hilbert filter lines in the off-center transaxial slices of the reconstructions. The image quality is assessed visually on reconstructed slices of simulated and clinical data. Quantitative evaluation studies are performed with the Forbild head phantom by calculating root-mean-squared-deviations (RMSDs) to the voxelized phantom for different detector overlap settings and by investigating the noise resolution trade-off with a wire phantom in the full detector and off-center scenario. Results: The noise-resolution behavior of all off-center reconstruction methods corresponds to their full detector performance with the best resolution for the FDK based methods with the given imaging geometry. With respect to RMSD and visual inspection, the proposed BPF with Katsevich-type differentiation outperforms all other methods for the smallest chosen detector overlap of about 15 mm. The best FBP method is the algorithm that is also based on the Katsevich-type differentiation and subsequent redundancy weighting. For wider overlap of about 40-50 mm, these two algorithms produce similar results outperforming the other three methods. The clinical case with a detector overlap of about 17 mm confirms these results. Conclusions: The BPF-type reconstructions with Katsevich differentiation are widely independent of the size of the detector overlap and give the best results with respect to RMSD and visual inspection for minimal detector overlap. The increased homogeneity will improve correct assessment of lesions in the entire field of view.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
NASA NTRS1990

Trees, bialgebras and intrinsic numerical algorithms

Preliminary work about intrinsic numerical integrators evolving on groups is described. Fix a finite dimensional Lie group G; let g denote its Lie algebra, and let Y(sub 1),...,Y(sub N) denote a basis of g. A class of numerical algorithms is presented that approximate solutions to differential equations evolving on G of the form: dot-x(t) = F(x(t)), x(0) = p is an element of G. The algorithms depend upon constants c(sub i) and c(sub ij), for i = 1,...,k and j is less than i. The algorithms have the property that if the algorithm starts on the group, then it remains on the group. In addition, they also have the property that if G is the abelian group R(N), then the algorithm becomes the classical Runge-Kutta algorithm. The Cayley algebra generated by labeled, ordered trees is used to generate the equations that the coefficients c(sub i) and c(sub ij) must satisfy in order for the algorithm to yield an rth order numerical integrator and to analyze the resulting algorithms.

Crouch, Peter↗
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗
DOE OSTI2014

Control of collective network chaos

Under certain conditions, the collective behavior of a large globally-coupled heterogeneous network of coupled oscillators, as quantified by the macroscopic mean field or order parameter, can exhibit low-dimensional chaotic behavior. Recent advances describe how a small set of “reduced” ordinary differential equations can be derived that captures this mean field behavior. Here, we show that chaos control algorithms designed using the reduced equations can be successfully applied to imperfect realizations of the full network. To systematically study the effectiveness of this technique, we measure the quality of control as we relax conditions that are required for the strict accuracy of the reduced equations, and hence, the controller. Although the effects are network-dependent, we show that the method is effective for surprisingly small networks, for modest departures from global coupling, and even with mild inaccuracy in the estimate of network heterogeneity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2011

p4est : Scalable Algorithms for Parallel Adaptive Mesh Refinement on Forests of Octrees

We present scalable algorithms for parallel adaptive mesh refinement and coarsening (AMR), partitioning, and 2:1 balancing on computational domains composed of multiple connected two-dimensional quadtrees or three-dimensional octrees, referred to as a forest of octrees. By distributing the union of octants from all octrees in parallel, we combine the high scalability proven previously for adaptive single-octree algorithms with the geometric flexibility that can be achieved by arbitrarily connected hexahedral macromeshes, in which each macroelement is the root of an adapted octree. A key concept of our approach is an encoding scheme of the interoctree connectivity that permits arbitrary relative orientations between octrees. Based on this encoding we develop interoctree transformations of octants. These form the basis for high-level parallel octree algorithms, which are designed to interact with an application code such as a numerical solver for partial differential equations. We have implemented and tested these algorithms in the p4est software library. We demonstrate the parallel scalability of p4est on its own and in combination with two geophysics codes. Using p4est we generate and adapt multioctree meshes with up to $5.13\times10^{11}$ octants on as many as 220,320 CPU cores and execute the 2:1 balance algorithm in less than 10 seconds per million octants per process.

Mathematics↗
DOE OSTI2025

Towards a Quantum Algorithm for the Incompressible Nonlinear Navier-Stokes Equations

In this work, we present novel concepts for quantum algorithms to solve transient, nonlinear partial differential equations (PDEs). The challenge lies in how to effectively represent, encode, process, and evolve the nonlinear system of PDEs on quantum computers. We will discuss the new techniques using the incompressible Navier-Stokes equations as an example, because it represents the fundamental nonlinear feature and yet removes certain complexity in physics, allowing us to focus on the design of quantum algorithms. Previous attempts solving nonlinear PDEs in quantum computation have often involved storing multiple copies of solutions or employing linearizations. Neither is practical due to exponential scaling with evolution time or insufficient solution accuracy. We propose a new framework based on matrix product states (MPSs) and matrix product operators (MPOs), in addition to the Krylov subspace methods. For example, the solution variables of the Navier-Stokes equations are represented by MPSs, and the linear and nonlinear terms are processed by MPOs. The time evolution of the operators is attained by a fast-forwarding algorithm using Krylov subspace methods. Furthermore, we discuss various techniques for efficient encoding of MPSs, measurement reduction for MPOs, and use of tensor operations to treat multi-variate, multi-physics characteristics of Navier-Stokes.

Gopalakrishnan Meena, Murali [ORNL] (ORCID:0000000↗
NASA NTRS1998

Numerical Investigation of Hot Gas Ingestion by STOVL Aircraft

This report compiles the various research activities conducted under the auspices of the NASA Grant NAG3-1026, "Numerical Investigation of Hot Gas Ingestion by STOVL Aircraft" during the period of April 1989 to April 1994. The effort involved the development of multigrid based algorithms and computer programs for the calculation of the flow and temperature fields generated by Short Take-off and Vertical Landing (STOVL) aircraft, while hovering in ground proximity. Of particular importance has been the interaction of the exhaust jets with the head wind which gives rise to the hot gas ingestion process. The objective of new STOVL designs to reduce the temperature of the gases ingested into the engine. The present work describes a solution algorithm for the multi-dimensional elliptic partial-differential equations governing fluid flow and heat transfer in general curvilinear coordinates. The solution algorithm is based on the multigrid technique which obtains rapid convergence of the iterative numerical procedure for the discrete equations. Initial efforts were concerned with the solution of the Cartesian form of the equations. This algorithm was applied to a simulated STOVL configuration in rectangular coordinates. In the next phase of the work, a computer code for general curvilinear coordinates was constructed. This was applied to model STOVL geometries on curvilinear grids. The code was also validated in model problems. In all these efforts, the standard k-Epsilon model was used.

Vanka, S. P.↗
DOE OSTI2019

Evaluating Optimization Strategies for Engine Simulations Using Machine Learning Emulators

This work evaluates different optimization algorithms for computational fluid dynamics (CFD) simulations of engine combustion. Due to the computational expense of CFD simulations, emulators built with machine learning algorithms were used as surrogates for the optimizers. Two types of emulators were used: a Gaussian process (GP) and a weighted variety of machine learning methods called SuperLearner (SL). The emulators were trained using a dataset of 2048 CFD simulations that were run concurrently on a supercomputer. The design of experiments (DOE) for the CFD runs was obtained by perturbing nine input parameters using a Monte-Carlo method. The CFD simulations were of a heavy duty engine running with a low octane gasoline-like fuel at a partially premixed compression ignition mode. Ten optimization algorithms were tested, including types typically used in research applications. Each optimizer was allowed 800 function evaluations and was randomly tested 100 times. The optimizers were evaluated for the median, minimum, and maximum merits obtained in the 100 attempts. Some optimizers required more sequential evaluations, thereby resulting in longer wall clock times to reach an optimum. The best performing optimization methods were particle swarm optimization (PSO), differential evolution (DE), GENOUD (an evolutionary algorithm), and micro-genetic algorithm (GA). These methods found a high median optimum as well as a reasonable minimum optimum of the 100 trials. Moreover, all of these methods were able to operate with less than 100 successive iterations, which reduced the wall clock time required in practice. Two methods were found to be effective but required a much larger number of successive iterations: the DIRECT and MALSCHAINS algorithms. A random search method that completed in a single iteration performed poorly in finding optimum designs but was included to illustrate the limitation of highly concurrent search methods. The last three methods, Nelder–Mead, bound optimization by quadratic approximation (BOBYQA), and constrained optimization by linear approximation (COBYLA), did not perform as well.

Probst, Daniel M.↗
NASA NTRS1987

A survey of parallel multigrid algorithms

A typical multigrid algorithm applied to well-behaved linear-elliptic partial-differential equations (PDEs) is described. Criteria for designing and evaluating parallel algorithms are presented. Before evaluating the performance of some parallel multigrid algorithms, consideration is given to some theoretical complexity results for solving PDEs in parallel and for executing the multigrid algorithm. The effect of mapping and load imbalance on the partial efficiency of the algorithm is studied.

Chan, Tony F.↗
DOE OSTI2013

A Note on Euler Approximations for Stochastic Differential Equations with Delay

An existence and uniqueness theorem for a class of stochastic delay differential equations is presented, and the convergence of Euler approximations for these equations is proved under general conditions. Moreover, the rate of almost sure convergence is obtained under local Lipschitz and also under monotonicity conditions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Using machine learning to predict the correlation of spectra using SDSS magnitudes as an improvement to the Locus Algorithm

The Locus Algorithm is a new technique to improve the quality of differential photometry by optimising the choices of reference stars. At the heart of this algorithm is a routine to assess how good each potential reference star is by comparing its SDSS magnitude values to those of the target star. In this way, the difference in wavelength-dependent effects of the Earth’s atmospheric scattering between target and reference can be minimised. This paper sets out a new way to estimate the quality of each reference star using machine learning. A random subset of stars from SDSS with spectra was chosen. For each one, a suitable reference star, also with a spectrum, was chosen. The correlation between the two spectra in the SDSS r band (between 550 nm and 700 nm) was taken to be the gold-standard measure of how well they match up for differential photometry. The five SDSS magnitude values for each of these stars were used as predictors. A number of supervised machine learning models were constructed on a training set of the stars and were each evaluated on a testing set. The model using Support Vector Regression had the best performance of these models. It was then tested on a final, hold-out, validation set of stars to get an unbiased measure of its performance. With an R 2 of 0.62, the SVR model presents enhanced performance for the Locus Algorithm technique.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2011

Moment transport equations for the primordial curvature perturbation

In a recent publication, we proposed that inflationary perturbation theory can be reformulated in terms of a probability transport equation, whose moments determine the correlation properties of the primordial curvature perturbation. In this paper we generalize this formulation to an arbitrary number of fields. We deduce ordinary differential equations for the evolution of the moments of ζ on superhorizon scales, which can be used to obtain an evolution equation for the dimensionless bispectrum, f{sub NL}. Our equations are covariant in field space and allow identification of the source terms responsible for evolution of f{sub NL}. In a model with M scalar fields, the number of numerical integrations required to obtain solutions of these equations scales like O(M{sup 3}). The performance of the moment transport algorithm means that numerical calculations with M >> 1 fields are straightforward. We illustrate this performance with a numerical calculation of f{sub NL} in Nflation models containing M ∼ 10{sup 2} fields, finding agreement with existing analytic calculations. We comment briefly on extensions of the method beyond the slow-roll approximation, or to calculate higher order parameters such as g{sub NL}.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Performance and Feature Improvements in Parareal-based Power System Dynamic Simulation

In recent years, a novel Parareal-based approach has been developed for fast transient simulations of large power system interconnections. Parareal belongs to the class of Parallel-in-time algorithms for solution of systems of differential-algebraic equations in parallel over an interval of time. The selection of a reasonably fast and accurate coarse solution is crucial to improve the performance of Parareal algorithm. Semi-analytical solution methods are one promising approach to achieve this goal. They have been investigated, and some preliminary results are presented here. In addition, Parareal-based simulator has been expanded to enable co-simulation with OpenDSS, a widely used open-source distribution system simulator. Preserving the parallel nature of the Parareal approach and taking advantage of the parallel capabilities of the latest versions of OpenDSS, each distribution system can be solved in their entirety on different processors in parallel within the main Parareal simulator. This paper also presents the structure of the transmission and distribution co-simulation and some results with different dynamic models of inverter-based resources in the distribution systems.

Park, Byungkwon↗