Search NASA⌕ Search

SEARCH · Search NASA

Results for “Automatic Differentiation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

On the feasibility of using physics-informed machine learning for underground reservoir pressure management

In this work, we evaluate the feasibility of using physics-informed machine learning (PIML) for underground energy-related pressure management. To this end, we develop a PIML framework to manage underground reservoir pressures by training neural networks to determine fluid extraction rates for dedicated extraction wells during fluid injection operations given a range of reservoir conditions (e.g., transmissivity and storativity). We implement an automatically-differentiable analytical physics model of fluid flow in porous media within the PIML framework as a proxy for more complicated models. This allows us to execute a sufficient number of training scenarios to fully evaluate the feasibility of using PIML to support pressure management activities. We quantify the number of physics-model parameters required for automatic differentiation to become more efficient than finite-difference gradient calculations. We use a simple scenario with a single injector, extractor, and critical location for our feasibility analysis. We evaluate the effect of the size of the training dataset (i.e., the number of reservoir condition samples) on the accuracy and efficiency of the PIML framework. For an equivalent number of model evaluations, the larger training dataset took less time to train and produced a neural network that was able to more accurately manage reservoir pressures. We also evaluate the effect of the training dataset batch size (i.e., number of reservoir condition samples used to update the neural network coefficients during training; i.e., how the training dataset is partitioned). While training ran faster with larger batch sizes, they produced neural networks that managed pressures less accurately. We demonstrate the approach on a more complex scenario involving 10 injectors, 10 extractors, and 4 critical locations (a relatively high well density of 20 wells/km2). We provide the number of forward and adjoint model evaluations required in each case as an indication of the feasibility of using PIML for pressure management when more complicated physics models with longer execution times are used.

54 ENVIRONMENTAL SCIENCES↗

Steady-State Computation of Constant Rotational Rate Dynamic Stability Derivatives

Dynamic stability derivatives are essential to predicting the open and closed loop performance, stability, and controllability of aircraft. Computational determination of constant-rate dynamic stability derivatives (derivatives of aircraft forces and moments with respect to constant rotational rates) is currently performed indirectly with finite differencing of multiple time-accurate computational fluid dynamics solutions. Typical time-accurate solutions require excessive amounts of computational time to complete. Formulating Navier-Stokes (N-S) equations in a rotating noninertial reference frame and applying an automatic differentiation tool to the modified code has the potential for directly computing these derivatives with a single, much faster steady-state calculation. The ability to rapidly determine static and dynamic stability derivatives by computational methods can benefit multidisciplinary design methodologies and reduce dependency on wind tunnel measurements. The CFL3D thin-layer N-S computational fluid dynamics code was modified for this study to allow calculations on complex three-dimensional configurations with constant rotation rate components in all three axes. These CFL3D modifications also have direct application to rotorcraft and turbomachinery analyses. The modified CFL3D steady-state calculation is a new capability that showed excellent agreement with results calculated by a similar formulation. The application of automatic differentiation to CFL3D allows the static stability and body-axis rate derivatives to be calculated quickly and exactly.

Park, Michael A.↗

Intracranial CT angiography obtained from a cerebral CT perfusion examination

CT perfusion (CTP) examinations of the brain are performed increasingly for the evaluation of cerebral blood flow in patients with stroke and vasospasm after subarachnoid hemorrhage. Of the same patient often also a CT angiography (CTA) examination is performed. This study investigates the possibility to obtain CTA images from the CTP examination, thereby possibly obviating the CTA examination. This would save the patient exposure to radiation, contrast, and time. Each CTP frame is a CTA image with a varying amount of contrast enhancement and with high noise. To improve the contrast-to-noise ratio (CNR) we combined all 3D images into one 3D image after registration to correct for patient motion between time frames. Image combination consists of weighted averaging in which the weighting factor of each frame is proportional to the arterial contrast. It can be shown that the arterial CNR is maximized in this procedure. An additional advantage of the use of the time series of CTP images is that automatic differentiation between arteries and veins is possible. This feature was used to mask veins in the resulting 3D images to enhance visibility of arteries in maximum intensity projection (MIP) images. With a Philips Brilliance 64 CT scanner (64x0.625 mm) CTP examinations of eight patients were performed on 80 mm of brain using the toggling table technique. The CTP examination consisted of a time series of 15 3D images (2x64x0.625 mm; 80 kV; 150 mAs each) with an interval of 4 s. The authors measured the CNR in images obtained with weighted averaging, images obtained with plain averaging, and images with maximal arterial enhancement. The authors also compared CNR and quality of the images with that of regular CTA examinations and examined the effectiveness of automatic vein masking in MIP images. The CNR of the weighted averaged images is, on the average, 1.73 times the CNR of an image at maximal arterial enhancement in the CTP series, where the use of plain averaging increases the CNR only with a factor of 1.49. The quality of the weighted averaged images approaches that of CTA images, although in the present study the image quality of CTA was not quite reached. The automatic masking of veins is effective and only small remnants of veins were sometimes present in the masked images. Weighted averaging makes it possible to create CTA images from a CTP examination with a CNR considerably higher than that of images with maximal arterial enhancement. The quality of the resulting images approaches that of CTA images and offers the additional advantages to automatically differentiate between arteries and veins.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Stabilized FE simulation of prototype thermal-hydraulics problems with integrated adjoint-based capabilities

A critical aspect of applying modern computational solution methods to complex multiphysics systems of relevance to nuclear reactor modeling, is the assessment of the predictive capability of specific proposed mathematical models. In this respect the understanding of numerical error, the sensitivity of the solution to parameters associated with input data, boundary condition uncertainty, and mathematical models is critical. Additionally, the ability to evaluate and or approximate the model efficiently, to allow development of a reasonable level of statistical diagnostics of the mathematical model and the physical system, is of central importance. In this study we report on initial efforts to apply integrated adjoint-based computational analysis and automatic differentiation tools to begin to address these issues. The study is carried out in the context of a Reynolds averaged Navier–Stokes approximation to turbulent fluid flow and heat transfer using a particular spatial discretization based on implicit fully-coupled stabilized FE methods. Initial results are presented that show the promise of these computational techniques in the context of nuclear reactor relevant prototype thermal-hydraulics problems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Differentiable programming for online training of a neural artificial viscosity function within a staggered grid Lagrangian hydrodynamics scheme

Lagrangian methods to solve the inviscid Euler equations produce numerical oscillations near shock waves. A common approach to reducing these oscillations is to add artificial viscosity (AV) to the discrete equations. The AV term acts as a dissipative mechanism that attenuates oscillations by smearing the shock across a finite number of computational cells. However, AV introduces several control parameters that are not determined by the underlying physical model, and hence, in practice are tuned to the characteristics of a given problem. We seek to improve the standard quadratic-linear AV form by replacing it with a learned neural function that reduces oscillations relative to exact solutions of the Euler equations, resulting in a hybrid numerical-neural hydrodynamic solver. Because AV is an artificial construct that exists solely to improve the numerical properties of a hydrodynamic code, there is no offline ‘viscosity data’ against which a neural network can be trained before inserting into a numerical simulation, thus requiring online training. We achieve this via differentiable programming, i.e. end-to-end backpropagation or adjoint solution through both the neural and differential equation code, using automatic differentiation of the hybrid code in the Julia programming language to calculate the necessary loss function gradients. A novel offline pre-training step accelerates training by initializing the neural network to the default numerical AV scheme, which can be learned rapidly by space-filling sampling over the AV input space. We find that online training over early time steps of simulation is sufficient to learn a neural AV function that reduces numerical oscillations in long-term hydrodynamic shock simulations. These results offer an early proof-of-principle that online differentiable training of hybrid numerical schemes with novel neural network components can improve certain performance aspects existing in purely numerical schemes.

97 MATHEMATICS AND COMPUTING↗

Physics-Informed Generative Adversarial Networks for Stochastic Differential Equations

We developed a new class of physics-informed generative adversarial networks (PI-GANs) to solve forward, inverse, and mixed stochastic problems in a unified manner based on a limited number of scattered measurements. Unlike standard GANs relying solely on data for training, here we encoded into the architecture of GANs the governing physical laws in the form of stochastic differential equations (SDEs) using automatic differentiation. In particular, we applied Wasserstein GANs with gradient penalty (WGAN-GP) for its enhanced stability compared to vanilla GANs. We first tested WGAN-GP in approximating Gaussian processes of different correlation lengths based on data realizations collected from simultaneous reads at sparsely placed sensors. We obtained good approximation of the generated stochastic processes to the target ones even if there is a mismatch between the input noise dimensionality and the effective dimensionality of the target stochastic processes. We also studied the overfitting issue for both the discriminator and the generator, and we found that overfitting occurs also in the generator in addition to the discriminator as previously reported. Subsequently, we considered the solution of elliptic SDEs requiring approximations of three stochastic processes, namely the solution, the forcing, and the diffusion coefficient. Here again, we assumed data collected from simultaneous reads at a limited number of sensors for the multiple stochastic processes. Furthermore, three generators were used for the PI-GANs: two of them were feed forward deep neural networks (DNNs), while the other one was the neural network induced by the SDE. For the case where we have one group of data, we employed one feed forward DNN as the discriminator, while for the case of multiple groups of data we employed multiple discriminators in PI-GANs. We solved forward, inverse, and mixed problems without changing the framework of PI-GANs, obtaining both the means and the standard deviations of the stochastic solution and the diffusion coefficient in good agreement with benchmarks. In this work, we have demonstrated the effectiveness of PI-GANs in solving SDEs for about 120 dimensions. In principle, PI-GANs could tackle very high dimensional problems given more sensor data with low-polynomial growth in computational cost.

97 MATHEMATICS AND COMPUTING↗

Learning and meta-learning of stochastic advection–diffusion–reaction systems from sparse measurements

Physics-informed neural networks (PINNs) were recently proposed in [18] as an alternative way to solve partial differential equations (PDEs). A neural network (NN) represents the solution, while a PDE-induced NN is coupled to the solution NN, and all differential operators are treated using automatic differentiation. Here, we first employ the standard PINN and a stochastic version, sPINN, to solve forward and inverse problems governed by a non-linear advection–diffusion–reaction (ADR) equation, assuming we have some sparse measurements of the concentration field at random or pre-selected locations. Subsequently, we attempt to optimise the hyper-parameters of sPINN by using the Bayesian optimisation method (meta-learning) and compare the results with the empirically selected hyper-parameters of sPINN. In particular, for the first part in solving the inverse deterministic ADR, we assume that we only have a few high-fidelity measurements, whereas the rest of the data is of lower fidelity. Hence, the PINN is trained using a composite multi-fidelity network, first introduced in [12], that learns the correlations between the multi-fidelity data and predicts the unknown values of diffusivity, transport velocity and two reaction constants as well as the concentration field. For the stochastic ADR, we employ a Karhunen–Loève (KL) expansion to represent the stochastic diffusivity, and arbitrary polynomial chaos (aPC) to represent the stochastic solution. Correspondingly, we design multiple NNs to represent the mean of the solution and learn each aPC mode separately, whereas we employ a separate NN to represent the mean of diffusivity and another NN to learn all modes of the KL expansion. For the inverse problem, in addition to stochastic diffusivity and concentration fields, we also aim to obtain the (unknown) deterministic values of transport velocity and reaction constants. The available data correspond to 7spatial points for the diffusivity and 20 space–time points for the solution, both sampled 2000 times. We obtain good accuracy for the deterministic parameters of the order of 1–2% and excellent accuracy for the mean and variance of the stochastic fields, better than three digits of accuracy. In the second part, we consider the previous stochastic inverse problem, and we use Bayesian optimisation to find five hyper-parameters of sPINN, namely the width, depth and learning rate of two NNs for learning the modes. Here, we obtain much deeper and wider optimal NNs compared to the manual tuning, leading to even better accuracy, i.e., errors less than 1% for the deterministic values, and about an order of magnitude less for the stochastic fields.

97 MATHEMATICS AND COMPUTING↗

Spatiotemporal control of laser intensity using differentiable programming

Optical techniques for spatiotemporal control can produce laser pulses with custom amplitude, phase, or polarization structure. In nonlinear optics and plasma physics, the use of structured pulses typically follows a forward design approach, in which the efficacy of a known structure is analyzed for a particular application. Inverse approaches, in contrast, enable the discovery of new structures with the potential for superior performance. Here, an implementation of the unidirectional pulse propagation equation that supports automatic differentiation is combined with gradient-based optimization to design structured pulses with features that are advantageous for a range of nonlinear optical and plasma-based applications: (1) a longitudinally uniform intensity over an extended region, (2) a superluminal intensity peak that travels many Rayleigh ranges with constant duration, spot size, and amplitude, and (3) a laser pulse that ionizes a gas to form a uniform column of plasma. In the final case, optimizing the full spatiotemporal structure improves the performance by a factor of 15 compared to optimizing only spatial or only temporal structure, highlighting the advantage of spatiotemporal control.

automatic differentiation↗

Dynamic energy system modeling using hybrid physics-based and machine learning encoder–decoder models

Three model configurations are presented for multi-step time series predictions of the heat absorbed by the water and steam in a thermal power plant. The models predict over horizons of 2, 4, and 6 steps into the future, where each step is a 5-minute increment. The evaluated models are a pure machine learning model, a novel hybrid machine learning and physics-based model, and the hybrid model with an incomplete dataset. The hybrid model deconstructs the machine learning into individual boiler heat absorption units: economizer, water wall, superheater, and reheater. Each configuration uses a gated recurrent unit (GRU) or a GRU-based encoder–decoder as the deep learning architecture. Mean squared error is used to evaluate the models compared to target values. The encoder–decoder architecture is over 11% more accurate than the GRU only models. The hybrid model with the incomplete dataset highlights the importance of the manipulated variables to the system. The hybrid model, compared to the pure machine learning model, is over 10% more accurate on average over 20 iterations of each model. Automatic differentiation is applied to the hybrid model to perform a local sensitivity analysis to identify the most impactful of the 72 manipulated variables on the heat absorbed in the boiler. The models and sensitivity analyses are used in a discussion about optimizing the thermal power plant.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Bison Robustness and Performance Improvements

In last year’s Bison robustness report, the authors demonstrated that improving the Jacobian matrix can drastically increase solution convergence rates. The development of an automatic differentiation (AD) capability in the MOOSE framework has created the possibility of having exact Jacobian matrices for all of our material models. A large focus in this year’s report is the application of AD to MOOSE physics modules, such as the tensor mechanics module, and the application to Bison fuel and cladding material models. A second main focus in this year’s report is the investigation of a set of algorithmic solver changes to improve the robustness and performance of Bison LWR simulations. Beyond providing increased solver robustness, the ability to solve strongly coupled multiphysics problems, and solving large deformation problems with ease, these automatically generated perfect Jacobians also drastically reduce development time for new models. In models converted to AD, Jacobian computation made up the majority of the code base. With this code removed, both readability and maintainability of the MOOSE/Bison code base are improved. The work documented here significantly improved the robustness of the Bison LWR assessment case suite. Through algorithmic changes in solver infrastructure, such as the application of predictors and improvements in timestepping, tangible improvements in performance were observed. These improvements are documented using a new standard set of metrics that are collected in all assessment cases, and a suite of statistical analysis tools developed for this report.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A comparative study of calibration techniques for finite strain elastoplasticity: Numerically-exact sensitivities for FEMU and VFM

Accurate identification of material parameters is crucial for predictive modeling in computational mechanics. Here, the two primary approaches in the experimental mechanics community for calibration from full-field digital image correlation data are known as finite element model updating (FEMU) and the virtual fields method (VFM). In VFM, the objective function is a squared mismatch between internal and external virtual work or power. In FEMU, the objective function quantifies the weighted mismatch between model predictions and corresponding experimentally measured quantities of interest. It is minimized by iteratively updating the parameters of an FE model. While FEMU is seen as more flexible, VFM is commonly used instead of FEMU due to its considerably greater computational expense. However, comparisons between the two methods usually involve approximations of gradients or sensitivities with finite difference schemes, thereby making direct assessments difficult. Hence, in this study, we compare VFM and FEMU in the context of numerically-exact sensitivities obtained through local sensitivity analyses and the application of automatic differentiation software. To this end, we conduct a series of test cases to assess both methods under practical challenges using a finite strain elastoplasticity model.

Automatic differentiation↗

A direct-adjoint approach for material point model calibration with application to plasticity

Here, this paper proposes a new approach for the calibration of material parameters in local elastoplastic constitutive models. The calibration is posed as a constrained optimization problem, where the constitutive model evolution equations for a single material point serve as constraints. The objective function quantifies the mismatch between the stress predicted by the model and corresponding experimental measurements. To improve calibration efficiency, a novel direct-adjoint approach is presented to compute the Hessian of the objective function, which enables the use of second-order optimization algorithms. Automatic differentiation is used for gradient and Hessian computations. Two numerical examples are employed to validate the Hessian matrices and to demonstrate that the Newton–Raphson algorithm consistently outperforms gradient-based algorithms such as L-BFGS-B.

36 MATERIALS SCIENCE↗

A set of parallel, implicit methods for a reconstructed discontinuous Galerkin method for compressible flows on 3D hybrid grids

A set of implicit methods are proposed for a third-order hierarchical WENO reconstructed discontinuous Galerkin method for compressible flows on 3D hybrid grids. An attractive feature in these methods are the application of the Jacobian matrix based on the P1 element approximation, resulting in a huge reduction of memory requirement compared with DG (P2). Also, three approaches -- analytical derivation, divided differencing, and automatic differentiation (AD) are presented to construct the Jacobian matrix respectively, where the AD approach shows the best robustness. A variety of compressible flow problems are computed to demonstrate the fast convergence property of the implemented flow solver. Furthermore, an SPMD (single program, multiple data) programming paradigm based on MPI is proposed to achieve parallelism. The numerical results on complex geometries indicate that this low-storage implicit method can provide a viable and attractive DG solution for complicated flows of practical importance.

97 MATHEMATICS AND COMPUTING↗

Density based topology optimization of turbulent flow heat transfer systems

The focus of this article is on topology optimization of heat sinks with turbulent forced convection. The goal is to demonstrate the extendibility, and the scalability of a previously developed fluid solver to coupled multi-physics and large 3D problems. The gradients of the objective and the constraints are obtained with the help of automatic differentiation applied on the discrete system without any simplifying assumptions. Thus, as demonstrated in earlier works of the authors, the sensitivities are exact to machine precision. The framework is applied to the optimization of 2D and 3D problems. Comparison between the simplified 2D setup and the full 3D optimized results is provided. A comparative study is also provided between designs optimized for laminar and turbulent flows. Overall, the comparisons highlight the importance and the benefits of full 3D optimization and including turbulence modeling in the optimization process, while also demonstrating extension of the methodology to include coupling of heat transfer with turbulent flows.

42 ENGINEERING↗