Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep neural network (DNN)”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

DeepClone: Lightweight State Replication of Deep Learning Models for Data Parallel Training

Training modern deep neural network (DNN) models involves complex workflows triggered by model exploration, sensitivity analysis, explainability, etc. A key primitive in this context is the ability to clone a model training instance, i.e. “fork” the training process in a potentially different direction, which enables comparisons of different evolution paths using variations of training data and model parameters. However, in a quest improve the training throughput, a mix of data parallel, model parallel, pipeline parallel and layer-wise parallel approaches are making the problem of cloning highly complex. In this paper, we explore the problem of efficient cloning under such circumstances. To this end, we leverage several properties of data-parallel training and layer-wise parallelism to design DeepClone, a cloning approach based on augmenting the execution graph to gain direct access to tensors, which are then sharded and reconstructed asynchronously in order to minimize runtime overhead, standby duration, readiness duration. Compared with state-of-art approaches, DeepClone shows orders of magnitude improvement for several classes of DNN models.

data-parallel training↗
DOE OSTI2021

Progressive transfer learning for low-frequency data prediction in full waveform inversion

To effectively overcome the cycle-skipping issue in full waveform inversion (FWI), we developed a deep neural network (DNN) approach to predict the absent low-frequency components by exploiting the hidden physical relation connecting the low- and the high-frequency data. To efficiently solve this challenging nonlinear regression problem, two novel strategies were proposed to design the DNN architecture and to optimize the learning process: (1) dual data feed structure; (2) progressive transfer learning. With the dual data feed structure, not only the high-frequency data, but also the corresponding beat tone data are fed into the DNN to relieve the burden of feature extraction. The second strategy, progressive transfer learning, enables us to train the DNN using a single evolving training dataset. Within the framework of the progressive transfer learning, the training dataset continuously evolves in an iterative manner by gradually retrieving the subsurface information through the physics-based inversion module, progressively enhancing the prediction accuracy of the DNN and propelling the inversion process out of the local minima. Here, the synthetic numerical experiments suggest that, without any a priori geological information, the low-frequency data predicted by the progressive transfer learning are sufficiently accurate for an FWI engine to produce reliable subsurface velocity models free of cycle-skipping artifacts.

02 PETROLEUM↗
DOE OSTI2025

An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture

This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.

42 ENGINEERING↗
DOE OSTI2020

Chapter 14: Machine Learning of Combustion LES Models from Reacting Direct Numerical Simulation

In this chapter we demonstrate how supervised deep learning techniques can be used to construct models for the filtered progress variable source term necessary for large eddy simulation (LES). The source data for the model is a direct numerical simulation (DNS) of a reacting flow in a low swirl burner configuration. Filtered quantities taken from the DNS data are used to train a deep neural network (DNN)-based model. An efficient data sampling strategy was devised to ensure that a uniform representation of all the states observed in the filtered DNS data are equally present in the training dataset. A-priori testing of the DNN-based model highlights the representative power of DNN to accurately reproduce the filtered reaction progress variable source term over a range of scales and various flame regimes as seen in an industrial burner.

combustion LES models↗
DOE OSTI2025

Multi-objective surrogate-assisted calibration of CPFEM models using macroscopic response and in situ EBSD measurements of grain reorientation trajectories

Crystal plasticity finite element method (CPFEM) models are widely used to simulate the deformation behaviour of polycrystalline materials, but their calibration is often limited by their high computational cost and the non-convexity of the optimisation landscape. Here, this study develops a multi-objective surrogate-assisted calibration workflow that couples a multi-objective genetic algorithm (MOGA) with an adaptively trained deep neural network (DNN) surrogate model to efficiently identify CPFEM parameters from experimental data. The workflow is demonstrated on three crystal plasticity (CP) formulations of increasing complexity — Voce hardening (VH), two-coefficient latent hardening (LH2), and six-coefficient latent hardening (LH6) — using in situ electron backscatter diffraction (EBSD) measurements of Alloy 617 under uniaxial tensile loading. The CPFEM models are calibrated against the experimentally observed stress–strain response and reorientation trajectories of eight grains, then validated against eight additional trajectories and overall texture evolution. Across the CP formulations, the macroscopic response was reproduced reliably, while differences emerged in the robustness and accuracy of the grain-scale predictions. Including grain reorientation trajectories in the multi-objective calibration improved texture evolution predictions and filtered out physically inconsistent parameter sets that can arise from calibrating against only the stress–strain data. The workflow also demonstrates good transferability of calibrated parameters from a low- to a high-fidelity microstructural model. These results provide practical guidance for integrating in situ microstructural data into CPFEM through efficient, repeatable, and physically meaningful multi-objective calibration.

Crystal plasticity finite element method↗
DOE OSTI2025

A hybrid numerical and machine learning framework for evaluating the performance of a 780 cm 2 aqueous organic redox flow battery

Aqueous organic redox flow battery (AORFB) is a promising cost-competitive technology for large-scale energy storage. Among existing work, the dihydroxyphenazine (DHP)-based AORFB has demonstrated high energy density and low capacity degradation in 10 cm2 cells during lab tests. However, its commercial-scale performance in more complex environments remains unknown, posing a barrier for commercialization. To address this gap, this work presents a comprehensive performance evaluation of a 780 cm 2 DHP-based AORFB by combining physics-based numerical model, machine learning (ML)-based surrogate models, and ML-derived sensitivity quantification. Specifically, we first select 12 key battery parameters that include 10 physicochemical quantities and 2 operation quantities, then select 6 performance metrics that include energy efficiency (EE), discharging capacity, charging energy, and power losses due to concentration, activation, and ohmic over-potentials. With such selection, 12800 combinations of the 12 parameters are subsequently generated using the Latin Hypercube Sampling method. These combinations, together with 38 pre-defined State of Charge, are then integrated to a validated AORFB model developed in COMSOL to compute the performance metrics. With both input parameters and performance metrics, 60 deep neural network (DNN) surrogate models are then trained to approximate the relationship between the 10 physicochemical quantities and 6 performance metrics at each flow rate and current density. Sensitivity scores are then calculated based on the DNN models. Two additional sensitivity analysis tools, i.e., MARS, and SHAP, are also used to cross-validate the sensitivity scores from the DNN. The results demonstrate that 1) the standard potential ranks the first in controlling EE and charging energy, 2) the membrane conductivity is most critical for power loss and EE, and 3) specific area and reaction rate control activation power loss.

25 ENERGY STORAGE↗
DOE OSTI2025

An investigation on machine learning predictive accuracy improvement and uncertainty reduction using VAE-based data augmentation

The confluence of ultrafast computers with large memory, rapid progress in Machine Learning (ML) algorithms, and the availability of large datasets place multiple engineering fields at the threshold of dramatic progress. However, a unique challenge in nuclear engineering is data scarcity because experimentation on nuclear systems is usually more expensive and time-consuming than most other disciplines. One potential way to resolve the data scarcity issue is deep generative learning, which uses certain ML models to learn the underlying distribution of existing data and generate synthetic samples that resemble the real data. In this way, one can significantly expand the dataset to train more accurate predictive ML models. In this study, our objective is to evaluate the effectiveness of data augmentation using variational autoencoder (VAE)-based deep generative models. We investigated whether the data augmentation leads to improved accuracy in the predictions of a deep neural network (DNN) model trained using the augmented data. Additionally, the DNN prediction uncertainties are quantified using Bayesian Neural Networks (BNN) and conformal prediction (CP) to assess the impact on predictive uncertainty reduction. To test the proposed methodology, we used TRACE simulations of steady-state void fraction data based on the NUPEC Boiling Water Reactor Full-size Fine-mesh Bundle Test (BFBT) benchmark. Here, we found that augmenting the training dataset using VAEs has improved the DNN model’s predictive accuracy, improved the prediction confidence intervals, and reduced the prediction uncertainties.

Bayesian neural network↗
DOE OSTI2023

Machine Learning Automated Analysis of Enormous Synchrotron X-ray Diffraction Datasets

X-ray diffraction (XRD) data analysis can be a time-consuming and laborious task. Deep neural network (DNN) based models trained with synthetic XRD patterns have been proven to be a highly efficient, accurate, and automated method for analyzing common XRD data collected from solid samples in ambient environments. However, it remains unclear whether synthetic XRD-based models can be effective in solving micro(μ)-XRD mapping data for in situ experiments involving liquid phases, which always have lower quality and significant artifacts. In this study, we collected μ-XRD mapping data from a LaCl 3 -calcite hydrothermal fluid system and trained two categories of models to analyze the experimental XRD patterns. Here, the models trained solely with synthetic XRD patterns showed low accuracy (as low as 64%) when solving experimental μ-XRD mapping data. However, the accuracy of the DNN models significantly improved (90% or above) when we trained them with a data set containing both synthetic and a small number of labeled experimental μ-XRD patterns. This study highlights the importance of labeled experimental patterns in training DNN models to solve μ-XRD mapping data from in situ experiments involving liquid phases.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2021

Condensed Phase Water Molecular Multipole Moments from Deep Neural Network Models Trained on Ab Initio Simulation Data

Ionic solvation phenomena in liquids involve intense interactions in the inner solvation shell. For interactions beyond the first shell, the ion–solvent interaction energies result from the sum of many smaller-magnitude contributions that can still include polarization effects. Deep neural network (DNN) methods have recently found wide application in developing efficient molecular models that maintain near-quantum accuracy. In this work, we extend the DeePMD-kit code to produce accurate molecular multipole moments in the bulk and near interfaces. The new method is validated by comparing the DNN moments with those generated by ab initio simulations. The moments are used to compute the electrostatic potential at the center of a molecular-sized hydrophobic cavity in water. The results show that the fields produced by the DNN models are in quantitative agreement with the AIMD-derived values. These efficient methods will open the door to more accurate solvation models for large solutes such as proteins.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Resistive Switching of Spinel Li 4 Ti 5 O 12 Lithium-Ion Battery Material for Neuromorphic Computing

The rapid rise of AI has exposed significant limitations in conventional Von Neumann computing architecture, particularly in regard to speed and energy efficiency. To address these challenges, researchers are exploring a brain-inspired neuromorphic architecture that mimics biological neural networks, enabling massive parallel processing with reduced power consumption for complex AI computational demands. Recent interest has focused on utilizing battery electrodes and solid electrolyte materials for their resistive switching properties in developing a neuromorphic architecture. These properties are precisely tuned through local- and bulk-level chemical composition modifications via voltage bias stimuli. In this study, we demonstrate fabricating a three-terminal lithium-ion electrochemical transistor based on lithium titanium oxide (Li 4 Ti 5 O 12 ), a popular lithium-ion battery anode material. We deposited and characterized LTO thin films using RF sputtering, demonstrating a 6 orders of magnitude increase in electronic conductivity upon lithiation, with conductivity plateauing after 20% lithiation. Density functional theory calculations revealed transformation from the insulating to conducting state, supported by experimental characterization through X-Ray Photoelectron Spectroscopy (XPS) and Direct Current (DC) polarization analyses. The fabricated transistor consisted of LTO as the channel layer, gold as source/drain terminals, lithium phosphorus oxynitride (LiPON) as the lithium-ion conductor, and copper as the gate terminal. The device exhibited clear hysteresis in transfer characteristics due to lithium insertion/extraction processes. Long-term potentiation (LTP) and long-term depression (LTD) measurements showed an asymmetric ratio of 1.425 and maximum/minimum conductance ratio of 7.83. When implemented in a deep neural network (DNN) for MNIST handwritten digit recognition, the device achieved 92.03% accuracy over 20 training epochs. Detailed transport mechanism analysis revealed the crucial role of oxygen vacancies and interface effects in device operation. Our preliminary findings establish LTO-based lithium-ion electrochemical transistors as promising candidates for energy-efficient neuromorphic computing applications, offering potential solutions to traditional Von Neumann architecture limitations.

25 ENERGY STORAGE↗
DOE OSTI2020

Physics-Informed Deep Neural Networks for Learning Parameters and Constitutive Relationships in Subsurface Flow Problems

In this paper, we present a physics informed deep neural network (DNN) method for estimating parameters and unknown physics (constitutive relationships) in partial differential equation (PDE) models. We use PDEs in addition to measurements to train DNNs to approximate unknown parameters and constitutive relationships as well as states. The proposed approach increases the accuracy of DNN approximations of partially known functions when a limited number of measurements is available and allows for training DNNs when no direct measurements of the functions of interest are available. We employ physics informed DNNs to estimate the unknown space-dependent diffusion coefficient in a linear diffusion equation and an unknown constitutive relationship in a non-linear diffusion equation. For the parameter estimation problem, we assume that partial measurements of the coefficient and states are available and demonstrate that under these conditions, the proposed method is more accurate than state-of-the-art methods. For the non-linear diffusion PDE model with a fully unknown constitutive relationship (i.e., no measurements of constitutive relationship are available), the physics informed DNN method can accurately estimate the non-linear constitutive relationship based on state measurements only. Finally, we demonstrate that the proposed method remains accurate in the presence of measurement noise.

42 ENGINEERING↗
DOE OSTI2021

Enhance Low Level Temperature and Moisture Profiles Through Combining NUCAPS, ABI Observations, and RTMA Analysis

Thermodynamic information from low levels in the atmosphere is crucial for operational weather forecasts and meteorological researchers. The NOAA Unique Combined Atmospheric Processing System (NUCAPS) sounding products have been proven beneficial to fill the data gap between synoptic radiosonde observations (RAOBs). However, compared with the upper troposphere, the accuracy of NUCAPS soundings in the low levels still needs improvement. In this study, a deep neural network (DNN) is applied to fuse multiple data sources to enhance the NUCAPS temperature and moisture profiles in the lower atmosphere. The network is developed by combining satellite observations, including NUCAPS sounding retrievals and high resolution geostationary satellite observations from the Advanced Baseline Imager, and surface analysis from the Real-Time Mesoscale Analysis (RTMA) as inputs, while collocated soundings from ECMWF re-analysis version 5 are used as the benchmark for the training. The performance of the model is evaluated by using the independent testing data set, data from a different year, as well as collocated RAOBs, showing improvement to the temperature and moisture profiles by reducing the root-mean-squared-error (RMSE) by more than 30% in the lower atmosphere (from 700 hPa to surface) in both clear sky and partially cloudy conditions. A convective event from June 18, 2017 is presented to illustrate the application of the enhanced low level soundings on high impact weather events. The enhanced soundings from fused data capture the large surface-based convective available potential energy structures in the preconvection environment, which is very useful for severe storm nowcasting and forecasting applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Thermal Experiments for Fractured Rock Characterization: Theoretical Analysis and Inverse Modeling

Abstract Field‐scale properties of fractured rocks play a crucial role in many subsurface applications, yet methodologies for identification of the statistical parameters of a discrete fracture network (DFN) are scarce. We present an inversion technique to infer two such parameters, fracture density and fractal dimension, from cross‐borehole thermal experiments data. It is based on a particle‐based heat‐transfer model, whose evaluation is accelerated with a deep neural network (DNN) surrogate that is integrated into a grid search. The DNN is trained on a small number of the heat‐transfer model runs and predicts the cumulative density function of the thermal field. The latter is used to compute fine posterior distributions of the (to be estimated) parameters. Our synthetic experiments reveal that fracture density is well constrained by data, while fractal dimension is harder to determine. Adding nonuniform prior information related to the DFN connectivity improves the inference of this parameter.

Zhou, Zitong↗
DOE OSTI2023

A Coupled Deep Learning Model for Estimating Surface NO 2 Levels from Remote Sensing Data: 15-Year Study Over the Contiguous United States

This study proposes a novel two-step deep learning (DL) model for estimating surface NO 2 concentrations using satellite data over the contiguous United States (CONUS) from 2005 to 2019. The first phase of the model uses partial convolutional neural network (PCNN), an advanced DL model that accurately imputes gaps between surface NO 2 stations and creates 5,478 daily-mean NO 2 grids (PCNN-NO 2 ) of the 2005-2019 period over the study area. We then feed the PCNN-NO 2 , along with other predictor variables, into a deep neural network (DNN) to estimate surface NO 2 levels, achieving exceptional performance with a correlation coefficient of 0.975 to 0.978, a mean absolute bias of 0.99 ppb to 1.38 ppb, and a root mean square error of 1.47 ppb to 1.97 ppb. Spatial cross-validation results also indicate strong spatial performance of PCNN-DNN surface NO 2 estimates. In addition to its accurate estimates, the PCNN-DNN model consistently generates estimated NO 2 grids without any missing values, improving the quality of various applications such as emission reduction strategies and public health studies. Between 2005 and 2019, the 5,478 daily estimated NO 2 grids over the CONUS reveal significant reductions in NO 2 levels in fourteen major urban environments: Washington D.C. (-43%), New York (-45%), Los Angeles (-38%), Chicago (-25%), Boston (-43%), Houston (-34%), Dallas (-40%), Philadelphia (-41%), Phoenix (-38%), Detroit (-20%), Denver (-23%), Atlanta (-0.7%), Cincinnati (-38%), and Pittsburgh (-56%). Furthermore, the study shows that the denser urban regions that in-situ stations are installed in, the higher the difference between in-situ observations and regional-mean NO 2 levels.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

A Medium‐Sized Paleo‐Tsunami Reconstruction by a Deep Neural Network Processing Sedimentary Deposits

Abstract Reconstructing the magnitude and recurrence time of tsunamis, one of the most destructive and unpredictable natural hazards impacting coastal communities, is essential. While major tsunamis are the most studied due to their disastrous impact, small/medium tsunamis (SMTs) are much more frequent and can still significantly impact the coast. Therefore, SMTs potentially provide an extensive archive of information preserved in the geological record. Analyzing the deposits of small/medium paleo‐tsunamis (SMPTs) opens a window into when their direct observation was unavailable. However, deposits of SMPTs are often degraded, traditional sediment deposition inversion models might fail. Recent research has shown that Deep Neural Networks (DNN) can effectively reconstruct the flow conditions of major tsunamis from their deposits. We evaluate the effectiveness of this approach in reconstructing the characteristics of a recent medium size tsunami (2006 Java) and of a medium paleo‐tsunami (1929 Grand Banks). We successfully reconstruct the flow characteristics of the 2006 Java event and show that an inversion of comparable quality is possible for the 1929 Grand Banks tsunami, despite greater uncertainties due to the deposit degradation. Our research shows that Machine Learning has the potential to unseal the meaning of data of thousands SMPTs.

Batubo, P.↗
DOE OSTI2025

Simulating Atmospheric Processes in Earth System Models and Quantifying Uncertainties With Deep Learning Multi‐Member and Stochastic Parameterizations

Abstract Deep learning is a powerful tool to represent subgrid processes in climate models, but many application cases have so far used idealized settings and deterministic approaches. Here, we develop stochastic parameterizations with calibrated uncertainty quantification to learn subgrid convective and turbulent processes and surface radiative fluxes of a superparameterization embedded in an Earth System Model (ESM). We explore three methods to construct stochastic parameterizations: (a) a single Deep Neural Network (DNN) with Monte Carlo Dropout; (b) a multi‐member parameterization; and (c) a Variational Encoder Decoder with latent space perturbation. We show that the multi‐member parameterization improves the representation of convective processes, especially in the planetary boundary layer, compared to individual DNNs. The respective uncertainty quantification illustrates that methods (b) and (c) are advantageous compared to a dropout‐based DNN parameterization regarding the spread of convective processes. Hybrid simulations with our best‐performing multi‐member parameterizations remained challenging and crash within the first days. Therefore, we develop a pragmatic partial coupling strategy relying on the superparameterization for condensate emulation. Partial coupling reduces the computational efficiency of hybrid Earth‐like simulations but enables model stability over 5 months with our multi‐member parameterizations. However, our hybrid simulations exhibit biases in thermodynamic fields and differences in precipitation patterns. Despite this, the multi‐member parameterizations enable improvements in reproducing tropical extreme precipitation compared to a traditional convection parameterization. Despite these challenges, our results indicate the potential of a new generation of multi‐member machine learning parameterizations leveraging uncertainty quantification to improve the representation of stochasticity of subgrid effects.

Behrens, Gunnar [Deutsches Zentrum für Luft‐ und R↗
DOE OSTI2020

Deep neural network improves the estimation of polygenic risk scores for breast cancer

Polygenic risk scores (PRS) estimate the genetic risk of an individual for a complex disease based on many genetic variants across the whole genome. Here, we compared a series of computational models for estimation of breast cancer PRS. A deep neural network (DNN) was found to outperform alternative machine learning techniques and established statistical algorithms, including BLUP, BayesA, and LDpred. In the test cohort with 50% prevalence, the Area Under the receiver operating characteristic Curve (AUC) were 67.4% for DNN, 64.2% for BLUP, 64.5% for BayesA, and 62.4% for LDpred. BLUP, BayesA, and LPpred all generated PRS that followed a normal distribution in the case population. However, the PRS generated by DNN in the case population followed a bimodal distribution composed of two normal distributions with distinctly different means. This suggests that DNN was able to separate the case population into a high-genetic-risk case subpopulation with an average PRS significantly higher than the control population and a normal-genetic-risk case subpopulation with an average PRS similar to the control population. This allowed DNN to achieve 18.8% recall at 90% precision in the test cohort with 50% prevalence, which can be extrapolated to 65.4% recall at 20% precision in a general population with 12% prevalence. Interpretation of the DNN model identified salient variants that were assigned insignificant p values by association studies, but were important for DNN prediction. These variants may be associated with the phenotype through nonlinear relationships.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2023

A deep neural network regressor for phase constitution estimation in the high entropy alloy system Al-Co-Cr-Fe-Mn-Nb-Ni

High Entropy Alloys (HEAs) are composed of more than one principal element and constitute a major paradigm in metals research. The HEA space is vast and an exhaustive exploration is improbable. Therefore, a thorough estimation of the phases present in the HEA is of paramount importance for alloy design. Machine Learning presents a feasible and non-expensive method for predicting possible new HEAs on-the-fly. A deep neural network (DNN) model for the elemental system of: Mn, Ni, Fe, Al, Cr, Nb, and Co is developed using a dataset generated by high-throughput computational thermodynamic calculations using Thermo-Calc. The features list used for the neural network is developed based on literature and freely available databases. A feature significance analysis matches the reported HEAs phase constitution trends on elemental properties and further expands it by providing so far-overlooked features. The final regressor has a coefficient of determination ( r 2 ) greater than 0.96 for identifying the most recurrent phases and the functionality is tested by running optimization tasks that simulate those required in alloy design. The DNN developed constitutes an example of an emulator that can be used in fast, real-time materials discovery/design tasks.

36 MATERIALS SCIENCE↗