Search NASA⌕ Search

SEARCH · Search NASA

Results for “deep neural network”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Predicting critical heat flux with uncertainty quantification and domain generalization using conditional variational autoencoders and deep neural networks

Deep generative models (DGMs) can generate synthetic data samples that closely resemble the original dataset, addressing data scarcity. In this work, we developed a conditional variational autoencoder (CVAE) to augment critical heat flux (CHF) data used for the 2006 Groeneveld lookup table. To compare with traditional methods, a fine-tuned deep neural network (DNN) regression model was evaluated on the same dataset. Both models achieved small mean absolute relative errors, with the CVAE showing more favorable results. Uncertainty quantification (UQ) was performed using repeated CVAE sampling and DNN ensembling. The DNN ensemble improved performance over the baseline, while the CVAE maintained consistent results with less variability and higher confidence. Both models achieved small errors inside and outside the training domain, with slightly larger errors outside. Altogether, the CVAE performed better than the DNN in predicting CHF and exhibited better uncertainty behavior.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Transfer learning nonlinear plasma dynamic transitions in low dimensional embeddings via deep neural networks

Deep learning algorithms provide a new paradigm to study high-dimensional dynamical behaviors, such as those in fusion plasma systems. Development of novel, data-driven model reduction methods, coupled with detection of abnormal modes with plasma physics, opens a unique opportunity to identify plasma instabilities through automated construction of parsimonious models that can be tuned to balance accuracy and cost. Our fusion transfer learning (FTL) model demonstrates success in rapidly reconstructing nonlinear kink mode structures by learning from a limited amount of nonlinear simulation data. The knowledge transfer process leverages a pre-trained neural encoder–decoder network, initially trained on linear simulations, to effectively capture nonlinear dynamics. The low-dimensional embeddings extract the coherent structures of interest, while preserving the inherent dynamics of the complex system. Experimental results highlight FTL’s capacity to capture transitional behaviors and dynamical features in plasma dynamics—a task often challenging for conventional methods. The model developed in this study is generalizable and can be extended broadly through transfer learning to address various magnetohydrodynamics modes.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Deep Neural Networks are Adaptive to Function Regularity and Data Distribution in Approximation and Estimation

Deep learning has exhibited remarkable results across diverse areas. To understand its success, substantial research has been directed towards its theoretical foundations. Nev- ertheless, the majority of these studies examine how well deep neural networks can model functions with uniform regularities. In this paper, we explore a different angle: how deep neural networks can adapt to varying degrees of smoothness in functions and nonuni- form data distributions across different locations and scales. More precisely, we focus on a broad class of functions defined by nonlinear tree-based approximation methods. This class encompasses a range of function types, such as functions with uniform regularities and discontinuous functions. We develop nonparametric approximation and estimation theories for this class using deep ReLU networks. Our results show that deep neural networks are adaptive to the nonuniform smoothness of functions and nonuniform data distributions at different locations and scales. We apply our results to several function classes, and derive the corresponding approximation and generalization errors. The validity of our results is demonstrated through numerical experiments.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Enhanced Ionization Charge Identification in the Short-Baseline Neutrino Program Neutrino Detectors with Deep Neural Networks

We present a deep neural net-based region of interest detection method (DNN ROI) for signal processing in the liquid argon time projection chambers of the Short-Baseline Neutrino (SBN) Program, SBND and ICARUS. DNN ROI addresses limitations of the traditional wire-by-wire thresholding algorithm by leveraging the full two-dimensional detector readout and cross-plane matching information. To account for detector performance variations, we explore training with augmented samples. We find that DNN ROI outperforms the traditional method in both low-level ROI identification performance and high-level reconstruction metrics for high-energy cosmic and accelerator neutrino interaction products, while also being more robust against detector variations, with or without sample augmentation.

Abratenko, P. [Tufts U.]↗
DOE OSTI2025

An adaptive and stability-promoting layerwise training approach for sparse deep neural network architecture

This work presents a two-stage adaptive framework for progressively developing deep neural network (DNN) architectures that generalize well for a given training data set. In the first stage, a layerwise training approach is adopted where a new layer is added each time and trained independently by freezing parameters in the previous layers. We impose desirable structures on the DNN by employing manifold regularization, sparsity regularization, and physics-informed terms. We introduce a ε – δ – stability-promoting concept as a desirable property for a learning algorithm and show that employing manifold regularization yields a ε – δ stability-promoting algorithm. Further, we also derive the necessary conditions for the trainability of a newly added layer and investigate the training saturation problem. In the second stage of the algorithm (post-processing), a sequence of shallow networks is employed to extract information from the residual produced in the first stage, thereby improving the prediction accuracy. Numerical investigations on prototype regression and classification problems demonstrate that the proposed approach can outperform fully connected DNNs of the same size. Moreover, by equipping the physics-informed neural network (PINN) with the proposed adaptive architecture strategy to solve partial differential equations, we numerically show that adaptive PINNs not only are superior to standard PINNs but also produce interpretable hidden layers with provable stability. As a result, we also apply our architecture design strategy to solve inverse problems governed by elliptic partial differential equations.

42 ENGINEERING↗
DOE OSTI2025

Bayesian Optimized Deep Ensemble for Uncertainty Quantification of Deep Neural Networks: a System Safety Case Study on Sodium Fast Reactor Thermal Stratification Modeling

Deep neural networks (DNNs) are increasingly important to scientific computing and engineering system simulations. Accurate uncertainty quantification (UQ) for DNNs is critical in safety-sensitive engineering domains. Traditional Deep Ensemble (DE) methods, while easy to implement, frequently suffer from poorly calibrated uncertainty estimates and limited predictive accuracy due to reliance on fixed architectures with varied weight initializations. To address these issues, we introduce a workflow that combines Bayesian Optimization (BO) and DE. The workflow is modular, scalable, and integrates parallel BO initialized with Sobol sequences to individually optimize the hyperparameters of each ensemble member. This method enhances ensemble diversity, improves predictive accuracy, and provides reliable uncertainty estimates. We evaluate the proposed BODE approach in a sodium fast reactor thermal stratification modeling case study, where we used a densely connected convolutional neural network to predict turbulent viscosity during the reactor transient with consideration of data noise. We benchmark its performance against several optimization approaches, including baseline deep ensemble, evolutionary algorithm-optimized ensemble, ensemble formed via random search combined with greedy selection, and a BO ensemble using random initialization. Here, our results demonstrate superior performance of the developed BODE approach. In noise-free scenarios, BODE notably reduces incorrect aleatoric uncertainty and significantly enhances predictive accuracy. Under conditions of 5% and 10% Gaussian noise, BODE adaptively quantifies uncertainty proportional to data noise, achieving up to an 80% reduction in root mean square error compared to baseline methods and producing well-calibrated prediction intervals.

Bayesian optimization↗
DOE OSTI2025

TorchBraid: High-Performance Layer-Parallel Training of Deep Neural Networks with MPI and GPU Acceleration

TorchBraid is a high-performance implementation of layer-parallel training for deep neural networks (DNNs) supporting MPI-based parallelism and GPU acceleration. Layer-parallel training has been developed to overcome the serialization inherent in forward and backward propagation of DNNs that limits utilization of computational resources in the strong scaling limit. To achieve this, TorchBraid integrates the PyTorch neural network framework with the state-of-the-art XBraid time-parallel library. Furthermore, this article presents the use and performance of TorchBraid, in addition to solutions for overcoming the algorithmic challenges inherent in combining automatic differentiation with layer-parallel. Results are presented with and without GPU acceleration for the Tiny ImageNet and MNIST image classification data sets, as well as recurrent neural networks. Overall, TorchBraid enables fast training of DNNs, both in a strong and weak scaling context. In addition to the TorchBraid software, several new advances in applying layer-parallel algorithms are detailed. Integration of layer-parallel with data-parallel algorithms is presented for the first time, showing the computational advantages of the combination. Standard deep learning techniques, like batch-normalization, are developed for layer-parallel training. Finally, a new approach combining layer-parallel with spatial coarsening in order to accelerate training for 3D image classification shows roughly a 10× speedup over serial execution.

Layer-parallel↗
DOE OSTI2024

Phase retrieval for refraction-enhanced x-ray radiography using a deep neural network

X-ray refraction-enhanced radiography (RER) or phase contrast imaging is widely used to study internal discontinuities within materials. The resulting radiograph captures both the decrease in intensity caused by material absorption along the x-ray path, as well as the phase shift, which is highly sensitive to gradients in density. A significant challenge lies in effectively analyzing the radiographs to decouple the intensity and phase information and accurately ascertain the density profile. Conventional algorithms often yield ambiguous and unrealistic results due to difficulties in including physical constraints and other relevant information. We have developed an algorithm that uses a deep neural network to address these issues and applied it to extract the detailed density profile from an experimental RER. To generalize the applicability of our algorithm, we have developed a technique that quantitatively evaluates the complexity of the phase retrieval process based on the characteristics of the sample and the configuration of the experiment. Accordingly, this evaluation aids in the selection of the neural network architecture for each specific case. Beyond RER, the model has potential applications for other diagnostics where phase retrieval analysis is required.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Automatically parallelizing batch inference on deep neural networks using Fiats and Fortran 2023 `do concurrent`

This paper introduces novel programming strategies that leverage features of the Fortran 2023 standard of the International Standards Organization (ISO) to automatically parallelize computations on deep neural networks. The paper focuses on the interplay of object-oriented, parallel, and functional programming paradigms in the Fiats deep learning library. We demonstrate how several infrequently used language features play a role in enabling efficient, parallel execution. Specifically, the ability to explicitly declare that a procedure is pure facilitates inference in the context of the language’s loop-parallelism construct `do concurrent`. Also, explicitly prohibiting the overriding of a parent type’s type-bound procedures eliminates the need for dynamic dispatch in performance-critical code. Finally, this paper uses batch inference calculations on a neural network surrogate for atmospheric aerosol dynamics to demonstrate that LLVM Flang compiler’s automatic parallelization of `do concurrent` achieves roughly the same performance and scalability as achieved by OpenMP compiler directives. We also demonstrate that double-precision inference costs 37–72% longer runtime than default-real precision with most values in the range 57-60%.

Rouson, Damian↗
DOE OSTI2024

Achieving precise multiparameter measurements with distributed optical fiber sensor using wavelength diversity and deep neural networks

The development of advanced distributed optical fiber sensing systems that are capable of performing accurate and spatially resolved multiparameter measurements is of great interest to a wide range of scientific and industrial applications. Here, in this paper, we propose and experimentally demonstrate a wavelength diversity based advanced distributed optical fiber sensor system to accomplish multiparameter sensing while greatly enhancing measurement accuracy. A suite of deep neural network (DNN) algorithms are developed and verified for data denoising, rapid Brillouin frequency shift estimation, and vibration data event classification. As a proof-of-concept, we demonstrate the effectiveness of the proposed advanced wavelength diversity distributed fiber sensor system assisted by DNN for simultaneous, independent measurements of static strain, temperature, and acoustic vibrations over a 25 km long sensing fiber at 3 m spatial resolution. These results suggest the potential for an intelligent multiparameter monitoring system with enhanced performance in advanced structural health monitoring applications.

47 OTHER INSTRUMENTATION↗
DOE OSTI2024

Modern deep neural networks for Direct Normal Irradiance forecasting: A classification approach

The escalating energy demand and the adverse environmental impacts of fossil-fuel use necessitate a shift towards cleaner and renewable alternatives. Concentrated Solar Power (CSP) technology emerges as a promising solution, offering a carbon-free alternative for power generation. The efficiency and profitability of CSP depend on the Direct Normal Irradiance (DNI) component of solar radiation; hence, accurate DNI forecasting can help optimize CSP plants’ operations and performance. The unpredictable nature of weather phenomena, particularly cloud cover, introduces uncertainty into DNI projections. Existing DNI forecasting models use meteorological factors, which are both challenging to estimate numerically over short prediction windows and expensive to model through data at a sufficiently high spatial and temporal resolution. This research addresses the challenge by presenting a novel approach that formulates DNI prediction as a multi-class classification problem, departing from conventional regression-based methods. The primary objective of this classification framework is to identify optimal periods aligning with specific operational thresholds for CSP plants, contributing to enhanced dispatch optimization strategies. We model the DNI classification problem using four advanced deep neural networks – rectified linear unit (ReLU) networks, 1D residual networks (ResNets), bidirectional long short-term memory (BiLSTM) networks, and transformers – achieving accuracies up to 93.5% without requiring meteorological parameters.

14 SOLAR ENERGY↗
DOE OSTI2025

Wide‐Field Bond Quality Evaluation Using Frequency Domain Thermoreflectance with Deep Neural Network Feature Reconstruction

Heterogeneous integration of microelectronic components provides a pathway to improve circuit/component performance; however, this comes with assembly challenges, in particular due to complex interfaces via subsurface bump bonds. The ability of these bonds to transmit electrical signals and conduct heat to the carrier substrate limits component performance. In this work, hyperspectral frequency‐domain thermoreflectance (FDTR) imaging is demonstrated as a robust technique for evaluating the quality of subsurface indium bump bonds in a surrogate microelectronic sample. By performing microscale FDTR imaging with coarse motion image stitching, thermal phase maps that cover a 4 mm by 4 mm field‐of‐view with subsurface feature sensitivity at depths greater than 50 µm are obtained. The resulting FDTR hyperspectral data contains more than three million pixels and reveal the quality of subsurface microbump arrays. Wide‐field analysis of bonded versus gap regions is enabled by deep neural network feature reconstruction, that after training, rapidly provides an interpretable representation of bond quality. Utility of noisy higher frequency FDTR phase maps, i.e., near the computationally predicted sensing depth limit, results in an average prediction error of 11%. Taken together, FDTR with neural network‐based analysis demonstrates subsurface bond monitoring at length scales relevant for heterogeneously integrated microelectronics.

FDTR↗
DOE OSTI2025

Deep Neural Network Assisted Distributed Strain and Temperature Fiber Sensor System for Natural Gas Pipeline Monitoring

Natural gas pipeline integrity monitoring is crucial to detect potential leaks, find structural issues, and prevent environmental damage. This article presents a system of natural gas pipeline monitoring that uses a specialized double Brillouin peak sensing fiber along with the Brillouin optical time domain analysis (BOTDAs) technique. The calibrated sensing fiber coefficients for strain and temperature are 41.8 kHz/ με and 0.9 MHz/°C for peak 1; and 47.2 kHz/ με , and 1.11 MHz/°C for peak 2, respectively. Initially, lab tests were performed by installing a short section of double Brillouin peak fiber (DBPF) on a 1-in steel pipe under pressure up to 1000 per square inch (psi) at elevated temperatures. Simultaneous distributed measurements of temperature and pressure-induced hoop strain were successfully measured. Considering the long processing speed to extract Brillouin frequency shift (BFS), we employ a novel probabilistic deep neural network (PDNN) framework for rapid BFS prediction. Additionally, using the Finite Element Method, the effects of the pipeline pressure on hoop strain were modeled and compared to the experimental hoop strain under the same set of pipeline conditions. Finally, an actual 4-in outer diameter steel natural gas pipeline was used for pilot-scale tests, where hoop strain was measured at various pressure levels. Leaks were simulated to demonstrate accurate pipeline integrity monitoring. At an internal pipe pressure of 1000 psi, hoop strain of approximately 300 με was observed, and the sensitivity was calculated as 0.28 με /psi. The results of this pilot-scale study demonstrated that the system is capable of performing distributed monitoring sufficient to detect pipeline pressure and the presence of leaks to ensure the safe operation of gas pipelines in the field.

03 NATURAL GAS↗
DOE OSTI2024

Simultaneous energy and mass calibration of large-radius jets with the ATLAS detector using a deep neural network

The energy and mass measurements of jets are crucial tasks for the Large Hadron Collider experiments. This paper presents a new calibration method to simultaneously calibrate these quantities for large-radius jets measured with the ATLAS detector using a deep neural network (DNN). To address the specificities of the calibration problem, special loss functions and training procedures are employed, and a complex network architecture, which includes feature annotation and residual connection layers, is used. The DNN-based calibration is compared to the standard numerical approach in an extensive series of tests. The DNN approach is found to perform significantly better in almost all of the tests and over most of the relevant kinematic phase space. In particular, it consistently improves the energy and mass resolutions, with a 30% better energy resolution obtained for transverse momenta $p$ T > $500$ GeV.

47 OTHER INSTRUMENTATION↗
DOE OSTI2025

Benchmarking Operators in Deep Neural Networks for Improving Performance Portability of SYCL

SYCL is a portable programming model for heterogeneous computing, so it is important to obtain reasonable performance portability of SYCL. Towards the goal of better understanding and improving performance portability of SYCL for machine learning workloads, we have been developing benchmarks for basic operators in deep neural networks (DNNs). These operators could be offloaded to heterogeneous computing devices such as graphics processing units (GPUs) to speed up computation. In this paper, we introduce the benchmarks, evaluate the performance of the operators on GPU-based systems, and describe the causes of the performance gap between the SYCL and Compute Unified Device Architecture (CUDA) kernels. We find that the causes are related to the utilization of the texture cache for read-only data, optimization of the memory accesses with strength reduction, use of local memory, and register usage per thread. We hope that the efforts of developing benchmarks for studying performance portability will stimulate discussion and interactions within the community.

Jin, Zheming [ORNL] (ORCID:000000027197780X)↗
DOE OSTI2026

Improving missing transverse momentum estimation with a deep neural network

At hadron colliders, the net transverse momentum of particles that do not interact with the detector (missing transverse momentum, $^→_𝑝$$^{miss}_{T}$) is a crucial observable in many analyses. In the standard model, $^→_𝑝$$^{miss}_{T}$ originates from neutrinos. Many beyond-the-standard-model particles, such as dark matter candidates, are also expected to leave the experimental apparatus undetected. This paper presents a novel deep neural network based $^→_𝑝$$^{miss}_{T}$ estimator, DeepMET, developed by the CMS Collaboration at the LHC. The DeepMET algorithm produces a weight for each reconstructed particle based on its properties. The estimator is based on the negative vector sum of the weighted transverse momenta of all reconstructed particles in an event. Compared with other estimators currently employed by CMS, DeepMET improves the $^→_𝑝$$^{miss}_{T}$ resolution by 10%–30%, shows improvement for a wide range of final states, is easier to train, and is more resilient against the effects of additional proton-proton interactions accompanying the collision of interest.

artificial neural networks↗
DOE OSTI2024

Distributed-Memory Sparse Deep Neural Network Inference Using Global Arrays

Partitioned Global Address Space (PGAS) models exhibit tremendous promise in developing efficient and productive distributed-memory parallel applications. They have been used extensively in scientific computations due to conveniently offering a ``shared-memory''-like model and convenient interfaces that separate communication with synchronization. Traditionally, PGAS communication models have been applied to dense/contiguously distributed data, but most modern applications depict varied levels of sparsity. Existing PGAS models require certain adaptations to support distributed sparse computations, since associated computations often require matrix arithmetic, in addition to data movement. The Global Arrays toolkit from Pacific Northwest National Laboratory (PNNL) is one of the earliest PGAS models to combine one-sided data communication and distributed matrix operations and is still used in the popular NWChem quantum chemistry suite. Recently, we have expanded the Global Arrays toolkit to support common sparse operations, like sparse matrix-dense matrix multiplies (SpMM), sparse matrix-sparse matrix multiplication (SpGEMM) and Sampled Dense-Dense Matrix Multiplication (SDDMM). As it turns out, these operations are the bedrock of sparse Deep Learning (DL); sparse deep neural networks and Graph Neural Networks (GNNs) have gained increasing attention recently in achieving speedups on training and inference with reduced memory footprints. Unlike scientific applications in High Performance Computing (HPC), modern (distributed-memory capable) DL toolkits often rely on non-standardized and closed-source vendor software optimizations, creating challenges in software-hardware co-design at scale. Our goal is to support a variety of distributed-memory sparse matrix operations and helper functions in the newly created Sparse Global Arrays (SGA), such that it is possible to build portable and productive Machine Learning scenarios for algorithm/software and hardware codesign purposes. Contemporary data-parallel schemes for training/inference are undergoing a major overhaul since model replication limits scalability and causes resource inefficiencies. As such, we have adopted tensor parallelism in decomposing the model and inputs, to mitigate memory issues. Current implementation is built on top of MPI and uses CPUs to maximize the portability across the platforms.

Distributed computing, machine learning↗
DOE OSTI2026

Measuring the Energy Consumption and Efficiency of Deep Neural Networks: An Empirical Analysis and Design Recommendations

Addressing the "Red-AI" trend of rising energy consumption by large-scale neural networks, this study investigates the measured energy consumption of training various fully connected neural network architectures. We introduce the BUTTER-E dataset, an augmentation to the BUTTER Empirical Deep Learning dataset, containing energy consumption and performance data from 41,129 individual experimental runs spanning 30,582 distinct configurations: 13 datasets, 20 sizes (trainable parameters), 8 "shapes", and 14 depths on both CPUs and GPUs using node-level watt-meters. This dataset reveals the complex relationship between dataset size, network structure, and energy use. Our analysis uncovers a surprising, hardware-mediated non-linear relationship between energy efficiency and network design, challenging the assumption that reducing the number of parameters or FLOPs is the best way to achieve greater energy efficiency. We propose a straightforward and effective energy model that accounts for network size, computing, and memory hierarchy. Highlighting the need for cache-considerate algorithm development, we suggest a codesign approach to energy efficient network, algorithm, and hardware design. This work contributes to the fields of sustainable computing and Green AI, offering practical guidance for creating more energy-efficient neural networks and promoting sustainable AI.

97 MATHEMATICS AND COMPUTING↗