Search NASA⌕ Search

SEARCH · Search NASA

Results for “Deep Neural Networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 199 records · Page 11
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Data and Code for Understanding Generative AI Content with Embedding Models

This repository contains code for the experiments in the paper "Understanding Generative AI Content with Embedding Models". Constructing high-quality features is critical to any quantitative data analysis. While feature engineering was historically addressed by carefully hand-crafting data representations based on domain expertise, deep neural networks (DNNs) now offer a radically different approach. DNNs implicitly engineer features by transforming their input data into hidden feature vectors called embeddings. For embedding vectors produced by foundation models -- which are trained to be useful across many contexts -- we demonstrate that simple and well-studied dimensionality-reduction techniques such as Principal Component Analysis uncover inherent heterogeneity in input data concordant with human-understandable explanations. Of the many applications for this framework, we find empirical evidence that there is intrinsic separability between real samples and those generated by artificial intelligence (AI).

Vargas, Max [Pacific Northwest National Laboratory↗
DOE OSTI2024

Mu2e: Modeling Drift of Ionized Particles with ML

The Mu2e experiment searches for charged lepton flavor violation through muon-to-electron conversion in the field of a nucleus. The signal is a monoenergetic electron with an energy of 104.97 MeV. Its momentum is reconstructed using information from drifting ionized particles in a straw tracker detector. This project analyzes the drift of ionized particles with a deep neural network to help improve the momentum reconstruction process. The model yields a 20% improvement in resolution from a reference linear model.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

Harnessing ML Privacy by Design Through Crossbar Array Non-idealities

Deep Neural Networks (DNNs), handling computeand data-intensive tasks, often utilize accelerators like Resistiveswitching Random-access Memory (RRAM) crossbar for energyefficient in-memory computation. Despite RRAM’s inherent nonidealities causing deviations in DNN output, this study transforms the weakness into strength. By leveraging RRAM non-idealities, the research enhances privacy protection against Membership Inference Attacks (MIAs), which reveal private information from training data. RRAM non-idealities disrupt MIA features, increasing model robustness and revealing a privacy-accuracy tradeoff. Empirical results with four MIAs and DNNs trained on different datasets demonstrate significant privacy leakage reduction with a minor accuracy drop (e.g., up to 2.8% for ResNet-18 with CIFAR-100).

artificial intelligence↗
DOE OSTI2024

Generating An Advanced Cross-section Library For HTGR Pebble Bed Depletion Calculations Using Reduced-Order Model Generation Techniques

For code development, Advanced Reactor Technologies - Gas Cooled Reactors Program (ART-GCR) rely on a collaboration with the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, but the cross sections generation and the methodology definition is part of this program area goals. Based on previous studies in FY23, the size of microscopic cross section libraries increases rapidly with the number of tabulations, requiring significant amount of memory and drastically slowing down the Griffin calculations when evaluating cross sections via the multivariate linear interpolation approach. Rising to these challenges, this work investigates constructing Reduced-order Models (ROMs) for the multi-group microscopic cross sections to accelerate the cross section evaluation in Griffin. A database of multigroup cross sections is first collected considering all possible parameters that a designer could change for optimization. Down-selection of the ROM techniques afterward shows Deep Neural Network (DNN) as the best candidate when jointly consider memory efficiency, predictive accuracy, computational cost, scalability, flexibility and ease of implementation of the algorithms in comparison to the multidimensional interpolation. This work develops a specific interface that enables the cross section predictions using pre-trained DNN models into Griffin leveraging the existing ROM capabilities. DNNs have been trained for all isotopes for use in Griffin. Preliminary Griffin testing shows that DNNs exhibit exceptional predictive accuracy and the use of DNNs provides orders of magnitude improvement in memory efficiency compared to conventional interpolation techniques. With such ROM techniques, it holds great promise to further increase the fidelity of the Pebble Bed Reactor (PBR) simulation by increasing the number of tabulations/state variables during cross section evaluation, while maintaining the computational cost affordable in Griffin.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

DS-TIDE: Harnessing Dynamical Systems for Efficient Time-Independent Differential Equation Solving

Time-Independent Differential Equations (TIDEs) are central to modeling equilibrium behavior across a wide range of scientific and engineering domains, from electrostatics to porous media flow. Conventional numerical solvers offer reliable solutions but incur significant computational costs due to fine-grained discretization and iterative procedures. Machine learning-based approaches address this by replacing iterative solving processes with one-time inference; however, their sophisticated models require extensive training resources that often exceed those of traditional solvers. Consequently, designing a TIDE solver that achieves high accuracy, broad applicability, and exceptional computational efficiency remains a fundamental challenge. In this paper, we propose DS-TIDE, a novel hardware solver that is inspired by, and subsequently leverages, the intrinsic connection between Dynamical Systems (DS) and Differential Equations (DEs) to efficiently and accurately solve TIDEs. DS-TIDE employs a CMOS-compatible DS-based processor, whose physical states evolve under carefully designed DE-driven dynamics and naturally converge to equilibrium -- the solution of the target TIDE -- within ~1µs on a ~1-watt DS-TIDE processor. To enhance expressivity, DS-TIDE incorporates Heterogeneous Dynamics with Temporal Layering (HDTL), which solves TIDEs through a three-stage DS evolution -- conditioning, solving, and decoding -- each governed by specialized dynamics. The entire evolution process is analogous to an infinitely deep neural network temporally unrolled, offering the system the capability of representing complex equations. Furthermore, DS-TIDE is equipped with an on-device DS-DE Auto-Alignment mechanism that dynamically adapts intrinsic hardware dynamics within milliseconds, effectively aligning the system’s dynamics to diverse target DEs. Experimental results across TIDEs from a wide range of scientific and engineering domains demonstrate that DS-TIDE achieves ~10^3× speedup, ~10^5× energy savings, and competitive or superior accuracy compared to state-of-the-art numerical and ML-based solvers.

Liu, Chuan↗
DOE OSTI2025

LUNA: LUT-Based Neural Architecture for Fast and Low-Cost Qubit Readout

Qubit readout is a critical operation in quantum computing systems, which maps the analog response of qubits into discrete classical states. Deep neural networks (DNNs) have recently emerged as a promising solution to improve readout accuracy . Prior hardware implementations of DNN-based readout are resource-intensive and suffer from high inference latency, limiting their practical use in low-latency decoding and quantum error correction (QEC) loops. This paper proposes LUNA, a fast and efficient superconducting qubit readout accelerator that combines low-cost integrator-based preprocessing with Look-Up Table (LUT) based neural networks for classification. The architecture uses simple integrators for dimensionality reduction with minimal hardware overhead, and employs LogicNets (DNNs synthesized into LUT logic) to drastically reduce resource usage while enabling ultra-low-latency inference. We integrate this with a differential evolution based exploration and optimization framework to identify high-quality design points. Our results show up to a 10.95x reduction in area and 30% lower latency with little to no loss in fidelity compared to the state-of-the-art. LUNA enables scalable, low-footprint, and high-speed qubit readout, supporting the development of larger and more reliable quantum computing systems.

Farooq, M. A. [Arizona State U., Tempe]↗
DOE OSTI2025

Demystifying the Resilience of Large Language Models: An End-to-End Perspective

Deep neural networks are known to be resilient to random bit-wise faults in their parameters. However, this resilience has primarily been established through evaluations of classification models. The extent to which this claim holds for large-language models remains underexplored. In this work, we conduct an extensive measurement study on the impact of random bitwise faults in commercial-scale language models. We perform an in-depth analysis of the resulting generation outputs. We first expose that these language models are not truly resilient to random bit-flips. While aggregate metrics such as accuracy may suggest resilience, an in-depth inspection of the generated outputs shows significant degradation in text quality. Our analysis also shows that tasks requiring more complex reasoning suffer more from performance and quality degradation. Moreover, we extend our analysis to models with augmented reasoning capabilities, such as Chain-of-Thought or Mixture of Experts architectures, and characterize their failure scenarios under random bit-flips.

Sun, Yu↗
DOE OSTI2025

Deep Learning without Global Optimization by Random Fourier Neural Networks

Here we introduce a new training algorithm for deep neural networks that utilize random complex exponential activation functions. Our approach employs a Markov chain Monte Carlo sampling procedure to iteratively train network layers, avoiding global and gradient-based optimization while maintaining error control. It consistently attains the theoretical approximation rate for residual networks with complex exponential activation functions, determined by network complexity. Additionally, it enables efficient learning of multiscale and high-frequency features, producing interpretable parameter distributions. Despite using sinusoidal basis functions, we do not observe Gibbs phenomena in approximating discontinuous target functions.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Overview of RFID Applications Utilizing Neural Networks

As Radio Frequency Identification (RFID) methods continue to evolve to higher levels of complexity, one form of machine learning is making its appearance. The use of Neural Networks (NN) in the RFID field is steadily increasing, and in the fields of localization and activity recognition, promising results are being shown from a variety of research. RFID applications fall primarily under two types of problems including regression and classification. We analyze RIFD localization techniques which fall under regression, and activity recognition which falls under classification. Many works don’t classify themselves as activity recognition methods, but because they fall under the classification category, we still consider them as activity recognition techniques. This research overviews the Neural Network models in the localization field based on whether they can perform independently of the environment in which they were tested. For activity recognition and accessory fields, the major methods involve tag-based and tag-free approaches. In conclusion, after the models are surveyed, a comparison study is given to examine what may be the cause for increased accuracy between different Neural Network models.

42 ENGINEERING↗
DOE OSTI2025

Three-dimensional reconstruction of inertial confinement fusion hot-spot plasma from x-ray and nuclear diagnostics on OMEGA

Multidimensional effects degrade the neutron yield and the compressed areal density of laser-direct-drive inertial confinement fusion implosions of layered deuterium–tritium cryogenic targets on the OMEGA Laser System with respect to 1D radiation-hydrodynamic simulation predictions. A comprehensive physics-informed 3D reconstruction effort is under way to infer hot-spot and shell conditions at stagnation from four x-ray and seven neutron detectors distributed around the OMEGA target chamber. Neutron diagnostics, providing measurements of the neutron yield, hot-spot flow velocity, and apparent ion-temperature distribution, are used to infer the mode-1 perturbation at stagnation. The x-ray imagers record the shape of the hot-spot plasma to diagnose mode-1 and mode-2 perturbations. A deep-learning convolutional neural network trained on an extensive set of 3D radiation-hydrodynamic simulations is used to interpret the x-ray and nuclear measurements to infer the 3D profiles of the hot-spot plasma conditions and the amount of laser energy coupled to the hot-spot plasma. A 3D simulation database shows that larger mode-1 asymmetries are correlated with higher hot-spot flow velocities and reduced laser-energy coupling and neutron yield. Three-dimensional hot-spot reconstructions from x-ray measurements indicate that higher amounts of residual kinetic energy are correlated with higher measured hot-spot flow velocities, consistent with 3D simulations.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

Physics-informed neural networks for heterogeneous poroelastic media

This study presents a novel physics-informed neural network (PINN) framework for modeling poroelasticity in heterogeneous media with material interfaces. The approach introduces a composite neural network (CoNN) where separate neural networks predict displacement and pressure variables for each material. While sharing identical activation functions, these networks are independently trained for all other parameters. To address challenges posed by heterogeneous material interfaces, the CoNN is integrated with the Interface-PINNs (I-PINNs) framework (Sarma et al., Comput. Methods Appl. Mech. Eng. 429: 117135, 2024), allowing different activation functions across material interfaces. Further, this ensures accurate approximation of discontinuous solution fields and gradients. Performance and accuracy of this combined architecture were evaluated against the conventional PINNs approach, a single neural network (SNN) architecture, and the eXtended PINNs (XPINNs) framework through two one-dimensional benchmark examples with discontinuous material properties. The results show that the proposed CoNN with I-PINNs architecture achieves an RMSE that is two orders of magnitude better than the conventional PINNs approach and is at least 40 times faster than the SNN framework. Compared to XPINNs, the proposed method achieves an RMSE at least one order of magnitude better and is 40% faster.

42 ENGINEERING↗
DOE OSTI2025

Learning and discovering multiple solutions using physics-informed neural networks with random initialization and deep ensemble

In this work we explore the capability of physics-informed neural networks (PINNs) to discover multiple solutions. Many real-world phenomena governed by nonlinear differential equations (DEs), such as fluid flow, exhibit multiple solutions under the same conditions, yet capturing this solution multiplicity remains a significant challenge. A key difficulty lies in providing appropriate initial conditions or guesses, as widely used time-marching schemes and Newton’s method are highly sensitive to these choices when solving complex computational problems. While machine learning models, particularly PINNs, have shown promise in solving DEs, their ability to capture multiple solutions remains underexplored. In this work, we propose a simple and practical approach using PINNs to learn and discover multiple solutions. We first demonstrate that PINNs, when combined with random initialization and deep ensemble method—originally developed for uncertainty quantification—can effectively uncover multiple solutions to nonlinear ordinary and partial DEs. Although training large ensembles of PINNs may appear computationally demanding, this can be done efficiently using vectorization techniques supported by modern deep learning frameworks, allowing many networks to be trained simultaneously. Our approach highlights the critical role of initialization in shaping solution diversity, addressing an often-overlooked aspect of machine learning for scientific computing. Furthermore, we propose utilizing PINN-generated solutions as initial conditions or initial guesses for conventional numerical solvers to enhance accuracy and efficiency in capturing multiple solutions. Extensive numerical experiments, including the Allen–Cahn equation and cavity flow, where our approach successfully identifies both stable and unstable solutions, validate the effectiveness of our method. These findings establish a general and efficient framework for addressing solution multiplicity in nonlinear DEs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Provable Repair of Vision Transformers

Vision Transformers have emerged as state-of-the-art image recognition tools, but may still exhibit incorrect behavior. Incorrect image recognition can have disastrous consequences in safety-critical real-world applications such as self-driving automobiles. In this paper, we present Provable Repair of Vision Transformers (PRoViT), a provable repair approach that guarantees the correct classification of images in a repair set for a given Vision Transformer without modifying its architecture. PRoViT avoids negatively affecting correctly classified images (drawdown) by minimizing the changes made to the Vision Transformer’s parameters and original output. Here, we observe that for Vision Transformers, unlike for other architectures such as ResNet or VGG, editing just the parameters in the last layer achieves correctness guarantees and very low drawdown. We introduce a novel method for editing these last-layer parameters that enables PRoViT to efficiently repair state-of-the-art Vision Transformers for thousands of images, far exceeding the capabilities of prior provable repair approaches.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

“Understanding Robustness Lottery”: A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches

Deep learning approaches have provided state-of-the-art performance in many applications by relying on large and overparameterized neural networks. However, such networks are very brittle and are difficult to deploy on resource-limited platforms. Model pruning, i.e., reducing the size of the network, is a widely adopted strategy that can lead to a more robust and compact model. Many heuristics exist for model pruning, but our understanding of the pruning process remains limited due to the black-box nature of a neural network model. Empirical studies show that some heuristics improve performance whereas others can make models more brittle. Here, this work aims to shed light on how different pruning methods alter the network’s internal feature representation and the corresponding impact on model performance. To facilitate a comprehensive comparison and characterization of the high-dimensional model feature space, we introduce a visual geometric analysis of feature representations. We evaluated a set of critical geometric concepts decomposed from the commonly adopted classification loss and used them to design a visualization system to compare and highlight the impact of pruning on model performance and feature representation. The proposed tool provides an environment for an in-depth comparison of pruning methods and a comprehensive understanding of how the model responds to common data corruption. By leveraging the proposed visualization, machine learning researchers can reveal the similarities between pruning methods and redundancy in robustness evaluation benchmarks, obtain geometric insights about the differences between pruned models that achieve superior robustness performance, and identify samples that are robust or fragile to model pruning and common data corruption.

Li, Zhimin [Univ. of Utah, Salt Lake City, UT (Uni↗
DOE OSTI2024

Infrared-safe energy weighting does not guarantee small nonperturbative effects

Infrared and collinear (IRC) safety has long been used a proxy for robustness when developing new jet substructure observables. This guiding philosophy has been carried into the deep learning era, where IRC-safe neural networks have been used for many jet studies. For graph-based neural networks, the most straightforward way to achieve IRC safety is to weight particle inputs by their energies. However, energy-weighting by itself does not guarantee that perturbative calculations of machine-learned observables will enjoy small nonperturbative corrections. Here, in this paper, we demonstrate the sensitivity of IRC-safe networks to nonperturbative effects, by training an energy flow network (EFN) to maximize its sensitivity to hadronization. We then show how to construct Lipschitz energy flow networks (L-EFNs), which are both IRC safe and relatively insensitive to nonperturbative corrections. We demonstrate the performance of L-EFNs on generated samples of quark and gluon jets, and showcase fascinating differences between the learned latent representations of EFNs and L-EFNs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2024

A physics-informed deep learning description of Knudsen layer reactivity reduction

A physics-informed neural network (PINN) is used to evaluate the fast ion distribution in the hot spot of an inertial confinement fusion target. The use of tailored input and output layers to the neural network is shown to enable a PINN to learn the parametric solution to the Vlasov–Fokker–Planck equation in the absence of any synthetic or experimental data. As an explicit demonstration of the approach, the specific problem of Knudsen layer fusion yield reduction is treated. Here, the predictions from the Vlasov–Fokker–Planck PINN are used to provide a non-perturbative solution of the fast ion tail in the vicinity of the hot spot, thus allowing the spatial profile of the fusion reactivity to be evaluated for a range of collisionalities and hot spot conditions. Excellent agreement is found between the predictions of the Vlasov–Fokker–Planck PINN and the results from traditional numerical solvers with respect to both the energy and spatial distribution of fast ions and the fusion reactivity profile, demonstrating that the Vlasov–Fokker–Planck PINN provides an accurate and efficient means of determining the impact of Knudsen layer yield reduction across a broad range of plasma conditions.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

ESM data downscaling: a comparison of super-resolution deep learning models

Abstract Climate projections at fine spatial resolutions are required to conduct accurate risk assessment for critical infrastructure and design adaptation planning. Generating these projections using advanced Earth system models (ESM) requires significant computational resources. To address this issue, various statistical downscaling techniques have been introduced to generate fine-resolution data from coarse-resolution simulations. In this study, we evaluate and compare five deep learning-based downscaling techniques, namely, super-resolution convolutional neural networks, fast super-resolution convolutional neural network ESM, efficient sub-pixel convolutional neural network, enhanced deep residual network (EDRN), and super-resolution generative adversarial network (SRGAN). These techniques are applied to a dataset generated by the Energy Exascale Earth System Model (E3SM), focusing on key surface variables such as surface temperature, shortwave heat flux, and longwave heat flux. Models are trained and validated using paired fine-resolution (0.25 $$^{\circ }$$ ∘ ) and coarse-resolution (1 $$^{\circ }$$ ∘ ) monthly data obtained from a 9-year simulation. Next, blind testing is performed using monthly data obtained from two different years outside of the training and validation set. To evaluate the efficiency of each technique, different statistical metrics are used, including mean squared error (MSE), peak signal-to-noise ratio (PSNR), structural similarity index measure (SSIM), and learned perceptual image patch similarity (LPIPS). The results show that EDRN outperforms other algorithms in terms of PSNR, SSIM, and MSE, but struggles to capture fine-scale features in the data. In contrast, SRGAN, a generative model that uses perceptual loss, excels in capturing fine details at boundaries and internal structures, resulting in lower LPIPS than other methods.

Pawar, Nikhil M. (ORCID:0000000211613289)↗