SEARCH · Search NASA
Results for “Neural network models”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Uncertainty Quantification in Computational Science: from Physical Models to Neural Networks
Tutorial at the upcoming BIRS workshop https://www.birs.ca/events/2025/5-day-workshops/25w5381/schedule
Establishing Digital Twins for Non-Proliferation of Secure Facilities under Low Data Availability with Agent-Based Modeling, Deep Neural Network Surrogates, and Virtual Reality
Explore the source record for details and available documents.
A Statistician’s Overview of Physics-Informed Neural Networks for Spatio-Temporal Data
The recent success of deep neural network models with physical constraints (so-called, Physics-Informed Neural Networks, PINNs) has led to renewed interest in the incorporation of mechanistic information in predictive models. Statisticians and others have long been interested in this problem, which has led to several practical and innovative solutions dating back decades. In this overview, we focus on the problem of data-driven prediction and inference of dynamic spatio-temporal processes that include mechanistic information, such as would be available from partial differential equations, with a strong focus on the quantification of uncertainty associated with data, process, and parameters. Here, we give a brief review of several paradigms and focus our attention on Bayesian implementations given they naturally accommodate uncertainty quantification. We then show that it is straight-forward to include the Bayesian PINN (B-PINN) within the Bayesian hierarchical model (BHM) framework that has long been considered for modeling dynamic spatio-temporal processes. Such a BHM-PINN is illustrated via a simulation study in which a latent nonlinear Burgers’ equation PDE governs the dynamics of Poisson distributed spatio-temporal data. Supplementary materials for this article are available online, including a standardized description of the materials available for reproducing the work.
Physics-Informed Recurrent Neural Networks to Predict Reactor Operations of the AGN-201 Nuclear Reactor
4 page paper submitted to ANS Student conference. Summary of paper similar to the following abstract: The ability to predict how a reactor will operate, understand when anomalous conditions arise, and ensure a reactor is being operated as expected is crucial for deploying new nuclear facilities. Digital twins serve as a unique solution to recognizing reactor behavior; however, they require data to be useful. For next-generation reactors, this data may not currently be available. To explore how synthetic physics-informed reactor data can be used to predict reactor operations, a recurrent neural network was implemented for the Idaho State University AGN-201 digital twin. The goal of this work is to determine how synthetic data can be used to train a recurrent neural network model for predicting the reactor power of the AGN-201. The recurrent neural network was validated using both synthetic and real operational data. We envision this approach will help bridge the gap between the virtual and physical sides of a digital twin, where reactor physics models based on as-built data can be corrected for actual operating parameters to ensure the virtual model mirrors reality.
DRDMannTurb: A Python package for scalable, data-driven synthetic turbulence
Synthetic turbulence models (STMs) are used in wind engineering to generate realistic flow fields and are employed as inputs to industrial wind simulations. Examples include prescribing inlet conditions in large eddy simulations that model loads on wind turbines and tall buildings. We are interested in STMs capable of generating fluctuations based on prescribed second-moment statistics since such models can simulate environmental conditions that closely resemble on-site observations. To this end, the widely used Mann model (see Mann, 1994, 1998) is the inspiration for DRDMannTurb. The Mann model is described by three physical parameters: a magnitude parameter influencing the global variance of the wind field and corresponding to the Kolmogorov constant multiplied by the rate of viscous dissipation of the turbulent kinetic energy to the two-thirds, αϵ 2/3 , a turbulence length scale parameter L, and a nondimensional parameter Γ related to the lifetime of the eddies. A number of studies, as well as international standards (e.g., those by the International Electrotechnical Commission (IEC)), include recommended values for these three parameters with the goal of standardizing wind simulations according to observed energy spectra. Yet, having only three parameters, the Mann model faces limitations in accurately representing the diversity of observable spectra. This Python package enables users to extend the Mann model and more accurately fit field measurements through flexible neural network models of the eddy lifetime function. Following Keith et al. (2021), we refer to this class of models as Deep Rapid Distortion (DRD) models. DRDMannTurb also includes a general module implementing an efficient method for synthetic turbulence generation based on a domain decomposition technique. This technique is also described in Keith et al. (2021).
Inverse model based error detection in beamline optics
Optics tuning in transfer lines and LINACs can be challenging due to the fact that multiple combinations of machine settings can lead to the same diagnostic output. Moreover, the lack of a periodic solution can limit the ability to infer optics in the same way as rings from BPM signals. Model based approaches are often used to assist with the optics tuning in combination with optimization or parameter estimation. Here we have developed a novel approach using machine learning inverse models trained on a known configuration to detect variations in quadrupole settings without explicitly including them in the model. This paper shows a comparison of neural network models and linear models on both a simulation based study and experimental studies conducted at the AGS to RHIC transfer line at Brookhaven National Lab.
A Data-Driven Framework for Predicting the Sorting and Screening Performance of an Integrated Biomass Feedstock Preprocessing System
The characteristics of mechanically sorted and screened lignocellulosic biomass, such as the mass contents of corn stover anatomical fractions (leaves, husks, stalks, cobs, etc.), can be used to calculate the intermediate feedstock quality attributes “yield” and “purity” that indicate the conversion efficiency of biocrude. No prior study has investigated the correlations from the characteristics of raw biomass and preprocessing unit operation parameters to those intermediate feedstock quality attributes. This work presents a data-driven framework for assessing and predicting the intermediate feedstock quality attributes in an integrated biomass feedstock preprocessing system. Our study used corn stover as a typical type of herbaceous biomass because of its abundance in the U.S. It began with data acquisition of moisture content, particle size distribution, and anatomical fractions of the materials after each unit operation in the system. The objective of this preprocessing system is to minimize husks and leaves and maximizing cobs and stalks by mechanically separating the materials into three streams via disc screen and air separator. Prototype neural network models were then developed to evaluate the feasibility of predicting process outcomes based on measurable parameters. It is found that incorporating physical constraints into these prediction models significantly enhances the accuracy of the predicted yield and purity against the ground truth data. The experimental data and model predictions indicate that decreasing throughput increases purity, while higher throughput results in lower purity. Finally, an optimization problem was introduced to search optimal combinations of feed material properties and preprocessing unit operation parameters, as the intermediate feedstock quality attributes – yield and purity, appeared to be competing factors. The study also suggests the continual need to improve the data-driven framework’s predictability by incorporating more accurate physical models to describe the dynamics in the preprocessing units such as the air separator.
Acoustic-based monitoring and machine learning of component status for microreactor applications
This report provides a description and assessment of recent efforts to couple acoustic-based experimental measurements and characterization with machine learning models in order to enhance structural health monitoring capabilities for nuclear microreactors. With resilient embedded sensors in development by others supported by programs funded by the US Department of Energy’s Office of Nuclear Energy, the work described herein builds upon ongoing efforts to improve non-destructive testing technology that relates measured acoustic signatures to component stresses and/or structural defects, using a combination of new experimental measurements and machine learning architectures. The experimental procedure remained similar to that developed for the previous year’s demonstration of damage detection by the authors, with the same damaged sample tested under similar applied stress conditions. Notably, a new mounting fixture was designed and implemented to improve measurement consistency and a more sophisticated laser Doppler vibrometer was employed to make high-fidelity vibration measurements. Two nominally identical sets of training data were collected for each experimental setup to better understand the repeatability of the experiment and to better test the generality of trained neural network models. Additionally, we obtained new high-quality 3D mode shapes of the damaged test article at various stress and excitation levels, providing greater insights into the physical response of the sample during testing. Previously, we demonstrated that a machine learning model based on a convolutional neural network can predict structural details of an artificially introduced interface (intact, rough cut, smooth cut), and the applied torque level. In this study, we have transitioned to graph-based neural network architectures to better develop and test a flexible framework that is more suitable to being transferred away from controlled benchtop experiments and into more applied settings where less-structured data inputs may be expected. In general, performance testing of a graph neural network on frequency-domain representations of the data indicates strong and consistent identification of test conditions for datasets recorded on damaged components. With goals of predicting damage location and other changing experimental conditions using limited datasets, predictive models using a graph neural network architecture correctly predicted the applied torque level with an accuracy of 85% using only a single measurement point and predicted within one torque level in 95% of test windows. Predictions of damage location had limited success due to the symmetry and minimal number of the damage scenarios presented during model training. Results were ambiguous as to whether the model could detect the location of the artificial damage, or if it was instead learning the location of a given measurement point on the part and subsequently detecting which points were closest to the location of the damage. This finding will be factored into upcoming planned work on damaged graphite components, where new experimental tests with a larger number and variety of damage scenarios are expected to provide improved validation of recent developments in monitoring methodology.
“Understanding Robustness Lottery”: A Geometric Visual Comparative Analysis of Neural Network Pruning Approaches
Deep learning approaches have provided state-of-the-art performance in many applications by relying on large and overparameterized neural networks. However, such networks are very brittle and are difficult to deploy on resource-limited platforms. Model pruning, i.e., reducing the size of the network, is a widely adopted strategy that can lead to a more robust and compact model. Many heuristics exist for model pruning, but our understanding of the pruning process remains limited due to the black-box nature of a neural network model. Empirical studies show that some heuristics improve performance whereas others can make models more brittle. Here, this work aims to shed light on how different pruning methods alter the network’s internal feature representation and the corresponding impact on model performance. To facilitate a comprehensive comparison and characterization of the high-dimensional model feature space, we introduce a visual geometric analysis of feature representations. We evaluated a set of critical geometric concepts decomposed from the commonly adopted classification loss and used them to design a visualization system to compare and highlight the impact of pruning on model performance and feature representation. The proposed tool provides an environment for an in-depth comparison of pruning methods and a comprehensive understanding of how the model responds to common data corruption. By leveraging the proposed visualization, machine learning researchers can reveal the similarities between pruning methods and redundancy in robustness evaluation benchmarks, obtain geometric insights about the differences between pruned models that achieve superior robustness performance, and identify samples that are robust or fragile to model pruning and common data corruption.
A Parallel Alternative for Energy-Efficient Neural Network Training and Inferencing
Energy efficiency of training and inferencing with large neural network models is a critical challenge facing the future of sustainable large-scale machine learning workloads. This paper introduces an alternative strategy, called phantom parallelism, to minimize the net energy consumption of traditional tensor (model) parallelism, the most energy-inefficient component of large neural network training. The approach is presented in the context of feed-forward network architectures as a preliminary, but comprehensive, proof-of-principle study of the proposed methodology. We derive new forward and backward propagation operators for phantom parallelism, implement them as custom autograd operations within an end-to-end phantom parallel training pipeline and compare its parallel performance and energy-efficiency against those of conventional tensor parallel training pipelines. Formal analyses that predict lower bandwidth and FLOP counts are presented with supporting empirical results on up to 256 GPUs that corroborate these gains. Experiments are shown to deliver ∼50% reduction in the energy consumed to train FFNs using the proposed phantom parallel approach when compared with conventional tensor parallel methods. Additionally, the proposed approach is shown to train smaller phantom models to the same model loss on smaller GPU counts as larger tensor parallel models on larger GPU counts offering the possibility for even greater energy savings.
a priori uncertainty quantification of reacting turbulence closure models using Bayesian neural networks
While many physics-based closure model forms have been posited for the sub-filter scale (SFS) in large eddy simulation (LES), vast amounts of data available from direct numerical simulations (DNS) create opportunities to leverage data-driven modeling techniques. Albeit flexible, data-driven models still depend on the dataset and the functional form of the model chosen. Increased adoption of such models requires reliable uncertainty estimates both in the data-informed and out-of-distribution regimes. Here, in this work, we employ Bayesian neural networks (BNNs) to capture both epistemic and aleatoric uncertainties in a reacting flow model. In particular, we model the filtered progress variable scalar dissipation rate which plays a key role in the dynamics of turbulent premixed flames. We demonstrate that BNN models can provide unique insights about the structure of uncertainty of the data-driven closure models. We also propose a method for the incorporation of out-of-distribution information in a BNN, which can be used for out-of-distribution query detection. The efficacy of the model is demonstrated by a priori evaluation on a dataset consisting of a variety of flame conditions and fuels.
Out-of-distribution detection with non-parametric density estimation for models predicting processing history of uranium ore concentrates
The rapid advancement in machine learning (ML) and computer vision (CV) coincides with the growth of interest in deploying these ML/CV models in numerous fields from medicine to social science. Similar to those areas, we have witnessed a great number of works in materials science employing ML/CV models – neural networks in particular – in their studies in recent years. These models have proven to obtain accurate performance in various tasks. However, these models struggle to attain a similar performance when encountering test samples coming from a distribution that is different from the training set. More importantly, they fail without providing any warning to the users. Therefore, we propose a framework for detecting out-of-distribution (OOD) samples to alert users when a human intervention might be necessary in this work. Specifically, we explore the use of a non-parametric density estimation method to detect OOD samples. Here, we assess OOD detection capability of the proposed framework on ML models developed for categorizing precipitation routes of U 3 O 8 when encountering OOD datasets that contain samples (1) undergone different imaging acquisition process, (2) undergone different material synthesis process, and (3) different materials than ID set. Through those experiments, we achieve an average area under the receiver operating characteristic (AUROC) of at least 91% on average in detecting OOD samples. With minimal overhead cost and superior performance, the proposed framework enables a reliable and safe system when deploying in real-world scenarios.
Enhancing Drinking Water Quality Modeling: Leveraging Physics Informed Neural Networks for Learning with Imperfect Reaction Models and Partial Data
Chemical kinetics models, typically formulated as systems of ordinary or partial differential equations, are valuable tools for simulating drinking water quality. However, these models often face inaccuracies due to discrepancies between the laboratory and the real-world conditions, as well as limitations in experimental analytical methods, hindering the accurate representation of the true underlying chemical mechanisms. In this study, we propose a Physics Informed Neural Network (PINN), using the eXtreme Theory of Functional Connections, to improve the prediction of chemical concentrations over time. The PINN method accounts for imperfect chemical models and incorporates partial data to improve predictions. Focusing on reactions describing water disinfection residual and disinfectant byproduct formation, which are crucial for public health and regulatory compliance, we demonstrate that the PINN model is able to accurately predict the concentrations of chemical species across various pH values. Notably, the model extends its accuracy to predict concentrations of chemical species not originally included in its training data. The developed method can be extended to a variety of chemical systems, offering a wide array of potential applications.
Machine learning neutrino-nucleus cross sections
Neutrino-nucleus scattering cross sections are critical theoretical inputs for long-baseline neutrino oscillation experiments. However, robust modeling of these cross sections remains challenging. For a simple but physically motivated toy model of the DUNE experiment, we demonstrate that an accurate neural-network model of the cross section—leveraging only Standard-Model symmetries—can be learned from near-detector data. We perform a neutrino oscillation analysis with simulated far-detector events, finding that oscillation analysis results enabled by our data-driven cross-section model approach the theoretical limit achievable with perfect prior knowledge of the cross section. We further quantify the effects of flux shape and detector resolution uncertainties as well as systematics from cross-section mismodeling. This proof-of-principle study highlights the potential of future neutrino near-detector datasets and data-driven cross-section models.
Machine Learning Neutrino-Nucleus Cross Sections
Neutrino-nucleus scattering cross sections are critical theoretical inputs for long-baseline neutrino oscillation experiments. However, robust modeling of these cross sections remains challenging. For a simple but physically motivated toy model of the DUNE experiment, we demonstrate that an accurate neural-network model of the cross section -- leveraging Standard Model symmetries -- can be learned from near-detector data. We then perform a neutrino oscillation analysis with simulated far-detector events, finding that the modeled cross section achieves results consistent with what could be obtained if the true cross section were known exactly. This proof-of-principle study highlights the potential of future neutrino near-detector datasets and data-driven cross-section models.
Machine Learning Neutrino-Nucleus Cross Sections
Neutrino-nucleus scattering cross sections are critical theoretical inputs for long-baseline neutrino oscillation experiments. However, robust modeling of these cross sections remains challenging. For a simple but physically motivated toy model of the DUNE experiment, we demonstrate that an accurate neural-network model of the cross section—leveraging only Standard-Model symmetries— can be learned from near-detector data. We perform a neutrino oscillation analysis with simulated far-detector events, finding that oscillation analysis results enabled by our data-driven cross-section model approach the theoretical limit achievable with perfect prior knowledge of the cross section. We further quantify the effects of flux shape and detector resolution uncertainties as well as systematics from cross-section mismodeling. This proof-of-principle study highlights the potential of future neutrino near-detector datasets and data-driven cross-section models.
A deep learning approach to fast analysis of collective Thomson scattering spectra
Fast analysis of collective Thomson scattering ion acoustic wave features using a deep convolutional neural network model is presented. The network was trained from spectra to predict the plasma parameters, including ion velocities, population fractions, and ion and electron temperatures. A fully kinetic particle-in-cell simulation was used to model a laboratory astrophysics experiment and simulate a diagnostic image of the ion acoustic wave feature. Network predictions were compared with Bayesian inference of the plasma model parameters for both the simulated and experimentally measured images. Both approaches were fairly accurate predicting the simulated image and the network predictions matched a good portion of the Bayesian results for the experimentally measured image. The Bayesian approach is more robust to noise and motivates future work to train deep learning models with realistic noise. The advantage of the deep learning model is making thousands of predictions in a few hundred milliseconds, compared to a few seconds to minutes per prediction for the optimization and Bayesian approaches presented here. The results demonstrate promising capabilities of deep learning models to analyze Thomson data orders of magnitude faster than conventional methods when using the neural network for standalone analysis. If more rigorous analysis is needed, neural network predictions can be used to quickly initialize other optimization methods and increase chances of success. This is especially useful when the dataset becomes very large or highly dimensional and manually refining initial conditions for the entire dataset are no longer tractable.