Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Network Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 379 records · Page 21
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Machine learning based reconstruction of intracardiac electrical behavior based on electrocardiograms

A computer-based system and process are disclosed for reconstructing the internal electrical behavior of a patient's heart based partly or wholly on the patient's electrocardiogram (ECG). The output of the process may include, for example, a cardiac activation map, and/or a representation of transmembrane potentials over time. The process advantageously does not require any medical imaging of the patient, and does not require any special medical equipment. For example, the patient's activation map and transmembrane potentials may be reconstructed based solely on a preexisting or newly-obtained 12-lead cardiac ECG of the patient. The process makes use of a machine learning model, such as a neural network based model, trained with actual and/or simulated ECGs and intracardiac electrical data (typically transmembrane potentials) of many thousands of patients. Because an insufficient quantity of such data exists for actual patients, model training may be performed using ECGs and intracardiac electrical data obtained through computer simulations.

Blake, Robert↗
DOE OSTI2021

Dynamical Large Deviations of Two-Dimensional Kinetically Constrained Models Using a Neural-Network State Ansatz

We use a neural-network ansatz originally designed for the variational optimization of quantum systems to study dynamical large deviations in classical ones. We use recurrent neural networks to describe the large deviations of the dynamical activity of model glasses, kinetically constrained models in two dimensions. We present the first finite size-scaling analysis of the large-deviation functions of the two-dimensional Fredrickson-Andersen model, and explore the spatial structure of the high-activity sector of the South-or-East model. These results provide a new route to the study of dynamical large-deviation functions, and highlight the broad applicability of the neural-network state ansatz across domains in physics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Hierarchical, rotation‐equivariant neural networks to select structural models of protein complexes

Abstract Predicting the structure of multi‐protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage predefined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any precomputed physics‐inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end‐to‐end learning from molecular structures containing tens of thousands of atoms: a point‐based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.

Eismann, Stephan↗
DOE OSTI2025

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling

Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabytescale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling.

Li, Chaojian [ORNL] (ORCID:0000000340309777)↗
DOE OSTI2021

nn-PINNs: Non-Newtonian physics-informed neural networks for complex fluid modeling

Time- and rate-dependent material functions in non-Newtonian fluids in response to different deformation fields pose a challenge in integrating different constitutive models into conventional computational fluid dynamic platforms. Considering their relevance in many industrial and natural settings alike, robust data-driven frameworks that enable accurate modeling of these complex fluids are of great interest. The main goal is to solve the coupled Partial Differential Equations (PDEs) consisting of the constitutive equations that relate the shear stress to the deformation and fully capture the behavior of the fluid under various flow protocols with different boundary conditions. Here, in this work, we present non-Newtonian physics-informed neural networks (nn-PINNs) for solving systems of coupled PDEs adopted for complex fluid flow modeling. The proposed nn-PINN method is employed to solve the constitutive models in conjunction with conservation of mass and momentum by benefiting from Automatic Differentiation (AD) in neural networks, hence avoiding the mesh generation step. nn-PINNs are tested for a number of different complex fluids with different constitutive models and for several flow protocols. These include a range of Generalized Newtonian Fluid (GNF) empirical constitutive models, as well as some phenomenological models with memory effects and thixotropic timescales. nn-PINNs are found to obtain the correct solution of complex fluids in spatiotemporal domains with good accuracy compared to the ground truth solution. We also present applications of nn-PINNs for complex fluid modeling problems with unknown boundary conditions on the surface, and show that our approach can successfully recover the velocity and stress fields across the domain, including the boundaries, given some sparse velocity measurements.

42 ENGINEERING↗
DOE OSTI2023

Optimization with Neural Network Feasibility Surrogates: Formulations and Application to Security-Constrained Optimal Power Flow

In many areas of constrained optimization, representing all possible constraints that give rise to an accurate feasible region can be difficult and computationally prohibitive for online use. Satisfying feasibility constraints becomes more challenging in high-dimensional, non-convex regimes which are common in engineering applications. A prominent example that is explored in the manuscript is the security-constrained optimal power flow (SCOPF) problem, which minimizes power generation costs, while enforcing system feasibility under contingency failures in the transmission network. In its full form, this problem has been modeled as a nonlinear two-stage stochastic programming problem. In this work, we propose a hybrid structure that incorporates and takes advantage of both a high-fidelity physical model and fast machine learning surrogates. Neural network (NN) models have been shown to classify highly non-linear functions and can be trained offline but require large training sets. In this work, we present how model-guided sampling can efficiently create datasets that are highly informative to a NN classifier for non-convex functions. We show how the resultant NN surrogates can be integrated into a non-linear program as smooth, continuous functions to simultaneously optimize the objective function and enforce feasibility using existing non-linear solvers. Overall, this allows us to optimize instances of the SCOPF problem with an order of magnitude CPU improvement over existing methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Predicting critical heat flux with uncertainty quantification and domain generalization using conditional variational autoencoders and deep neural networks

Deep generative models (DGMs) can generate synthetic data samples that closely resemble the original dataset, addressing data scarcity. In this work, we developed a conditional variational autoencoder (CVAE) to augment critical heat flux (CHF) data used for the 2006 Groeneveld lookup table. To compare with traditional methods, a fine-tuned deep neural network (DNN) regression model was evaluated on the same dataset. Both models achieved small mean absolute relative errors, with the CVAE showing more favorable results. Uncertainty quantification (UQ) was performed using repeated CVAE sampling and DNN ensembling. The DNN ensemble improved performance over the baseline, while the CVAE maintained consistent results with less variability and higher confidence. Both models achieved small errors inside and outside the training domain, with slightly larger errors outside. Altogether, the CVAE performed better than the DNN in predicting CHF and exhibited better uncertainty behavior.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2021

Verification of Data-Driven Models of Physical Phenomena using Interpretable Approximation

Machine-learned models, specifically neural networks, are increasingly used as “closures” or “constitutive models” in engineering simulators to represent fine-scale physical phenomena that are too computationally expensive to resolve explicitly. However, these neural net models of unresolved physical phenomena tend to fail unpredictably and are therefore not used in mission-critical simulations. In this report, we describe new methods to authenticate them, i.e., to determine the (physical) information content of their training datasets, qualify the scenarios where they may be used and to verify that the neural net, as trained, adhere to physics theory. We demonstrate these methods with neural net closure of turbulent phenomena used in Reynolds Averaged Navier-Stokes equations. We show the types of turbulent physics extant in our training datasets, and, using a test flow of an impinging jet, identify the exact locations where the neural network would be extrapolating i.e., where it would be used outside the feature-space where it was trained. Using Generalized Linear Mixed Models, we also generate explanations of the neural net (à la Local Interpretable Model agnostic Explanations) at prototypes placed in the training data and compare them with approximate analytical models from turbulence theory. Finally, we verify our findings by reproducing them using two different methods.

42 ENGINEERING↗
DOE OSTI2024

Voltage Calculations in Secondary Distribution Networks via Physics-Inspired Neural Network Using Smart Meter Data

The increasing penetration of distributed energy resources (DERs) leads to voltage issues across distribution networks, necessitating voltage calculations by utilities. Electric model-free voltage calculation offers an enticing solution. However, most researches mainly focus on primary distribution networks ignoring secondary distribution networks and commonly overlook extreme voltage case calculations, which require the model’s extrapolation abilities. Here, in addressing the gaps, this paper presents a customized physics-inspired neural network (PINN) model, the structure of which is inspired by the derived coupled power flow model of primary-secondary distribution networks. To ensure precision and rapid convergence, a crafted training framework for the PINN model is proposed. The PINN’s “structure-mimetic” design enables superior extrapolation for unseen scenarios and enhances physical information awareness. We demonstrate this through two applications: hosting capacity analysis and customer-transformer connectivity. The effectiveness and advantages of the proposed PINN model are validated on two public testing systems and one utility distribution feeder model.

Distribution network↗
DOE OSTI2022

Physics-informed machine learning modeling for predictive control using noisy data

Due to the occurrence of over-fitting at the learning phase, the modeling of chemical processes via artificial neural networks (ANN) by using corrupted data (i.e., noisy data) is an ongoing challenge. Therefore, this work investigates the effect of both Gaussian and non-Gaussian noise on the performance of process-structure based recurrent neural networks (RNN) models, which take the form of partially-connected RNN models in this work, that are used to approximate a class of multi-input-multi-outputs nonlinear systems. Furthermore, two different techniques, specifically Monte Carlo dropout and co-teaching, are utilized in the development of partially-connected RNN models. Here, these two techniques are employed to reduce the over-fitting in ANNs when noisy data is used in the training process and, hence, to improve the open-loop accuracy as well as the closed-loop performance under a Lyapunov-based model predictive controller (MPC). Aspen Plus Dynamics, a well-known high-fidelity process simulator, is used to simulate a large-scale chemical process application in order to demonstrate the anticipated improvements in both open-loop approximation and closed-loop controller performance in the presence of Gaussian and non-Gaussian noise in the data set using physics-informed RNNs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Deep Image Prior Enabled Full Waveform Inversion (Final Technical Report)

MS Student Naveen Gupta worked on the problem of full waveform inversion (FWI) using neural networks as shown in Figure 1. Our goal was to learn a neural network to represent the subsurface velocity model, which when fed into the FWI module (implemented using a numerical forward model of wave equations) produces amplitude estimates that match with ground-truth observations of amplitude. We used neural networks to solve the inverse problem of estimating velocity distributions for a given seismic amplitude data such that, once trained, our neural network model can generate a distribution of velocity profiles for different random vectors fed as inputs to the neural network model.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Thermodynamic Modeling of Complex Solid Solutions in the Lu-H-N System via Graph Neural Network Accelerated Monte Carlo Simulations

Metal hydrides are important across diverse applications, such as hydrogen storage, batteries, gas sensors, nuclear reactions, and high-temperature superconductivity. Previous computational studies of metal hydrides under extreme pressures, e.g., 𝑂⁡(10 2 ) ⁢GPa, usually treat them as stoichiometric compounds without considering interstitial lattice disorder. As pressures become more moderate in the 𝑂⁡(10 0 ) ⁢GPa and below range, hydrogen disorder at interstitial lattice sites becomes prominent, e.g., in the N-doped Lu hydride that was recently claimed superconducting near 1 GPa. Further adding compositional complexity from alloying and/or multielement interstitial occupation makes elucidating pressure- and temperature-dependent observables intractable by first-principles calculations alone. We therefore propose a lattice graph neural-network surrogate modeling approach to predict configuration- and pressure-dependent equation-of-state properties. Their efficiency permits Monte Carlo simulations to calculate Gibbs energies and pressure-dependent phase diagrams, thereby revealing insights into the synthesis conditions required for achieving desired phase equilibria. We demonstrate this concept for the compositionally complex cubic Lu(H,N,Va) 3 system where three constituents (hydrogen, nitrogen and vacancy) have disordered multielement interstitial occupancies and insights into pressure-dependent phase equilibria are critically needed, e.g., N-doping levels can significantly lower dehydrogenation temperatures and provide a new strategy to optimize hydrogen-storage alloys. This work can improve the thermodynamic understanding of the Lu-H-N system and help rational synthesis of N-doped Lu hydrides, but more generally demonstrates an efficient approach to model pressure-dependent thermodynamics of multicomponent solid solutions.

Monte Carlo methods↗
DOE OSTI2022

Machine Learning-Assisted High-Temperature Reservoir Thermal Energy Storage Optimization: Numerical Modeling and Machine Learning Input and Output Files

This data set includes the numerical modeling input files and output files used to synthesize data, and the reduced-order machine learning models trained from the synthesized data for reservoir thermal energy storage site identification. In this study, a machine-learning-assisted computational framework is presented to identify High-Temperature Reservoir Thermal Energy Storage (HT-RTES) site with optimal performance metrics by combining physics-based simulation with stochastic hydrogeologic formation and thermal energy storage operation parameters, artificial neural network regression of the simulation data, and genetic algorithm-enabled multi-objective optimization. A doublet well configuration with a layered (aquitard-aquifer-aquitard) generic reservoir is simulated for cases of continuous operation and seasonal-cycle operation scenarios. Neural network-based surrogate models are developed for the two scenarios and applied to generate the Pareto fronts of the HT-RTES performance for four potential HT-RTES sites. The developed Pareto optimal solutions indicate the performance of HT-RTES is operation-scenario (i.e., fluid cycle) and reservoir-site dependent, and the performance metrics have competing effects for a given site and a given fluid cycle. The developed neural network models can be applied to identify suitable sites for HT-RTES, and the proposed framework sheds light on the design of resilient HT-RTES systems. All the simulations and the neural network model were done by Idaho National Laboratory. A detailed description of the work was reported in publication linked below.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

A publicly available PyTorch-$\mathrm{ABAQUS}$ $\mathrm{UMAT}$ deep-learning framework for level-set plasticity

Here this paper introduces a publicly available PyTorch-ABAQUS deep-learning framework of a family of plasticity models where the yield surface is implicitly represented by a scalar-valued function. In particular, our focus is to introduce a practical framework that can be deployed for engineering analysis that employs a user-defined material subroutine (UMAT/VUMAT) for ABAQUS, which is written in FORTRAN. To accomplish this task while leveraging the back-propagation learning algorithm to speed up the neural-network training, we introduce an interface code where the weights and biases of the trained neural networks obtained via the PyTorch library can be automatically converted into a generic FORTRAN code that can be a part of the UMAT/VUMAT algorithm. To enable third-party validation, we purposely make all the data sets, source code used to train the neural-network-based constitutive models, and the trained models available in a public repository. Furthermore, the practicality of the workflow is then further tested on a dataset for anisotropic yield function to showcase the extensibility of the proposed framework. A number of representative numerical experiments are used to examine the accuracy, robustness and reproducibility of the results generated by the neural network models.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Advancements on Multi-Fidelity Random Fourier Neural Networks: Application to Hurricane Modeling for Wind Energy

Multi-fidelity approaches are emerging as effective strategies in computational science to handle otherwise intractable tasks like Uncertainty Quantification (UQ), training of Machine Learning (ML) models, and optimization, for expensive high-fidelity applications in which the amount of available simulations or data is limited. The main idea is simple: large datasets generated for low-fidelity approximations of the problem at hand are fused with a much sparser dataset for the target (high-fidelity) system. In this paper, we build on our recent success in designing random Fourier Neural Networks (rFNNs) [1] to target problems arising in wind energy applications and in particular problems of interest for hurricane modeling. In this context, data for the high-fidelity models are limited and lower fidelity alternatives are needed. In this work, we introduce a novel multi-fidelity training approach for our rFNNs and demonstrate its use on a simple verification problem and on a hurricane modeling problem in which high-fidelity data are generated via Large-Eddy Simulations (LES), while low-fidelity data are given by a mesoscale model. Initial results demonstrate how the multi-fidelity training approach can improve the quality of the resulting surrogate.

Fourier Neural Networks↗
DOE OSTI2024

Survey of gravitationally lensed objects in HSC imaging (SuGOHI) – X. Strong lens finding in the HSC-SSP using convolutional neural networks

ABSTRACT We apply a novel model based on convolutional neural networks (CNN) to identify gravitationally lensed galaxies in multiband imaging of the Hyper Suprime-Cam Subaru Strategic Program (HSC-SSP) Survey. The trained model is applied to a parent sample of 2350 061 galaxies selected from the $\sim$ 800 deg$^2$ Wide area of the HSC-SSP Public Data Release 2. The galaxies in HSC Wide are selected based on stringent pre-selection criteria, such as multiband magnitudes, stellar mass, star formation rate, extendedness limit, photometric redshift range, etc. The trained CNN assigns a score from 0 to 1, with 1 representing lenses and 0 representing non-lenses. Initially, the CNN selects a total of 20 241 cutouts with a score greater than 0.9, but this number is subsequently reduced to 1522 cutouts after removing definite non-lenses for further visual inspection. We discover 43 grade A (definite) and 269 grade B (probable) strong lens candidates, of which 97 are completely new. In addition, we also discover 880 grade C (possible) lens candidates, 289 of which are known systems in the literature. We identify 143 candidates from the known systems of grade C that had higher confidence in previous searches. Our model can also recover 285 candidate galaxy-scale lenses from the Survey of Gravitationally lensed Objects in HSC Imaging (SuGOHI), where a single foreground galaxy acts as the deflector. Even though group-scale and cluster-scale lens systems are not included in the training, a sample of 32 SuGOHI-c (i.e. group/cluster-scale systems) lens candidates is retrieved. Our discoveries will be useful for ongoing and planned spectroscopic surveys, such as the Subaru Prime Focus Spectrograph project, to measure lens and source redshifts in order to enable detailed lens modelling.

Jaelani, Anton T. (ORCID:0000000162825778)↗