Search NASA⌕ Search

SEARCH · Search NASA

Results for “Neural Network Models”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 523 records · Page 29
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Woven ceramic matrix composite surrogate model based on physics-informed recurrent neural network

A recurrent neural network (RNN) based surrogate model is developed to emulate the nonlinear constitutive behavior of woven ceramic matrix composites (CMCs) driven by matrix damage at multiple length scales. Physics-informed constraints are introduced into the surrogate model through regularization to ground the prediction in physics and improve its predictive capabilities. Training data is generated using the multiscale generalized method of cells (MSGMC) approach coupled with a matrix damage model. This coupling permits simulating the nonlinear behavior of woven CMCs based on constituent response at the micro-, meso-, and macroscales. The multiscale repeating unit cell is loaded under non-monotonic conditions including multiple load / unload cycles and tension / compression. The fiber volume fraction as well as the intra- and intertow void volume fractions are also varied in the generation of training data. Therefore, the RNN-based surrogate model is tasked with predicting, as a function of variable input strain sequence and fiber and void volume fractions, the resulting stress versus strain response while satisfying physical constraints such as positive semi-definiteness of the tangent stiffness matrix and linear elastic unloading. Further, the trained surrogate model effectively matches the stress versus strain response and successfully predicts the tangent modulus throughout the loading regime. Neural network based surrogate models can offer efficient alternatives to running computationally intensive multiscale material models to simulate the nonlinear response of large structural models. Therefore the presented work provides evidence towards the feasibility of developing, training, and running such models for CMCs with complex architectures, nonlinear multiaxial material response, and under non-monotonic loading conditions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Predictive Skill of Deep Learning Models Trained on Limited Sequence Data

In this report we investigate the utility of one-dimensional convolutional neural network (CNN) models in epidemiological forecasting. Deep learning models, especially variants of recurrent neural networks (RNNs) have been studied for influenza forecasting, and have achieved higher forecasting skill compared to conventional models such as ARIMA models. In this study, we adapt two neural networks that employ one-dimensional temporal convolutional layers as a primary building block temporal convolutional networks and simple neural attentive meta-learner for epidemiological forecasting and test them with influenza data from the US collected over 2010-2019. We find that epidemiological forecasting with CNNs is feasible, and their forecasting skill is comparable to, and at times, superior to, RNNs. Thus CNNs and RNNs bring the power of nonlinear transformations to purely data-driven epidemiological models, a capability that heretofore has been limited to more elaborate mechanistic/compartmental disease models.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗
DOE OSTI2024

Acceleration of Graph Neural Network-Based Prediction Models in Chemistry via Co-Design Optimization on Intelligence Processing Units

Atomic structure prediction and associated property calculations are the bedrock of chemical physics. Since high-fidelity ab initio modeling techniques for computing the structure and properties can be prohibitively expensive, this motivates the development of machine-learning (ML) models that make these predictions more efficiently. Training graph neural networks over large atomistic databases introduces unique computational challenges such as the need to process millions of small graphs with variable size and support communication patterns that are distinct from learning over large graphs such as social networks. We demonstrate a novel hardware-software co-design approach to scale up the training of atomistic graph neural networks (GNN) for structure and property prediction. First, to eliminate redundant computation and memory associated with alternative padding techniques and to improve throughput via minimizing communication, we formulate the effective coalescing of the batches of variable-size atomistic graphs as the bin packing problem and introduce a hardware-agnostic algorithm to pack these batches. In addition, we propose hardware-specific optimizations including a planner and vectorization for the gather-scatter operations targeted for Graphcore’s Intelligence Processing Unit (IPU), as well as model-specific optimizations such as merged communication collectives and optimized softplus. Putting these all together, we demonstrate the effectiveness of the proposed co-design approach by providing an implementation of a well-established atomistic GNN on the Graphcore IPUs. We evaluate the training performance on multiple atomistic graph databases with varying degrees of graph counts, sizes and sparsity. Here, we demonstrate that such a co-design approach can reduce the training time of atomistic GNNs and can improve the performance by up to 1.5× compared to the baseline implementation of the model on the IPUs. Additionally, we compare our IPU implementation with a Nvidia GPU-based implementation and show that our atomistic GNN implementation on the IPUs can run 1.8× faster on average compared to the execution time on the GPUs.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Neural networks for parameter estimation in intractable models

The goal is to use deep learning models to estimate parameters in statistical models when standard likelihood estimation methods are computationally infeasible. For instance, inference for max-stable processes is exceptionally challenging even with small datasets, but simulation is straightforward. Data from model simulations are used to train deep neural networks and learn statistical parameters from max-stable models. The proposed neural network-based method provides a competitive alternative to current approaches, as demonstrated by considerable accuracy and computational time improvements. Finally, it serves as a proof of concept for deep learning in statistical parameter estimation and can be extended to other estimation problems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Waveform processing using neural network algorithms on the front-end electronics

In a multi-channel radiation detector readout system, waveform sampling, digitization, and raw data transmission to the data acquisition system constitute a conventional processing chain. The deposited energy on the sensor is estimated by extracting peak amplitudes, area under pulse envelopes from the raw data, and starting times of signals or time of arrivals. However, such quantities can be estimated using machine learning algorithms on the front-end Application-Specific Integrated Circuits (ASICs), often termed as “edge computing”. Edge computation offers enormous benefits, especially when the analytical forms are not fully known or the registered waveform suffers from noise and imperfections of practical implementations. In this work, we aim to predict peak amplitude from a single waveform snippet whose rising and falling edges containing only 3 to 4 samples. We thoroughly studied two well-accepted neural network algorithms, Multi-Layer Perceptron (MLP) and Convolutional Neural Network (CNN) by varying their model sizes. Further, to better fit front-end electronics, neural network model reduction techniques, such as network pruning methods and variable-bit quantization approaches, were also studied. By combining pruning and quantization, our best performing model has the size of 1.5 KB, reduced from 16.6 KB of its full model counterpart. It can reach mean absolute error of 0.034 comparing to that of a naive baseline of 0.135. Such parameter-efficient and predictive neural network models established feasibility and practicality of their deployment on front-end ASICs.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Machine Learning for Mapping Multipactor Susceptibility in RF Systems: Capabilities and Generalization Constraints

Multipactor is a surface-driven electron avalanche phenomenon that degrades the performance and reliability of radio-frequency (RF) systems in particle accelerator and vacuum electronics applications. Multipactor behavior in a given device structure is conventionally assessed through susceptibility charts, which provide a parameter-space characterization of the instability. In this work, we assess the capabilities of machine-learning (ML) models to learn and predict such susceptibility charts and analyze the constraints governing their generalization across materials. Using a simulation-derived dataset spanning six distinct secondary-electron-yield material profiles in a canonical two-surface planar geometry, we train supervised regression models and artificial neural networks to predict the time-averaged electron growth rate, δavg, across the relevant parameter space. Model performance is evaluated using metrics that explicitly probe the structure of susceptibility charts, including Intersection over Union, Structural Similarity Index, and correlation analysis. Tree-based ensemble models outperform neural-network models in reconstructing susceptibility regions and in generalizing across material domains. Principal-component analysis reveals disjoint material feature distributions, indicating that the piecewise mode structure of multipactor susceptibility is difficult to represent with a single global model and that generalization is constrained by data coverage rather than by model complexity. An exhaustive reduced-coverage study further shows that sparse material-space coverage can yield mean performance in the same general range but producing large variability in the susceptibility-region overlap. These results clarify the capabilities of ML-based surrogate models for parameter-space characterization of multipactor discharge. They also provide guidance for their appropriate use in RF system design.

43 PARTICLE ACCELERATORS↗
DOE OSTI2025

Machine learning guided selection of broad-spectrum epitope-specific functional antibodies for "Disease X"

Our project established and demonstrated a transfer learning framework that enables prediction of antibody–antigen interactions across related viruses. The approach focused on three major activities: 1. Conserved region and epitope identification – We compared viral protein structures and sequences to identify shared receptor-binding domains and neutralizing epitope regions across variants and related viruses. These conserved features formed the foundation for discovering broadly functional antibodies. 2. Machine learning model development – We built neural network–based models that integrate epitope features with antibody sequence information. Instead of relying solely on structural or physical properties, the models learned transferable patterns that describe antibody binding potential across different viral families. 3. Transfer learning and validation – Using SARS-CoV-2 and Ebola as source systems, we successfully transferred learned epitope features to predict antibody interactions for SARS CoV-1 and Marburg virus. Iterative cycles of dataset generation, retraining, and evaluation improved generalization and predictive power, ensuring the framework can adapt to new threats.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2024

Machine learning based reconstruction of intracardiac electrical behavior based on electrocardiograms

A computer-based system and process are disclosed for reconstructing the internal electrical behavior of a patient's heart based partly or wholly on the patient's electrocardiogram (ECG). The output of the process may include, for example, a cardiac activation map, and/or a representation of transmembrane potentials over time. The process advantageously does not require any medical imaging of the patient, and does not require any special medical equipment. For example, the patient's activation map and transmembrane potentials may be reconstructed based solely on a preexisting or newly-obtained 12-lead cardiac ECG of the patient. The process makes use of a machine learning model, such as a neural network based model, trained with actual and/or simulated ECGs and intracardiac electrical data (typically transmembrane potentials) of many thousands of patients. Because an insufficient quantity of such data exists for actual patients, model training may be performed using ECGs and intracardiac electrical data obtained through computer simulations.

Blake, Robert↗
NASA NTRS1992

Fault detection and diagnosis using neural network approaches

Neural networks can be used to detect and identify abnormalities in real-time process data. Two basic approaches can be used, the first based on training networks using data representing both normal and abnormal modes of process behavior, and the second based on statistical characterization of the normal mode only. Given data representative of process faults, radial basis function networks can effectively identify failures. This approach is often limited by the lack of fault data, but can be facilitated by process simulation. The second approach employs elliptical and radial basis function neural networks and other models to learn the statistical distributions of process observables under normal conditions. Analytical models of failure modes can then be applied in combination with the neural network models to identify faults. Special methods can be applied to compensate for sensor failures, to produce real-time estimation of missing or failed sensors based on the correlations codified in the neural network.

Kramer, Mark A.↗
NASA NTRS1992

Creative-Dynamics Approach To Neural Intelligence

Paper discusses approach to mathematical modeling of artificial neural networks exhibiting complicated behaviors reminiscent of creativity and intelligence of biological neural networks. Neural network treated as non-Lipschitzian dynamical system - as described in "Non-Lipschitzian Dynamics For Modeling Neural Networks" (NPO-17814). System serves as tool for modeling of temporal-pattern memories and recognition of complicated spatial patterns.

Zak, Michail A.↗
DOE OSTI2021

Dynamical Large Deviations of Two-Dimensional Kinetically Constrained Models Using a Neural-Network State Ansatz

We use a neural-network ansatz originally designed for the variational optimization of quantum systems to study dynamical large deviations in classical ones. We use recurrent neural networks to describe the large deviations of the dynamical activity of model glasses, kinetically constrained models in two dimensions. We present the first finite size-scaling analysis of the large-deviation functions of the two-dimensional Fredrickson-Andersen model, and explore the spatial structure of the high-activity sector of the South-or-East model. These results provide a new route to the study of dynamical large-deviation functions, and highlight the broad applicability of the neural-network state ansatz across domains in physics.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Hierarchical, rotation‐equivariant neural networks to select structural models of protein complexes

Abstract Predicting the structure of multi‐protein complexes is a grand challenge in biochemistry, with major implications for basic science and drug discovery. Computational structure prediction methods generally leverage predefined structural features to distinguish accurate structural models from less accurate ones. This raises the question of whether it is possible to learn characteristics of accurate models directly from atomic coordinates of protein complexes, with no prior assumptions. Here we introduce a machine learning method that learns directly from the 3D positions of all atoms to identify accurate models of protein complexes, without using any precomputed physics‐inspired or statistical terms. Our neural network architecture combines multiple ingredients that together enable end‐to‐end learning from molecular structures containing tens of thousands of atoms: a point‐based representation of atoms, equivariance with respect to rotation and translation, local convolutions, and hierarchical subsampling operations. When used in combination with previously developed scoring functions, our network substantially improves the identification of accurate structural models among a large set of possible models. Our network can also be used to predict the accuracy of a given structural model in absolute terms. The architecture we present is readily applicable to other tasks involving learning on 3D structures of large atomic systems.

Eismann, Stephan↗
DOE OSTI2025

Scaling Laws of Graph Neural Networks for Atomistic Materials Modeling

Atomistic materials modeling is a critical task with wide-ranging applications, from drug discovery to materials science, where accurate predictions of the target material property can lead to significant advancements in scientific discovery. Graph Neural Networks (GNNs) represent the state-of-the-art approach for modeling atomistic material data thanks to their capacity to capture complex relational structures. While machine learning performance has historically improved with larger models and datasets, GNNs for atomistic materials modeling remain relatively small compared to large language models (LLMs), which leverage billions of parameters and terabyte-scale datasets to achieve remarkable performance in their respective domains. To address this gap, we explore the scaling limits of GNNs for atomistic materials modeling by developing a foundational model with billions of parameters, trained on extensive datasets in terabytescale. Our approach incorporates techniques from LLM libraries to efficiently manage large-scale data and models, enabling both effective training and deployment of these large-scale GNN models. This work addresses three fundamental questions in scaling GNNs: the potential for scaling GNN model architectures, the effect of dataset size on model accuracy, and the applicability of LLM-inspired techniques to GNN architectures. Specifically, the outcomes of this study include (1) insights into the scaling laws for GNNs, highlighting the relationship between model size, dataset volume, and accuracy, (2) a foundational GNN model optimized for atomistic materials modeling, and (3) a GNN codebase enhanced with advanced LLM-based training techniques. Our findings lay the groundwork for large-scale GNNs with billions of parameters and terabyte-scale datasets, establishing a scalable pathway for future advancements in atomistic materials modeling.

Li, Chaojian [ORNL] (ORCID:0000000340309777)↗
DOE OSTI2021

nn-PINNs: Non-Newtonian physics-informed neural networks for complex fluid modeling

Time- and rate-dependent material functions in non-Newtonian fluids in response to different deformation fields pose a challenge in integrating different constitutive models into conventional computational fluid dynamic platforms. Considering their relevance in many industrial and natural settings alike, robust data-driven frameworks that enable accurate modeling of these complex fluids are of great interest. The main goal is to solve the coupled Partial Differential Equations (PDEs) consisting of the constitutive equations that relate the shear stress to the deformation and fully capture the behavior of the fluid under various flow protocols with different boundary conditions. Here, in this work, we present non-Newtonian physics-informed neural networks (nn-PINNs) for solving systems of coupled PDEs adopted for complex fluid flow modeling. The proposed nn-PINN method is employed to solve the constitutive models in conjunction with conservation of mass and momentum by benefiting from Automatic Differentiation (AD) in neural networks, hence avoiding the mesh generation step. nn-PINNs are tested for a number of different complex fluids with different constitutive models and for several flow protocols. These include a range of Generalized Newtonian Fluid (GNF) empirical constitutive models, as well as some phenomenological models with memory effects and thixotropic timescales. nn-PINNs are found to obtain the correct solution of complex fluids in spatiotemporal domains with good accuracy compared to the ground truth solution. We also present applications of nn-PINNs for complex fluid modeling problems with unknown boundary conditions on the surface, and show that our approach can successfully recover the velocity and stress fields across the domain, including the boundaries, given some sparse velocity measurements.

42 ENGINEERING↗
NASA NTRS2012

Reduced-Order Modeling for Flutter/LCO Using Recurrent Artificial Neural Network

The present study demonstrates the efficacy of a recurrent artificial neural network to provide a high fidelity time-dependent nonlinear reduced-order model (ROM) for flutter/limit-cycle oscillation (LCO) modeling. An artificial neural network is a relatively straightforward nonlinear method for modeling an input-output relationship from a set of known data, for which we use the radial basis function (RBF) with its parameters determined through a training process. The resulting RBF neural network, however, is only static and is not yet adequate for an application to problems of dynamic nature. The recurrent neural network method [1] is applied to construct a reduced order model resulting from a series of high-fidelity time-dependent data of aero-elastic simulations. Once the RBF neural network ROM is constructed properly, an accurate approximate solution can be obtained at a fraction of the cost of a full-order computation. The method derived during the study has been validated for predicting nonlinear aerodynamic forces in transonic flow and is capable of accurate flutter/LCO simulations. The obtained results indicate that the present recurrent RBF neural network is accurate and efficient for nonlinear aero-elastic system analysis

Yao, Weigang↗
DOE OSTI2023

Optimization with Neural Network Feasibility Surrogates: Formulations and Application to Security-Constrained Optimal Power Flow

In many areas of constrained optimization, representing all possible constraints that give rise to an accurate feasible region can be difficult and computationally prohibitive for online use. Satisfying feasibility constraints becomes more challenging in high-dimensional, non-convex regimes which are common in engineering applications. A prominent example that is explored in the manuscript is the security-constrained optimal power flow (SCOPF) problem, which minimizes power generation costs, while enforcing system feasibility under contingency failures in the transmission network. In its full form, this problem has been modeled as a nonlinear two-stage stochastic programming problem. In this work, we propose a hybrid structure that incorporates and takes advantage of both a high-fidelity physical model and fast machine learning surrogates. Neural network (NN) models have been shown to classify highly non-linear functions and can be trained offline but require large training sets. In this work, we present how model-guided sampling can efficiently create datasets that are highly informative to a NN classifier for non-convex functions. We show how the resultant NN surrogates can be integrated into a non-linear program as smooth, continuous functions to simultaneously optimize the objective function and enforce feasibility using existing non-linear solvers. Overall, this allows us to optimize instances of the SCOPF problem with an order of magnitude CPU improvement over existing methods.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2025

Predicting critical heat flux with uncertainty quantification and domain generalization using conditional variational autoencoders and deep neural networks

Deep generative models (DGMs) can generate synthetic data samples that closely resemble the original dataset, addressing data scarcity. In this work, we developed a conditional variational autoencoder (CVAE) to augment critical heat flux (CHF) data used for the 2006 Groeneveld lookup table. To compare with traditional methods, a fine-tuned deep neural network (DNN) regression model was evaluated on the same dataset. Both models achieved small mean absolute relative errors, with the CVAE showing more favorable results. Uncertainty quantification (UQ) was performed using repeated CVAE sampling and DNN ensembling. The DNN ensemble improved performance over the baseline, while the CVAE maintained consistent results with less variability and higher confidence. Both models achieved small errors inside and outside the training domain, with slightly larger errors outside. Altogether, the CVAE performed better than the DNN in predicting CHF and exhibited better uncertainty behavior.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗