Search NASA⌕ Search

SEARCH · Search NASA

Results for “graph convolutional neural networks”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Geometric learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity

The history-dependent behaviors of classical plasticity models are often driven by internal variables evolved according to phenomenological laws. The difficulty to interpret how these internal variables represent a history of deformation, the lack of direct measurement of these internal variables for calibration and validation, and the weak physical underpinning of those phenomenological laws have long been criticized as barriers to creating realistic models. In this work, geometric machine learning on graph data (e.g. finite element solutions) is used as a means to establish a connection between nonlinear dimensional reduction techniques and plasticity models. Geometric learning-based encoding on graphs allows the embedding of rich time-history data onto a low-dimensional Euclidean space such that the evolution of plastic deformation can be predicted in the embedded feature space. Finally, a corresponding decoder can then convert these low-dimensional internal variables back into a weighted graph such that the dominating topological features of plastic deformation can be observed and analyzed.

42 ENGINEERING↗
DOE OSTI2022

On the integration of molecular dynamics, data science, and experiments for studying solvent effects on catalysis

Computational workflows that combine molecular dynamics (MD) simulations and emerging data-centric (DC) methods can accelerate the screening and analysis of solvent systems experimentally and computationally. Here, MD simulations provide atomic positions and velocities of reactant, solvent, and catalyst materials that can be manipulated into data representations that in turn can be used by DC techniques to conduct predictive modeling, feature extraction, and experimental design. For liquid-phase catalytic applications, emerging DC techniques such as Convolutional and Graph Neural Networks (CNN/GNN), Topological Data Analysis (TDA), and Active Learning (AL) can leverage MD and experimental data to quickly predict solvent effects on reaction outcomes. For instance, in recent studies, 3D solvent environments obtained with MD have been exploited by CNNs to predict experimental reaction rates for homogeneous acid-catalyzed lignocellulosic processes. In this perspective, we discuss basic principles of DC methods and how these can be combined with MD to enable high-throughput screening of solvent selection for diverse catalysis applications.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Lorentz group equivariant autoencoders

Abstract There has been significant work recently in developing machine learning (ML) models in high energy physics (HEP) for tasks such as classification, simulation, and anomaly detection. Often these models are adapted from those designed for datasets in computer vision or natural language processing, which lack inductive biases suited to HEP data, such as equivariance to its inherent symmetries. Such biases have been shown to make models more performant and interpretable, and reduce the amount of training data needed. To that end, we develop the Lorentz group autoencoder (LGAE), an autoencoder model equivariant with respect to the proper, orthochronous Lorentz group $$\textrm{SO}^+(3,1)$$ SO + ( 3 , 1 ) , with a latent space living in the representations of the group. We present our architecture and several experimental results on jets at the LHC and find it outperforms graph and convolutional neural network baseline models on several compression, reconstruction, and anomaly detection metrics. We also demonstrate the advantage of such an equivariant model in analyzing the latent space of the autoencoder, which can improve the explainability of potential anomalies discovered by such ML models.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

A Taxonomy for Classification and Comparison of Dataflows for GNN Accelerators

Recently, Graph Neural Networks (GNNs) have received a lot of interest because of their success in learning representations from graph structured data. However, GNNs exhibit different compute and memory characteristics compared to traditional Deep Neural Networks (DNNs). Graph convolutions require feature aggregations from neighboring nodes (known as the aggregation phase), which leads to highly irregular data accesses. GNNs also have a very regular compute phase that can be broken down to matrix multiplications (known as the combination phase). All recently proposed GNN accelerators utilize different dataflows and microarchitecture optimizations for these two phases. Different communication strategies between the two phases have been also used. However, as more custom GNN accelerators are proposed, the harder it is to qualitatively classify them and quantitatively contrast them. In this work, we present a taxonomy to describe several diverse dataflows for running GNN inference on accelerators. This provides a structured way to describe and compare the design-space of GNN accelerators.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Comparative study of machine learning techniques for post-combustion carbon capture systems

Computational analysis of countercurrent flows in packed absorption columns, often used in solvent-based post-combustion carbon capture systems (CCSs), is challenging. Typically, computational fluid dynamics (CFD) approaches are used to simulate the interactions between a solvent, gas, and column's packing geometry while accounting for the thermodynamics, kinetics, heat, and mass transfer effects of the absorption process. These simulations can then be used explain a column's hydrodynamic characteristics and evaluate its CO 2 -capture efficiency. However, these approaches are computationally expensive, making it difficult to evaluate numerous designs and operating conditions to improve efficiency at industrial scales. In this work, we comprehensively explore the application of statistical ML methods, convolutional neural networks (CNNs), and graph neural networks (GNNs) to aid and accelerate the scale-up and design optimization of solvent-based post-combustion CCSs. We apply these methods to CFD datasets of countercurrent flows in absorption columns with structured packings characterized by several geometric parameters. We train models to use these parameters, inlet velocity conditions, and other model-specific representations of the column to estimate key determinants of CO 2 -capture efficiency without having to simulate additional CFD datasets. We also evaluate the impact of different input types on the accuracy and generalizability of each model. We discuss the strengths and limitations of each approach to further elucidate the role of CNNs, GNNs, and other machine learning approaches for CO 2 -capture property prediction and design optimization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Analyzing inference workloads for spatiotemporal modeling

Ensuring power grid resiliency, forecasting climate conditions, and optimization of transportation infrastructure are some of the many application areas where data is collected in both space and time. Spatiotemporal modeling is about modeling those patterns for forecasting future trends and carrying out critical decision-making by leveraging machine learning/deep learning. Once trained offline, field deployment of trained models for near real-time inference could be challenging because performance can vary significantly depending on the environment, available compute resources and tolerance to ambiguity in results. Users deploying spatiotemporal models for solving complex problems can benefit from analytical studies considering a plethora of system adaptations to understand the associated performance-quality trade-offs. To facilitate the co-design of next-generation hardware architectures for field deployment of trained models, it is critical to characterize the workloads of these deep learning (DL) applications during inference and assess their computational patterns at different levels of the execution stack. In this paper, we develop several variants of deep learning applications that use spatiotemporal data from dynamical systems. We study the associated computational patterns for inference workloads at different levels, considering relevant models (Long short-term Memory, Convolutional Neural Network and Spatio-Temporal Graph Convolution Network), DL frameworks (Tensorflow and PyTorch), precision (FP16, FP32, AMP, INT16 and INT8), inference runtime (ONNX and AI Template), post-training quantization (TensorRT) and platforms (Nvidia DGX A100 and Sambanova SN10 RDU). Overall, our findings indicate that although there is potential in mixed-precision models and post-training quantization for spatiotemporal modeling, extracting efficiency from contemporary GPU systems might be challenging. Instead, co-designing custom accelerators by leveraging optimized High Level Synthesis frameworks (such as SODA High-Level Synthesizer for customized FPGA/ASIC targets) can make workload-specific adjustments to enhance the efficiency.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Hyperparameter Optimization and Feature Inclusion in Graph Neural Networks for Spiking Implementation

Graph convolutional networks leverage both graph structures and features on nodes and edges for improved learning performance in comparison with classical machine learning approaches. Spiking neuromorphic computers natively implement network-like computation and have been shown to be successful at implementing graph learning without features. Incorporating graph features brings the challenge of efficient feature representation and balancing the contribution of topology and features in learning. In this work, we present our design of a simulated network of spiking neurons to perform semi-supervised learning on graph data using both the graph structure and the node features. We explore various design choices, present preliminary results, and discuss the opportunities for using neuromorphic computers for this task in the future.

Cong, Guojing↗
DOE OSTI2025

Graph neural networks for mechanical property prediction of 2D fiber composites

This work investigates the ability of graph neural networks (GNNs) to homogenize 2D fiber composite microstructures. We use different inhomogeneity and anisotropy indices to motivate and show that the Volume Elements (VEs) used in ML methods should ideally be far from their Representative Volume Element (RVE) size limit and, consequently, are notably anisotropic. Hence, training only the isotropic limit properties may not be acceptable. Another aspect is the need to normalize elastic stiffness values for ML, especially when high elastic contrast ratios are encountered between composite phases or in the material set. We introduce a normalization technique based on the mean-field method (MFM) to handle such high contrast ratios and train for the entire stiffness tensor. We show that the proposed GNN approaches exhibit high accuracy and efficiency compared to traditional methods and convolutional neural networks, utilizing unstructured graphs constructed from microstructure topology. Our model successfully predicts the stiffness tensor, peak strength under bulk damage, and brittle fracture initiation strength across diverse microstructure configurations while maintaining high accuracy even for extreme material contrasts and volume fractions. We also present a method to improve prediction accuracy for small dataset sizes using Voronoi partitioning.

Brittle strength↗
DOE OSTI2021

Scaling Inference Using Triton to Accelerate Particle Physics at the LHC and DUNE

DUNE and the LHC experiments consist of unique and cutting-edge particle detectors that create massive, complex, and rich datasets with billions of events. They require sophisticated algorithms to reconstruct and interpret the data. Modern machine learning algorithms provide a powerful toolset to detect and classify particles, from familiar image processing convolutional neural networks to newer graph neural network architectures. A full reconstruction of these particle collisions requires novel approaches to handle the computing challenge of processing so much raw data. In a series of studies, physicists from Fermilab, CERN, and university groups explored how to accelerate their data processing using the Triton Inference Server.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Predicting Failure Using Deep Learning SAND Report

Accurate prediction of ductile failure is critical to Sandia’s NW mission, but the models are computationally heavy. The costs of including high-fidelity physics and mechanics that are germane to the failure mechanisms are often too burdensome for analysts either because of the person-hours it requires to input them or because of the additional computational time, or both. In an effort to deliver analysts a tool for representing these phenomena with minimal impact to their existing workflow, our project sought to develop modern data-driven methods that would add microstructural information to business-as-usual calculations and expedite failure predictions. The goal is a tool that receives as input a structural model with stress and strain fields, as well as a machine-learned model, and output predictions of structural response in time, including failure. As such, our project spent substantial time performing high-fidelity, three-dimensional experiments to elucidate materials mechanisms of void nucleation and evolution. We developed crystal-plasticity finite-element models from the experimental observations to enrich the findings with fields not readily measured. We developed engineering length-scale simulations of replicated test specimens to understand how the engineering fields evolve in the presence of fine-scale defects. Finally, we developed deep learning convolutional neural networks, and graph-based neural networks to encode the findings of the experiments and simulations and make forward predictions in time for structural performance. This project demonstrated the power of data-driven methods for model development, which have the potential to vastly increase both the accuracy and speed of failure predictions. These benefits and the methods necessary to develop them are highlighted in this report. However, many challenges remain to implementing these in real applications, and these are discussed along with potential methods for overcoming them.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

SBND Shower Reconstruction with SPINE

The Short-Baseline Near Detector (SBND) is a liquid argon time projection chamber (LArTPC) neutrino detector in the Short-Baseline Neutrino (SBN) program at Fermilab. SBND is designed to investigate the Low-Energy Excess (LEE), an unexplained excess of electron-like events observed by previous short-baseline neutrino experiments that may point to physics beyond the Standard Model. In LArTPC detectors, precise shower reconstruction is essential for distinguishing electrons from photons, a key requirement for testing possible explanations of the LEE and improving $\nu_e$ event selection. In this poster, the reconstruction studies using the Scalable Particle Imaging with Neural Embeddings (SPINE), a machine learning based reconstruction framework for particle imaging detectors will be presented. SPINE combines sparse convolutional neural networks (CNN) and graph neural networks (GNN) to enable detailed reconstruction and characterization of neutrino interactions in LArTPC detectors. Shower calorimetry and kinematic reconstruction are performed in dedicated post-processing stages. Strong agreement between data and Monte Carlo simulation will be demonstrated, indicating high-precision detector calibration and reconstruction performance. The agreement between reconstructed and true electron shower energy will also be discussed, emphasizing the robustness of the shower reconstruction performance. These results demonstrate the unprecedented precision achievable with SPINE in SBND, highlighting their potential for future high-resolution neutrino measurements.

Fan, Castaly [Florida U.; Fermilab] (ORCID:0000000↗
DOE OSTI2025

Data Quality Monitoring for the Hadron Calorimeters Using Transfer Learning for Anomaly Detection

The proliferation of sensors brings an immense volume of spatio-temporal (ST) data in many domains, including monitoring, diagnostics, and prognostics applications. Data curation is a time-consuming process for a large volume of data, making it challenging and expensive to deploy data analytics platforms in new environments. Transfer learning (TL) mechanisms promise to mitigate data sparsity and model complexity by utilizing pre-trained models for a new task. Despite the triumph of TL in fields like computer vision and natural language processing, efforts on complex ST models for anomaly detection (AD) applications are limited. In this study, we present the potential of TL within the context of high-dimensional ST AD with a hybrid autoencoder architecture, incorporating convolutional, graph, and recurrent neural networks. Motivated by the need for improved model accuracy and robustness, particularly in scenarios with limited training data on systems with thousands of sensors, this research investigates the transferability of models trained on different sections of the Hadron Calorimeter of the Compact Muon Solenoid experiment at CERN. The key contributions of the study include exploring TL’s potential and limitations within the context of encoder and decoder networks, revealing insights into model initialization and training configurations that enhance performance while substantially reducing trainable parameters and mitigating data contamination effects.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

A Look Inside the Black Box: Using graph-theoretical descriptors to interpret a Continuous-Filter Convolutional Neural Network (CF-CNN) trained on the global and local minimum energy structures of neutral water clusters

A Continuous Filter Convolutional Neural Network (CF-CNN) was trained to predict the potential energy of water cluster networks \ce{(H2O)_{\textit{N}}}, \textit{N}=10--30, corresponding to local minima lying within 5 kcal/mol from the putative minima taken from a newly published database containing over 5 million unique networks. The chemical sampling space of the database was characterized using chemical descriptors derived from graph theory, which led to the identification of important trends in the topology, connectivity, polygon structures associated with the various networks as a function of cluster size. The resulting graphs are available alongside the original database at \url{https://sites.uw.edu/wdbase/}. The CF-CNN trained on a subset of 500,000 networks for (\textit{N}=10, 30) yielded a mean absolute error of 0.002$\pm$0.002 kcal/mol per water molecule, giving the trained CF-CNN the highest accuracy of any neural network-based surrogate model to date. In addition, clusters of sizes not included in the training set exhibited errors of the same magnitude, indicating that the CF-CNN ptotocol is general enough to accurately predict energies of networks for both smaller and larger sizes than those used during training. The graph-theoretical descriptors were developed in order to analyze the properties of the full database and interpret the predictive power of the CF-CNN. Using topology measures, such as the Wiener index and the average shortest path length along with two similarity measures, we showed that all networks from the test set were within the range of the ones from the training set, suggesting that the training set covered the chemical space of interest quite well. Our graph analysis suggests that the mean degree and number of polygons for networks with larger errors tend to lie further from the mean than those with lower errors. The generality of the used CF-CNN was thus demonstrated, while the use of the graph-theoretical descriptors assisted in interpreting the predicted results.

Bilbrey, Jenna A.↗
DOE OSTI2022

Spatiotemporal Graph Convolutional Networks for Earthquake Source Characterization

Abstract Accurate earthquake location and magnitude estimation play critical roles in seismology. Recent deep learning frameworks have produced encouraging results on various seismological tasks (e.g., earthquake detection, phase picking, seismic classification, and earthquake early warning). Many existing machine learning earthquake location methods utilize waveform information from a single station. However, multiple stations contain more complete information for earthquake source characterization. Inspired by recent successes in applying graph neural networks (GNNs) in graph‐structured data, we develop a Spatiotemporal Graph Neural Network (STGNN) for estimating earthquake locations and magnitudes. Our graph neural network leverages geographical and waveform information from multiple stations to construct graphs automatically and dynamically by adaptive message passing based on graphs' edges. Using a recent graph neural network and a fully convolutional neural network as baselines, we apply STGNN to earthquakes recorded by the Southern California Seismic Network from 2000 to 2019 and earthquakes collected in Oklahoma from 2014 to 2015. STGNN yields more accurate earthquake locations than those obtained by the baseline models and performs comparably in terms of depth and magnitude prediction, though the ability to predict depth and magnitude remains weak for all tested models. Our work demonstrates the potential of using GNNs and multiple stations for better automatic estimation of earthquake epicenters.

58 GEOSCIENCES↗
DOE OSTI2022

Improving Variational Autoencoders for New Physics Detection at the LHC With Normalizing Flows

We investigate how to improve new physics detection strategies exploiting variational autoencoders and normalizing flows for anomaly detection at the Large Hadron Collider. As a working example, we consider the DarkMachines challenge dataset. We show how different design choices (e.g., event representations, anomaly score definitions, network architectures) affect the result on specific benchmark new physics models. Once a baseline is established, we discuss how to improve the anomaly detection accuracy by exploiting normalizing flow layers in the latent space of the variational autoencoder.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

H-GCN: A Graph Convolutional Network Accelerator on Versal ACAP Architecture

Recently Graph Neural Networks (GNNs) have drawn tremendous attentions due to their unique capability to extend the Machine Learning (ML) approaches to broadly defined applications with unstructured data, especially graphs. Comparing with other ML modalities, the acceleration of GNNs is as critical but even more challenging due to the irregularity and heterogeneity from graph typologies that together limit the performance. Existing efforts mainly focus on handling graphs’ irregularity, however, have not studied the heterogeneity. To this end, in this work, we propose H-GCN, a PL-AIE-based hybrid accelerator that leverages the emerging heterogeneity of Xilinx Versal ACAPs to achieve high-performance GNN inference. In particular, H-GCN partitions each graph into three subgraphs based on its inherent heterogeneity and processes them using PL and the newly emerged AIE respectively. To further improve the performance, we explore the sparsity support of AIE and develop an efficient density-aware method to map tiles of SpMM onto the systolic tensor array automatically. Compared with the current state-of-the-art GCN accelerator, HGCN achieves on average 1.5× speedups.

Zhang, Chengming↗
DOE OSTI2021

Explore Spatio‐Temporal Learning of Large Sample Hydrology Using Graph Neural Networks

Abstract Streamflow forecasting over gauged and ungauged basins play a vital role in water resources planning, especially under the changing climate. Increased availability of large sample hydrology data sets, together with recent advances in deep learning techniques, has presented new opportunities to explore temporal and spatial patterns in hydrological signatures for improving streamflow forecasting. The purpose of this study is to adapt and benchmark several state‐of‐the‐art graph neural network (GNN) architectures, including ChebNet, Graph Convolutional Network (GCN), and GraphWaveNet, for end‐to‐end graph learning. We explicitly represent river basins as nodes in a graph, learn the spatiotemporal nodal dependencies, and then use the learned relations to predict streamflow simultaneously across all nodes in the graph. The efficacy of the developed GNN models is investigated using the Catchment Attributes and MEteorology for Large‐sample Studies (CAMELS) data set under two settings, fixed graph topology (transductive learning), and variable graph topology (inductive learning), with the latter applicable to prediction in ungauged basins (PUB). Results indicate that GNNs are generally robust and computationally efficient, achieving similar or better performance than a baseline model trained using the long short‐term memory (LSTM) network. Further analyses are conducted to interpret the graph learning process at the edge and node levels and to investigate the effect of different model configurations. We conclude that graph learning constitutes a viable machine learning‐based method for aggregating spatiotemporal information from a multitude of sources for streamflow forecasting

Sun, Alexander Y.↗
DOE OSTI2024

Muon Neutrino Reconstruction at ICARUS with Machine Learning

The ICARUS T600 LArTPC detector successfully ran for three years at the underground LNGS laboratories, providing a first sensitive search for LSND-like anomalous electron neutrino appearance in the CNGS beam. After a significant overhauling at CERN, the T600 detector has been placed in its experimental hall at Fermilab, fully commissioned, and the first events observed with full detector readout. Regular data-taking began in May 2021 with neutrinos from the Booster Neutrino Beam (BNB) and neutrinos six degrees off-axis from the Neutrinos at the Main Injector (NuMI). Modern developments in machine learning have allowed for the development of an end-to-end machine learning-based event reconstruction for ICARUS data. This reconstruction folds in 3D voxel-level feature extraction using sparse convolutional neural networks and particle clustering using graph neural networks to produce outputs suitable for physics analyses. This poster will summarize the performance of a high-purity and high-efficiency end-to-end machine learning-based selection of muon neutrinos from the BNB and highlight studies of electromagnetic shower reconstruction from a neutral pion selection.

43 PARTICLE ACCELERATORS↗