Search NASA⌕ Search

SEARCH · Search NASA

Results for “graph embeddings”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

TransformerG2G: Adaptive time-stepping for learning temporal graph embeddings using transformers

Dynamic graph embedding has emerged as a very effective technique for addressing diverse temporal graph analytic tasks (i.e., link prediction, node classification, recommender systems, anomaly detection, and graph generation) in various applications. Such temporal graphs exhibit heterogeneous transient dynamics, varying time intervals, and highly evolving node features throughout their evolution. Hence, incorporating long-range dependencies from the historical graph context plays a crucial role in accurately learning their temporal dynamics. In this paper, we develop a graph embedding model with uncertainty quantification, TransformerG2G, by exploiting the advanced transformer encoder to first learn intermediate node representations from its current state (t) and previous context (over timestamps [t–1,t–l], l is the length of context). Moreover, we employ two projection layers to generate lower-dimensional multivariate Gaussian distributions as each node's latent embedding at timestamp t. We consider diverse benchmarks with varying levels of "novelty" as measured by the TEA (Temporal Edge Appearance) plots. Here, our experiments demonstrate that the proposed TransformerG2G model outperforms conventional multi-step methods and our prior work (DynG2G) in terms of both link prediction accuracy and computational efficiency, especially for high degree of novelty. Furthermore, the learned time-dependent attention weights across multiple graph snapshots reveal the development of an automatic adaptive time stepping enabled by the transformer. Importantly, by examining the attention weights, we can uncover temporal dependencies, identify influential elements, and gain insights into the complex interactions within the graph structure. For example, we identified a strong correlation between attention weights and node degree at the various stages of the graph topology evolution.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

A graph embedding‐based approach for automatic cyber‐physical power system risk assessment to prevent and mitigate threats at scale

Abstract Power systems are facing an increasing number of cyber incidents, potentially leading to damaging consequences to both physical and cyber aspects. However, the development of analytical methods for the study of large‐scale power infrastructures as cyber‐physical systems is still in its early stages. Drawing inspiration from machine‐learning techniques, the authors introduce a method inspired by the principles of graph embedding that is tailored for quantitative risk assessment and the exploration of possible mitigation strategies of large‐scale cyber‐physical power systems. The primary advantage of the graph embedding approach lies in its ability to generate numerous random walks on a graph, simulating potential access paths. Meanwhile, it enables capturing high‐dimensional structures in low‐dimensional spaces, facilitating advanced machine‐learning applications, and ensuring scalability and adaptability for comprehensive network analysis. By employing this graph embedding‐based approach, the authors present a structured and methodical framework for risk assessment in cyber‐physical systems. The proposed graph embedding‐based risk analysis framework aims to provide a more insightful perspective on cyber‐physical risk assessment and situation awareness for power systems. To validate and demonstrate its applicability, the method has been tested on two cyber‐physical power system models: the Western System Coordinating Council (WSCC) 9‐Bus System and the Illinois 200‐Bus System , thereby showing its advantages in enhancing the accuracy of risk analysis and comprehensiveness of situational awareness.

Sun, Shining↗
DOE OSTI2024

SNNVis: Visualizing Graph Embedding of Evolutionary Optimization for Spiking Neural Networks

While Spiking Neural Networks (SNNs) show a lot of promise, it is difficult to optimize them because applying traditional gradient-based optimization techniques is difficult. Even though evolutionary algorithms (EAs) have been shown to promise to optimize SNNs, understanding the relationship between evolving the characteristics of SNNs and their performance to improve the optimization algorithm is challenging because of the complex characteristics and huge population size. We propose visual analytics with novel graph embedding for evolutionary SNNs to address the challenges. While existing graph embedding techniques have limitations in preserving the specific features of the nodes and edges, our approach maintains them. Also, we develop visual analytics for understanding the relationship between the network performance and the features of nodes and edges and exploring and analyzing the evolving SNNs to build insights into improving the EA.

Chae, Junghoon [ORNL] (ORCID:0000000206016746)↗
DOE OSTI2023

An finite element analysis surrogate model with boundary oriented graph embedding approach for rapid design

Abstract In this work, we present a boundary oriented graph embedding (BOGE) approach for the graph neural network to assist in rapid design and digital prototyping. The cantilever beam problem has been solved as an example to validate its potential of providing physical field results and optimized designs using only 10 ms. Providing shortcuts for both boundary elements and local neighbor elements, the BOGE approach can embed unstructured mesh elements into the graph and performs an efficient regression on large-scale triangular-mesh-based finite element analysis (FEA) results, which cannot be realized by other machine-learning-based surrogate methods. It has the potential to serve as a surrogate model for other boundary value problems. Focusing on the cantilever beam problem, the BOGE approach with 3-layer DeepGCN model achieves the regression with mean square error (MSE) of 0.011 706 (2.41% mean absolute percentage error) for stress field prediction and 0.002 735 MSE (with 1.58% elements having error larger than 0.01) for topological optimization. The overall concept of the BOGE approach paves the way for a general and efficient deep-learning-based FEA simulator that will benefit both industry and Computer Aided Design (CAD) design-related areas.

42 ENGINEERING↗
DOE OSTI2025

Knowledge Graph Entity Linking using Graph Embeddings

Details the use of a custom embedding model on knowledge graphs to aid in downstream natural language processing (NLP) models for Derivative Classification Assist. Motivations, algorithms, and results were discussed.

Mahesh, Aarav [Sandia National Laboratories (SNL-N↗
DOE OSTI2022

Geometric learning for computational mechanics Part II: Graph embedding for interpretable multiscale plasticity

The history-dependent behaviors of classical plasticity models are often driven by internal variables evolved according to phenomenological laws. The difficulty to interpret how these internal variables represent a history of deformation, the lack of direct measurement of these internal variables for calibration and validation, and the weak physical underpinning of those phenomenological laws have long been criticized as barriers to creating realistic models. In this work, geometric machine learning on graph data (e.g. finite element solutions) is used as a means to establish a connection between nonlinear dimensional reduction techniques and plasticity models. Geometric learning-based encoding on graphs allows the embedding of rich time-history data onto a low-dimensional Euclidean space such that the evolution of plastic deformation can be predicted in the embedded feature space. Finally, a corresponding decoder can then convert these low-dimensional internal variables back into a weighted graph such that the dominating topological features of plastic deformation can be observed and analyzed.

42 ENGINEERING↗
DOE OSTI2022

Graph Embeddings for CEBAF Operations: Progress and Future Plans

We describe research towards leveraging deep learning on graph representations of the injector beamline at the Continuous Electron Beam Accelerator Facility (CEBAF) in order to create a tool for improving the efficiency of beam tuning tasks. Specifically, we use graphs to represent the injector beamline at any arbitrary date and time and invoke a graph neural network to extract a low-dimensional, informative representation that can be visualized in two-dimensions. By analyzing years of operational data from the CEBAF archiver, good and bad regions of parameter space can be identified. The goal is to exercise this framework as a real-time tool to aid beam tuning, which represents the dominant source of machine downtime.

Tennant, C.↗
DOE OSTI2024

Efficient graph representation framework for chemical molecule similarity tasks

Graph data has emerged in numerous scientific domains and machine learning techniques have been widely used for analysis and learning of diverse data for prediction and decision. Machine learning techniques can readily address complex problems by leveraging their structural information. But graphs cannot be directly used for existing machine learning algorithms unless encoded as vectors. The problem of efficient representation of graphs is a substantial challenge in graph machine learning. In this paper, we propose a novel two-stage framework for the representation of chemical molecule graphs based on the strengths of Graph Isomorphism Networks (GINs) and Siamese autoencoders. In the first stage, the GIN model is constructed and trained using the structural information of chemical molecule graphs. Node attributes, edge attributes, and edge indices are used as input data, while graph attributes are used as labels. The GIN model effectively captures the structural characteristics of graphs and can accurately predict graph attributes, i.e., molecular properties. It also generates Graph Embeddings, represented as vectors that encode the structural information of graphs. In the second stage, Graph Embedding vectors are further optimized for downstream similarity tasks while preserving the graph structural information. The Siamese autoencoder is constructed and trained, which reduces the dimensionality of the Graph Embedding vectors, while maximizing the preservation of structural information in the original high-dimensional vectors. The resulting low-dimensional Graph Embeddings can be effectively utilized for tasks such as approximate nearest neighbor search. The experimental results demonstrate the effectiveness of our proposed framework in accurately predicting graph similarity.

Ma, Jiaji↗
DOE OSTI2021

Versatile feature learning with graph convolutions and graph structures

Graphs represent real world relationships, and graph embedding projects nodes in a graph to a latent space that can help simplify downstream tasks. Recent development of graph convolutions in deep learning significantly improves the performance of many learning tasks on graphs. Unfortunately, prior embedding methods either do not embed graphs with node features, or fail to produce high-quality embeddings for downstream learning tasks that result in large performance gap in comparison to direct learning on graphs.We present a versatile and effective embedding method, Conv2Vec, for embedding graphs with or without node features. It is based on graph convolutions with objective functions motivated by concepts and structures from classical graph algorithms. Conv2Vec produce high-quality embedding for both plain graphs and graphs with node features for downstream tasks.We evaluate the embeddings generated by Conv2Vec with a transductive node classification task. With the generated embeddings and very simple machine learning approaches, we are able to achieve accuracies similar to those achieved by direct learning with graph convolutions. Interestingly, if we strip the node features from the graph and thus learning an embedding has to rely entirely on the graph topology, node classification with our embedding significantly outperforms direct learning with various graph convolutions. This suggests that structures from classical graph algorithms may play an important role in learning on graphs.

Cong, Guojing↗
DOE OSTI2021

TX$^2$: Transformer eXplainability and eXploration

The Transformer eXplainability and eXploration (Martindale & Stewart, 2021), or TX 2 software package, is a library designed for artificial intelligence researchers to better understand the performance of transformer models (Vaswani et al., 2017) used for sequence classification. The tool is capable of integrating with a trained transformer model and a dataset split into training and testing populations to produce an ipywidget (Project Jupyter Contributors, 2021) dashboard with a number of visualizations to understand model performance with an emphasis on explainability and interpretability. The TX 2 package is primarily intended to integrate into a workflow centered around Jupyter Notebooks (Kluyver et al., 2016), and currently assumes the use of PyTorch (Paszke et al., 2019) and Hugging Face transformers library (Wolf et al., 2020). The dashboard includes visualization and data exploration features to aid researchers, including an interactive UMAP embedding graph (McInnes et al., 2018) to understand classification clusters, a word salience map that can be updated as researchers alter textual entries in near real time, a set of tools to understand word frequency and importance based on the clusters in the UMAP embedding graph, and a set of traditional confusion matrix analysis tools.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Detecting Masquerade Attacks in Controller Area Networks Using Graph Machine Learning

Modern vehicles rely on a myriad of electronic control units (ECUs) interconnected via controller area networks (CANs) for critical operations. Despite their ubiquitous use and reliability, CANs are susceptible to sophisticated cyberattacks, particularly masquerade attacks, which inject false data that mimic legitimate messages at the expected frequency. These attacks pose severe risks such as unintended acceleration, brake deactivation, and rogue steering. Traditional intrusion detection systems (IDS) often struggle to detect these subtle intrusions due to their seamless integration into normal traffic. This paper introduces a novel framework for detecting masquerade attacks in the CAN bus using graph machine learning (ML). We hypothesize that the integration of shallow graph embeddings with time series features derived from CAN frames enhances the detection of masquerade attacks. We show that by representing CAN bus frames as message sequence graphs (MSGs) and enriching each node with contextual statistical attributes from time series, we can enhance detection capabilities across various attack patterns compared to using graph-based features only. Our method ensures a comprehensive and dynamic analysis of CAN frame interactions, improving robustness and efficiency. Extensive experiments on the ROAD dataset validate the effectiveness of our approach, demonstrating statistically significant improvements in the detection rates of masquerade attacks compared to a baseline that uses graph-based features only as confirmed by Mann-Whitney U and Kolmogorov-Smirnov tests (p < 0.05) .

Marfo, William [Univ. of Texas, El Paso, TX (Unite↗
DOE OSTI2023

Towards Geospatial Knowledge Graph Infused Neuro-Symbolic AI for Remote Sensing Scene Understanding

Deep learning has proven its effectiveness in numerous tasks for remote sensing scene understanding. However there is an increasing interest to explore fusion of domain-specific background information to the deep neural network to further improve its performance. Remote sensing researchers are also working towards developing models that generalize and adapt to multiple applications. Generalization challenges coupled with the scarcity of large corpora of high-quality noise-free labelled data, have together fueled an interest for leveraging background information. Knowledge graphs serve as excellent choice to represent domain-specific information in a structured, standardized and extensible manner. Integrating symbolic knowledge representations in the form of Knowledge Graph Embedding (KGE) to perform neuro-symbolic reasoning is an emerging research direction promising significant impacts. This vision paper seeks to position ideas and provoke early thoughts toward advancing neuro-symbolic artificial intelligence in the context of geospatial challenges. Specifically, it conceptualizes and elaborates on an architecture for infusing geospatial knowledge from knowledge graph in a deep neural network pipeline. As guiding case studies - land-use land-cover classification, object detection and instance segmentation can benefit from infusing spatio-contextual information with remote sensing imagery. The discussion further reflects on and articulates the challenges and explainable AI opportunities anticipated when scaling and maintaining large-scale geospatial knowledge graphs.

Potnis, Abhishek↗
DOE OSTI2022

A resilient network recovery framework against cascading failures with deep graph learning

Because of the increasing importance and dependencies of infrastructure networks and the potential for massive cascading failures in real-world network systems, maintenance optimization to effectively reduce system performance loss caused by diverse disruptions is of significant interest among researchers and practitioners. In this work, a new recovery framework was developed to rapidly identify important system components for maintenance to improve network resilience against cascading failures. Here this work provides distinct advantages to determine an optimal maintenance priority by combining real-time network structure importance with other maintenance prioritization based on customer preference. This approach adopts structural graph embedding and deep reinforcement learning to extract real-time network topology information (such as minimum vertex cover) to update the maintenance priority during the recovery process. Based on the case studies on synthetic networks and a US airport network, the proposed recovery framework with real-time network topology awareness shows better performance than other maintenance prioritization strategies regarding resilience enhancement. This work improves the understanding of how the changing network structure influences maintenance effects. It also provides insights of the practical usefulness of advanced deep learning on helping optimal maintenance prioritization to effectively reduce the intensity and extent of cascading failures.

42 ENGINEERING↗
DOE OSTI2022

NaroNet: Discovery of tumor microenvironment elements from highly multiplexed images

Understanding the spatial interactions between the elements of the tumor microenvironment -i.e. tumor cells. fibroblasts, immune cells- and how these interactions relate to the diagnosis or prognosis of a tumor is one of the goals of computational pathology. We present NaroNet, a deep learning framework that models the multi-scale tumor microenvironment from multiplex-stained cancer tissue images and provides patient-level interpretable predictions using a seamless end-to-end learning pipeline. Trained only with multiplex-stained tissue images and their corresponding patient-level clinical labels, NaroNet unsupervisedly learns which cell phenotypes, cell neighborhoods, and neighborhood interactions have the highest influence to predict the correct label. To this end, NaroNet incorporates several novel and state-of-the-art deep learning techniques, such as patch-level contrastive learning, multi-level graph embeddings, a novel max-sum pooling operation, or a metric that quantifies the relevance that each microenvironment element has in the individual predictions. We validate NaroNet using synthetic data simulating multiplex-immunostained images where a patient label is artificially associated to the -adjustable- probabilistic incidence of different microenvironment elements. We then apply our model to two sets of images of human cancer tissues: 336 seven-color multiplex-immunostained images from 12 high-grade endometrial cancer patients; and 382 35-plex mass cytometry images from 215 breast cancer patients. In both synthetic and real datasets, NaroNet provides outstanding predictions of relevant clinical information while associating those predictions to the presence of specific microenvironment elements.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

HydraGNN v4.0

The new version of HydraGNN v4.0 provides additional core capabilities, such as: Inclusion of multi-body atomistic cluster expansion MACE, polarizable atom interaction neural network PAINN, and equivariant principal neighborhood aggregation (PNAEq) among the message passing layers supported -Inclusion of graph transformers to directly model long-range interactions between nodes that are distant in the graph topology Integration of graph transformers with message passing layers by combining the graph embedding generated by the two mechanisms, which allows for an improved expressivity of the HydraGNN architecture Improved re-implementation of multi-task learning (MTL) to allow its use for stabilized training across imbalanced, multi-source, multi-fidelity data Introduction of multi-task parallelism, a newly proposed type of model parallelism specifically for MTL architectures, which allows to dispatch different output decoding heads to different GPU devices Integration of multi-task parallelism with pre-existing distributed data parallelism to enable a 2D parallelization for distributed training Improved portability of the distributed training across Intel GPUs, which has been testes on ALCF exascale supercomputer Aurora Inclusion of 2-level fine-grained energy profilers portable across NVIDIA, AMD, and Intel GPUs to monitor the power and energy consumption associated with different functions executed by the HydraGNN code during data pre-load and training Restructuring of previous examples and inclusion of new sets of examples to illustrate the download, preprocess, and training of HydraGNN models on new large-scale open-source datasets for atomistic materials modeling (e.g., Alexandria, Transition1x, OMat24, OMol25)

Lupo Pasini, Massimiliano [Oak Ridge National Labo↗