SEARCH · Search NASA

Results for “community detection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2024

DyG-DPCD: A Distributed Parallel Community Detection Algorithm for Large-Scale Dynamic Graphs

Dynamic (Temporal) graphs capture the valuable evolution of real-world systems, from the continuously evolving patterns of social interactions and genetic pathways to the dynamic fluctuations of economic forces. Detecting communities for such evolving networks poses unique challenges. Detecting and analyzing the evolution of communities within dynamic graphs unlocks valuable insights into the underlying structural and temporal patterns of real-world systems. However, the sheer volume of modern graph data and the inherent complexity of the temporal dimension pose significant challenges to scalable community detection algorithms. Addressing this gap, our work explores the limited landscape of scalable distributed-memory parallel methods specifically designed for dynamic network community detection. We propose a novel parallel algorithm, DyG-DPCD (Dynamic Graph Distributed Parallel Community Detection), to detect communities in dynamic networks using the Message Passing Interface (MPI) framework. We present a vertex-centric approach, allowing us to detect communities through local optimization. Furthermore, we enhance our baseline algorithm by incorporating three heuristics, which improve the algorithm’s performance significantly while maintaining the quality of the solutions. We demonstrate the efficiency of our algorithm by experimenting on several real-world large-scale networks with hundreds of millions of edges spanning diverse domains. Notably, DyG-DPCD achieves speedups between 25× and 30× for large networks that we experimented on using NERSC compute nodes. In conclusion, our algorithm outperforms the STINGER parallel re-agglomeration algorithm by 30×.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (distributed parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

graph algorithms, high performance comptuing↗

DOE OSTI2024

Distributed Multi-GPU Community Detection on Exascale Computing Platforms

Community detection is a fundamental operation in graph mining, and by uncovering hidden structures and patterns within complex systems it helps solve fundamental problems pertaining to social networks, such as information diffusion, epidemics, and recommender systems. Scaling graph algorithms for massive networks becomes challenging on modern distributed-memory multi-GPU (Graphics Processing Unit) systems due to limitations such as irregular memory access patterns, load imbalances, higher communication-computation ratios, and cross-platform support. We present a novel algorithm HiPDPL-GPU (Distributed Parallel Louvain) to address these challenges. We conduct experiments involving different partitioning techniques to achieve an optimized performance of HiPDPL-GPU on the two largest supercomputers: Frontier and Summit. Remarkably, HiPDPL-GPU processes a graph with 4.2 billion edges in less than 3 minutes using 1024 GPUs. Qualitatively, the performance of HiPDPL-GPU is similar or better compared to other state-of-the-art CPU- and GPU-based implementations. While prior GPU implementations have predominantly employed CUDA, our first-of-its-kind implementation for community detection is cross-platform, accommodating both AMD and NVIDIA GPUs.

Sattar, Naw Safrin↗

DOE OSTI2026

Community detection robustness of graph neural networks

Graph neural networks (GNNs) are increasingly widely used for community detection in attributed networks. They combine structural topology with node attributes through message passing and pooling. However, their robustness or lack thereof with respect to different perturbations and targeted attacks in conjunction with community detection tasks is not well understood. To shed light on latent mechanisms behind GNN sensitivity on community detection tasks, we conduct a systematic computational evaluation of six widely adopted GNN architectures graph convolutional network, graph attention network, graph sample and aggregate (GraphSAGE), differentiable pooling (DiffPool), minimum cut pooling (MinCUT), and deep modularity networks (DMoN). The analysis covers three perturbation categories: node attribute manipulations, edge topology distortions, and adversarial attacks. We use element-centric similarity as the evaluation metric on synthetic benchmarks and real-world citation networks. Our findings indicate that supervised GNNs tend to achieve higher baseline accuracy, while unsupervised methods, particularly DMoN, maintain stronger resilience under targeted and adversarial perturbations. Furthermore, robustness appears to be strongly influenced by community strength, with well-defined communities reducing performance loss. Across all models, node attribute perturbations associated with targeted edge deletions and shifts in attribute distributions tend to cause the largest degradation in community recovery. These findings highlight important trade-offs between accuracy and robustness in GNN-based community detection and offer insights into selecting architectures resilient to noise and adversarial attacks.

Goel, Jaidev [Virginia Polytechnic Inst. and State↗

DOE OSTI2024

Community detection in hypergraphs via mutual information maximization

Abstract The hypergraph community detection problem seeks to identify groups of related vertices in hypergraph data. We propose an information-theoretic hypergraph community detection algorithm which compresses the observed data in terms of community labels and community-edge intersections. This algorithm can also be viewed as maximum-likelihood inference in a degree-corrected microcanonical stochastic blockmodel. We perform the compression/inference step via simulated annealing. Unlike several recent algorithms based on canonical models, our microcanonical algorithm does not require inference of statistical parameters such as vertex degrees or pairwise group connection rates. Through synthetic experiments, we find that our algorithm succeeds down to recently-conjectured thresholds for sparse random hypergraphs. We also find competitive performance in cluster recovery tasks on several hypergraph data sets.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Exploring temporal community evolution: algorithmic approaches and parallel optimization for dynamic community detection

Abstract Dynamic (temporal) graphs are a convenient mathematical abstraction for many practical complex systems including social contacts, business transactions, and computer communications. Community discovery is an extensively used graph analysis kernel with rich literature for static graphs. However, community discovery in a dynamic setting is challenging for two specific reasons. Firstly, the notion of temporal community lacks a widely accepted formalization, and only limited work exists on understanding how communities emerge over time. Secondly, the added temporal dimension along with the sheer size of modern graph data necessitates new scalable algorithms. In this paper, we investigate how communities evolve over time based on several graph metrics under a temporal formalization. We compare six different algorithmic approaches for dynamic community detection for their quality and runtime. We identify that a vertex-centric (local) optimization method works as efficiently as the classical modularity-based methods. To its advantage, such local computation allows for the efficient design of parallel algorithms without incurring a significant parallel overhead. Based on this insight, we design a shared-memory parallel algorithm DyComPar , which demonstrates between 4 and 18 fold speed-up on a multi-core machine with 20 threads, for several real-world and synthetic graphs from different domains.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Exploring the Landscape of Distributed Graph Clustering on Leadership Supercomputers

The rapid growth of large-scale datasets in fields like biology and social networks has driven the need for advanced graph analytics techniques. Community detection, a fundamental task in graph analytics, identifies closely connected groups of nodes within a network, providing valuable insights across various disciplines. This study focuses on two classic community detection methods, the Louvain algorithm and Markov Clustering (MCL), and evaluates the performance of two prominent distributed community detection algorithms: HiPDPL-GPU, our prior implementation, and HipMCL. We conduct experiments on GPU-accelerated heterogeneous HPC systems, Summit and Frontier, to assess their performance under varying conditions. Our objective is to identify the strengths and weaknesses of these algorithms in terms of scalability, and quality of solutions. We evaluate these algorithms on a diverse set of 70+ networks spanning 13 domains, with sizes ranging up to 4.2 billion edges. Our results demonstrate that HiPDPL-GPU consistently outperforms HipMCL, especially for large-scale networks. HiPDPL-GPU achieves significantly faster runtimes (47x to 1439x), higher modularity scores, and improved scalability. These findings highlight HiPDPL-GPU as a promising solution for efficient and effective large-scale graph analytics in diverse application domains, and provide insights into the feasibility of using MCL-based approaches for certain application domains.

Community detection, graph algorithms↗

DOE OSTI2026

Simultaneous global and local clustering in multiplex networks with covariate information

Understanding both global and layer-specific group structures is useful for uncovering complex patterns in networks with multiple interaction types. In this work, we introduce a new model, the hierarchical multiplex stochastic blockmodel, which simultaneously detects communities within individual layers of a multiplex network while inferring a global node clustering across the layers. A stochastic blockmodel is assumed in each layer, with probabilities of layer-level group memberships determined by a node’s global group assignment. Our model uses a Bayesian framework, employing a probit stick-breaking process to construct node-specific mixing proportions over a set of shared Griffiths–Engen–McCloseky distributions. These proportions determine layer-level community assignment, allowing for an unknown and varying number of groups across layers, while incorporating nodal covariate information to inform the global clustering. We propose a scalable variational inference procedure with parallelisable updates for application to large networks. Extensive simulation studies demonstrate our model’s ability to accurately recover both global and layer-level clusters in complicated settings, and applications to real data showcase the model’s effectiveness in uncovering interesting latent network structure.

community detection↗

DOE OSTI2024

Detection of Overlapping Communities in Undirected Graphs with Personalized PageRank

In this document, we investigate a method to detect overlapping communities in undirected graphs. This project is ongoing work in collaboration1 with Trevor Steil at Lawrence Livermore National Laboratory (LLNL).

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Hunting Sub-GeV Dark Matter with Diamonds and Magnetic Microcalorimeters Principle

This project is to study feasibility of a new concept for the sub-GeV dark matter (DM) detection using diamond crystals and magnetic microcalorimeters (MMCs). Diamond crystals are made of low mass carbon constituent that maximize kinetic energy of nuclear recoils from sub-GeV DM scattering due to their relatively similar masses. MMC is employed as a sensitive phonon sensor to measure athermal phonons that are produced by DM scattering in diamond crystals with 100 ns timing resolution and ~10 eV energy resolution. MMC’s fast timing resolution allows high precision phonon pulse shape analysis to separate out unwanted background or noise signals in the low energy region at E < 1 keV. Especially, the low energy excess (LEE) problems have been reported in the dark matter detection community and the proposed diamond-MMC detector might be able to provide important information to understand the origins of LEE signals in DM detectors. In this project a diamond-MMC detector has been built for proof-of-concept experiments and athermal phonon collection efficiencies have been investigated for single crystal and poly-crystal chemical-vapour-diamonds (SC and PC CVDs) as well as reference sapphire crystals are tested in the same geometries to quatify athermal phonon propagation and their collection to the MMC phonon sensors. Athermal phonon collection efficiencies and their lifetime in the diamond and sapphire crystals are successfully measured in the same geometry and experimental setup for comparison. SC CVD crystals exhibit poorer performance than the sapphire reference crystals in athermal phonon collection, while the PC CVD crystal exhibit better result than the sapphire. PC CVD with MMC phonon sensors would be feasible for sub GeV DM detection.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2025

Toward computing bounds for Ramsey numbers using quantum annealing

Quantum annealing is a powerful tool for solving and approximating combinatorial optimization problems, such as graph partitioning, community detection, centrality, routing problems, and more. In this paper we explore the use of quantum annealing as a tool for use in exploring combinatorial mathematics research problems. We consider the monochromatic triangle problem and the Ramsey number problem, both examples of graph coloring. Conversion to quadratic unconstrained binary optimization (QUBO) form is required to run on quantum hardware. While the monochromatic triangle problem is quadratic by nature, the Ramsey number problem requires the use of order reduction methods for a quadratic formulation. The goal is to provide a method for producing special colorings of graphs which if successful would provide lower bounds for certain Ramsey numbers. We discuss implementations, limitations, and results when running on the D-Wave Advantage quantum annealer.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Network Analysis of Academic Medical Center Websites in the United States

Healthcare resources are published annually in repositories such as the AHA Annual Survey Database TM . However, these data repositories are created via manual surveying techniques which are cumbersome in collection and not updated as frequently as website information of the respective hospital systems represented. Also, this resource is not widely available to patients in an easy-to-use format. Network analysis techniques have the potential to create topological maps which serve to aid in pathfinding for patients in their search for healthcare services. This study explores the topological structure of forty United States academic health center websites. Network analysis is utilized to analyze and visualize 48,686 webpages. Several elements of network structure are examined including basic network properties, and centrality measures distributions. The Louvain community detection algorithm is used to examine the extent to which these techniques allow identification of healthcare resources within networks. The results indicate that websites with related healthcare services tend to form observable clusters useful in mapping key resources within a hospital system.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Neutrino signatures of 100 2D Axisymmetric Core-Collapse Supernova Simulations

ABSTRACT We present in this paper a public data release of an unprecedentedly large set of core-collapse supernova (CCSN) neutrino emission models, comprising 100 detailed 2D axisymmetric radiation-hydrodynamic simulations evolved out to as late as ∼5 s post-bounce and spanning an extensive range of massive-star progenitors. The motivation for this paper is to provide a physically and numerically uniform benchmark data set to the broader neutrino detection community to help it characterize and optimize subsurface facilities for what is likely to be a once-in-a-lifetime galactic supernova burst event. With this release, we hope to (1) help the international experiment and modelling communities more efficiently optimize the retrieval of physical information about the next galactic CCSN, (2) facilitate the better understanding of core-collapse theory and modelling among interested experimentalists, and (3) help further integrate the broader supernova neutrino community.

Astronomy & Astrophysics↗

DOE OSTI2023

A Data Processing Pipeline for Socio-Technical Network Analysis [Slides]

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Use of Graph Theory and Neural Networks for Microstructural Classification

Recent advances in materials data analytics have provided new avenues for determining process-structure-property (PSP) linkages in a variety of materials. Machine learning techniques including few-shot learning have increased the efficiency of classifying microscopy images for the purposes of material characterization. Modifications in segmentation also show potential in improving the accuracy of our current pyCHIP classifier. Replacing previous encoders trained on ImageNet with those trained on microscopy images like MicroNet has initially shown better performance at classifying images of irradiated samples. Additionally, different normalization approaches were tested to show no discernable effect on classification. The Louvain method for community detection is analyzed on a set of irradiated samples with different parameters to determine which proved beneficial under what circumstances. We suggest that microscopy experiments be automated in the future using a combination of these techniques to enable high-throughput analyses.

36 MATERIALS SCIENCE↗

DOE OSTI2023

A Data Processing Pipeline for Adversarial Socio-Technical Network Analysis

With the rapid adoption of emerging technologies, there is a need to catalog and model sociotechnical interdependencies that have been historically used to influence the operation of Critical Infrastructure networks including the impacts of mergers and acquisitions, hostile takeovers, and foreign investment. Our research intends to address this need with two primary contributions. First, we have developed a data curation and processing pipeline to generate sociotechnical networks extracted from a variety of data sources including SEC filings and infrastructure asset databases. The pipeline, implemented in Apache Airflow, extracts and normalizes the representation of entities and relations, specified within ontologies. Our intent is to provide an extensible, machine-actionable approach to quickly communicate such models, reproduce previous results, and adapt them to new, unanticipated situations. Second, networks produced by our pipeline enable the development of graph-theoretic metrics that consider the properties of network components in addition to its topology. Metadata associated with network components---whether semantic, temporal, or geospatial---affects the alignment of generated networks with assumptions underlying complexity metrics. Validation of generated networks relative to component types defined by an ontology, may allow the research community to adapt metrics to the semantics of the domains being studied. Generated networks may be processed as knowledge, dynamic, or spatial graphs and enables a variety of analyses including automated reasoning and measures of network complexity. Automated reasoning views extracted entities and relations as a knowledge graph; this enables application of inference rules that represent historically-attested adversarial business methods and applies that behavior to a specific geographic context. Measures of network complexity, including degree distribution, reachability analyses, temporal analysis, and community detection can be adapted to indicate adversarial organizational influence.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

4th TDAMM Workshop White Paper

Time-Domain and Multi-Messenger Astrophysics (TDAMM) is entering a fundamentally new phase characterized by an unprecedented increase in the rate and diversity of astrophysical transient detections. The community is transitioning from a discovery-limited to a follow-up-limited era, driven by major investments across electromagnetic, gravitational-wave, and neutrino observatories. Upcoming facilities such as the Vera C. Rubin Observatory, the Nancy Grace Roman Space Telescope, and wide-field survey instruments will produce a deluge of time-domain alerts, reaching millions of events per night. Simultaneously, upgrades to the gravitational-wave network (LVK O5 and beyond) and neutrino observatories (IceCube Gen2) will significantly increase the detection rates of non-electromagnetic messengers. New high-energy missions and expansions of the InterPlanetary Network (IPN) will further enhance discovery capabilities across the gamma-ray and X-ray regimes. This convergence of capabilities represents a transformative opportunity: for the first time, the community will routinely detect rare and high-impact events across multiple messengers. However, the scientific return from these discoveries will depend critically on the ability to rapidly identify, prioritize, and coordinate follow-up observations across a heterogeneous and globally distributed set of facilities.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Metatranscriptomic analysis reveals dissimilarity in viral community activity between an ice-free and ice-covered winter in Lake Erie

Winter is a relatively under-studied season in freshwater ecology. The paucity of wintertime surveys has led to a lack of knowledge regarding microbial community activity during the winter in Lake Erie, a North American Great Lake. Viruses shape microbial communities and regulate biogeochemical cycles by acting as top-down controls, yet very few efforts have been made to examine active virus populations during the winter in Lake Erie. Furthermore, climate change-driven declines in seasonal ice cover have been shown to influence microbial community structure, but no studies have compared viral community activity between different ice cover conditions. We surveyed surface water metatranscriptomes for viral hallmark genes as a proxy for active virus populations and compared activity metrics between ice-covered and ice-free conditions from two sampled winters. Transcriptionally active viral communities were detected in both winters, spanning diverse phylogenetic clades of putative bacteriophage (Caudoviricetes), giant viruses (Nucleocytoviricota, or NCLDV), and RNA viruses (Orthornavirae). However, viral community activity metrics revealed pronounced differences between the ice-covered and ice-free winters. Viral community composition was distinct between winters and viral hallmark gene richness was reduced in the ice-covered relative to the ice-free conditions. In addition, the observed differences in viral communities correlated with microbial community activity metrics. Overall, these findings contribute to our understanding of the viral populations that are active during the winter in Lake Erie and suggest that viral community activity may be associated with ice cover extent.

59 BASIC BIOLOGICAL SCIENCES↗