Search NASA⌕ Search

SEARCH · Search NASA

Results for “security visualization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 217 records · Page 12

Synchrophasors-based Master State Awareness Estimator for Cybersecurity in Power Grid: Testbed Implementation & Field Demonstration

The integration of distributed energy resources(DERs) and expansion of complex network in the distribution grid requires an advanced distributed state estimator to monitor the grid health at micro-level. The distribution state estimator will improve the situational awareness and resiliency of distributed power system. This paper proposes a synchrophasors-based master state awareness (MSA) estimator to enhance the cybersecurity in distribution grid by providing a real-time estimation of system operating states to control center operators. In this paper, the proposed MSA estimator utilizes only phasor measurements, bus magnitudes and angles, from phasor measurement units (PMUs),deployed in local substations, to estimate the system states and also detects data integrity attacks, such as load tripping attack that disconnects the load. To validate the proof of concept, we implement the proposed methodology in cyber-physical testbed environment at the Idaho National Laboratory (INL) Electric Grid Security Testbed. Further, to address the “valley of death” and support technology commercialization, field demonstration is also performed at the Critical Infrastructure Test Range Complex(CITRC) at the INL. Our experimental results reveal a promising performance in detecting load tripping attack and providing an accurate situational awareness through an alert visualization dashboard in real-time

42 ENGINEERING↗

Scalable Reasoning System (SRS)

SRS is a web-based and mobile toolkit for information visualization and collaboration. It includes server-side components in the form of web services that provide analytical functions such as data clustering and temporal analysis. The SRS client is accessed via a web browser and uses asynchronous javascript and XML (AJAX) to interact with and visualize the results of these web services. Users of SRS can query data sources that are connected to the SRS backend, and query results are presented in the form of coordinated visual displays in a web browser that depict temporal, topical, and spatial relationships in the results. SRS also includes an early stage graphical reasoning diagram interface that helps analysts express and test hypotheses. This software was developed with funding from the Science and Technology Directorate of the U.S. Department of Homeland Security.

Pike, William↗

Changes in Exercise Data Management

The suite of exercise hardware aboard the International Space Station (ISS) generates an immense amount of data. The data collected, treadmill, cycle ergometer, and resistance strength training hardware, are basic exercise parameters (time, heart rate, speed, load, etc.). The raw data are processed in the laboratory and more detailed parameters are calculated from each exercise data file. Updates recently have been made to how these valuable data are stored, adding an additional level of security, increasing accessibility, and resulting in overall increased efficiency of medical report delivery. Questions regarding exercise performance or how exercise may influence other variables of crew health frequently arise within the crew health care community. Inquiries regarding the health of the exercise hardware often need quick analysis and response to ensure the exercise system is operable on a continuous basis. Consolidating all of the exercise system data in a single repository enables a quick response to both the medical and engineering communities. A SQL server database is currently in use, and provides a secure location for all of the exercise data starting at ISS Expedition 1 to current date. The database has been structured to update derived metrics automatically, making analysis and reporting available within minutes of dropping the in-flight data into the database. Commercial tools were evaluated to help aggregate and visualize data from the SQL database. The Tableau software provides manageable interface, which has improved the laboratory’s output time of crew reports by 67%. Expansion of the SQL database, to be inclusive of additional medical requirement metrics, addition of ‘app-like’ tools for mobile visualization, and collaborative use (e.g., operational support teams, research groups, and International Partners) of the data system, is currently being explored.

Buxton, R. E.↗

Design and Analysis of a Region-Wide Remotely Controllable Electrical Lock-Out System

Electric utilities have a main responsibility to protect the lives and safety of their workers when they are working on low-, medium-, and high-voltage power lines and distribution circuits. With the anticipated widespread deployment of smart grids, a secure and highly reliable means of maintaining isolation of customer-owned distributed generation (DG) from the affected distribution circuits during maintenance is necessary to provide a fully de-energized work area, ensure utility personnel safety, and prevent hazards that can lead to accidents such as accidental electrocution from unanticipated power sources. Some circuits are serviced while energized (live line work) while others are de-energized for maintenance. For servicing de-energized circuits and equipment, lock-out tag-out (LOTO) programs provide a verifiable procedure for ensuring that circuit breakers are locked in the off state and tagged to indicate that status to operational personnel so that the lines will be checked for voltage to verify they are de-energized. The de-energized area is isolated from any energized sources, which traditionally are the substations. This procedure works well when all power sources and their interconnections are known armed with this knowledge, utility personnel can determine the appropriate circuits to de-energize for isolating the target line or equipment. However, with customer-owned DG tied into the grid, the risk of inadvertently reenergizing a circuit increases because circuit connections may not be adequately documented and are not under the direct control of the local utility. Thus, the active device may not be properly de-energized or isolated from the work area. Further, a remote means of de-energizing and locking out energized devices provides an opportunity for greatly reduced safety risk to utility personnel compared to manual operations. In this paper, we present a remotely controllable LOTO system that allows individual workers to determine the configuration and status of electrical system circuits and permit them to lock out customer-owned DG devices for safety purposes using a highly secure and ultra-reliable radio signal. The system consists of: (1) individual personal lockout devices, (2) lockout communications and logic module at circuit breakers, which are located at all DG devices, and (3) a database and configuration control process located at the utility operations center. The lockout system is a close permissive, i.e., loss of control power or communications will cause the circuit breaker to open. Once the DG device is tripped open, a visual means will provide confirmation of a loss of voltage and current that verifies the disconnected status of the DG. Further the utility personnel will be able to place their own lock electronically on the system to ensure a lockout functionally. The proposed LOTO system provides enhanced worker safety and protection against unintended energized lines when DG is present. The main approaches and challenges encountered through designing the proposed region-wide LOTO system are discussed in this paper. These approaches include: (1) evaluating the reliability of the proposed approach under N-modular redundancy with voter/spares configurations and (2) conducting a system level risk assessment study using the failure modes and effects analysis (FMEA) technique to identify and rank failure modes by probability of occurrence, probability of detection, and severity of consequences. This ranking allows a cost benefits analysis to be conducted such that dollars and efforts will be applied to the failures that provide greatest incremental gains in system capability (resilience, survivability, security, reliability, availability, etc.) per dollar spent whether capital, operations, or investment. Several simulation scenarios and their results are presented to demonstrate the viability of these approaches.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Bayesian Segmentation (BCNN) v.1

Deep learning has been applied with great success to the segmentation of 3D X-Ray Computed Tomography (CT) scans. Establishing the credibility of these segmentations requires uncertainty quantification (UQ) to identify problem areas. Bayesian neural networks (BNNs), which use variational inference to learn the posterior distribution of the neural network weights, have been proposed to incorporate UQ into deep learning models. This software is an implementation of a novel 3D Bayesian convolutional neural network (BCNN) that provides accurate binary segmentations and uncertainty maps for 3D volumes. In particular, the uncertainty maps generated by this BCNN capture continuity and visual gradients, making them interpretable as confidence intervals for segmentation usable in numerical simulations. SAND2019-12927 M Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

LaBonte, TylerMatthew↗

mvBayesR

SAND2025-11559O The mvBayesR tool performs multivariate Bayesian analysis on generic data. It includes tools for regression modeling, diagnosis, basis decomposition, sensitivity analysis, and visualization. The tool compiles state-of-the-art methodology into one easy-to-use package. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Tucker, James [Sandia National Lab. (SNL-CA), Live↗

Iterative ML and Experiments for Emerging VOCs

SAND2026-17074O Iterative ML and Experiments for Emerging VOCs is a tool that analyzes and predicts the behaviors of SARS-CoV-2 variants. It processes experimental data on ACE2 (the receptor for the SARS-CoV-2 virus that allows it to infect the cell) and antibody binding using machine learning models, including neural networks, to forecast ACE2 interactions and variant expression. The tool employs transfer learning and global epistasis modeling, integrating public datasets with proprietary data to enhance prediction accuracy. Additionally, it fits concentration-response curves to determine dissociation constants and generates visualizations to support research findings, thereby aiding in the identification of new antibodies for emerging variants of concern. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Sheffield, Thomas [Sandia National Lab. (SNL-NM), ↗

Mapping Support for Targeted Critical Minerals Exploration and Extraction

The United States’ dependency on imported minerals poses significant risks to economic stability and national security due to potential supply disruptions. Recognizing the strategic importance of critical minerals, the Department of Energy (DOE) emphasizes the need for a secure and resilient supply chain to support emissions reduction, technology development, and capitalization on clean energy opportunities. The DOE’s Office of Manufacturing and Energy Supply Chains (MESC), in collaboration with the Office of Policy (OP), addresses these vulnerabilities by focusing on upstream domestic critical minerals production, balancing extraction with social and environmental goals, including conservation, environmental justice, and respect for Tribal sovereignty. This report showcases a collaborative effort involving Idaho National Laboratory (INL), Argonne National Laboratory (Argonne), National Renewable Energy Laboratory (NREL), and the U.S. Geological Survey (USGS) to map mineral development potential along with key social and environmental datasets. A geographical information system (GIS)-based web map application was developed as a preliminary tool for environmental analysis, integrating 158 geospatial data layers such as critical habitat, land ownership, economic indicators, and environmental concerns. Data were sourced from agencies like the Bureau of Land Management (BLM) and USGS and processed using GIS technology to enhance visualization and analysis. The proposed analysis framework categorizes areas into high, mid, and low concern based on withdrawn lands, special status species, the Economic Development Capacity Index (EDCI) Mining Composite Index, and the Climate and Economic Justice Screening Tool (CEJST). While the application provides broad visualizations, it is not a substitute for detailed environmental reviews required under the National Environmental Policy Act (NEPA). Users must conduct further analyses and engage with tribal entities and other stakeholders for comprehensive planning. A case study of the Idaho Cobalt Belt (ICB) in Lemhi County, Idaho, has been provided in the report to illustrate the tool's practical use. This report introduces a GIS application and framework to support stakeholders in identifying and prioritizing areas for critical mineral exploration, promoting secure supply chains, and advancing the nation's energy independence through responsible resource stewardship.

54 ENVIRONMENTAL SCIENCES↗

Mesh Computing Remote Automatic Workflow

The software suite uses a microservice architecture using Docker and `docker-compose`. The microservices are as follows: 1. User interface. This interface is written in JavaScript using the Svelte framework. It exposes form elements and a 3D visualizer to prompt the user through the definition of microstructure parameters, and setting parameters for mesh generation and refinement. 2. Mesh generator. This is a container running the Python package for DREAM3D to generate a voxelized mesh that represents a microstructure defined by the user in the interface. 3. Cubit runner. This is a secure shell protocol tool that makes the submitting the DREAM mesh to an HPC instance and starts to run Cubit shell commands to smooth the grain boundaries with its `sculpt` library, applies user-defined boundary node sets, and bundles and returns the simulation-ready meshes and input files as a zipped directory.

Harris, BrennanKay↗

MCNPy

SAND2026-20425O MCNPy runs and analyzes simulations from MCNP, a software that models radiation transport of neutrons and gamma rays. MCNPy uses Python to start MCNP, retrieve event data files, and convert them into graph structures for detailed analysis. It offers visualization tools, including 2D views of particle histories, making complex simulation data easier to interpret for researchers and engineers. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy's National Nuclear Security Administration under contract DE-NA0003525.

Nowack, Aaron [Sandia National Lab. (SNL-CA), Live↗

(ODIN): An Open Source, Low-Latency Data Integration & Visualization Framework for the NASA System Wide Safety Project's Disaster Response Safety Demonstration Series

The Open Data Integration Framework (ODIN) is an open source, low latency data integration and visualization framework (https://github.com/NASARace/race-odin) developed under NASA’s System WideSafety Program to demonstrate new safety capabilities designed to improve US airspace operations. Safety demonstrations are a set of increasingly complex (from public safety perspective) disaster response scenarios under which air systems must operate with increased capacity and include: 1) Wildland fire response, 2) Hurricane relief and recovery, 4) Emergency medical delivery via UAS and 4) Urban disaster relief. To accommodate disaster response, ODIN is field deployable and can scale on one or more multi-core, commodity laptops operating with full to limited or intermittent internet connectivity, conditions likely encountered during operations. ODIN runs as webserver with local, persistent data storage to serve either public or a secured, ad hoc network (e.g., an incident command post). The current released ODIN, ODIN-Fire is tailored for wildland fire management incorporating information on satellite overpasses with links to the near real-time data and imagery from the respective agencies. Included are winds data, an important variable for emergency responders and airspace operations, and high-resolution wind forecasts generated by super-computing resources and ingested into ODIN. As an open-source project, ODIN has attracted interest from multiple entities. We will show how 1) a commercial field instrument and data provider uses ODIN to help users visualize, publish and integrate their in-situ sensor network data and 2) ODIN’s capabilities to ingest, integrate and display near-real time satellite data with air traffic and a USFS winds forecast model used in fire response and post-fire assessment. Within NASA ODIN demonstrated novel, near terminal airspace safety capabilities for a project close-out event and previously it monitored the national airspace in real-time to meet an agency milestone. ODIN is presently under development for the anticipated hurricane relief and response demonstration notionally scheduled for the 2025-27 time frame and is available from NASA's github at the above link.

Aeronautics↗

Fuel Cycle Research & Development: Technical Monthly – June 2012

Topics are: (1) MPACT Campaign - (a) Management and Integration • Coordination meetings between NE and NA-22, NA-24, and NA-82 were conducted the week of June 11th. Preparations are being made for the next MPACT working group meeting, scheduled for Aug 28-30 at Idaho Falls. In addition to covering accomplishments and discussing future plans, a site tour of INL facilities (MFC, EBR, ATR, INTEC) is being organized. (2) Accounting and Control Technologies - (a) Microcalorimetry • Now operating 256-pixel array at LANL. We are in the process of tuning detector parameters to improve and optimize performance. Preliminary measurements show approximate number of live pixels is similar to that observed previously at NIST. Continuing to study contribution to systematic error from uncertainties in tabulated gamma-ray energies. (b) Electrochemical Sensor • Testing of sensors fashioned from different precursor materials continued. SEM analysis of all used sensors has been or will be performed. (c) Lead Slowing Down Spectrometer • Ongoing perturbation calculations are providing information on the fundamental systematic error limits of LSDS. In order to achieve separating the contribution of Pu and 235U to the signal, there will need to be tight controls on systematic errors. Continuing to look into a He4 detector. Research into local construction of a He4 detector continued. We have started to apply the algorithm to test the LSDS using experimental data from previous RPI measurements. PNNL also developed a plan to address the lack of statistics in the MCNP modeling of the NGSI 64 assemblies. The ISU graduate student built and tested a fission chamber to gain experience with them. (d) Fast Neutron Imaging to Quantify Nuclear Materials • The imaging detector design was modified for each pixel to have an 8 x 8 pixel array. Quotations and purchasing process for components, including the new PSD scintillator are in progress. (e) Fast Neutron Multiplicity Analysis • The team submitted two papers to the upcoming INMM meeting that are related to the fast neutron multiplicity R&D effort. Progress was made on the project's main goal of designing a concept for a prototype fast-neutron multiplicity counter. We started laying out the outline for the final report. We have been working with our ORNL collaborators to develop a new digitizer system to support our experimental campaign planned for next year. (3) MPACT Analysis Tools - (a) Multi-isotope Process Monitor • Fuel characterization framework development continued during June. A report describing the methodologies is being completed. Kenneth Dayman, from University of Texas, spent a week at PNNL wrapping up his master's research and working on a journal submission covering that work. The target journal is the IEEE transactions on Nuclear Science; submission is planned for the end of July. A proposal to instrument H-Canyon is being prepared in conjunction with SRNL and the NNSA's NGSI program. The impact of gamma-ray spectrum counting statistics on the precision of relative radioisotope component intensities as reconstructed via Principal Component Regression (PCR) continued in June with Monte Carlo simulations of a two-component (i.e., two radioisotope) system. This work generalizes earlier studies in FY12 in which Poisson counting variations of only a single spectrum component were simulated. (b) Modeling and Simulation for Analysis of Safeguards Performance (Electrochemical) • Preliminary insights into safeguards challenges and the initial design for an electrochemical plant have been written up into an INMM paper and will be presented at the INMM Summer Meeting. Work is currently adding a new visualization capability for integrating materials accountancy with physical protection. (c) Material Control including Process Monitoring (Pattern Recognition, Sensors) • Fabrication of quartz chips continued at an external foundry. Awaiting delivery of the heat exchange manifold and chip holder. (d) MPACT System Integration and Technical Support • The initial report on cost-basis metrics for nuclear material security risk and proliferation risk was completed as scheduled. (4) Safeguards and Security by Design - (a) Used fuels storage security analysis, guidance and best practices • RFP was finalized and sent out. Awaiting response from World Institute for Nuclear Security. Coordinated with Scott Demuth on joint report • LANL milestone on best practices. Attended Nuclear Energy Institute National Nuclear Security Conference. Developed plan for soliciting input on prioritized issues for used fuel storage security. Worked on setting up a meeting with the Nuclear Regulatory Commission. Worked on FY2013 work package planning.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Space-based Operations Grid Prototype

The Space based Operations Grid is intended to integrate the "high end" network services and compute resources that a remote payload investigator needs. This includes integrating and enhancing existing services such as access to telemetry, payload commanding, payload planning and internet voice distribution as well as the addition of services such as video conferencing, collaborative design, modeling or visualization, text messaging, application sharing, and access to existing compute or data grids. Grid technology addresses some of the greatest challenges and opportunities presented by the current trends in technology, i.e. how to take advantage of ever increasing bandwidth, how to manage virtual organizations and how to deal with the increasing threats to information technology security. We will discuss the pros and cons of using grid technology in space-based operations and share current plans for the prototype. It is hoped that early on the prototype can incorporate many of the existing as well as future services that are discussed in the first paragraph above to cooperating International Space Station Principle Investigators both nationally and internationally.

Bradford, Robert N.↗

Computational Tools and Workflows for Quantitative Risk Assessment and Decision Support for Geologic Carbon Storage Sites: Progress and Insights from the U.S. DOE’s National Risk Assessment Partnership

The 2005 Intergovernmental Panel on Climate Change (IPCC) Special Report on CCS raised the profile of CO2 capture and storage (CCS) as an important technology for reducing greenhouse gas (GHG) emissions. CCS is now recognized as a key component of most climate change mitigation scenarios. Since publication of that report the international research, development, and deployment (RD&D) community has advanced key technical aspects, clarified regulatory requirements, explored value chain and infrastructure solutions, and developed incentive paradigms to enable and promote large-scale deployment of CCS. These efforts have included research to better characterize geologic storage resources, to improve injection performance and storage efficiency, to assess and manage subsurface environmental risks, and to advance monitoring technologies to assure system conformance. These efforts have helped to build confidence in the viability of geologic carbon storage (GCS), but stakeholder concerns about long-term risks and liability associated with GCS remain a hurdle to broad acceptance and large-scale deployment of CCS. Since 2010, the U.S. DOE’s National Risk Assessment Partnership (NRAP) – a research collaboration between five contributing national laboratories – has worked to establish and demonstrate methods and tools to quantify and manage the subsurface environmental risks associated with GCS, amidst uncertainty. This work supports the Office of Fossil Energy and Carbon Management Carbon Transport and Storage Program’s goal of advancing safe and secure commercial-scale GCS deployment. To address the technical challenge of simulating the physical response of the GCS site to large-scale CO2 injection, NRAP has adopted an approach that relies on coupling computationally efficient reduced-order and/or data-driven proxy models of important system components (i.e., storage reservoir, sealing caprock, leakage pathways, intermediate formations, overlying groundwater aquifers, and the atmosphere) in integrated assessment framework. That integrated model of the physical system is complemented with fit-for purpose functionality to support site characterization and risk-related decisions. The recently released NRAP Phase II toolset includes the Open-Source Integrated Assessment Model (NRAP-Open-IAM) for evaluation of trends in leakage risk and potential impact, tools to support monitoring design optimization (Designs for Risk Evaluation and Management – DREAM v3.0 and Passive Seismic Monitoring Tool - PSMT), and tools for state of stress evaluation (State-of-Stress Analysis Tool - SOSAT) and forecasting induced seismicity risk. The NRAP team has also released a pair of reports describing conceptual workflows to incorporate physics-based, quantitative risk assessment into many of the design, planning, operation, and closure decisions for GCS projects. An online catalogue highlights published studies where these tools and methods are demonstrated. In this presentation, the utility of these products to assess risks and address key stakeholder questions will be highlighted through examples, and related insights about the safety and security of geologic carbon storage in qualified storage sites will be discussed. The prospect of rapid, large-scale deployment of GCS technology to aggressively reduce anthropogenic CO2 emissions requires careful consideration of interference between multiple commercial-scale storage projects within a basin. Going forward, NRAP is expanding and adapting site-scale risk quantification tools and methods to enable assessment of risks and inform management decisions for basin-scale deployment. Increasingly, this work will leverage next-generation approaches for surrogate modelling, fast prediction, and advanced visualization enabled by machine learning and artificial intelligence to promote virtual learning, scenario evaluation, and augment risk-based decision making.

quantitative risk assessment, geologic carbon stor↗

Survey Report for the Characterization of the Five Tanks Located Near the Old Salvage Yard at the Y-12 National Security Complex, Oak Ridge, Tennessee

This summary report presents analytical results, radiological survey data, and other data/information for disposition planning of the five tanks located west of the Old Salvage Yard (OSY) at the Y-12 National Security Complex (Y-12) in Oak Ridge, Tennessee. Field personnel from Oak Ridge Associated Universities (ORAU) and URS?CH2M Oak Ridge LLC completed data collection in May 2012 per the project-specific plan (PSP) (ORAU 2012). Deviations from the PSP are addressed in the body of this report. Characterization activities included three data collection modes: visual inspection, radiological survey, and volumetric sampling/analysis. This report includes the final validated dataset and updates associated with the Tank 2 residues originally thought to be a biological bloom (e.g., slime mold) but ultimately identified as iron sulfate crystals.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

In-Device Delay: Introduction to Obscurant Physcial Protection System

The Office of Radiological Security (ORS) In-Device Delay (IDD) program has undertaken a project to research and develop a novel protection system for industrial irradiators that contain high-activity Co-60 sources. Based on adversary testing conducted by ORS, it is was determined that to successfully accomplish the theft of the target material, the adversary will require visual contact of the sources and source rack located at the bottom of the pool. Therefore, if a means of obscuring or visually hiding the sources in the pool can be achieved (while adhering to facility operations, safety, and regulatory requirements), then illicit source theft will be significantly hindered. This project aims to develop a low-cost, non-propriety obscurant that, when an adversary action is detected, the obscurant will be deployed into the pool quickly, rendering visual observation of the source problematic; however, this obscurant will not otherwise disturb the sources, source rack, and filtration system. The obscurant will remain in the pool until removed by another process.

61 RADIATION PROTECTION AND DOSIMETRY↗

mvBayes

SAND2026-16980O mvBayes implements multivariate Bayesian regression using MATLAB and decomposes a multivariate or functional response into components based on a user-specified orthogonal basis. This allows for independent modeling of each component with any chosen univariate Bayesian regression model. This tool includes methods for prediction and visualization, facilitating the evaluation of Bayesian surrogate models through the application of Bayesian theory and Markov Chain Monte Carlo (MCMC) sampling techniques. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Tucker, J. Derek [Sandia National Lab. (SNL-CA), L↗

Scalable edge clustering of dynamic graphs via weighted line graphs

Timestamped relational datasets consisting of records (or connections) between pairs of entities are ubiquitous in network science. For applications like peer-to-peer communication, email, various social network interactions, and computer network security, it is useful to organize these records into groups based on how and when they are occurring. Weighted line graphs offer a natural way to model how records are related in such datasets but for large real-world graph topologies, building and utilizing the line graph is prohibitively expensive. Here, we present the framework to cluster the edges of a dynamic graph via the associated line graph that contains two major contributions. The first is a method to work with the line graph implicitly and the second is a distributed scale implementation of an agglomerative hierarchical graph clustering algorithm. We outline a novel hierarchical dynamic graph edge clustering approach that efficiently breaks massive relational datasets into small sets of edges containing events at various timescales. This is in stark contrast to traditional graph clustering algorithms that prioritize highly connected (clique-like) community structures. Our approach relies on constructing a sufficient subgraph of a weighted line graph and applying a hierarchical agglomerative clustering. This approach is related to scalable techniques from spatial clustering, nonlinear-dimension reduction, topological data analysis, and draws particular inspiration from HDBSCAN. As an edge clustering, this method yields an overlapping node clustering. Our algorithm is parallelizable and we demonstrate efficient clustering of a billion-scale, real-world dynamic graph into small edge sets that correlate in topology and time. The entire clustering process for a graph with tens of billions of edges takes just a few minutes of run time on 256 nodes of a distributed compute environment. We argue how the output of the edge clustering is useful for a multitude of data visualization and powerful machine learning tasks, both involving the original massive dynamic graph data and metadata associated with the nodes and edges. Finally, we describe how this approach can be extended to dynamic hypergraphs and dynamic graphs/hypergraphs with unstructured data living on vertices and edges.

Data Analysis↗