Search NASA⌕ Search

SEARCH · Search NASA

Results for “Graph analysis systems”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

BCSR on GPU: A Way Forward Extreme-scale Graph Processing on Accelerator-enabled Frontier Supercomputer

Handling large graphs in a distributed environment requires effective partitioning across processors and efficient management of local partitions. In 2D partitioning, local graphs often become too sparse, making memory-efficient data structures crucial. Using the Compressed Sparse Row (CSR) format wastes space, especially for > 83% of vertices with empty edges for the sparse graphs. This study explores bit-CSR (BCSR), a modified CSR representation, on GPUs to reduce memory usage in graph computations. We achieved 16.67% memory savings on a sparse rmat dataset with 268 million vertices and 357 million edges, without performance degradation, supported by both theoretical and experimental storage savings of 33%. However, we observed a 1.7× slowdown in degree lookup times due to bitwise operations on AMD CPUs. This analysis highlights the potential of BCSR on GPUs for improving Graph500 benchmark performance on GPU-accelerated systems, such as the Frontier supercomputer.

Sattar, Naw Safrin↗
DOE OSTI2012

System Advisor Model, SAM 2011.12.2: General Description

This document describes the capabilities of the U.S. Department of Energy and National Renewable Energy Laboratory's System Advisor Model (SAM), Version 2011.12.2, released on December 2, 2011. SAM is software that models the cost and performance of renewable energy systems. Project developers, policy makers, equipment manufacturers, and researchers use graphs and tables of SAM results in the process of evaluating financial, technology, and incentive options for renewable energy projects. SAM simulates the performance of solar, wind, geothermal, biomass, and conventional power systems. The financial model can represent financing structures for projects that either buy and sell electricity at retail rates (residential and commercial) or sell electricity at a price determined in a power purchase agreement (utility). Advanced analysis options facilitate parametric, sensitivity, and statistical analyses, and allow for interfacing SAM with Microsoft Excel or with other computer programs. SAM is available as a free download at http://sam.nrel.gov. Technical support and more information about the software are available on the website.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

An intelligent Data Delivery Service for and beyond the ATLAS experiment

The intelligent Data Delivery Service (iDDS) has been developed to cope with the huge increase of computing and storage resource usage in the coming LHC data taking. It has been designed to intelligently orchestrate workflows and data management systems, decoupling data pre-processing, delivery, and primary processing in large scale workflows. It is an experiment-agnostic service that has been deployed to serve data carousel (orchestrating efficient processing of tape-resident data), machine learning hyperparameter optimization, active learning, and other complex multi-stage workflows defined via DAG (Directed Acyclic Graph), CWL (Common Workflow Language) and other descriptions, including a growing number of analysis workflows. We will at first introduce some deployed use cases in a summary. Then we will focus on new improvements and use cases under developments in ATLAS, Rubin Observatory and sPHENIX, together with future efforts.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Emerging Jets Search, Triton Server Deployment, and Track Quality Development: Machine Learning Applications in High Energy Physics

Machine learning is becoming prevalent in high energy physics, with numerous applications in physics analyses and event reconstruction showing great improvements compared to traditional computing methods. This thesis studies three projects which each propose new avenues for machine learning applications within the high energy physics CMS experiment located at CERN. In the first project, a search for a dark matter signal called “emerging jets” is performed, using graph neural networks to greatly increase sensitivity to the signal’s signature within the data. The result of this dark matter search sets the most stringent exclusion limits to date on theoretical emerging jet models. Motivated by inefficiencies encountered when processing the emerging jet graph neural network at Fermi National Accelerator Laboratory’s computing centers, the second project re-optimizes the computing centers for machine learning inference. This re-optimization uses NVIDIA Triton Inference Servers to process users’ analysis code heterogeneously, therefore achieving high processing throughput and decreasing user time-to-insight. The last project focuses on an upgrade to the CMS experiment’s real-time event selection system which improves physics object reconstruction under harsh processing conditions. A boosted decision tree is used to quickly and efficiently quantify a reconstructed particle’s “track quality” in order to remove particle tracks reconstructed erroneously. In summary, this thesis will not only present examples of how high energy physics can greatly benefit by leveraging machine learning techniques for physics analysis and reconstruction, but will also provide guidance on how the field can prepare for the inevitable increase in machine learning applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2017

Comparative Study of Clustering Techniques for Real-Time Dynamic Model Reduction

Dynamic model reduction in power systems is necessary for improving computational efficiency. Traditional model reduction using linearized models or offline analysis would not be adequate to capture power system dynamic behaviors, especially the new mix of intermittent generation and intelligent consumption makes the power system more dynamic and non-linear. Real time dynamic model reduction emerges as an important need. This paper explores the use of clustering techniques to analyze real-time phasor measurements to determine generator groups and representative generators for dynamic model reduction. Two clustering techniques – graph clustering and evolutionary clustering – are studied in this paper. Various implementations of these techniques are compared and also compared with a previously developed Singular Value Decomposition (SVD)-based dynamic model reduction approach. Various methods exhibit different levels of accuracy when comparing the reduced model simulation against the original model. But some of them are consistently accurate. From this comparative perspective, this paper provides a good reference point for practical implementations.

model reduction, graph clustering, svd, evolutiona↗
DOE OSTI2026

Classification of dynamical Lie algebras generated by spin interactions on undirected graphs

Dynamical Lie algebras (DLAs) are a versatile tool for various topics that span from the expressibility-trainability of variational quantum algorithms (VQAs), to simulation of many body Hamiltonians. Quantum gates and most of the Hamiltonians of interest consist of local interactions; therefore, the analysis of all possible DLAs generated by 1- and 2-local operators is crucial for quantum simulation and VQAs on current hardware. Previously in [R. Wiersema et al ., npj Quantum Inf. 10 , 110 (2024)], we analyzed the DLAs on linear, circular and all-to-all topologies, and obtained results about their dimensions and algebraic structure. Here, in this work, we extend our analysis into any possible hardware topology and provide a classification of all DLAs generated by Pauli strings on any undirected interaction graph. Our results indicate that the DLAs depend solely on whether the connectivity or interaction graph is bipartite or not. In addition, we find that the non-trivial polynomially scaling DLAs appear only on 1D line or circle topologies, and all other DLAs have dimensions scaling exponentially with the system size. Together with the current VQA literature, our results imply that either the majority of VQAs are non-trainable, or we are yet to understand the role of DLAs on the trainability of VQAs.

Algebraic structures↗
DOE OSTI2018

Numerically approximating centrality for graph ranking guarantees

Many real-world datasets can be represented as graphs. Using iterative solvers to approximate graph centrality measures allows us to obtain a ranking vector on the nodes of the graph, consisting of a number for each vertex in the graph identifying its relative importance. In this study the centrality measures we use are Katz Centrality and PageRank. Given an approximate solution, we use the residual to accurately estimate how much of the ranking matches the ranking given by the exact solution. Using probabilistic matrix norms, we obtain bounds on the accuracy of the approximation compared to the exact solution with respect to the highly ranked nodes and apply numerical analysis to the computation of centrality with iterative methods. This relates the numerical accuracy of the linear solver to the data analysis accuracy of finding the correct ranking. In particular, we answer the question of which pairwise rankings are reliable given an approximate solution to the linear system. Experiments on many real-world undirected and directed networks up to several million vertices and several hundred million edges validate our theory and show that we are able to accurately estimate large portions of the approximation. We also analyze the difference between global centrality scores and personalized scores (w.r.t. specific seed vertices). By analyzing convergence error, we develop confidence in the ranking schemes of data mining. We show we are able to accurately guarantee ranking of vertices with an approximation to centrality metrics faster than current methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2011

Coherent random lasing in diffusive resonant media

We investigate diffusive propagation of light and consequent random lasing in a medium comprising resonant spherical scatterers. A Monte-Carlo calculation based on photon propagation via three-dimensional random walks is employed to obtain the dwell-times of light in the system. We compare the inter-scatterer and intra-scatterer dwell-times for representative resonant and non-resonant wavelengths. Our results show that more efficient random lasing, with intense coherent modes, is obtained when the gain is present inside the scatterers. Further, a larger reduction in frequency fluctuations is achieved by the system with intra-scatterer gain.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2019

Topology Inference of Unknown Networks Based on Robust Virtual Coordinate Systems

Learning and exploring the connectivity of unknown networks represent an important problem in practical applications of communication networks and social-media networks. Modeling large-scale networks as connected graphs is highly desirable to extract their connectivity information among nodes to visualize network topology, disseminate data, and improve routing efficiency. This paper investigates a simple measurement model in which a small subset of source nodes collect hop distance information from networked nodes in order to generate a virtual coordinate system (VCS) for networks of unknown topology. We establish the VCS to define logical distance among nodes based on principal component analysis and to determine connectivity relationship and effective routing methods. More importantly, we present a robust analytical algorithm to derive the VCS against practical issues of missing and corrupted measurements. We also develop a connectivity inference method which classifies nodes into layers based on the hop distances and derives partial information on network connectivity.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2019

A Knowledge Graph Approach for the Secondary Use of Cancer Registry Data

Population-based central cancer registries collect valuable structured and unstructured data primarily for cancer surveillance and research, enhancing insights into clinical features associated with cancer occurrence, cancer treatment, and cancer outcomes to guide interventions which reduce the cancer burden. Cancer registries primarily collect data on (1) cancer type (case or tumor); (2) patient demographics such as age, gender, and residential address at time of diagnosis; (3) planned first course of treatment; and (4) date of last contact, vital status, and cause of death. Cancer registry data is dynamic, structured data, which is extracted from many unstructured sources such as electronic healthcare records, and consolidated for reporting and other purposes. While available advanced analytic tools such as SEER*Stat have the ability to build SAS queries, we, however, explore an innovative knowledge graph approach to organizing cancer registry data for advanced analytics and visualization, which has unique advantages over approaches of existing tools. This innovative knowledge graph approach semantically enriches the data and easily enables linkage with third-party data, which can better explain variation in outcomes. We have developed a prototype knowledge graph based on data from the Louisiana Tumor Registry and other publicly available datasets including Behavioral Risk Factor Surveillance System, Clinical Trials, DBpedia, GeoNames, Rural-Urban Continuum Codes, and Semantic MEDLINE. The resource description framework (RDF) data model was selected to represent our knowledge graph, which contains more than 25 billion triples and is ~4TB in storage size. To exhibit the benefits of the knowledge graph approach, we used scenario specific queries, which find the relationships between cancer treatment sequences and outcomes. To illustrate its ease of use in iterative analysis, the knowledge graph was linked to external datasets for performing complex queries across multiple datasets. In addition, we used knowledge graphs to identify data discrepancies and to handle schema changes. Finally, we visualized the knowledge graph to discover data patterns. Our results demonstrate this graph-based solution enables cancer researchers to execute complex queries and more easily perform iterative analyses to improve understanding of cancer registry data. In the future, we would like to use high-performance computing (HPC) resources for faster-generating hypotheses with clinical potential from our knowledge graph.

Hasan, S M Shamimul↗
DOE OSTI2011

MicroCT: Automated Analysis of CT Reconstructed Data of Home Made Explosive Materials Using the Matlab MicroCT Analysis GUI

This Standard Operating Procedure (SOP) provides the specific procedural steps for analyzing reconstructed CT images obtained under the IDD Standard Operating Procedures for data acquisition and MicroCT image reconstruction, per the IDD Quality Assurance Plan for MicroCT Scanning. Although intended to apply primarily to MicroCT data acquired in the HEAFCAT Facility at LLNL, these procedures may also be applied to data acquired at Tyndall from the YXLON cabinet and at TSL from the HEXCAT system. This SOP also provides the procedural steps for preparing the tables and graphs to be used in the reporting of analytical results. This SOP applies to production work - for R and D there are two other semi-automated methods as given in [4, 5].

42 ENGINEERING↗
DOE OSTI2011

MicroCT: Semi-Automated Analysis of CT Reconstructed Data of Home Made Explosive Materials Using the Matlab MicroCT Analysis GUI

This Standard Operating Procedure (SOP) provides the specific procedural steps for analyzing reconstructed CT images obtained under the IDD Standard Operating Procedures for data acquisition and MicroCT image reconstruction, per the IDD Quality Assurance Plan for MicroCT Scanning. Although intended to apply primarily to MicroCT data acquired in the HEAFCAT Facility at LLNL, these procedures may also be applied to data acquired at Tyndall from the YXLON cabinet and at TSL from the HEXCAT system. This SOP also provides the procedural steps for preparing the tables and graphs to be used in the reporting of analytical results. This SOP applies to R and D work - for production applications, use [4].

42 ENGINEERING↗
DOE OSTI2023

Gaps labeling theorem for the bubble-diamond self-similar graphs

Abstract Motivated by the appearance of fractals in several areas of physics, especially in solid state physics and the physics of aperiodic order, and in other sciences, including the quantum information theory, we present a detailed spectral analysis for a new class of fractal-type diamond graphs, referred to as bubble-diamond graphs, and provide a gap-labeling theorem in the sense of Bellissard for the corresponding probabilistic graph Laplacians using the technique of spectral decimation. Labeling the gaps in the Cantor set by the normalized eigenvalue counting function, also known as the integrated density of states, we describe the gap labels as orbits of a second dynamical system that reflects the branching parameter of the bubble construction and the decimation structure. The spectrum of the natural Laplacian on limit graphs is shown generically to be pure point supported on a Cantor set, though one particular graph has a mixture of pure point and singularly continuous components.

Physics↗
DOE OSTI2015

Graph processing platforms at scale: practices and experiences

Graph analysis unveils hidden associations of data in many phenomena and artifacts, such as road network, social networks, genomic information, and scientific collaboration. Unfortunately, a wide diversity in the characteristics of graphs and graph operations make it challenging to find a right combination of tools and implementation of algorithms to discover desired knowledge from the target data set. This study presents an extensive empirical study of three representative graph processing platforms: Pegasus, GraphX, and Urika. Each system represents a combination of options in data model, processing paradigm, and infrastructure. We benchmarked each platform using three popular graph operations, degree distribution, connected components, and PageRank over a variety of real-world graphs. Our experiments show that each graph processing platform shows different strength, depending the type of graph operations. While Urika performs the best in non-iterative operations like degree distribution, GraphX outputforms iterative operations like connected components and PageRank. In addition, we discuss challenges to optimize the performance of each platform over large scale real world graphs.

Lim, Seung-Hwan↗
DOE OSTI2017

Multi-scale visual analysis of time-varying electrocorticography data via clustering of brain regions

There exists a need for effective and easy-to-use software tools supporting the analysis of complex Electrocorticography (ECoG) data. Understanding how epileptic seizures develop or identifying diagnostic indicators for neurological diseases require the in-depth analysis of neural activity data from ECoG. Such data is multi-scale and is of high spatio-temporal resolution. Comprehensive analysis of this data should be supported by interactive visual analysis methods that allow a scientist to understand functional patterns at varying levels of granularity and comprehend its time-varying behavior. We introduce a novel multi-scale visual analysis system, ECoG ClusterFlow, for the detailed exploration of ECoG data. Our system detects and visualizes dynamic high-level structures, such as communities, derived from the time-varying connectivity network. The system supports two major views: 1) an overview summarizing the evolution of clusters over time and 2) an electrode view using hierarchical glyph-based design to visualize the propagation of clusters in their spatial, anatomical context. We present case studies that were performed in collaboration with neuroscientists and neurosurgeons using simulated and recorded epileptic seizure data to demonstrate our system's effectiveness. ECoG ClusterFlow supports the comparison of spatio-temporal patterns for specific time intervals and allows a user to utilize various clustering algorithms. Neuroscientists can identify the site of seizure genesis and its spatial progression during various the stages of a seizure. Our system serves as a fast and powerful means for the generation of preliminary hypotheses that can be used as a basis for subsequent application of rigorous statistical methods, with the ultimate goal being the clinical treatment of epileptogenic zones.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2018

Multiphysics Lattice Discrete Particle Modeling (M-LDPM) for the Simulation of Shale Fracture Permeability

A three-dimensional multiphysics lattice discrete particle model (M-LDPM) framework is formulated to investigate the fracture permeability behavior of shale. The framework features a dual lattice system mimicking the mesostructure of the material and simulates coupled mechanical and flow behavior. The mechanical lattice model simulates the granular internal structure of shale, and describes heterogeneous deformation by means of discrete compatibility and equilibrium equations. The network of flow lattice elements constitutes a dual graph of the mechanical lattice system. A discrete formulation of mass balance for the flow elements is presented to model fluid flow along cracks and intact materials. The overall computational framework is implemented with a mixed explicit–implicit integration scheme and a staggered coupling method that makes use of the dual lattice topology enabling the seamless two-way coupling of the mechanical and flow behaviors. The proposed model is used for the computational analysis of shale fracture permeability behavior by simulating triaxial direct shear tests on Marcellus shale specimens under various confining pressures. The simulated mechanical response is calibrated against the experimental data, and the predicted permeability values are also compared with the experimental measurements. Moreover, the paper presents the scaling analysis of both the mechanical response and permeability measurements based on simulations performed on geometrically similar specimens with increasing size. The simulated stress strain curves reflect a significant size effect in the post-peak due to the presence of localized fractures. The scaling analysis of permeability measurements enables prediction of permeability for large specimens by extrapolating the numerical results of small ones.

58 GEOSCIENCES↗
DOE OSTI2020

A supernodal all-pairs shortest path algorithm

We show how to exploit graph sparsity in the Floyd-Warshall algorithm for the all-pairs shortest path (Apsp) problem. Floyd-Warshall is an attractive choice for Apsp on high-performing systems due to its structural similarity to solving dense linear systems and matrix multiplication. However, if sparsity of the input graph is not properly exploited, Floyd-Warshall will perform unnecessary asymptotic work and thus may not be a suitable choice for many input graphs. To overcome this limitation, the key idea in our approach is to use the known algebraic relationship between Floyd-Warshall and Gaussian elimination, and import several algorithmic techniques from sparse Cholesky factorization, namely, fill-in reducing ordering, symbolic analysis, supernodal traversal, and elimination tree parallelism. When combined, these techniques reduce computation, improve locality and enhance parallelism. We implement these ideas in an efficient shared memory parallel prototype that is orders of magnitude faster than an efficient multi-threaded baseline Floyd-Warshall that does not exploit sparsity. Our experiments suggest that the Floyd-Warshall algorithm can compete with Dijkstra's algorithm (the algorithmic core of Johnson's algorithm) for several classes sparse graphs.

Sao, Piyush↗
DOE OSTI2026

Risk-Aware Measurement Synchronization and Recovery for DSSE With Heterogeneous Data Sources

Power distribution systems are increasingly integrating heterogeneous sensors with varying data reporting rates and types, which pose challenges to achieving observability at the desired temporal resolution of distribution system state estimation (DSSE). Multisensor failures caused by extreme events exacerbate these issues, introducing substantial uncertainties into DSSE. This article proposes a novel solution to these challenges by ensuring high-resolution system observability despite heterogeneous data sources and multisensor failures. First, a deep learning architecture combining long short-term memory (LSTM) and graph convolutional network (GCN) is employed to synchronize meters with different reporting rates, aiming to achieve system observability. A random-walk-model-based approach is introduced to generate pseudo-measurements while properly characterizing their uncertainties under multisensor failures. Finally, a disaster-risk-informed observability metric (RiOM) is defined to quantify the uncertainty associated with state estimation results. The proposed framework offers deeper insights into the system observability on the fly compared with conventional analysis. The effectiveness of the framework is demonstrated on an IEEE standard test case and a large-scale real-world distribution feeder in mid-Minnesota in the U.S.

97 MATHEMATICS AND COMPUTING↗