Search NASA⌕ Search

SEARCH · Search NASA

Results for “data locality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

Data-Driven Event Detection of Power Systems Based on Unequal-Interval Reduction of PMU Data and Local Outlier Factor

With the deployment of phasor measurement units (PMU) and wide area measurement system (WAMS), it is feasible to have an insight into the events occurred in power systems based on measured data. Thus, a novel data-driven algorithm based on local outlier factor (LOF) is proposed in this work to detect and locate events in power systems using reduced PMU data. First, the unequal-interval reduction method is presented to reduce the scale of PMU data in sub-stations and reconstruct it in master station of WAMS, which can relieve the burden of communication systems. Then, principle component analysis (PCA)-based similarity search method is proposed to measure the differences of operation state between any two buses. Next, LOF is presented to detect the abnormal events in power systems, and employed to determine the region of the event source. Finally, six cases from the Western electricity coordinating council (WECC) 179-bus power system, a case from the South China power system (SCPS), and a case from the Guangdong power system (GDPS) are utilized to demonstrate the effectiveness of the proposed algorithm. Overall, the results show that proposed algorithm is effective and can be applied to event detection, event location, and online monitoring, which can enhance the situation awareness ability of power system operators.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2018

Partnership For Edge Physics Simulation

In this effort, we will extend our prior work as part of CPES (i.e., DART and DataSpaces) to support in-situ tight coupling between application codes that exploits data locality and core-level parallelism to maximize on-chip data exchange and reuse. This will be accomplished by mapping coupled simulations so that the data exchanges are more localized within the nodes. Coupled simulation workflows can more effectively utilize the resources available on emerging HEC platforms if they can be mapped and executed to exploit data locality as well as the communication patterns between application components. Scheduling and running such workflows requires an extended framework that should provide a unified hybrid abstraction to enable coordination and data sharing across computation tasks that run on the heterogeneous multi-core-based systems, and develop a data-locality based dynamic tasks scheduling approach to increase on-chip or intra-node data exchanges and in-situ execution. This effort will extend our prior work as part of CPES (i.e., DART and DataSpaces), which provided a simple virtual shared-space abstraction hosted at the staging nodes, to support application coordination, data sharing and active data processing services. Moreover, it will transparently manage the low-level operations associated with the inter-application data exchange, such as data redistributions, and will enable running coupled simulation workflow on multi-cores computing platforms.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2017

Global and adaptive K-nearest neighbor graphs in a spectral target detector based on Schroedinger Eigenmaps

The Schroedinger Eigenmaps (SE) embedding has been previously introduced and applied to spectral target detection problems in hyperspectral imagery (HSI). The proposed SEbased detection approach combines the spectral and spatial connectivity of target-like pixels into the Schroedinger operator by using a “knowledge propagation” scheme. Likewise, it has been noted the impact that the local data structure modeling has over the SE-based detector. This local data structure is modeled by using an adjacency graph, i.e, set of vertices and edges, that in most of the applications is a k nearest neighbor (knn) graph. In this paper, the impact of k in the graph-building process is analyzed and assessed from a target detection point of view. Here, two ways to estimate the parameter k in the graph construction are considered and their assessment is performed by using two HSI data sets and the Receiver Optimal Curve (ROC) as a detection performance metric

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Locality-aware and load-balanced static task scheduling for MapReduce

Task scheduling for MapReduce jobs has been an active area of research with the objective of decreasing the amount of data transferred during the shuffle phase via exploiting data locality. In the literature, generally only the scheduling of reduce tasks is considered with the assumption that scheduling of map tasks is already determined by the input data placement. However, in cloud or HPC deployments of MapReduce, the input data is located in a remote storage and scheduling map tasks gains importance. Here, we propose models for simultaneous scheduling of map and reduce tasks in order to improve data locality and balance the processors’ loads in both map and reduce phases. Our approach is based on graph and hypergraph models which correctly encode the interactions between map and reduce tasks. Partitions produced by these models are decoded to schedule map and reduce tasks. A two-constraint formulation utilized in these models enables balancing processors’ loads in both map and reduce phases. The partitioning objective in the hypergraph models correctly encapsulates the minimization of data transfer when a local combine step is performed prior to shuffle, whereas the partitioning objective in the graph models achieve the same feat when a local combine is not performed. Here, we show the validity of our scheduling on the MapReduce parallelizations of two important kernel operations – sparse matrix–vector multiplication (SpMV) and generalized sparse matrix–matrix multiplication (SpGEMM) – that are widely encountered in big data analytics and scientific computations. Compared to random scheduling, our models lead to tremendous savings in data transfer by reducing data traffic from several hundreds of megabytes to just a few megabytes in the shuffle phase and consequently leading up to 2.6x and 4.2x speedup for SpMV and SpGEMM, respectively.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Customizable Scale-Out Key-Value Stores

Enterprise KV stores are often not well suited for HPC applications, and thus cumbersome end-to-end KV design customization is required to meet the needs of modern HPC applications. To this end, in this article we present bespoKV, an adaptive, extensible, and scale-out KV store framework. bespoKV decouples the KV store design into the control plane for distributed management and the data plane for local data store. For the control plane, bespoKVprovides pre-built modules, called controlets, supporting common distributed functionalities (e.g., replication, consistency, and topology) and their various combinations. This decoupling allows bespoKV to take a user-provided single-server KV store, called a datalet, and transparently enables a scalable and fault-tolerant distributed KV store service. The resulting distributed stores are also adaptive to consistency or topology requirement changes and can be easily extended for new types of services. Such specializations enable innovative uses of KV stores in HPC applications, especially for emerging applications that utilize KV-friendly workloads. We evaluate bespoKV in a local testbed as well as in a public cloud settings. Experiments show that bespoKV-enabled distributed KV stores scale horizontally to a large number of nodes, and performs comparably and sometimes 1.2× to 2.6× better than the state-of-the-art systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

BESPOKV: Application Tailored Scale-Out Key-Value Stores

Enterprise KV stores are not well suited for HPC applications, and entail customization and cumbersome end-to-end KV design to extract the HPC application needs. To this end, in this paper we present BESPOKV, an adaptive, extensible, and scale-out KV store framework. BESPOKV decouples the KV store design into the control plane for distributed management and the data plane for local data store. BESPOKV takes as input a single-server KV store, called a datalet, and transparently enables a scalable and fault-tolerant distributed KV store service. The resulting distributed stores are also adaptive to consistency or topology requirement changes and can be easily extended for new types of services. Experiments show that BESPOKV-enabled distributed KV stores scale horizontally to a large number of nodes, and performs comparably and sometimes better than the state-of-the-art systems.

Anwar, Ali↗
DOE OSTI2023

FPGA Acceleration of GCN in Light of the Symmetry of Graph Adjacency Matrix

Graph Convolutional Neural Networks (GCNs) are widely used to process large-scale graph data. Different from deep neural networks (DNNs), GCNs are sparse, irregular, and unstructured, posing unique challenges to hardware acceleration with regular processing elements (PEs). In particular, the adjacency matrix of a GCN is extremely sparse, leading to frequent but irregular memory access, low spatial/temporal data locality and poor data reuse. Furthermore, a realistic graph usually consists of unstructured data (e.g., unbalanced distributions), creating significantly different processing times and imbalanced workload for each node in GCN acceleration. To overcome these challenges, we propose an end-to-end hardware-software co-design to accelerate GCNs on resource-constrained FPGAs with the features including: (1) A custom dataflow that leverages symmetry along the diagonal of the adjacency matrix to accelerate feature aggregation for undirected graphs. We utilize either the upper or the lower triangular matrix of the adjacency matrix to perform aggregation in GCN to improve data reuse. (2) Unified compute cores for both aggregation and transform phases, with full support to the symmetry-based dataflow. These cores can be dynamically reconfigured to the systolic mode for transformation or as individual accumulators for aggregation in GCN processing. (3) Preprocessing of the graph in software to rearrange the edges and features to match the custom dataflow. This step improves the regularity in memory access and data reuse in the aggregation phase. Moreover, we quantize the GCN precision from FP32 to INT8 to reduce the memory footprint without losing the inference accuracy. We implement our accelerator design in Intel Stratix10 MX FPGA board with HBM2, and demonstrate 1.3x-110.5x improvement in end-to-end GCN latency as compared to the state-of the-art FPGA implementations, on the graph datasets of Cora, Pubmed, Citeseer and Reddit.

Nair, Gopikrishnan R.↗
DOE OSTI2020

Redirecting data to improve page locality in a scalable data fabric

A data processing system includes a host processor, a local memory coupled to the host processor, a plurality of remote memory media, and a scalable data fabric coupled to the host processor and to the plurality of remote memory media. The scalable data fabric includes a filter for storing information indicating a location of data that is stored by the data processing system. The host processor includes a hardware sequencer coupled to the filter for selectively moving data stored by the filter to the local memory.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2007

The Business Case for Automated Software Engineering

Adoption of advanced automated SE (ASE) tools would be more favored if a business case could be made that these tools are more valuable than alternate methods. In theory, software prediction models can be used to make that case. In practice, this is complicated by the 'local tuning' problem. Normally. predictors for software effort and defects and threat use local data to tune their predictions. Such local tuning data is often unavailable. This paper shows that assessing the relative merits of different SE methods need not require precise local tunings. STAR 1 is a simulated annealer plus a Bayesian post-processor that explores the space of possible local tunings within software prediction models. STAR 1 ranks project decisions by their effects on effort and defects and threats. In experiments with NASA systems. STARI found one project where ASE were essential for minimizing effort/ defect/ threats; and another project were ASE tools were merely optional.

model evaluation↗
NASA NTRS1987

A hypermatrix formulation for subspace iteration

The computational efficiency of subspace iteration is addressed relative to the data structures adopted for the very large and generally sparse coefficient matrices. The frequent triangulations and matrix multiplications demand that access to the terms in the coefficient matrices be unbiased. Reliance on virtual memory (paging) operating systems with no special considerations for localized data access is not adequate. Specific data structures must be designed that accommodate the needs of the numerical algorithm yet eliminate unnecessary paging. An implementation of the subspace iteration method using hypermatrix data structures is presented. Use of hypermatrices is shown to provide unbiased and localized data access. The various modifications to the conventional formulation are described and an example problem illustrates the potential benefits of the hypermatrix formulation. Possibilities for adapting hypermatrix data structures to new supercomputer architectures are discussed.

Schmidt, Richard J.↗
DOE OSTI2015

SU-E-T-144: Bayesian Inference of Local Relapse Data Using a Poisson-Based Tumour Control Probability Model

Purpose: The purpose of this project is to develop a robust method of parameter estimation for a Poisson-based TCP model using Bayesian inference. Methods: Bayesian inference was performed using the PyMC3 probabilistic programming framework written in Python. A Poisson-based TCP regression model that accounts for clonogen proliferation was fit to observed rates of local relapse as a function of equivalent dose in 2 Gy fractions for a population of 623 stage-I non-small-cell lung cancer patients. The Slice Markov Chain Monte Carlo sampling algorithm was used to sample the posterior distributions, and was initiated using the maximum of the posterior distributions found by optimization. The calculation of TCP with each sample step required integration over the free parameter α, which was performed using an adaptive 24-point Gauss-Legendre quadrature. Convergence was verified via inspection of the trace plot and posterior distribution for each of the fit parameters, as well as with comparisons of the most probable parameter values with their respective maximum likelihood estimates. Results: Posterior distributions for α, the standard deviation of α (σ), the average tumour cell-doubling time (Td), and the repopulation delay time (Tk), were generated assuming α/β = 10 Gy, and a fixed clonogen density of 10{sup 7} cm−{sup 3}. Posterior predictive plots generated from samples from these posterior distributions are in excellent agreement with the observed rates of local relapse used in the Bayesian inference. The most probable values of the model parameters also agree well with maximum likelihood estimates. Conclusion: A robust method of performing Bayesian inference of TCP data using a complex TCP model has been established.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

High-order hydrodynamic algorithms for exascale computing

Hydrodynamic algorithms are at the core of many laboratory missions ranging from simulating ICF implosions to climate modeling. The hydrodynamic algorithms commonly employed at the laboratory and in industry (1) typically lack requisite accuracy for complex multi- material vortical flows and (2) are not well suited for exascale computing due to poor data locality and poor FLOP/memory ratios. Exascale computing requires advances in both computer science and numerical algorithms. We propose to research the second requirement and create a new high-order hydrodynamic algorithm that has superior accuracy, excellent data locality, and excellent FLOP/memory ratios. This proposal will impact a broad range of research areas including numerical theory, discrete mathematics, vorticity evolution, gas dynamics, interface instability evolution, turbulent flows, fluid dynamics and shock driven flows. If successful, the proposed research has the potential to radically transform simulation capabilities and help position the laboratory for computing at the exascale.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2010

Phased Array Noise Source Localization Measurements of an F404 Nozzle Plume at Both Full and Model Scale

A 48-microphone planar phased array system was used to acquire jet noise source localization data on both a full-scale F404-GE-F400 engine and on a 1/4th scale model of a F400 series nozzle. The full-scale engine test data show the location of the dominant noise sources in the jet plume as a function of frequency for the engine in both baseline (no chevron) and chevron configurations. Data are presented for the engine operating both with and without afterburners. Based on lessons learned during this test, a set of recommendations are provided regarding how the phased array measurement system could be modified in order to obtain more useful acoustic source localization data on high-performance military engines in the future. The data obtained on the 1/4th scale F400 series nozzle provide useful insights regarding the full-scale engine jet noise source mechanisms, and document some of the differences associated with testing at model-scale versus fullscale.

Podboy, Gary G.↗
DOE OSTI2020

Investigation of countercurrent flow profile and liquid holdup in random packed column with local CFD data

Liquid holdup and mass transfer area are critical parameters used in packed column design and CO 2 capture efficiency prediction. Compared to experimental methods, 3D computational fluid dynamics (CFD) can provide high-resolution data for studying countercurrent flow hydrodynamics in a packed column. In this paper, a framework was established for modeling the liquid–gas countercurrent flow in a random packed column with pall rings. Besides the column-averaged information, the radial pall ring distribution, velocity, and liquid holdup profiles are obtained in the simulations. The radial two-phase velocity and liquid holdup profiles show significant fluctuations caused by the non-uniform pall ring distribution. The entrance effect and wall influence are also observed in the packed column. By using local countercurrent flow information, the packing specific area $a_p$ and liquid velocity $u_L$ range are significantly expanded for liquid holdup hL prediction. This enables us to develop the liquid holdup correlation with one packing size and six CFD runs only. The proposed power law of the liquid holdup correlation $h_L~u_L^{0.44}$ indicates the random packed column falls in a viscous to turbulent transition regime with a Reynolds Number range of [6.7–40.2]. The derived liquid holdup correlation is in good agreement with existing correlations that are developed using the column-averaged experimental data.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Enhancing Network Anomaly Detection Using Graph Neural Networks

In the world of Internet of Things (IoT) networks, where devices are constantly communicating, keeping them secure from cyber threats is critical. This paper introduces a novel approach to detecting unusual and potentially harmful activities in these networks using graph neural networks (GNNs). We combine two specific types of GNNs-GraphSAGE and graph attention networks (GAT)-to create a model that understands and represents the behaviors and interactions in a network. GraphSAGE creates an embedding of network activities by examining local data interactions, while GAT directs the model's focus to the most critical interactions. By integrating these two methods in a single model that considers different types of interactions (both host and flow nodes), we aim to create a system that accurately represents the current state of a network and can also spot anomalies effectively while reducing false positives and negatives. Our innovative approach has demonstrated promising results, achieving an accuracy of 98% on the UNSW-NB15 dataset, significantly outperforming standalone GraphSAGE and GAT models. This underscores its potential as a robust framework for securing IoT networks against cyber threats and anomalies.

Marfo, William↗
NASA NTRS2004

Telescience Resource Kit Software Capabilities and Future Enhancements

The Telescience Resource Kit (TReK) is a suite of PC-based software applications that can be used to monitor and control a payload on board the International Space Station (ISS). This software provides a way for payload users to operate their payloads from their home sites. It can be used by an individual or a team of people. TReK provides both local ground support system services and an interface to utilize remote services provided by the Payload Operations Integration Center (POIC). by the POIC and to perform local data functions such as processing the data, storing it in local files, and forwarding it to other computer systems. TReK can also be used to build, send, and track payload commands. In addition to these features, work is in progress to add a new command management capability. This capability will provide a way to manage a multi- platform command environment that can include geographically distributed computers. This is intended to help those teams that need to manage a shared on-board resource such as a facility class payload. The environment can be configured such that one individual can manage all the command activities associated with that payload. This paper will provide a summary of existing TReK capabilities and a description of the new command management capability. For example, 7'ReK can be used to receive payload data distributed

Schneider, Michelle↗
DOE OSTI2024

h5bench: A unified benchmark suite for evaluating HDF5 I/O performance on pre‐exascale platforms

Summary Parallel I/O is a critical technique for moving data between compute and storage subsystems of supercomputers. With massive amounts of data produced or consumed by compute nodes, high‐performant parallel I/O is essential. I/O benchmarks play an important role in this process; however, there is a scarcity of I/O benchmarks representative of current workloads on HPC systems. Toward creating representative I/O kernels from real‐world applications, we have created h5bench , a set of I/O kernels that exercise hierarchical data format version 5 (HDF5) I/O on parallel file systems in numerous dimensions. Our focus on HDF5 is due to the parallel I/O library's heavy usage in various scientific applications running on supercomputing systems. The various tests benchmarked in the h5bench suite include I/O operations (read and write), data locality (arrays of basic data types and arrays of structures), array dimensionality (one‐dimensional arrays, two‐dimensional meshes, three‐dimensional cubes), I/O modes (synchronous and asynchronous). In this paper, we present the observed performance of h5bench executed along several of these dimensions on existing supercomputers (Cori and Summit) and pre‐exascale platforms (Perlmutter, Theta, and Polaris). h5bench measurements can be used to identify performance bottlenecks and their root causes and evaluate I/O optimizations. As the I/O patterns of h5bench are diverse and capture the I/O behaviors of various HPC applications, this study will be helpful to the broader supercomputing and I/O community.

97 MATHEMATICS AND COMPUTING↗