Search NASA⌕ Search

SEARCH · Search NASA

Results for “structured data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

RDMA vs. RPC for Implementing Distributed Data Structures

Distributed data structures are key to implementing scalable applications for scientific simulations and data analysis. In this paper we look at two implementation styles for distributed data structures: remote direct memory access (RDMA) and remote procedure call (RPC). We focus on operations that require individual accesses to remote portions of a distributed data structure, e.g., accessing a hash table bucket or distributed queue, rather than global operations in which all processors collectively exchange information. We look at the trade-offs between the two styles through microbenchmarks and a performance model that approximates the cost of each. The RDMA operations have direct hardware support in the network and therefore lower latency and overhead, while the RPC operations are more expressive but higher cost and can suffer from lack of attentiveness from the remote side. We also run experiments to compare the real-world performance of RDMA- and RPC-based data structure operations with the predicted performance to evaluate the accuracy of our model, and show that while the model does not always precisely predict running time, it allows us to choose the best implementation in the examples shown. We believe this analysis will assist developers in designing data structures that will perform well on current network architectures, as well as network architects in providing better support for this class of distributed data structures.

distributed data structures↗

A Visual Analytics Approach to Structured Data Analysis to Enhance Nonproliferation and Arms Control Verification Activities

Analysis activities for Nonproliferation and Arms Control verification require the use of many types of data. Tabular structured data, such as Excel spreadsheets and relational databases, have traditionally been used for data mining activities, where specific queries are issued against data to look for matching results. The application of visual analytics tools to structured data enables further exploration of datasets to promote discovery of previously unknown results. This paper discusses the application of a specific visual analytics tool to datasets related to the field of Arms Control and Nonproliferation to promote the use of visual analytics more broadly in this domain. Visual analytics focuses on analytical reasoning facilitated by interactive visual interfaces (Wong and Thomas 2004). It promotes exploratory analysis of data, and complements data mining technologies where known patterns can be mined for. Also with a human in the loop, they can bring in domain knowledge and subject matter expertise. Visual analytics has not widely been applied to this domain. In this paper, we will focus on one type of data: structured data, and show the results of applying a specific visual analytics tool to answer questions in the Arms Control and Nonproliferation domain. We chose to use the T.Rex tool, a visual analytics tool developed at PNNL, which uses a variety of visual exploration patterns to discover relationships in structured datasets, including a facet view, graph view, matrix view, and timeline view. The facet view enables discovery of relationships between categorical information, such as countries and locations. The graph tool visualizes node-link relationship patterns, such as the flow of materials being shipped between parties. The matrix visualization shows highly correlated categories of information. The timeline view shows temporal patterns in data. In this paper, we will use T.Rex with two different datasets to demonstrate how interactive exploration of the data can aid an analyst with arms control and nonproliferation verification activities. Using a dataset from PIERS (PIERS 2014), we will show how container shipment imports and exports can aid an analyst in understanding the shipping patterns between two countries. We will also use T.Rex to examine a collection of research publications from the IAEA International Nuclear Information System (IAEA 2014) to discover collaborations of concern. We hope this paper will encourage the use of visual analytics structured data analytics in the field of nonproliferation and arms control verification. Our paper outlines some of the challenges that exist before broad adoption of these kinds of tools can occur and offers next steps to overcome these challenges.

arms control↗

An XML-hierarchical data structure for ENSDF

A data structure based on an eXtensible Markup Language (XML) hierarchy according to experimental nuclear structure data in the Evaluated Nuclear Structure Data File (ENSDF) is presented. A Python-coded translator has been developed to interpret the standard one-card records of the ENSDF datasets, together with their associated quantities defined according to field position, and generate corresponding representative XML output. The quantities belonging to this mixed-record format are described in the ENSDF manual. Of the 16 ENSDF records in total, XML output has been successfully generated for 15 records. An XML-translation for the Comment Record is yet to be implemented; this will be considered in a separate phase of the overall translation effort. Continuation records, not yet implemented, will also be treated in a future phase of this work. Several examples are presented in this document to illustrate the XML schema and methods for handling the various ENSDF data types. However, the proposed nomenclature for the XML elements and attributes need not necessarily be considered as a fixed set of constructs. Indeed, better conventions may be suggested and a consensus can be achieved amongst the various groups of people interested in this project. The main purpose here is to present an initial phase of the translation effort to demonstrate the feasibility of interpreting ENSDF datasets and creating a representative XML-structured hierarchy for data storage.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A Combined Fit of Total Scattering and Extended X-ray Absorption Fine Structure Data for Local-Structure Determination in Crystalline Materials

Reverse Monte Carlo (RMC) refinements of local structure using a simultaneous fit of X-ray/neutron total scattering and extended X-ray absorption fine structure (EXAFS) data were developed to incorporate an explicit treatment of both single- and multiple-scattering contributions to EXAFS. The refinement algorithm, implemented as an extension to the public domain computer software RMCProfile, enables accurate modeling of EXAFS over distances encompassing several coordination shells around the absorbing species. The approach was first tested on Ni, which exhibits extensive multiple scattering in EXAFS, and then applied to perovskite-like SrAl{sub 1/2}Nb{sub 1/2}O{sub 3}. This compound crystallizes with a cubic double-perovskite structure but presents a challenge for local-structure determination using a total pair-distribution function (PDF) alone because of overlapping peaks of the constituent partial PDFs (e.g. Al-O and Nb-O or Sr-O and O-O). The results obtained here suggest that the combined use of the total scattering and EXAFS data provides sufficient constraints for RMC refinements to recover fine details of local structure in complex perovskites. Among other results, it was found that the probability density distribution for Sr in SrAl{sub 1/2}Nb{sub 1/2}O{sub 3} adopts Td point-group symmetry for the Sr sites, determined by the ordered arrangement of Al and Nb, as opposed to a spherical distribution commonly assumed in traditional Rietveld refinements.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

A combined fit of total scattering and extended x-ray absorption fine structure data for local-structure determination in crystalline materials

Reverse Monte Carlo (RMC) refinements of local structure using a simultaneous fit of X-ray/neutron total scattering and extended X-ray absorption fine structure (EXAFS) data were developed to incorporate an explicit treatment of both single- and multiple-scattering contributions to EXAFS. The refinement algorithm, implemented as an extension to the public domain computer software RMCProfile, enables accurate modeling of EXAFS over distances encompassing several coordination shells around the absorbing species. The approach was first tested on Ni, which exhibits extensive multiple scattering in EXAFS, and then applied to perovskite-like SrAl{sub 1/2}Nb{sub 1/2}O{sub 3}. This compound crystal1izes with a cubic double-perovskite structure but presents a challenge for local-structure determination using a total pair-distribution function (PDF) alone because of overlapping peaks of the constituent partial PDFs (e.g. Al-O and Nb-O or Sr-O and O-O). The results obtained here suggest that the combined use of the total scattering and EXAFS data provides sufficient constraints for RMC refinements to recover fine details of local structure in complex perovskites. Among other results, it was found that the probability density distribution for Sr in SrAl{sub 1/2}Nb{sub 1/2}O{sub 3} adopts T{sub d} point-group symmetry for the Sr sites, determined by the ordered arrangement of Al and Nb, as opposed to a spherical distribution commonly assumed in traditional Rietveld refinements.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A comparative study of multi-material data structures for computational physics applications

The data structures used to represent the multi-material state of a computational physics application can have a significant impact on its performance. Here, we look at efficient data structures for applications where there may be many materials overall, but where most computational cells contain only one or a few of these materials. We develop simple performance models for selecting possible data structures and programming patterns. We verify the analytic performance models with a small test program of the representative cases. We discuss the impact of these techniques and analysis for multi-material physics applications, which are also applicable to a wide range of sparse computational data structures.

97 MATHEMATICS AND COMPUTING↗

Optimizing Management of Persistent Data Structures in High-Performance Analytics

Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.

Computer science↗

Modifiable Array Data Structures for Mesh Topology

Topological data structures are useful in many areas, including the various mesh data structures used in finite element and finite volume applications. Here, we present a mesh data structure which is both flexible enough to support general mesh adaptivity and is compactly stored in a few large arrays. This structure can efficiently store full topological information for meshes with mixed element types. It allows both constant-time adjacency queries and constant-time addition and removal of mesh entities. We compare this structure to similar structures for finite elements and graphics and present its performance for adjacency queries. We also present its use in adaptive simulation applications.

97 MATHEMATICS AND COMPUTING↗

Structural Data for the Columbus Salt Marsh Geothermal Area - GIS Data

Shapefiles and spreadsheets of structural data, including attitudes of faults and strata and slip orientations of faults. - Detailed geologic mapping of ~30 km2 was completed in the vicinity of the Columbus Marsh geothermal field to obtain critical structural data that would elucidate the structural controls of this field. - Documenting E‐ to ENE‐striking left lateral faults and N‐ to NNE‐striking normal faults. - Some faults cut Quaternary basalts. - This field appears to occupy a displacement transfer zone near the eastern end of a system of left‐lateral faults. ENE‐striking sinistral faults diffuse into a system of N‐ to NNE‐striking normal faults within the displacement transfer zone. - Columbus Marsh therefore corresponds to an area of enhanced extension and contains a nexus of fault intersections, both conducive for geothermal activity.

15 GEOTHERMAL ENERGY↗

A Comparative Study of Multi-material Data Structures for Computational Physics Applications

The data structures used to represent the multi-material state of a computational physics application can have a drastic impact on the performance of the application. We look at efficient data structures for sparse applications where there may be many materials, but only one or few in most computational cells. We develop simple performance models for use in selecting possible data structures and programming patterns. We verify the analytic models of performance through a small test program of the representative cases.

97 MATHEMATICS AND COMPUTING↗

SHAD: the Scalable High-performance Algorithms and Data-structures Library

The unprecedented amount of data that needs to be processed in emerging data analytics applications poses novel challenges to industry and academia. Scalability and high performance become more than a desirable feature because, due to the scale and the nature of the problems, they draw the line between what is achievable and what is unfeasible. In this paper, we propose SHAD, the Scalable High-performance Algorithms and Data-structures library. SHAD adopts a modular design that confines low level details and promotes reuse. SHAD's core is built on an Abstract Runtime Interface which enhances portability and identifies the minimal set of features of the underlying system required by the framework. The core library includes common data-structures such as: Array, Vector, Map and Set. These are designed to accommodate significant amount of data which can be accessed in massively parallel environments, and used as building blocks for SHAD extensions, i.e. higher level software libraries. We have validated and evaluated our design with a performance and scalability study of the core components of the library. We have validated the design flexibility by proposing a Graph Library as an example of SHAD extension, which implements two different graph data-structures; we evaluate their performance with a set of graph applications. Experimental results show that the approach is promising in terms of both performance and scalability. On a distributed system with 320 cores, SHAD Arrays are able to sustain a throughput of 65 billion operations per second, while SHAD Maps sustain 1 billion of operations per second. Algorithms implemented using the Graph Library exhibit performance and scalability comparable to a custom solution, but with smaller development effort.

Castellana, Vito G.↗

LBMD : a layer-based mesh data structure tailored for generic API infrastructures.

A new mesh data structure is introduced for the purpose of mesh processing in Application Programming Interface (API) infrastructures. This data structure utilizes a reduced mesh representation to increase its ability to handle significantly larger meshes compared to full mesh representation. In spite of the reduced representation, each mesh entity (vertex, edge, face, and region) is represented using a unique handle, with no extra storage cost, which is a crucial requirement in most API libraries. The concept of mesh layers makes the data structure more flexible for mesh generation and mesh modification operations. This flexibility can have a favorable impact in solver based queries of finite volume and multigrid methods. The capabilities of LBMD make it even more attractive for parallel implementations using Message Passing Interface (MPI) or Graphics Processing Units (GPUs). The data structure is associated with a new classification method to relate mesh entities to their corresponding geometrical entities. The classification technique stores the related information at the node level without introducing any ambiguities. Several examples are presented to illustrate the strength of this new data structure.

99 GENERAL AND MISCELLANEOUS↗

Device and method for constructing and displaying high quality images from imaging data by transforming a data structure utilizing machine learning techniques

Constructing a computer image from raw imaging data or encoded imaging data by transforming a first data structure in which the raw imaging data or the encoded imaging data is stored into a second data structure storing reorganized imaging data. The raw imaging data or the encoded imaging data is received, stored in the first data structure. The computer reorganizes the raw imaging data or the encoded imaging data into the reorganized data and stores the reorganized data in the second data structure, which is a multi-dimensional array having subarrays containing local information needed by a convolutional neural network for processing the reorganized data. Other portions of the multi-dimensional array store other portions of the raw imaging data or the encoded imaging data. The computer also processes the reorganized data using the convolutional neural network to construct the image, whereby a constructed image is formed.

Korbin, John P.↗

Advanced Data Structures for Improved Cyber Resilience and Awareness in Untrusted Environments: LDRD Report

This report summarizes the work performed under the project "Advanced Data Structures for Improved Cyber Resilience and Awareness in Untrusted Environments." The goal of the project was to design, analyze, and test new data structures for cybersecurity applications. We had two major thrusts: 1) using new/improved write-optimized data structures and/or algorithms to better man- age and analyze high-speed massive-volume cyberstreams, and 2) adding security features to data structures at minimum cost. Write optimization allows data structures to better use secondary memory to store and search a larger amount of useful information. Secondary memory is large compared to main memory, but data movement to and from secondary memory must be carefully managed to run quickly enough to keep up with fast streams. The first thrust included managing cyberstreams in parallel, both multi-threaded and distributed, and improving the benchmarking infrastructure for testing new streaming data structures. We considered both (near) real-time discovery of particular patterns, and improved logging for improved forensics. We considered two kinds of security-feature problem. The first was high-performance history- independent external-memory data structures. These provide certain protections to data if a disk is stolen. We also prove some trade-offs between speed and security in this setting. The second data-security problem is more secure data look-up in secret-shared data bases. This report summarizes the project's major accomplishments, with the background to under- stand these accomplishments. It gathers the abstracts and references for the six refereed publications that have appeared as part of this work. We summarize several accomplishments that will be submitted for publication. We then archive one piece of partial work that is not likely to be published in the near future: validation of history-independent data structure implementations.

97 MATHEMATICS AND COMPUTING↗

Understanding Data Structures by Extracting Memory Access Graphs

Understanding the data structures employed by a program is important for reverse engineering activities and can improve the results of automated software analysis techniques. In a compiled binary, access to data structure fields and array indices defined in the source program are replaced by raw pointer arithmetic. We present a representation for capturing the essential details of how a program accesses memory regions, which we call a Memory Access Graph (MAG), and a static analysis for automatically extracting this information from a program binary. The static analysis to extract the MAGs from the program is straightforward and does not require sophisticated integer or pointer analysis. The MAGs are readily understood by reverse engineers; they are generally able to perceive the data structure definition corresponding to a MAG. We briefly discuss automatic extraction of structure definitions outlining some of the difficulties in doing so.

97 MATHEMATICS AND COMPUTING↗

AHF: Array-Based Half-Facet Data Structure for Mixed-Dimensional and Non-Manifold Meshes

We present an Array-based Half-Facet mesh data structure, or AHF, for efficient mesh query and modification operations. The AHF extends the compact array-based half-edge and half-face data structures (T.J. Alumbaugh and X. Jiao, Compact array-based mesh data structures, IMR, 2005) to support mixed-dimensional and non-manifold meshes. The design goals of our data structure include generality to support such meshes, efficiency of neighborhood queries and mesh modification, compactness of memory footprint, and facilitation of interoperability of mesh-based application codes. To accomplish these goals, our data structure uses sibling half-facets as a core abstraction, coupled with other explicit and implicit representations of entities. A unique feature of our data structure is a comprehensive implementation in MATLAB, which allows rapid prototyping, debugging, testing, and deployment of meshing algorithms and other mesh-based numerical methods. We have also developed C++ implementation built on top of MOAB (T.J. Tautges, R. Meyers, and K. Merkley, MOAB: A Mesh-Oriented Database, Sandia National Laboratories, 2004). We present some comparisons of the memory requirements and computational costs, and also demonstrate its effectiveness with a few sample applications.

Dyedov, Volodymyr↗