Building Fast Concurrent Data Structures through Data Structure Families.
Abstract not provided.
SEARCH · Search NASA
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Abstract not provided.
This paper discusses the use of computer data structures in finite-element structural analysis programs. A number of data structure types that have been shown to be useful in such programs are introduced and described. A simple finite-element model is used to demonstrate how the given set of data structure types naturally lend themselves to developing software for the model. Different methods of implementing data structures in the context of a program are discussed.
Distributed data structures are key to implementing scalable applications for scientific simulations and data analysis. In this paper we look at two implementation styles for distributed data structures: remote direct memory access (RDMA) and remote procedure call (RPC). We focus on operations that require individual accesses to remote portions of a distributed data structure, e.g., accessing a hash table bucket or distributed queue, rather than global operations in which all processors collectively exchange information. We look at the trade-offs between the two styles through microbenchmarks and a performance model that approximates the cost of each. The RDMA operations have direct hardware support in the network and therefore lower latency and overhead, while the RPC operations are more expressive but higher cost and can suffer from lack of attentiveness from the remote side. We also run experiments to compare the real-world performance of RDMA- and RPC-based data structure operations with the predicted performance to evaluate the accuracy of our model, and show that while the model does not always precisely predict running time, it allows us to choose the best implementation in the examples shown. We believe this analysis will assist developers in designing data structures that will perform well on current network architectures, as well as network architects in providing better support for this class of distributed data structures.
The automated analysis of land surface features is increasingly important to earth scientists. User-friendly algorithms for studying these features can be integrated into geographic information systems through the use of topological grid structure, which maintains the simplicity and transportability of standard grid structure while providing the essential capability to treat groups of contiguous, identically-classified pixels (corresponding to lakes, forests, fields, etc.) as distinct spatial entities.
Analysis activities for Nonproliferation and Arms Control verification require the use of many types of data. Tabular structured data, such as Excel spreadsheets and relational databases, have traditionally been used for data mining activities, where specific queries are issued against data to look for matching results. The application of visual analytics tools to structured data enables further exploration of datasets to promote discovery of previously unknown results. This paper discusses the application of a specific visual analytics tool to datasets related to the field of Arms Control and Nonproliferation to promote the use of visual analytics more broadly in this domain. Visual analytics focuses on analytical reasoning facilitated by interactive visual interfaces (Wong and Thomas 2004). It promotes exploratory analysis of data, and complements data mining technologies where known patterns can be mined for. Also with a human in the loop, they can bring in domain knowledge and subject matter expertise. Visual analytics has not widely been applied to this domain. In this paper, we will focus on one type of data: structured data, and show the results of applying a specific visual analytics tool to answer questions in the Arms Control and Nonproliferation domain. We chose to use the T.Rex tool, a visual analytics tool developed at PNNL, which uses a variety of visual exploration patterns to discover relationships in structured datasets, including a facet view, graph view, matrix view, and timeline view. The facet view enables discovery of relationships between categorical information, such as countries and locations. The graph tool visualizes node-link relationship patterns, such as the flow of materials being shipped between parties. The matrix visualization shows highly correlated categories of information. The timeline view shows temporal patterns in data. In this paper, we will use T.Rex with two different datasets to demonstrate how interactive exploration of the data can aid an analyst with arms control and nonproliferation verification activities. Using a dataset from PIERS (PIERS 2014), we will show how container shipment imports and exports can aid an analyst in understanding the shipping patterns between two countries. We will also use T.Rex to examine a collection of research publications from the IAEA International Nuclear Information System (IAEA 2014) to discover collaborations of concern. We hope this paper will encourage the use of visual analytics structured data analytics in the field of nonproliferation and arms control verification. Our paper outlines some of the challenges that exist before broad adoption of these kinds of tools can occur and offers next steps to overcome these challenges.
A data structure based on an eXtensible Markup Language (XML) hierarchy according to experimental nuclear structure data in the Evaluated Nuclear Structure Data File (ENSDF) is presented. A Python-coded translator has been developed to interpret the standard one-card records of the ENSDF datasets, together with their associated quantities defined according to field position, and generate corresponding representative XML output. The quantities belonging to this mixed-record format are described in the ENSDF manual. Of the 16 ENSDF records in total, XML output has been successfully generated for 15 records. An XML-translation for the Comment Record is yet to be implemented; this will be considered in a separate phase of the overall translation effort. Continuation records, not yet implemented, will also be treated in a future phase of this work. Several examples are presented in this document to illustrate the XML schema and methods for handling the various ENSDF data types. However, the proposed nomenclature for the XML elements and attributes need not necessarily be considered as a fixed set of constructs. Indeed, better conventions may be suggested and a consensus can be achieved amongst the various groups of people interested in this project. The main purpose here is to present an initial phase of the translation effort to demonstrate the feasibility of interpreting ENSDF datasets and creating a representative XML-structured hierarchy for data storage.
Both principal component analysis (PCA) and principal factor analysis (PFA) were used to analyze an experimental multispectral data structure in terms of common and unique variance. Only the common variance of the multispectral data was associated with the principal factor, while higher-order principal components were associated with both common and unique variance. The unique variance was found to represent small spectral variations within each cover type as well as noise vectors, and was most abundant in the lower-order principal components. The lower-order principal components can be useful in research designed to discriminate minor physical variations within features, and to highlight localized change when using multitemporal-multispectral data. Conversely, PFA of the multispectral data provided an insight into a great potential for discriminating basic land-cover types by excluding the unique variance which was related to the noise and minor spectral variations.
Reverse Monte Carlo (RMC) refinements of local structure using a simultaneous fit of X-ray/neutron total scattering and extended X-ray absorption fine structure (EXAFS) data were developed to incorporate an explicit treatment of both single- and multiple-scattering contributions to EXAFS. The refinement algorithm, implemented as an extension to the public domain computer software RMCProfile, enables accurate modeling of EXAFS over distances encompassing several coordination shells around the absorbing species. The approach was first tested on Ni, which exhibits extensive multiple scattering in EXAFS, and then applied to perovskite-like SrAl{sub 1/2}Nb{sub 1/2}O{sub 3}. This compound crystallizes with a cubic double-perovskite structure but presents a challenge for local-structure determination using a total pair-distribution function (PDF) alone because of overlapping peaks of the constituent partial PDFs (e.g. Al-O and Nb-O or Sr-O and O-O). The results obtained here suggest that the combined use of the total scattering and EXAFS data provides sufficient constraints for RMC refinements to recover fine details of local structure in complex perovskites. Among other results, it was found that the probability density distribution for Sr in SrAl{sub 1/2}Nb{sub 1/2}O{sub 3} adopts Td point-group symmetry for the Sr sites, determined by the ordered arrangement of Al and Nb, as opposed to a spherical distribution commonly assumed in traditional Rietveld refinements.
Reverse Monte Carlo (RMC) refinements of local structure using a simultaneous fit of X-ray/neutron total scattering and extended X-ray absorption fine structure (EXAFS) data were developed to incorporate an explicit treatment of both single- and multiple-scattering contributions to EXAFS. The refinement algorithm, implemented as an extension to the public domain computer software RMCProfile, enables accurate modeling of EXAFS over distances encompassing several coordination shells around the absorbing species. The approach was first tested on Ni, which exhibits extensive multiple scattering in EXAFS, and then applied to perovskite-like SrAl{sub 1/2}Nb{sub 1/2}O{sub 3}. This compound crystal1izes with a cubic double-perovskite structure but presents a challenge for local-structure determination using a total pair-distribution function (PDF) alone because of overlapping peaks of the constituent partial PDFs (e.g. Al-O and Nb-O or Sr-O and O-O). The results obtained here suggest that the combined use of the total scattering and EXAFS data provides sufficient constraints for RMC refinements to recover fine details of local structure in complex perovskites. Among other results, it was found that the probability density distribution for Sr in SrAl{sub 1/2}Nb{sub 1/2}O{sub 3} adopts T{sub d} point-group symmetry for the Sr sites, determined by the ordered arrangement of Al and Nb, as opposed to a spherical distribution commonly assumed in traditional Rietveld refinements.
The data structures used to represent the multi-material state of a computational physics application can have a significant impact on its performance. Here, we look at efficient data structures for applications where there may be many materials overall, but where most computational cells contain only one or a few of these materials. We develop simple performance models for selecting possible data structures and programming patterns. We verify the analytic performance models with a small test program of the representative cases. We discuss the impact of these techniques and analysis for multi-material physics applications, which are also applicable to a wide range of sparse computational data structures.
Large-scale data analytics workflows ingest massive input data into various data structures, including graphs and key-value datastores. These data structures undergo multiple transformations and computations and are typically reused in incremental and iterative analytics workflows. Persisting in-memory views of these data structures enables reusing them beyond the scope of a single program run while avoiding repetitive raw data ingestion overheads. Memory-mapped I/O enables persisting in-memory data structures without data serialization and deserialization overheads. However, memory-mapped I/O lacks the key feature of persisting consistent snapshots of these data structures for incremental ingestion and processing. The obstacles to efficient virtual memory snapshots using memory-mapped I/O include background writebacks outside the application’s control, and the significantly high storage footprint of such snapshots. To address these limitations, we present Privateer, a memory and storage management tool that enables storage-efficient virtual memory snapshotting while also optimizing snapshot I/O performance. Here, we integrated Privateer into Metall, a state-of-the-art persistent memory allocator for C++, and the Lightning Memory-Mapped Database (LMDB), a widely-used key-value datastore in data analytics and machine learning. Privateer optimized application performance by 1.22× when storing data structure snapshots to node-local storage, and up to 16.7× when storing snapshots to a parallel file system. Privateer also optimizes storage efficiency of incremental data structure snapshots by up to 11× using data deduplication and compression.
Topological data structures are useful in many areas, including the various mesh data structures used in finite element and finite volume applications. Here, we present a mesh data structure which is both flexible enough to support general mesh adaptivity and is compactly stored in a few large arrays. This structure can efficiently store full topological information for meshes with mixed element types. It allows both constant-time adjacency queries and constant-time addition and removal of mesh entities. We compare this structure to similar structures for finite elements and graphics and present its performance for adjacency queries. We also present its use in adaptive simulation applications.
Thematic Mapper data, simulated from field and laboratory spectrometer measurements of a variety of agricultural crops and a wide range of soils, are analyzed to determine their dispersion in the six space defined by the reflective TM bands (i.e., excluding the thermal band). While similar analyses of MSS data from agricultural scenes show that the vast majority of the MSS data occupy a single plane, the simulated TM data primarily occupy three dimensions, defining two intersecting planes and a zone of transition between the two. Viewing the plane of Vegetation head on provides a projection comparable to the single plane of MSS data. The Plane of Soils and transition zone represent new information made available largely as a result of the longer infrared bands included in the Thematic Mapper. A transformation, named the Thematic Mapper Tasseled Cap, is presented which rotates the TM data such that the described data structure is most readily accessible to view.
A possible Data Relay Satellite System (DRSS) topology and network architecture is introduced. An asynchronous network concept, whereby each link (Inter-orbit, Inter-satellite, Feeder) is allowed to operate on its own clock, without causing loss of information, in conjunction with packet data structures, such as those specified by the CCSDS for advanced orbiting systems is discussed. A matching OBP payload architecture is described, highlighting the advantages provided by the OBP-based concept and then giving some indications on the OBP mass/power requirements.
Shapefiles and spreadsheets of structural data, including attitudes of faults and strata and slip orientations of faults. - Detailed geologic mapping of ~30 km2 was completed in the vicinity of the Columbus Marsh geothermal field to obtain critical structural data that would elucidate the structural controls of this field. - Documenting E‐ to ENE‐striking left lateral faults and N‐ to NNE‐striking normal faults. - Some faults cut Quaternary basalts. - This field appears to occupy a displacement transfer zone near the eastern end of a system of left‐lateral faults. ENE‐striking sinistral faults diffuse into a system of N‐ to NNE‐striking normal faults within the displacement transfer zone. - Columbus Marsh therefore corresponds to an area of enhanced extension and contains a nexus of fault intersections, both conducive for geothermal activity.
The data structures used to represent the multi-material state of a computational physics application can have a drastic impact on the performance of the application. We look at efficient data structures for sparse applications where there may be many materials, but only one or few in most computational cells. We develop simple performance models for use in selecting possible data structures and programming patterns. We verify the analytic models of performance through a small test program of the representative cases.
The unprecedented amount of data that needs to be processed in emerging data analytics applications poses novel challenges to industry and academia. Scalability and high performance become more than a desirable feature because, due to the scale and the nature of the problems, they draw the line between what is achievable and what is unfeasible. In this paper, we propose SHAD, the Scalable High-performance Algorithms and Data-structures library. SHAD adopts a modular design that confines low level details and promotes reuse. SHAD's core is built on an Abstract Runtime Interface which enhances portability and identifies the minimal set of features of the underlying system required by the framework. The core library includes common data-structures such as: Array, Vector, Map and Set. These are designed to accommodate significant amount of data which can be accessed in massively parallel environments, and used as building blocks for SHAD extensions, i.e. higher level software libraries. We have validated and evaluated our design with a performance and scalability study of the core components of the library. We have validated the design flexibility by proposing a Graph Library as an example of SHAD extension, which implements two different graph data-structures; we evaluate their performance with a set of graph applications. Experimental results show that the approach is promising in terms of both performance and scalability. On a distributed system with 320 cores, SHAD Arrays are able to sustain a throughput of 65 billion operations per second, while SHAD Maps sustain 1 billion of operations per second. Algorithms implemented using the Graph Library exhibit performance and scalability comparable to a custom solution, but with smaller development effort.
A new mesh data structure is introduced for the purpose of mesh processing in Application Programming Interface (API) infrastructures. This data structure utilizes a reduced mesh representation to increase its ability to handle significantly larger meshes compared to full mesh representation. In spite of the reduced representation, each mesh entity (vertex, edge, face, and region) is represented using a unique handle, with no extra storage cost, which is a crucial requirement in most API libraries. The concept of mesh layers makes the data structure more flexible for mesh generation and mesh modification operations. This flexibility can have a favorable impact in solver based queries of finite volume and multigrid methods. The capabilities of LBMD make it even more attractive for parallel implementations using Message Passing Interface (MPI) or Graphics Processing Units (GPUs). The data structure is associated with a new classification method to relate mesh entities to their corresponding geometrical entities. The classification technique stores the related information at the node level without introducing any ambiguities. Several examples are presented to illustrate the strength of this new data structure.