SEARCH · Search NASA

Results for “parallel data structures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

All sources NASA NTRS DOE OSTI

DOE OSTI2018

Advanced Data Structures for Improved Cyber Resilience and Awareness in Untrusted Environments: LDRD Report

This report summarizes the work performed under the project "Advanced Data Structures for Improved Cyber Resilience and Awareness in Untrusted Environments." The goal of the project was to design, analyze, and test new data structures for cybersecurity applications. We had two major thrusts: 1) using new/improved write-optimized data structures and/or algorithms to better man- age and analyze high-speed massive-volume cyberstreams, and 2) adding security features to data structures at minimum cost. Write optimization allows data structures to better use secondary memory to store and search a larger amount of useful information. Secondary memory is large compared to main memory, but data movement to and from secondary memory must be carefully managed to run quickly enough to keep up with fast streams. The first thrust included managing cyberstreams in parallel, both multi-threaded and distributed, and improving the benchmarking infrastructure for testing new streaming data structures. We considered both (near) real-time discovery of particular patterns, and improved logging for improved forensics. We considered two kinds of security-feature problem. The first was high-performance history- independent external-memory data structures. These provide certain protections to data if a disk is stolen. We also prove some trade-offs between speed and security in this setting. The second data-security problem is more secure data look-up in secret-shared data bases. This report summarizes the project's major accomplishments, with the background to under- stand these accomplishments. It gathers the abstracts and references for the six refereed publications that have appeared as part of this work. We summarize several accomplishments that will be submitted for publication. We then archive one piece of partial work that is not likely to be published in the near future: validation of history-independent data structure implementations.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2019

Parallel PIPS-SBB: multi-level parallelism for stochastic mixed-integer programs

PIPS-SBB is a distributed-memory parallel solver with a scalable data distribution paradigm. It is designed to solve MIPs with a dual-block angular structure, which is characteristic of deterministic-equivalent Stochastic Mixed-Integer Programs (SMIPs). In this paper, we present two different parallelizations of Branch & Bound (B&B), implementing both as extensions of PIPS-SBB, thus adding an additional layer of parallelism. In the first of the proposed frameworks, PIPS-PSBB, the coordination and load-balancing of the different optimization workers is done in a decentralized fashion. This new framework is designed to ensure all available cores are processing the most promising parts of the B&B tree. The second, ug[PIPS-SBB,MPI], is a parallel implementation using the Ubiquity Generator (UG), a universal framework for parallelizing B&B tree search that has been sucessfully applied to other MIP solvers. We show the effects of leveraging multiple levels of parallelism in potentially improving scaling performance beyond thousands of cores.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Phoenix: A Scalable Streaming Hypergraph Analysis Framework

We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.

Kurte, Kuldeep↗

DOE OSTI2020

Scalable line and plane relaxation in a parallel structured multigrid solver

The efficient solution of sparse, linear systems that arise through the discretization of partial differential equations remains a key challenge for a range of high performance scientific simulations. One approach for reducing data movement and improving performance is by exposing and exploiting structure in a problem through the use of robust structured multilevel solvers. By choosing coarsening that preserves the structure of the problem, these methods maintain efficient structured computation and communication throughout the multigrid hierarchy. However, when coarsening is not permitted to be dependent on the operator, anisotropy must be addressed by the smoother — producing error compatible for coarse-grid correction with structured coarsening. Here, the components required in a scalable parallel structured solver are described with a focus on memory and communication efficiency of robust smoothers. While the implementation of communication and memory reduction techniques in smoothers integrated in a complete 3D solver present a significant engineering challenge, a novel approach is proposed that addresses these challenges systematically through a change to the solver’s execution model. Enabled by user-level threading paired with a set of data and communication abstractions, this approach permits seamless aggregation of communication in plane smoothers — directly reusing code for a 2D distributed multilevel cycle. Results show an effective reduction in communication costs for coarse-grid problems, and result in a speedup of 8.7x in smoothing routines shown in Fig. 12 using this approach. This produces a significant improvement to strong scalability while maintaining favorable weak scaling behavior. Finally, a parallel scaling study using a series of refined meshes is included that demonstrates the effectiveness of this approach in an application of interest.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

Fast correlation function calculator: A high-performance pair-counting toolkit

A novel high-performance exact pair-counting toolkit called fast correlation function calculator (FCFC) is presented. With the rapid growth of modern cosmological datasets, the evaluation of correlation functions with observational and simulation catalogues has become a challenge. High-efficiency pair-counting codes are thus in great demand. We introduce different data structures and algorithms that can be used for pair-counting problems, and perform comprehensive benchmarks to identify the most efficient algorithms for real-world cosmological applications. We then describe the three levels of parallelisms used by FCFC, SIMD, OpenMP, and MPI, and run extensive tests to investigate the scalabilities. Finally, we compare the efficiency of FCFC with alternative pair-counting codes. The data structures and histogram update algorithms implemented in FCFC are shown to outperform alternative methods. FCFC does not benefit greatly from SIMD because the bottleneck of our histogram update algorithm is mainly cache latency. Nevertheless, the efficiency of FCFC scales well with the numbers of OpenMP threads and MPI processes, even though speedups may be degraded with over a few thousand threads in total. FCFC is found to be faster than most (if not all) other public pair-counting codes for modern cosmological pair-counting applications.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2020

The hTERT core promoter forms three parallel G-quadruplexes

The structure of the 68 nt sequence with G-quadruplex forming potential within the hTERT promoter is disputed. One model features a structure with three stacked parallel G-quadruplex units, while another features an unusual duplex hairpin structure adjoined to two stacked parallel and antiparallel quadruplexes. We report here the results of an integrated structural biology study designed to distinguish between these possibilities. As part of our study, we designed a sequence with an optimized hairpin structure and show that its biophysical and biochemical properties are inconsistent with the structure formed by the hTERT wild-type sequence. By using circular dichroism, thermal denaturation, nuclear magnetic resonance spectroscopy, analytical ultracentrifugation, small-angle X-ray scattering, molecular dynamics simulations and a DNase I cleavage assay we found that the wild type hTERT core promoter folds into a stacked, three-parallel G-quadruplex structure. The hairpin structure is inconsistent with all of our experimental data obtained with the wild-type sequence. All-atom models for both structures were constructed using molecular dynamics simulations. These models accurately predicted the experimental hydrodynamic properties measured for each structure. We found with certainty that the wild-type hTERT promoter sequence does not form a hairpin structure in solution, but rather folds into a compact stacked three-G-quadruplex conformation.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2017

2017 publication guidelines for structural modelling of small-angle scattering data from biomolecules in solution: an update

In 2012, preliminary guidelines were published addressing sample quality, data acquisition and reduction, presentation of scattering data and validation, and modelling for biomolecular small-angle scattering (SAS) experiments. Biomolecular SAS has since continued to grow and authors have increasingly adopted the preliminary guidelines. In parallel, integrative/hybrid determination of biomolecular structures is a rapidly growing field that is expanding the scope of structural biology. For SAS to contribute maximally to this field, it is essential to ensure open access to the information required for evaluation of the quality of SAS samples and data, as well as the validity of SAS-based structural models. To this end, the preliminary guidelines for data presentation in a publication are reviewed and updated, and the deposition of data and associated models in a public archive is recommended. These guidelines and recommendations have been prepared in consultation with the members of the International Union of Crystallography (IUCr) Small-Angle Scattering and Journals Commissions, the Worldwide Protein Data Bank (wwPDB) Small-Angle Scattering Validation Task Force and additional experts in the field.

Biochemistry & Molecular Biology↗

DOE OSTI2022

Galaxy Alignments with Surrounding Structure in the Sloan Digital Sky Survey

Using data from the Sloan Digital Sky Survey Legacy Survey, we study the alignment of luminous galaxies with spectroscopic data with the surrounding larger-scale structure as defined by galaxies with only photometric data. We find that galaxies from the red sequence have a statistically significant tendency for their apparent long axes to align parallel to the projected surrounding structure. Red galaxies more luminous than the median of our sample (M r < –21.78) have a mean alignment angle $\langle$Φ$\rangle$ < 45°, indicating preferred parallel alignment, at a significance level >4.5σ on projected scales 0.1 Mpc < r p ≤ 7.5 Mpc. Fainter red galaxies have $\langle$Φ$\rangle$ < 45° at a significance level >4.3σ at scales 1 Mpc < r p < 3 Mpc. At a projected scale r p = 3.0 Mpc, the mean alignment angle decreases steadily with increasing luminosity for red galaxies with M r ≲ –22.5, reaching $\langle$Φ$\rangle$ = 40$^{°}_{.}$49 ± 0$^{°}_{.}$56 for the most luminous 1% (M r ~ –23.57). Galaxies from the blue sequence show no statistically significant tendency for their axes to align with larger-scale structure, regardless of galaxy luminosity. Galaxies in higher-density regions do not show a statistically significant difference in the mean alignment angle from galaxies in lower-density regions; this holds true for the faint blue, luminous blue, faint red, and luminous red subsets.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2020

Protein folding from heterogeneous unfolded state revealed by time-resolved X-ray solution scattering

One of the most challenging tasks in biological science is to understand how a protein folds. In theoretical studies, the hypothesis adopting a funnel-like free-energy landscape has been recognized as a prominent scheme for explaining protein folding in views of both internal energy and conformational heterogeneity of a protein. Despite numerous experimental efforts, however, comprehensively studying protein folding with respect to its global conformational changes in conjunction with the heterogeneity has been elusive. Here we investigate the redox-coupled folding dynamics of equine heart cytochrome c (cyt-c) induced by external electron injection by using time-resolved X-ray solution scattering. A systematic kinetic analysis unveils a kinetic model for its folding with a stretched exponential behavior during the transition toward the folded state. With the aid of the ensemble optimization method combined with molecular dynamics simulations, we found that during the folding the heterogeneously populated ensemble of the unfolded state is converted to a narrowly populated ensemble of folded conformations. These observations obtained from the kinetic and the structural analyses of X-ray scattering data reveal that the folding dynamics of cyt-c accompanies many parallel pathways associated with the heterogeneously populated ensemble of unfolded conformations, resulting in the stretched exponential kinetics at room temperature. This finding provides direct evidence with a view to microscopic protein conformations that the cyt-c folding initiates from a highly heterogeneous unfolded state, passes through still diverse intermediate structures, and reaches structural homogeneity by arriving at the folded state.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2020

Sage: parallel semi-asymmtric graph algorithms for NVRAMs

Non-volatile main memory (NVRAM) technologies provide an attractive set of features for large-scale graph analytics, including byte-addressability, low idle power, and improved memory-density. NVRAM systems today have an order of magnitude more NVRAM than traditional memory (DRAM). NVRAM systems could therefore potentially allow very large graph problems to be solved on a single machine, at a modest cost. However, a significant challenge in achieving high performance is in accounting for the fact that NVRAM writes can be much more expensive than NVRAM reads. In this paper, we propose an approach to parallel graph analytics using the Parallel Semi-Asymmetric Model (PSAM), in which the graph is stored as a read-only data structure (in NVRAM), and the amount of mutable memory is kept proportional to the number of vertices. Similar to the popular semi-external and semi-streaming models for graph analytics, the PSAM approach assumes that the vertices of the graph fit in a fast read-write memory (DRAM), but the edges do not. In NVRAM systems, our approach eliminates writes to the NVRAM, among other benefits. To experimentally study this new setting, we develop Sage, a parallel semi-asymmetric graph engine with which we implement provably-efficient (and often work-optimal) PSAM algorithms for over a dozen fundamental graph problems. We experimentally study Sage using a 48--core machine on the largest publicly-available real-world graph (the Hyperlink Web graph with over 3.5 billion vertices and 128 billion edges) equipped with Optane DC Persistent Memory, and show that Sage outperforms the fastest prior systems designed for NVRAM. Importantly, we also show that Sage nearly matches the fastest prior systems running solely in DRAM, by effectively hiding the costs of repeatedly accessing NVRAM versus DRAM.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2020

A view into FleCSI [Slides]

FleCSI is largely based around sharing of memory spaces. FleCSI facilitates parallel execution and sharing of memory across memory spaces. Execute() transforms sender-side data structures into receiver-side data structures.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

SBIR Phase I Final Report, TACO: Distributed and Heterogeneous Sparse Compiler

Tensor algebra is a powerful tool for computing, but writing optimized codes that operate on sparse tensors can be very complex. This project enables a Tensor Algebra Compiler (TACO) that simplifies this task from man-years to man-days and extends TACO to support complex and large distributed systems. This report details the hypotheses, approaches used, and findings in this project.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2023

BOXKIT

SF-23-067 BoxKit is a library that provides building blocks to parallelize and scale data science, high performance computing, and machine learning applications for block-structured datasets. Spatial data from simulations and experiments can be accessed and managed using tools available in this library when working with more data analysis oriented packages like SciKit (https://github.com/scikit-learn/scikit-learn) and FlowNet (https://github.com/NVIDIA/flownet2-pytorch)

DHRUV, AKASH↗

DOE OSTI2018

System and method of storing and analyzing information

A system and method of storing and analyzing information is disclosed. The system includes a compiler layer to convert user queries to data parallel executable code. The system further includes a library of multithreaded algorithms, processes, and data structures. The system also includes a multithreaded runtime library for implementing compiled code at runtime. The executable code is dynamically loaded on computing elements and contains calls to the library of multithreaded algorithms, processes, and data structures and the multithreaded runtime library.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

System and method of storing and analyzing information

Feo, John T.↗

DOE OSTI2020

Reduction of blob-filament radial propagation by parallel variation of flows: Analysis of a gyrokinetic simulation

Data from the XGC1 gyrokinetic simulation is analyzed to understand the three-dimensional spatial structure and the radial propagation of blob-filaments generated by quasisteady turbulence in the tokamak edge pedestal and scrape-off layer plasma. Spontaneous toroidal flows vary in the poloidal direction and shear the filaments within a flux surface resulting in a structure that varies in the parallel direction. Here, this parallel structure allows the curvature and grad-B induced polarization charge density to be shorted out via parallel electron motion. As a result, it is found that the blob-filament radial velocity is significantly reduced from estimates which neglect parallel electron kinetics, broadly consistent with experimental observations. Conditions for when this charge shorting effect tends to dominate blob dynamics are derived and compared with the simulation.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2021

cuTS: Scaling Subgraph Isomorphism on Distributed Multi-GPUSystems Using Trie Based Data Structure

Subgraph isomorphism is a pattern-matching algorithm widely used in many domains such as chem-informatics, bioinformatics, databases, and social network analysis. It is computationally expensive and is a proven NP-hard problem. The massive parallelism offered by the GPU hardware is well suited for solving the subgraph isomorphism. However, current GPU implementations are far from the achievable performance. Moreover, the enormous memory requirement of current approaches limits the problem size that can be handled. This work analyzes the fundamental challenges associated with processing the subgraph isomorphism on GPUs and develops an efficient GPU hardware-aware implementation. We also develop a new GPU-friendly trie-based data structure to drastically reduce the intermediate storage space requirement. Hence, our approach runs larger benchmarks than the competitors. We also develop the first distributed sub-graph isomorphism algorithm for GPUs. Our experimental evaluation section demonstrates the efficacy of our approach by comparing the execution time and number of cases that we can handle against the state-of-the-art GPU implementations.

Xiang, Lizhi↗

DOE OSTI2018

Scaling Structured Multigrid to 500K+ Cores Through Coarse-Grid Redistribution

The effcient solution of sparse, linear systems resulting from the discretization of partial differential equations is crucial to the performance of many physics-based simulations. The algorithmic optimality of multilevel approaches for common discretizations makes them a good candidate for an efficient parallel solver. Yet, modern architectures for high-performance computing systems continue to challenge the parallel scalability of multilevel solvers. While algebraic multigrid methods are robust for solving a variety of problems, the increasing importance of data locality and cost of data movement in modern architectures motivates the need to carefully exploit structure in the problem. Robust logically structured variational multigrid methods, such as Black Box Multigrid (BoxMG), maintain structure throughout the multigrid hierarchy. This avoids indirection and increased coarse- grid communication costs typical in parallel algebraic multigrid. Nevertheless, the parallel scalability of structured multigrid is challenged by coarse-grid problems where the overhead in communication dominates computation. In this paper, an algorithm is introduced for redistributing coarse-grid problems through incremental agglomeration. Guided by a predictive performance model, this algorithm provides robust redistribution decisions for structured multilevel solvers. A two-dimensional diffusion problem is used to demonstrate the significant gain in performance of this algorithm over the previous approach that used agglomeration to one processor. In addition, the parallel scalability of this approach

97 MATHEMATICS AND COMPUTING↗