SEARCH · Search NASA

Results for “distributed data structures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,117 records · Page 62

All sources NASA NTRS DOE OSTI

DOE OSTI2026

Toward a standardization of cryostructure and cryogenic soil structure terminology for the field description of permafrost‐affected soils

This paper establishes standardized terminology and field documentation protocols for cryostructures and cryogenic soil structures in permafrost‐affected soils and provides brief guidance on descriptions of ground ice morphology and ice volume estimates. We consolidate permafrost terminology from Russian and North American literature, clarify long‐standing ambiguities, and provide explicit guidelines that align with US Department of Agriculture‐Natural Resources Conservation Service soil description standards. Our scheme makes critical distinctions between cryostructure, the distribution of ice within soil, and cryogenic soil structure, the morphological structure of soil resulting from ice formation. The scheme organizes cryostructures into three main categories: non‐segregated ice, visible segregated ice, and ice matrices. We introduce standardized codes and parameters for field descriptions of ice and soil that enable machine‐readable data collection compatible with existing soil information systems. This standardization will significantly enhance the integration of field observations into landscape‐scale assessments of permafrost stability, infrastructure vulnerability, and ecosystem response to permafrost thaw, addressing an urgent need for quantitative data to inform modeling and decision‐making in rapidly changing Arctic and subarctic environments.

Andersen, Megan L. [University of Minnesota, Saint↗

DOE OSTI2021

Phoenix: A Scalable Streaming Hypergraph Analysis Framework

We present Phoenix, a scalable hypergraph analytics framework for data analytics and knowledge discovery that was implemented on the leadership class computing platforms at Oak Ridge National Laboratory (ORNL). Our software framework comprises a distributed implementation of a streaming server architecture which acts as a gateway for various hypergraph generators/external sources to connect. Phoenix has the capability to utilize diverse hypergraph generators, including HyGen, a very large-scale hypergraph generator developed by ORNL. Phoenix incorporates specific algorithms for efficient data representation by exploiting hidden structures of the hypergraphs. Our experimental results demonstrate Phoenix’s scalable and stable performance on massively parallel computing platforms. Phoenix’s superior performance is due to the merging of high-performance computing with data analytic.

Kurte, Kuldeep↗

DOE OSTI2026

Multi-task Parallelism for Robust Pre-training of Graph Foundation Models on Multi-source, Multi-fidelity Atomistic Modeling Data

Graph foundation models using graph neural networks promise sustainable, efficient atomistic modeling. To tackle challenges of processing multi-source, multi-fidelity data during pre-training, recent studies employ multi-task learning, in which shared message passing layers initially process input atomistic structures regardless of source, then route them to multiple decoding heads that predict data-specific outputs. This approach stabilizes pre-training and enhances a model’s transferability to unexplored chemical regions. Preliminary results on approximately four million structures are encouraging, yet questions remain about generalizability to larger, more diverse datasets and scalability on supercomputers. We propose a multi-task parallelism method that distributes each head across computing resources with GPU acceleration. Implemented in the open-source HydraGNN architecture, our method was trained on over 24 million structures from five datasets and tested on the Perlmutter, Aurora, and Frontier supercomputers, demonstrating efficient scaling on all three highly heterogeneous super-computing architectures.

Lupo Pasini, Massimiliano [ORNL] (ORCID:0000000249↗

DOE OSTI2018

Fractal Dimension Calculation for Big Data Using Box Locality Index

The box-counting approach for fractal dimension calculation is scaled up for big data using a data structure named box locality index (BLI). The BLI is constructed as key-value pair style with the key (an integer array) indexing the location of a “box” (i.e., a gird cell on the multi-dimensional space embedding a given dataset) and the value counting the number of data points inside the box (called “box occupancy”). Compared with the traditionally used E-dim tree, the BLI avoids complex hierarchical structure and encodes only necessary information required by box-counting approach for fractal dimension calculation. Moreover, the key-value pair nature of BLI, together with the fact that the box occupancy is aggregatable, grants box-counting approach the needed scalability for fractal dimension calculation of big data using distributed computing techniques (e.g., MapReduce and Spark). Taking the advantage of the BLI, MapReduce and Spark methods for fractal dimension calculation of big data are developed, which conduct box-counting for each grid level as a cascade of MapReduce/Spark jobs in a bottom-up fashion. In an empirical validation, the MapReduce and Spark methods demonstrated good effectiveness and efficiency in fractal calculation of a big synthetic dataset. In summary, this work provides an efficient solution for the estimation of intrinsic dimension of big data that is essential for many machine learning methods and data analytics.

Fractal dimension, Intrinsic dimension, Box-counti↗

DOE OSTI2019

Multiscale Based Characterization and Classification of Urban Land-Use

Machine learning and deep learning provide a means for generating urban land-use maps with relatively little human effort compared to manually digitizing images. This is especially important for supporting global and regional initiatives focused on sustainability, planning, health, pro-poor policy, infrastructure, and population distribution estimates. Many of these initiatives work in areas where geospatial data is scarce, such as the global south, and often use land-use maps to help achieve their goals. In this study, we develop a typology for automated labeling of urban land-use data that captures the variation in structural patterns within cities. A comparison of classification accuracy between convolutional neural networks (CNNs) and support vector machines (SVMs) coupled with handcrafted features is conducted. Through experimental validation on two highly dense cities in Africa, we report on new insights and the potential benefits offered by both multiscale handcrafted features and multiscale-CNNs even with limited training data.

Arndt, Jacob↗

DOE OSTI2015

Implementing High-Performance Geometric Multigrid Solver with Naturally Grained Messages

Structured-grid linear solvers often require manually packing and unpacking of communication data to achieve high performance.Orchestrating this process efficiently is challenging, labor-intensive, and potentially error-prone.In this paper, we explore an alternative approach that communicates the data with naturally grained messagesizes without manual packing and unpacking. This approach is the distributed analogue of shared-memory programming, taking advantage of the global addressspace in PGAS languages to provide substantial programming ease. However, its performance may suffer from the large number of small messages. We investigate theruntime support required in the UPC ++ library for this naturally grained version to close the performance gap between the two approaches and attain comparable performance at scale using the High-Performance Geometric Multgrid (HPGMG-FV) benchmark as a driver.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2024

Resolved photons in S HERPA

We present the first complete simulation framework, in the SHERPA event generator, for resolved photon interactions at next-to leading order accuracy. It includes photon spectra obtained through the equivalent-photon approximation, parton distribution functions to parametrize the hadronic structure of quasi-real photons, the matching of the parton shower to next-to leading order QCD calculations for resolved photon cross sections, and the modelling of multiple-parton interactions. We validate our framework against a wide range of photo-production data from LEP and HERPA experiments, observing good overall agreement. We identify important future steps relevant for high-quality simulations at the planned Electron–Ion Collider.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2016

Towards a Scalable and Adaptive Application Support Platform for Large-Scale Distributed E-Sciences in High-Performance Network Environments

The advent of large-scale collaborative scientific applications has demonstrated the potential for broad scientific communities to pool globally distributed resources to produce unprecedented data acquisition, movement, and analysis. System resources including supercomputers, data repositories, computing facilities, network infrastructures, storage systems, and display devices have been increasingly deployed at national laboratories and academic institutes. These resources are typically shared by large communities of users over Internet or dedicated networks and hence exhibit an inherent dynamic nature in their availability, accessibility, capacity, and stability. Scientific applications using either experimental facilities or computation-based simulations with various physical, chemical, climatic, and biological models feature diverse scientific workflows as simple as linear pipelines or as complex as a directed acyclic graphs, which must be executed and supported over wide-area networks with massively distributed resources. Application users oftentimes need to manually configure their computing tasks over networks in an ad hoc manner, hence significantly limiting the productivity of scientists and constraining the utilization of resources. The success of these large-scale distributed applications requires a highly adaptive and massively scalable workflow platform that provides automated and optimized computing and networking services. This project is to design and develop a generic Scientific Workflow Automation and Management Platform (SWAMP), which contains a web-based user interface specially tailored for a target application, a set of user libraries, and several easy-to-use computing and networking toolkits for application scientists to conveniently assemble, execute, monitor, and control complex computing workflows in heterogeneous high-performance network environments. SWAMP will enable the automation and management of the entire process of scientific workflows with the convenience of a few mouse clicks while hiding the implementation and technical details from end users. Particularly, we will consider two types of applications with distinct performance requirements: data-centric and service-centric applications. For data-centric applications, the main workflow task involves large-volume data generation, catalog, storage, and movement typically from supercomputers or experimental facilities to a team of geographically distributed users; while for service-centric applications, the main focus of workflow is on data archiving, preprocessing, filtering, synthesis, visualization, and other application-specific analysis. We will conduct a comprehensive comparison of existing workflow systems and choose the best suited one with open-source code, a flexible system structure, and a large user base as the starting point for our development. Based on the chosen system, we will develop and integrate new components including a black box design of computing modules, performance monitoring and prediction, and workflow optimization and reconfiguration, which are missing from existing workflow systems. A modular design for separating specification, execution, and monitoring aspects will be adopted to establish a common generic infrastructure suited for a wide spectrum of science applications. We will further design and develop efficient workflow mapping and scheduling algorithms to optimize the workflow performance in terms of minimum end-to-end delay, maximum frame rate, and highest reliability. We will develop and demonstrate the SWAMP system in a local environment, the grid network, and the 100Gpbs Advanced Network Initiative (ANI) testbed. The demonstration will target scientific applications in climate modeling and high energy physics and the functions to be demonstrated include workflow deployment, execution, steering, and reconfiguration. Throughout the project period, we will work closely with the science communities in the fields of climate modeling and high energy physics including Spallation Neutron Source (SNS) and Large Hadron Collider (LHC) projects to mature the system for production use.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2026

DEM Modeling and Validation of Pebble Bed Packing Using Chrono::GPU

Accurate prediction of pebble packing structure is important for pebble bed reactors because the spatial distribution of void fraction directly affects coolant flow, pressure drop, heat transfer, and neutronic behavior. However, experimentally validated DEM studies that directly evaluate local void-fraction structure in reactor-relevant pebble beds remain limited. In this work, the pebble bed experiment conducted at Missouri University of Science and Technology is simulated using the graphics processing unit (GPU)-based discrete element method (DEM) code Chrono::GPU. The study focuses on evaluating the ability of Chrono::GPU to reproduce the packing arrangement and void-fraction distribution of a randomly packed spherical pebble bed. The DEM results are first verified against established radial void-fraction correlations, including the Mueller and Vortmeyer-Schuster models, to assess the predicted bulk porosity, near-wall behavior, and oscillatory packing structure. The simulation is then verified against reference DEM data and validated against gamma-ray computed tomography (CT) experimental data at three axial locations. The Chrono::GPU results reproduce the main features of the experimental packing, including the high void fraction near the wall, the first near-wall trough, and the damped oscillatory radial profile caused by wall-induced ordering. Quantitative comparison with DEM data and the CT-based radial profiles shows good agreement, with mean absolute errors on the order of 0.07 and root-mean-square errors below 0.09 for the averaged profiles. These results demonstrate that Chrono::GPU can accurately capture the void-fraction structure of spherical pebble beds and provides a reliable DEM framework for future pebble bed reactor packing, recycling, and thermal-hydraulic studies.

97 - MATHEMATICS AND COMPUTING↗

DOE OSTI2016

Crystal and magnetic structures of Cr{sub 1∕3}NbSe{sub 2} from neutron diffraction

Neutron diffraction measurements of the Cr intercalated niobium diselenide Cr{sub 1∕3}NbSe{sub 2} together with magnetization measurements have revealed that this compound exhibits ferromagnetic ordering below T{sub C} = 96 K unlike a chiral helimagnetic order observed in the sulfide compound Cr{sub 1∕3}NbS{sub 2}. As derived from neutron diffraction data, the Cr magnetic moments μ{sub Cr} = 2.83 ± 0.03 μ{sub B} in Cr{sub 1∕3}NbSe{sub 2} are aligned within basal plane. The discrepancy in the magnetic states of Cr{sub 1∕3}NbS{sub 2} and Cr{sub 1∕3}NbSe{sub 2} is ascribed to the difference in the preferential site occupation of Cr ions in crystal lattices. In Cr{sub 1∕3}NbSe{sub 2}, the Cr ions are predominantly distributed over 2b Wyckoff site, which determines a centrosymmetric character of the crystal structure unlike Cr{sub 1∕3}NbS{sub 2}, where the Cr ions are mainly located in 2c position and the crystal structure is non-centrosymmetric.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2015

Representational analysis of extended disorder in atomistic ensembles derived from total scattering data

With the increased availability of high-intensity time-of-flight neutron and synchrotron X-ray scattering sources that can access wide ranges of momentum transfer, the pair distribution function method has become a standard analysis technique for studying disorder of local coordination spheres and at intermediate atomic separations. In some cases, rational modeling of the total scattering data (Bragg and diffuse) becomes intractable with least-squares approaches, necessitating reverse Monte Carlo simulations using large atomistic ensembles. However, the extraction of meaningful information from the resulting atomistic ensembles is challenging, especially at intermediate length scales. Representational analysis is used here to describe the displacements of atoms in reverse Monte Carlo ensembles from an ideal crystallographic structure in an approach analogous to tight-binding methods. Rewriting the displacements in terms of a local basis that is descriptive of the ideal crystallographic symmetry provides a robust approach to characterizing medium-range order (and disorder) and symmetry breaking in complex and disordered crystalline materials. Lastly, this method enables the extraction of statistically relevant displacement modes (orientation, amplitude and distribution) of the crystalline disorder and provides directly meaningful information in a locally symmetry-adapted basis set that is most descriptive of the crystal chemistry and physics.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2018

Duality-Free Decomposition Based Data-Driven Stochastic Security-Constrained Unit Commitment

To incorporate the superiority of both stochastic and robust approaches, a data-driven stochastic optimization is employed to solve the security-constrained unit commitment model. This approach makes the most use of the historical data to generate a set of possible probability distributions for wind power outputs and then it optimizes the unit commitment under the worst-case probability distribution. However, this model suffers from huge computational burden, as a large number of scenarios are considered. To tackle this issue, a duality-free decomposition method is proposed in this paper. This approach does not require doing duality, which can save a large set of dual variables and constraints, and therefore reduces the computational burden. In addition, the inner max-min problem has a special mathematical structure, where the scenarios have the similar constraint. Thus, the max-min problem can be decomposed into independent sub-problems to be solved in parallel, which further improves the computational efficiency. A numerical study on an IEEE 118-bus system with practical data of a wind power system has demonstrated the effectiveness of the proposal.

42 ENGINEERING↗

DOE OSTI2018

Structure of a Deformed Inhomogeneous Material on the Data of Acoustic Emission and X-Ray Computer Microtomography

Accumulation of defects at various stages of the deformation of a structurally inhomogeneous material (granite) has been studied using two nondestructive methods: acoustic emission (AE) and X-ray computer microtomorgaphy (CT). The quasi-static testing of cylindrical samples of a Westerly granite was carried out under uniaxial compression. The control of the defect formation was realized using the real-time monitoring of acoustic emission. For each sample under study, several steps of the loading and tomographic imaging have been performed. We have found that an exponential or power-law function of the energy distribution of the AE signals makes it possible to select a sample region, in which the system of defects has transited into a self-organized criticality state and large cracks have been formed. This result coincides with the data of the X-ray tomography Computer Microtomography.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

DOE OSTI2020

Postglacial migration shaped the genomic diversity and global distribution of the wild ancestor of lager-brewing hybrids

The wild, cold-adapted parent of hybrid lager-brewing yeasts, Saccharomyces eubayanus, has a complex and understudied natural history. The exploration of this diversity can be used both to develop new brewing applications and to enlighten our understanding of the dynamics of yeast evolution in the wild. Here, we integrate whole genome sequence and phenotypic data of 200 S. eubayanus strains, the largest collection known to date. S. eubayanus has a multilayered population structure, consisting of two major populations that are further structured into six subpopulations. Four of these subpopulations are found exclusively in the Patagonian region of South America; one is found predominantly in Patagonia and sparsely in Oceania and North America; and one is specific to the Holarctic ecozone. Plant host associations differed between subpopulations and between S. eubayanus and its sister species, Saccharomyces uvarum. S. eubayanus is most abundant and genetically diverse in northern Patagonia, where some locations harbor more genetic diversity than is found outside of South America, suggesting that northern Patagonia east of the Andes was a glacial refugium for this species. All but one subpopulation shows isolation-by-distance, and gene flow between subpopulations is low. However, there are strong signals of ancient and recent outcrossing, including two admixed lineages, one that is sympatric with and one that is mostly isolated from its parental populations. Using our extensive biogeographical data, we build a robust model that predicts all known and a handful of additional regions of the globe that are climatically suitable for S. eubayanus, including Europe where host accessibility and competitive exclusion by other Saccharomyces species may explain its continued elusiveness. We conclude that this industrially relevant species has rich natural diversity with many factors contributing to its complex distribution and natural history.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Boosting H I -Galaxy Cross-Clustering Signal through Higher-Order Cross-Correlations

After reionization, neutral hydrogen (${\rm H\, \small {I}}$) traces the large-scale structure (LSS) of the Universe, enabling ${\rm H\, \small {I}}$ intensity mapping (IM) to capture the LSS in 3D and constrain key cosmological parameters. We present a new framework utilizing higher-order cross-correlations to study ${\rm H\, \small {I}}$ clustering around galaxies, tested using real-space data from the IllustrisTNG300 simulation. This approach computes the joint distributions of k-nearest neighbor (kNN) optical galaxies and the ${\rm H\, \small {I}}$ brightness temperature field smoothed at relevant scales (the kNN-field framework), providing sensitivity to all higher-order cross-correlations, unlike two-point statistics. To simulate ${\rm H\, \small {I}}$ data from actual surveys, we add random thermal noise and apply a simple foreground cleaning model, filtering out Fourier modes of the brightness temperature field with k ∥ < k min,∥ . Under current levels of thermal noise and foreground cleaning, typical of a Canadian Hydrogen Intensity Mapping Experiment (CHIME)-like survey, the ${\rm H\, \small {I}}$-galaxy cross-correlation signal in our simulations, using the kNN-field framework, is detectable at >30σ across r = [3, 12] h –1 Mpc. In contrast, the detectability of the standard two-point correlation function (2PCF) over the same scales depends strongly on the foreground filter: a sharp k ∥ filter can spuriously boost detection to 8σ due to position-space ringing, whereas a less sharp filter yields no detection. Nonetheless, we conclude that kNN-field cross-correlations are robustly detectable across a broad range of foreground filtering and thermal noise conditions, suggesting their potential for enhanced constraining power over 2PCFs.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2019

Beyond Badger’s Rule: The Origins and Generality of the Structure–Spectra Relationship of Aqueous Hydrogen Bonds

The structure of hydrogen bonded networks is intimately intertwined with their dynamics. Despite the incredibly wide range of hydrogen bond strengths encountered in water clusters, ion-water clusters and liquid water, we demonstrate that the previously reported correlation between the change in the equilibrium bond length of the hydrogen bonded OH covalent bond and the corresponding shift in its harmonic frequency in water clusters is much more broadly applicable. Surprisingly, this correlation describes the ratios for both the equilibrium OH bond length/harmonic frequency, and the vibrationally averaged bond length/anharmonic frequency in water, hydronium water and halide water clusters. Consideration of harmonic and anaharmonic data leads to a correlation of -19 ± 1 cm -1 /0.001 Å. The basic nature of this correlation is further confirmed through the analysis of ab initio Molecular Dynamics (AIMD) trajectories for liquid water. We show that this simple correlation for both harmonic and anharmonic systems can be modeled by the response of an OH bond to an external field. Treating the OH bond as a Morse oscillator, we develop analytic expressions, which relate the ratio of the shift in the vibrational frequency of the hydrogen-bonded OH bond to the shift in OH bond length, to parameters in the Morse potential and the ratio of the first and second derivatives of the field-dependent projection of the dipole moment of water onto the hydrogen-bonded OH bond. Based on our analysis, we develop a protocol for reconstructing the AIMD spectra of liquid water from the sampled distribution of the OH bond lengths. Our conclusions elucidate the origins of the relationship between the molecular structure of the fleeting hydrogen-bonded network and the ensuing dynamics, which can be probed by vibrational spectroscopy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2024

A Bayesian Multi-fidelity Neural Network to Predict Nonlinear Frequency Backbone Curves

The use of structural mechanics models during the design process often leads to the development of models of varying fidelity. Often low-fidelity models are efficient to simulate but lack accuracy, while the high-fidelity counterparts are accurate with less efficiency. Here, this paper presents a multi-fidelity surrogate modeling approach that combines the accuracy of a high-fidelity finite element model with the efficiency of a low-fidelity model to train an even faster surrogate model that parameterizes the design space of interest. The objective of these models is to predict the nonlinear frequency backbone curves of the Tribomechadynamics Research Challenge benchmark structure which exhibits simultaneous nonlinearities from frictional contact and geometric nonlinearity. The surrogate model consists of an ensemble of neural networks that learn the mapping between low and high-fidelity data through nonlinear transformations. Bayesian neural networks are used to assess the surrogate model's uncertainty. Once trained, the multi-fidelity neural network is used to perform sensitivity analysis to assess the influence of the design parameters on the predicted backbone curves. Additionally, Bayesian calibration is performed to update the input parameter distributions to correlate the model parameters to the collection of experimentally measured backbone curves.

42 ENGINEERING↗

DOE OSTI2023

High-Resolution Near-Surface Imaging at the Basin Scale Using Dark Fiber and Distributed Acoustic Sensing: Toward Site Effect Estimation in Urban Environments

Near-surface seismic structure, particularly the shear wave velocity (V s ), can strongly affect local site response, and should be accurately estimated for ground motion prediction during seismic hazard assessment. The Imperial Valley (California), occupying the southern end of the Salton Trough, is a seismically active basin with thick surficial lacustrine sedimentary deposits. In this study, we utilize ambient noise records and local earthquake events for high-resolution near-surface characterization and site effect estimation with an unlit fiber-optic telecommunication infrastructure (dark fiber) in Imperial Valley by using the distributed acoustic sensing (DAS) technique. We apply ambient noise interferometry to retrieve coherent surface waves from DAS records, and evaluate performances of three different surface wave methods on DAS ambient noise dispersion imaging. We develop a quality control workflow to improve the dispersion measurement of noisy portions of the DAS data set by using a data selection strategy. Using the joint inversion of both the fundamental mode and higher overtones of Rayleigh waves, a high resolution two-dimensional (2D) V s structure down to 70 m depth is obtained. We successfully achieve an improved V s 30 (the time-averaged shear-wave velocity in the top 30 m) model with higher spatial-resolution and reliability compared to the existing community model for the area. We also explore the potential for utilizing DAS earthquake events for site amplification estimation. The preliminary results reveal a clear anti-correlation between the approximated site response and the V s 30 profile. In conclusion, our results indicate the potential utility of DAS deployed on dark fiber for near-surface characterization in appropriate contexts.

58 GEOSCIENCES↗