Search NASA⌕ Search

SEARCH · Search NASA

Results for “distributed data structures”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

TuckerMPI: A Parallel C++/MPI Software Package for Large-scale Data Compression via the Tucker Tensor Decomposition

With this study, our goal is compression of massive-scale grid-structured data, such as the multi-terabyte output of a high-fidelity computational simulation. For such data sets, we have developed a new software package called TuckerMPI, a parallel C++/MPI software package for compressing distributed data. The approach is based on treating the data as a tensor, i.e., a multidimensional array, and computing its truncated Tucker decomposition, a higher-order analogue to the truncated singular value decomposition of a matrix. The result is a low-rank approximation of the original tensor-structured data. Compression efficiency is achieved by detecting latent global structure within the data, which we contrast to most compression methods that are focused on local structure. In this work, we describe TuckerMPI, our implementation of the truncated Tucker decomposition, including details of the data distribution and in-memory layouts, the parallel and serial implementations of the key kernels, and analysis of the storage, communication, and computational costs. We test the software on 4.5 and 6.7 terabyte data sets distributed across 100 s of nodes (1,000 s of MPI processes), achieving compression ratios between 100 and 200,000×, which equates to 99--99.999% compression (depending on the desired accuracy) in substantially less time than it would take to even read the same dataset from a parallel file system. Moreover, we show that our method also allows for reconstruction of partial or down-sampled data on a single node, without a parallel computer so long as the reconstructed portion is small enough to fit on a single machine, e.g., in the instance of reconstructing/visualizing a single down-sampled time step or computing summary statistics. The code is available at https://gitlab.com/tensors/TuckerMPI.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

A D-term Modeling Code (DMC) for Simultaneous Calibration and Full-Stokes Imaging of Very Long Baseline Interferometric Data

In this paper we present DMC, a model and associated tool for polarimetric imaging of very long baseline interferometry data sets that simultaneously reconstructs the full-Stokes emission structure along with the station-based gain and leakage calibration terms. DMC formulates the imaging problem in terms of posterior exploration, which is achieved using Hamiltonian Monte Carlo sampling. The resulting posterior distribution provides a natural quantification of uncertainty in both the image structure and the data calibration. We run DMC on both synthetic and real data sets, the results of which demonstrate its ability to accurately recover both the image structure and calibration quantities, as well as to assess their corresponding uncertainties. The framework underpinning DMC is flexible, and its specific implementation is under continued development.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Review of Current Software for Analyzing Total X-ray Scattering Data from Liquids

The popularity of the pair distribution function (PDF) analysis of X-ray total scattering data has steadily grown as access to ex situ synchrotron data has expanded. Due to the broadening of the PDF user community, there is a growing demand for software that can be used to extract PDFs and is accessible to non-expert users. While user-friendly options have been developed over the past decade for fast, streamlined data analysis, care must be taken in both processing the data and understanding any limitations, especially in the case of liquids. In this review, the same scattering data are analyzed using different total X-ray scattering software, in order to compare the accuracy of the extracted structure factors and associated pair distribution functions. The goal is to assess the best practices for extracting the most accurate liquid data for each software package. The importance of absolute normalization and the application of the most appropriate corrections are emphasized via quantitative comparisons between liquid sulfur and water. Additionally, an awareness of the competing conventions used to define the PDF in crystallography and liquids/glasses is crucial for both the downstream analyses of the data and a comparison with the previous results in the literature.

Gallington, Leighanne C. (ORCID:0000000203837522)↗
DOE OSTI2024

Development of Steady-State and Dynamic Mass and Energy Constrained Neural Networks for Distributed Chemical Systems Using Noisy Transient Data

The paper presents the development of algorithms for mass and energy constrained neural network models that can exactly conserve the overall mass and energy of distributed chemical process systems, even though the noisy transient data used for optimal model training violate the same. In contrast to approximately satisfying mass and energy balance constraints of a system by soft penalization of objective function, algorithms have been developed for solving equality-constrained nonlinear optimization problems, thus providing the guarantee of exactly satisfying the system mass and energy conservation laws. For developing dynamic mass-energy constrained network models for distributed systems, hybrid series and parallel dynamic-static neural networks have been leveraged. The developed algorithms for solving both the training and forward problems are validated using both steady-state and dynamic data in the presence of various noise characteristics. The developed data-driven algorithms are flexible to exactly satisfy mass and energy balance constraints for dynamic chemical processes if the system holdup information is available. The proposed network structures and algorithms are applied to the development of data-driven lumped and distributed models of an adiabatic superheater/reheater system, a nonisothermal continuous stirred tank reactor, as well as an electrically heated plug-flow reactor system where one form of energy gets transformed to another. It has been observed that the mass-energy constrained neural networks yield a root mean squared error of <1% with respect to the system truth for the case studies evaluated in this work.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2018

Interpretation of solution scattering data from lipid nanodiscs

The structural information contained in solution scattering data from empty lipid nanodiscs is examined in the context of a multi-component geometric model. X-ray scattering data were collected on nanodiscs of different compositions at scattering vector magnitudes up to 2.0 Å −1 . Through the calculation of the partial form factor for each of the nanodisc components before the isotropic average, structural parameters in the model were correlated to the features observed in the X-ray scattering data and to the corresponding distance distribution function. It is shown that, in general, the features at ∼0.3–0.6 Å −1 in the scattering data correlate to the bilayer structure. The data also support the argument that the elliptical shape of nanodiscs found in model fitting is physical, rather than an artefact due to the nanodisc size distribution. The lipid chain packing peak at ∼1.5 Å −1 is visible in the data and reflects the lipid bilayer phase transition. The shape change in the distance distribution function across the phase transition suggests that the nanodiscs are more circular in the fluid phase. The implication of these findings for model fitting of empty and protein-loaded nanodiscs is discussed.

36 MATERIALS SCIENCE↗
DOE OSTI2018

Luminescence, structure and antiferroelectric-type phase transition in Ca8ZnEu(PO4)7

Highlights: • Thermal, dielectric and luminescence properties were obtained for Ca{sub 8}ZnEu(PO{sub 4}){sub 7}. • Ca{sub 8}ZnEu(PO{sub 4}){sub 7} belongs to antiferroelectric with T{sub c} = 836 K. • First-order antiferroelectric/paraelectric phase transitions have been found. • Influence of composition on the luminescence properties has been shown. • Structure of Ca{sub 8}ZnEu(PO{sub 4}){sub 7} was refined by the Rietveld method. - Abstract: Photoluminescence excitation and emission spectra of new phosphate Ca{sub 8}ZnEu(PO{sub 4}){sub 7} have been investigated. The influence of the composition on the luminescence intensity has been shown. The substitution of Zn{sup 2+} for Ca{sup 2+} in Ca{sub 9}Eu(PO{sub 4}){sub 7} with the formation of Ca{sub 8}ZnEu(PO{sub 4}){sub 7} leads to the increased emission intensity {sup 5}D{sub 0} → {sup 7}F{sub 2} by about 1.5 times and to decreased number of bands from three to one in the range of {sup 5}D{sub 0} → {sup 7}F{sub 0} transition. Conditions of preparation do not influence the luminescent properties and distribution of Eu{sup 3+} cations in the structure. DSC, SHG data and dielectric spectroscopy reveal that Ca{sub 8}ZnEu(PO{sub 4}){sub 7} belongs to antiferroelectrics (T{sub c} = 836 K). The first-order antiferroelectric/paraelectric transformations are accompanied by a changing symmetry (R3-barC to R3-barm). The structure of Ca{sub 8}ZnEu(PO{sub 4}){sub 7} (sp. gr. R3-barc) was refined by the Rietveld method. The Zn{sup 2+} cations occupy the M5 site. The Eu{sup 3+} and Ca{sup 2+} ions are located in the M1 and M3 sites of β-Ca{sub 3}(PO{sub 4}){sub 2}-type structure.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Probing lead acetate in solution using X-ray diffraction

Here, for the purpose of water purification involving toxic metal sensing and removal, high energy X-ray diffraction experiments have been performed on lead acetate Pb(Ac) 2 solutions, liquid and amorphous glutathione disulfide (GSSG), and their mixtures. The data have been interpreted using Empirical Potential Structure Refinement and pair distribution function analysis. At the highest concentration of 1 M Pb(Ac) 2 in water, the lead molecules are found to cluster and the second shell in the water structure becomes slightly more ordered as the water molecules become compressed. Liquid and amorphous GSSG are found to hydrogen bond primarily via OH-O interactions, while NH-O bonds are much more distorted. Aqueous solutions of Pb(Ac) 2 + GSSG show the closest Pb-O (carboxyl) and Pb-N (amine) bonds both at a distance of 2.5 ± 0.1 Å at a concentration of 1 M Pb(Ac) 2 in water. At 0.75 M Pb(Ac) 2 in water, strong Pb-S bonds are found at a distance of 2.8 ± 0.1 Å. The implications for lead removal using capacitive deionization are discussed.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Achievements and Prospects of Global Broadband Seismographic Networks After 30 Years of Continuous Geophysical Observations

Global seismographic networks (GSNs) emerged during the late nineteenth and early twentieth centuries, facilitated by seminal international developments in theory, technology, instrumentation, and data exchange. The mid- to late-twentieth century saw the creation of the World-Wide Standardized Seismographic Network (1961) and International Deployment of Accelerometers (1976), which advanced global geographic coverage as seismometer bandwidth increased greatly allowing for the recording of the Earth's principal seismic spectrum. The modern era of global observations and rapid data access began during the 1980s, and notably included the inception of the GEOSCOPE initiative (1982) and GSN (1988). Through continual improvements, GEOSCOPE and the GSN have realized near-real time recording of ground motion with state-of-art data quality, dynamic range, and timing precision to encompass 180 seismic stations, many in very remote locations. Data from GSNs are increasingly integrated with other geophysical data (e.g., space geodesy, infrasound and Interferometric Synthetic Aperture Radar). Globally distributed seismic data are critical to resolving crust, mantle, and core structure; illuminating features of the plate tectonic and mantle convection system; rapid characterization of earthquakes; identification of potential tsunamis; global nuclear test verification; and provide sensitive proxies for environmental changes. As the global geosciences community continues to advance our understanding of Earth structure and processes controlling elastic wave propagation, GSN infrastructure offers a springboard to realize increasingly multi-instrument geophysical observatories. Here, we review the historical, scientific, and monitoring heritage of GSNs, summarize key discoveries, and discuss future associated opportunities for Earth Science.

58 GEOSCIENCES↗
DOE OSTI2021

Generating synthetic occupants for use in building performance simulation

Occupant behaviour simulation frameworks can employ synthetic populations to characterize occupancy and behavioural patterns in buildings based on observed demographic data at a certain geographical location. For buildings, very few synthetic occupant populations have been generated. This paper uses a Bayesian Networks (BN) structural learning approach to synthesize populations of occupants in a multi-family housing case study. Two additional cases of office occupants and senior housing residents are considered as a cross-case comparison. Furthermore, we draw upon the extended version of drivers-needs-actions-systems (DNAS) framework to guide the selection of variables and data imputation. Our results show that the BN approach is powerful in learning the structure of data sets. The synthetic data sets successfully match the joint distributions of the underlying combined data sets. Experiments on the multi-family housing particularly show better performance than the office and senior housing cases.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2023

Building a genome-based understanding of bacterial pH preferences

The environmental preferences of many microbes remain undetermined. This is the case for bacterial pH preferences, which can be difficult to predict a priori despite the importance of pH as a factor structuring bacterial communities in many systems. We compiled data on bacterial distributions from five datasets spanning pH gradients in soil and freshwater systems (1470 samples), quantified the pH preferences of bacterial taxa across these datasets, and compiled genomic data from representative bacterial taxa. While taxonomic and phylogenetic information were generally poor predictors of bacterial pH preferences, we identified genes consistently associated with pH preference across environments. We then developed and validated a machine learning model to estimate bacterial pH preferences from genomic information alone, a model that could aid in the selection of microbial inoculants, improve species distribution models, or help design effective cultivation strategies. More generally, we demonstrate the value of combining biogeographic and genomic data to infer and predict the environmental preferences of diverse bacterial taxa.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2020

National Cancer Institute Think-Tank Meeting Report on Proteomic Cartography and Biomarkers at the Single-Cell Level: Interrogation of Premalignant Lesions

A Think-Tank Meeting was convened by the National Cancer Institute (NCI) to solicit experts’ opinion on the development and application of multi-omic single-cell analyses, and especially single-cell proteomics, to improve the development of a new generation of biomarkers for cancer risk, early detection, diagnosis, prognosis as well as to discuss the discovery of new targets for prevention and therapy. It is anticipated that such markers and targets will be based on cellular, subcellular, molecular and functional aberrations within the lesion and within individual cells. Additionally, single-cell proteomic data will be essential for the establishment of new tools that with searchable and scalable features that include spatial and temporal cartographies of premalignant and malignant lesions. Challenges and potential solutions that were discussed included: • The best way/s to analyze single-cells from fresh and preserved tissue • Detection and analysis of secreted molecules and from single cells, especially from a tissue slice • Detection of new, previously undocumented cell type/s in the premalignant and early stage cancer tissue microenvironment • Multi-omic integration of data to support and inform proteomic measurements • Subcellular organelles – identifying abnormal structure, function, distribution and location within individual premalignant and malignant cells • How to improve the dynamic range of single-cell proteomic measurements for discovery of differentially expressed proteins and their post translational modifications (PTM) • The depth of coverage measured concurrently using single-cell techniques • Quantitation - absolute or semiquantitative? • Single methodology or multiplexed combinations? • Application of analytical methods for identification of biologically significant subsets • Data visualization of N-dimensional datasets • How to construct intercellular signaling networks in individual cells within premalignant tumor microenvironments (TME) • Associations between intrinsic cellular processes and extrinsic stimuli • How to predict cellular responses to stress inducing stimuli • Identification of new markers for prediction of progression from precursor, benign and localized lesions to invasive cancer, based on spatial and temporal changes within individual cells • Identification of new targets for immunoprevention or immunotherapy – identification of neoantigens and surfactome of individual cells within a lesion

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2017

A Review of Power Distribution Test Feeders in the United States and the Need for Synthetic Representative Networks

Under the increasing penetration of distributed energy resources and new smart network technologies, distribution utilities face new challenges and opportunities to ensure reliable operations, manage service quality, and reduce operational and investment costs. Simultaneously, the research community is developing algorithms for advanced controls and distribution automation that can help to address some of these challenges. However, there is a shortage of realistic test systems that are publically available for development, testing, and evaluation of such new algorithms. Concerns around revealing critical infrastructure details and customer privacy have severely limited the number of actual networks published and that are available for testing. In recent decades, several distribution test feeders and US-featured representative networks have been published, but the scale, complexity, and control data vary widely. This paper presents a first-of-a-kind structured literature review of published distribution test networks with a special emphasis on classifying their main characteristics and identifying the types of studies for which they have been used. As a result, this both aids researchers in choosing suitable test networks for their needs and highlights the opportunities and directions for further test system development. In particular, we highlight the need for building large-scale synthetic networks to overcome the identified drawbacks of current distribution test feeders.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2022

Müllerian mimicry and the coloration patterns of sympatric coral snakes

Abstract Coral snakes in the genus Micrurus are venomous, aposematic organisms that signal danger to predators through vivid coloration. Previous studies found that they serve as models to several harmless species of Batesian mimics. However, the extent to which Micrurus species engage in Müllerian mimicry remains poorly understood. We integrate detailed morphological and geographical distribution data to investigate if coral snakes are Müllerian mimics. We found that coloration is spatially structured and that Micrurus species tend to be more similar where they co-occur. Though long supposed, we demonstrate for the first time that coral snakes might indeed be Müllerian mimics as they show some convergence in coloration patterns. Additionally, we found that the length of red-coloured rings in Micrurus is conserved, even at large geographic scales. This finding suggests that bright red rings may be under more substantial stabilizing selection than other aspects of coloration and probably function as a generalized signal for deterring predators.

Evolutionary Biology↗
DOE OSTI2024

Up and down quark structure of the proton

We report an improved measurement of the valence u and d quark distributions from the forward-backward asymmetry in the Drell-Yan process using 8.6 fb − 1 of data collected with the D0 detector in p p ¯ collisions at s = 1.96 . This analysis provides the values of new structure parameters that are directly related to the valence up and down quark distributions in the proton. In other experimental results measuring the quark content of the proton, d quark contributions are mixed with those from other quark flavors. In this measurement, the u and d quark contributions are separately extracted by applying a factorization of the QCD and electroweak portions of the forward-backward asymmetry. Published by the American Physical Society 2024

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Distributed directory of named data elements in coordination namespace

An approach is described that provides a distributed directory structure within a storage of an information handling system (a local node). A request is received with the request corresponding to a shared virtual address. The shared virtual address that is shared amongst a number of nodes that includes the local node and some remote nodes. A Global Address Space Directory (GASD) is retrieved that corresponds to a global virtual address space. The GASD is stored in a Coordination Namespace that is stored in a memory that is distributed amongst the nodes. A mapping that is included in the GASD is used to determine the node where the shared virtual address currently resides. The shared virtual address is then accessed from the node where it currently resides.

Johns, Charles Ray↗
DOE OSTI2018

Distributed Fiber Optic Arrays: Integrated Temperature and Seismic Sensing for Detection of CO 2 Flow, Leakage and Subsurface Distribution (Final Report)

This report describes the seismic and heat-pulse monitoring equipment, surveys, results and accomplishments of the fiber optic (FO) project funded by the U.S. Department of Energy (DOE) under Agreement Number DE-FE0012700. The formal title of the project is “Distributed Fiber Optic Arrays: Integrated Temperature and Seismic Sensing for Detection of CO 2 Flow, Leakage and Subsurface Distribution.” The project focuses on the field acquisition of FO data using Distributed Acoustic Sensing (DAS) arrays for subsurface imaging of geologic structure and to track the CO 2 migration in the injection interval using time-lapse techniques. A second component of the project uses a novel heat-pulse monitoring method that relies on Distributed Temperature Sensing (DTS) to evaluate near-field leakage of CO 2 or brine in and around the wellbore. The Southeast Regional Carbon Sequestration (SECARB) partnership hosted the project at the Anthropogenic Test site near Citronelle, Alabama after the original site host, the Kansas Geological Survey (KGS), experienced significant delays in permitting its CO 2 injection well at the Wellington Oil Field site in Kansas. The KGS project was later terminated in 2017 before an injection permit issued. A second field trial was planned for the Livingston Oil Field, located in Louisiana, but the site host ran into financial troubles and the project was terminated in 2015. A replacement field-site host was not found for the Livingston field trial. Both the Wellington, Kansas and Livingston, Louisiana projects were funded by DOE. A novel sensor platform referred to as the Modular Borehole Monitoring (MBM) system deployed at the Citronelle field site was utilized by the FO project. Two different seismic sensor technologies were incorporated in the Citronelle MBM system design including: (1) a semi-permanent, tubing-deployed, 18-level geophone array with custom hydraulic clamps; and (2) two, single mode optical fibers for Distributed Acoustic Sensing (DAS). Incorporation of the 18-level geophone array in the MBM system provided an ideal opportunity to compare the DAS response to conventional geophones. A heat-pulse monitoring system consisting of two, multi-mode optical fibers for Distributed Temperature Sensing (DTS) and copper heater element were also incorporated into the MBM package. This report provides an overview of the design of the MBM, DAS, and DTS equipment and the surveys performed during the study. The heat-pulse monitoring technique was used to diagnose a completion problem with observation well D-9-8 #2 where the MBM system was installed in 2012, thus successfully demonstrating its application for leak detection. In 2015, CO 2 unexpectedly arrived at the observation well approximately one year after CO 2 injection and heat-pulse monitoring ended. It was determined at this time that saltwater intrusion had likely penetrated the stainless-steel capillary tube containing the fibers and copper heater elements, thus preventing the heater from being used. DTS measurements indicated that the fiber had also degraded probably due to hydrogen darkening. It is recommended that the capillary tube be constructed of a more corrosive resistant material (e.g., Inconel) in the future should it be deployed in similar harsh downhole environments. Geophysical imaging of the subsurface and monitoring of the CO 2 were successful using DAS in the vertical seismic profile (VSP) survey configuration. A high resolution VSP image of the subsurface was obtained in 2014 using DAS, which exceeded our expectations in comparison to a lower resolution image obtained by SECARB in 2012 using an 80-level geophone array. A time-lapse image of the redistribution of CO 2 after injection ended in September 2014 was obtained using two DAS VSP surveys from June 2014 and December 2015, thus successfully demonstrating its application. DAS data were also acquired during a crosswell seismic survey conducted in 2014. Unfortunately, the DAS technique was not success in the crosswell survey configuration because the noise level was too high, increasing by a factor of ten, over the high frequency output (100-1,200) of the piezoelectric source.

58 GEOSCIENCES↗
DOE OSTI2024

e RPCA : Robust Principal Component Analysis for Exponential Family Distributions

Abstract Robust principal component analysis (RPCA) is a widely used method for recovering low‐rank structure from data matrices corrupted by significant and sparse outliers. These corruptions may arise from occlusions, malicious tampering, or other causes for anomalies, and the joint identification of such corruptions with low‐rank background is critical for process monitoring and diagnosis. However, existing RPCA methods and their extensions largely do not account for the underlying probabilistic distribution for the data matrices, which in many applications are known and can be highly non‐Gaussian. We thus propose a new method called RPCA for exponential family distributions (), which can perform the desired decomposition into low‐rank and sparse matrices when such a distribution falls within the exponential family. We present a novel alternating direction method of multiplier optimization algorithm for efficient decomposition, under either its natural or canonical parametrization. The effectiveness of is then demonstrated in two applications: the first for steel sheet defect detection and the second for crime activity monitoring in the Atlanta metropolitan area.

Zheng, Xiaojun↗
DOE OSTI2021

Local structure of Mott insulating iron oxychalcogenides La 2 O 2 Fe 2 OM2(M=S,Se)

Here, we describe the local structural properties of the iron oxychalcogenides, La 2 O 2 Fe 2 OM 2 (M=S,Se), by using pair distribution function analysis applied to total scattering data. Our results from neutron powder diffraction show that M = S and Se possess similar nuclear structures at low and room temperatures. The local crystal structures were studied by investigating deviations in atomic positions and the extent of the formation of orthorhombicity. Analysis of the total scattering data suggests that buckling of the Fe 2 O plane occurs below 100 K. The buckling may occur concomitantly with a change in octahedral height. Furthermore, within a typical range of 1-2 nm, we observed a short-range orthorhombiclike structure suggestive of nematic fluctuations in both of these materials.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗