Search NASA⌕ Search

SEARCH · Search NASA

Results for “data storage systems buffer data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Global Corn Heat Stress: Mean and SD of Degree Days Above 29°C based on NEX-GDDP-CMIP6 Climate Projections

Description This global dataset provides the estimated mean and standard deviation (SD) of corn heat stress (degree days above 29°C) for a set of climate models in NEX-GDDP-CMIP6 at 0.25-degree resolution. The NEX-GDDP-CMIP6 dataset is comprised of global downscaled climate scenarios derived from the General Circulation Model (GCM) runs conducted under the Coupled Model Intercomparison Project Phase 6 (CMIP6). The current dataset includes: Long-Term Average Degree Days Above 29°C- Historical Long-Term Average Degree Days Above 29°C- SSP245 Long-Term Standard Deviation of Degree Days Above 29°C- Historical Long-Term Standard Deviation of Degree Days Above 29°C- SSP245 The mean and SD are calculated over 1985-2014 for the historical period and over 2035-2064 for future projections. A full description of methods, including growing season, daily temperature distribution, and statistical coefficients, can be found in Haqiqi (2024). The source climate data are obtained from https://ds.nccs.nasa.gov/thredds2/catalog/catalog.html and are described in Thrasher et al (2022). The codes used to create this dataset are available at https://github.com/ihaqiqi/dd29c_nex_cmip6. Acknowledgments This work was supported by the US Department of Energy, Office of Science, Biological and Environmental Research Program, Earth and Environmental Systems Modeling, MultiSector Dynamics under Cooperative Agreement DE-SC0022141. The data processing, computation, and storage were completed on Purdue Anvil supercomputer and cyberinfrastructure supported by the National Science Foundation HDR award # 2118329: "NSF Institute for Geospatial Understanding through an Integrative Discovery Environment (I-GUIDE)". References Haqiqi. I. (2024). Trade can buffer climate-induced risks and volatilities in crop supply. Environmental Research: Food Systems. https://doi.org/10.1088/2976-601X/ad7d12 Thrasher, B., Wang, W., Michaelis, A., Melton, F., Lee, T., & Nemani, R. (2022). NASA global daily downscaled projections, CMIP6. Scientific Data, 9(1), 262. https://doi.org/10.1038/s41597-022-01393-4

Climate Change↗
DOE OSTI2017

Performance characterization of scientific workflows for the optimal use of Burst Buffers

Scientific discoveries are significantly rely on the analysis of large volumes of data from observations and simulations of complex phenomena. Scientists compose the complex analyses as workflows and execute them on large-scale HPC systems. The workflow structures are in contrast with monolithic single simulations that have often been the primary use case on HPC systems. At the same time, new storage paradigms such as Burst Buffers are becoming available on HPC platforms. In this paper, we analyze the performance characteristics of a Burst Buffer and two representative scientific workflows with the aim of optimizing the usage of a Burst Buffer, extending our previous analyses (Daley et al., 2016). Our key contributions are (a) developing a performance analysis methodology pertinent to Burst Buffers, (b) improving the use of a Burst Buffer in workflows with bandwidth-sensitive and metadata-sensitive I/O workloads, (c) highlighting the key data management challenges when incorporating a Burst Buffer in the studied scientific workflows.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2016

Parallel compression of data chunks of a shared data object using a log-structured file system

Techniques are provided for parallel compression of data chunks being written to a shared object. A client executing on a compute node or a burst buffer node in a parallel computing system stores a data chunk generated by the parallel computing system to a shared data object on a storage node by compressing the data chunk; and providing the data compressed data chunk to the storage node that stores the shared object. The client and storage node may employ Log-Structured File techniques. The compressed data chunk can be de-compressed by the client when the data chunk is read. A storage node stores a data chunk as part of a shared object by receiving a compressed version of the data chunk from a compute node; and storing the compressed version of the data chunk to the shared data object on the storage node.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Analysis of controls for integrated energy storage system in energy arbitrage configuration with concrete thermal energy storage

Novel highly detailed NuScale reactor and turbo-generator models have been built in Modelica to analyze potential controls for an integrated energy system (IES) configuration combining a nuclear reactor and concrete thermal energy storage (CTES) operating in energy arbitrage. A shakedown test demonstrates the viability of the control schemes to operate across expected operating conditions while maintaining safety limits and buffers. A five-day data set from Idaho is used to show a load-following application of this system that increases the average reactor power level by 20% over the data set and matching demand at nearly all points. Here, by integrating these models, feedback mechanisms across the primary and secondary sides of the nuclear reactor are integrated, and realistic control evaluation is done. All models are consistent with the Hybrid repository, a suite of Modelica models, maintained at Idaho National Laboratory.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2019

Learning concave-convex profiles of data transport over dedicated connections

Dedicated data transport infrastructures are increasingly being deployed to support distributed big-data and high-performance computing scenarios. These infrastructures employ data transfer nodes that use sophisticated software stacks to support network transport among sites, which often house distributed file and storage systems. Throughput measurements collected over such infrastructures for a range of round trip times (RTTs) reflect the underlying complex end-to-end connections, and have revealed dichotomous throughput profiles as functions of RTT. In particular, concave regions of throughput profiles at lower RTTs indicate near-optimal performance, and convex regions at higher RTTs indicate bottlenecks due to factors such as buffer or credit limits. We present a machine learning method that explicitly infers these concave and convex regions and transitions between them using sigmoid functions. We also provide distribution-free confidence estimates for the generalization error of these concave-convex profile estimates. Throughput profiles for data transfers over 10 Gbps connections with 0–366 ms RTT provide important performance insights, including the near optimality of transfers performed with the XDD tool between XFS filesystems, and the performance limits of wide-area Lustre extensions using LNet routers. A direct application of generic machine learning packages does not adequately highlight these critical performance regions or provide as precise confidence estimates.

Rao, Nageswara↗
DOE OSTI2019

Parallel storage system with burst buffer appliance for storage of partitioned key-value store across a plurality of storage tiers

Parallel storage systems are provided with a burst buffer appliance for storage of a partitioned key-value store across a plurality of storage tiers. The burst buffer appliance is configured to store key-value data on first and second storage tiers comprising respective disjoint subsets of storage. One or more servers in the partitioned key-value store provide (i) an interface between one or more local key-value stores and one or more additional storage architectures, and/or (ii) a key value interface that provides access to at least one hardware key-value store. The local key-values are optionally sorted both locally by the local store and globally by the overall MDHIM framework. The local and global sorting can be leveraged to provide batch input/output (IO) operations that aggregate multiple requests into batch operations on one or more sorted ranges.

Bent, John M.↗
DOE OSTI2018

Enforcing End-to-end I/O Policies for Scientific Workflows using Software-Defined Storage Resource Enclaves

Data-intensive knowledge discovery requires scientific applications to run concurrently with analytics and visualization codes executing in situ for timely output inspection and knowledge extraction. Consequently, I/O pipelines of scientific workflows can be long and complex because they comprise many stages of analytics across different layers of the I/O stack of high-performance computing systems. Performance limitations at any I/O layer or stage can cause an I/O bottleneck resulting in greater than expected end-to-end I/O latency. In this paper, we present the design and implementation of a novel data management infrastructure called Software-Defined Storage Resource Enclaves (SIREN) at system level to enforce end-to-end policies that dictate an I/O pipeline's performance. SIREN provides an I/O performance interface for users to specify the desired storage resources in the context of in-situ analytics. If suboptimal performance of analytics is caused by an I/O bottleneck when data are transferred between simulations and analytics, schedulers in different layers of the I/O stack automatically provide the guaranteed lower bounds on I/O throughput. Lastly, our experimental results demonstrate that SIREN provides performance isolation among scientific workflows sharing multiple storage servers across two I/O layers (burst buffer and parallel file systems) while maintaining high system scalability and resource utilization.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Solution scattering at the Life Science X-ray Scattering (LiX) beamline

This work reports the instrumentation and software implementation at the Life Science X-ray Scattering (LiX) beamline at NSLS-II in support of biomolecular solution scattering. For automated static measurements, samples are stored in PCR tubes and grouped in 18-position sample holders. Unattended operations are enabled using a six-axis robot that exchanges sample holders between a storage box and a sample handler, transporting samples from the PCR tubes to the X-ray beam for scattering measurements. The storage box has a capacity of 20 sample holders. At full capacity, the measurements on all samples last for ∼9 h. For in-line size-exclusion chromatography, the beamline-control software coordinates with a commercial high-performance liquid chromatography (HPLC) system to measure multiple samples in batch mode. The beamline can switch between static and HPLC measurements instantaneously. In all measurements, the scattering data span a wide q -range of typically 0.006–3.2 Å −1 . Functionalities in the Python package py4xs have been developed to support automated data processing, including azimuthal averaging, merging data from multiple detectors, buffer scattering subtraction, data storage in HDF5 format and exporting the final data in a three-column text format that is acceptable by most data analysis tools. These functionalities have been integrated into graphical user interfaces that run in Jupyter notebooks, with hooks for external data analysis software.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2016

Development of a Hydrogen Energy System as a Grid Frequency Management Tool

The Hawai‘i Natural Energy Institute (HNEI) is conducting research to assess the technical potential of using an electrolyzer-based hydrogen (H2) production and storage system as a grid demand response tool using battery data from a 200 MW grid to show the kind of response required. The hydrogen produced by the electrolyzer is used for transportation. A 65 kg/day hydrogen energy system (HES) consisting of a PEM electrolyzer, 35 bar buffer tank, 450 bar compressor, and associated chiller systems was purchased and installed at the Hawaii Natural Energy Laboratory Hawaii Authority (NELHA) to demonstrate long-term durability of the electrolyzer under cyclic operation required for frequency regulation on an island grid system. The excess hydrogen was stored for use by three fuel-cell buses to be operated at Hawai‘i Volcanoes National Park (HAVO) and by the County of Hawai‘i Mass Transit Agency (MTA). This paper describes the site selection and equipment commissioning, plus a comprehensive test plan that was developed to characterize the performance and durability of the electrolyzer under dynamic load conditions. The controls were modified for the operating envelope and dynamic limits of the electrolyzer. While the data showed these modifications significantly improved the system response time, it is not fast enough to match a BESS response time for grid frequency management. The electrolyzer can only be used for slower acting changes (1 to 0.5 Hz). A potential solution is to design an electrolyzer/BESS hybrid system and develop a modeling program to find the optimum mix of battery and electrolyzer to provide the maximum grid regulation services at minimum cost.

08 HYDROGEN↗
DOE OSTI2024

BioSiting Tool (BioSiting) v2

The BioSiting Tool provides a geospatial interface for analyzing bioeconomy resources and infrastructure across the continental U.S. The tool integrates empirical and modeled data from a broad range of sources. Bioeconomy resources mapped in the tool include agricultural residues, forest residues, municipal solid waste streams, food waste, manure, fats, oils and greases and potential yields of energy crops. Infrastructure mapped in the tool includes biorefineries, material recovery facilities, anaerobic digesters, wastewater treatment plants, combustion plants, district energy systems, crude oil pipelines, petroleum pipelines, natural gas pipelines, railways and freight terminals. Additional data layers include environmental justice indicators at the census tract level and carbon dioxide geologic storage potential. Users can select a location on the map, define a buffer radius in kilometers and generate an inventory of all bioecomony resources within the buffer zone. Data from the tool can be downloaded from individual buffer zones, or at the state or national level.

Huntington, Tyler↗
DOE OSTI2019

ATLAS Data Carousel

The ATLAS experiment at CERN's LHC stores detector and simulation data in raw and derived data formats across more than 150 Grid sites world-wide, currently in total about 200PB on disk and 250PB on tape. Data have different access characteristics due to various computational workflows, and can be accessed from different media, such as remote I/O, disk cache on hard disk drives or SSDs. Also, larger data centers provide the majority of offline storage capability via tape systems. For the High-Luminosity LHC (HL-LHC), the estimated data storage requirements are several factors bigger than the present forecast of available resources, based on a flat budget assumption. On the computing side, ATLAS Distributed Computing was very successful in the last years with high performance and high throughput computing integration and in using opportunistic computing resources for the Monte Carlo simulation. On the other hand, equivalent opportunistic storage does not exist. ATLAS started the "Data Carousel" project to increase the usage of less expensive storage, i.e. tapes or even commercial storage, so it is not limited to tape technologies exclusively. Data Carousel orchestrates data processing between workload management, data management, and storage services with the bulk data resident on offline storage. The processing is executed by staging and promptly processing a sliding window of inputs onto faster buffer storage, such that only a small percentage of input data are available at any one time. With this project, we aim to demonstrate that this is the natural way to dramatically reduce our storage cost. The first phase of the project was started in the fall of 2018 and was related to I/O tests of the sites archiving systems. Phase II now requires a tight integration of the workload and data management systems. Additionally, the Data Carousel studies the feasibility to run multiple computing workflows from tape. The project is progressing very well and the results presented in this document will be used before the LHC Run 3.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

DoCeph: DPU-Offloaded Messaging in Ceph for Reduced Host CPU Utilization

Ceph is a widely used distributed object store, but its messenger layer imposes substantial CPU overhead on the host. To address this limitation, we propose DoCeph, a DPU-offloaded storage architecture for Ceph that disaggregates the system by offloading the communication-intensive messaging component to the DPU while retaining the storage backend on the host. The DPU efficiently manages communication, using lightweight RPC for metadata operations and DMA for data transfer. Moreover, DoCeph introduces a pipelining technique that overlaps data transmission with buffer preparation, mitigating hardware-imposed transfer size limitations. We implemented DoCeph on a Ceph cluster with NVIDIA BlueField-3 DPUs. Evaluation results indicate that DoCeph cuts host CPU usage by up to 92% while sustaining stable throughput and providing larger performance benefits for object writes over 1 MB.

Park, Kuri [Sogang University]↗
DOE OSTI2025

A buffering heat exchanger/thermal energy storage system for desalination applications

A buffering heat exchanger/thermal energy storage (BHXTES) system was designed using a fast-response phase-change material/graphite foam (PCM/GF) medium for desalination and potentially other industrial applications. Unlike a store-now-and-use-later TES system, this study focused on developing a dynamically balanced, continuously used, system that optimized three functions. A lab-scale prototype was designed and fabricated with a PureTemp 151/GF storage medium. The experimental data showed excellent thermal performance repeatability indicating minimal effects of PCM expansion and contraction on the GF as well as negligible PCM redistribution effects. The experimental data also served as a validation of the numerical model of the BHXTES system, after which optimization of the BHXTES system was conducted through numerical simulations for both PureTemp 151/GF and solar salt/GF as the storage medium. The results show that, with a Therminol 55 heat transfer fluid, continuous operation of the system can be realized with a solar energy source through cycles of 8-h changing for desalination and energy storage with the supplied heat followed by 16-h discharging for desalination with the stored energy. As a result, the working fluid outlet temperatures are generally in the ranges of approximately 128–160 °C for a PureTemp 151/GF medium and 198–220 °C for a solar salt/GF medium.

Buffering↗
DOE OSTI2025

HDF5 in the exascale era: Delivering efficient and scalable parallel I/O for exascale applications

Accurately modeling real-world systems requires scientific applications at exascale to generate massive amounts of data and manage data storage efficiently. However, parallel input and output (I/O) faces challenges due to new application workflows and the state-of-the-art memory, interconnect, and storage architectures considered in exascale designs. The storage hierarchy has expanded with node-local persistent memory, solid-state storage, and traditional disk and tape-based storage, thus requiring efficiency at each layer and much more efficient data movement among these layers. This paper discusses how the ExaHDF5 project improved the I/O performance and data management for exascale architectures by enhancing HDF5, a widely used parallel I/O library. The team developed an Asynchronous I/O Virtual Object Layer (VOL) connector that allowed overlapping I/O with computation. They also created a Cache VOL to complement asynchronous I/O by incorporating fast storage layers, such as burst buffer and node-local storage, into the parallel I/O workflow through caching and staging data. Additionally, the team enabled data aggregation and I/O at the node level by using a Subfiling Virtual File Driver (VFD). To demonstrate superior I/O performance with HDF5 at exascale, the ExaHDF5 team collaborated with several exascale applications. In this paper, we show I/O performance improvements for three applications: Cabana (a particle-based simulation library), EQSIM (a regional earthquake simulation software), and E3SM (a climate system modeling library).

Asynchronous I/Ol↗
DOE OSTI2021

A control oriented comprehensive degradation model for battery energy storage system life prediction

In order to ensure that stationary battery energy storage systems (BESSs) provide reliable energy buffering, both for power quality and economic purposes, the degradation must be considered. Cell degradation involves various side reactions and is highly dependent on its operating conditions. To accurately track cell degradation and predict its impact on battery behavior, a comprehensive physics-based degradation model based on an electrolyte phase-enhanced single particle (SP) model is developed. Key degradation physics, namely solid electrolyte interphase (SEI) layer formation and growth, Li plating on the graphite anode, and Mn dissolution on cathode of nickel-cobalt-manganese oxide (NMC622) are considered. The model is validated against experimental capacity data. The results reveal that the deposition rate of both SEI layer and Li metal increase as the charge voltage increases. At the cathode side, the solvent oxidation rate determines the Mn dissolution rate. As a result, the volume fraction of NMC622 in the cathode continually decreases at a gradually increasing rate.

25 ENERGY STORAGE↗
DOE OSTI2017

Scalable Analysis Methods and In Situ Infrastructure for Extreme Scale Knowledge Discovery

High performance computers have for many years been on a trajectory that gives them extraordinary compute power with the addition of more and more compute cores. At the same time, other system parameters such as the amount of memory per core and bandwidth to storage have remained constant or have barely increased. This creates an imbalance in the computer, giving it the ability to compute a lot of data that it cannot reasonably save out due to time and storage constraints. While technologies have been invented to mitigate this problem (burst buffers, etc.), software has been adapting to employ in situ libraries which perform data analysis and visualization on simulation data while it is still resident in memory. This avoids the need to ever have to pay the costs of writing many terabytes of data files. Instead, in situ enables the creation of more concentrated data products such as statistics, plots, and data extracts, which are all far smaller than the full-sized volume data. With the increasing popularity of in situ, multiple in situ infrastructures have been created, each with its own mechanism for integrating with a simulation. To make it easier to instrument a simulation with multiple in situ infrastructures and include custom analysis algorithms, this project created the SENSEI framework.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Side-by-Side Testing of Water Heating Systems: Results from the 2013–2014 Evaluation

The Florida Solar Energy Center (FSEC) completed a fourth year-long evaluation on residential hot water heating systems in a laboratory environment (east central Florida, hot-humid climate). The evaluation studied the performance of five hot water systems (HWS) plus a reference baseline system for each fuel, (i.e., electric and natural gas). Electric HWS consisted of two residential electric heat pump water heaters (HPWHs, 60 and 80 gallons), a solar thermal system using a polymer absorber (glazed) collector with 80-gallon storage and a duplicate 50-gallon standard electric water heater with added cap and wrap insulation. Baseline performance data were collected from a standard 50-gallon electric water heater of minimum code efficiency to compare energy savings. Similarly, a standard 40-gallon upright vented natural gas water heater served as baseline for the natural gas fuel category. The latter, having a larger jacket diameter [18 in., with an energy factor (EF) of 0.62] with increased insulation, replaced a former baseline (17 in. diameter, EF = 0.59) that served during three previous testing rotations (2009–2013). A high-efficiency, condensing natural gas hybrid water heater with 27-gallon buffered tank was also tested and compared against the gas baseline. All systems underwent testing simultaneously side-by-side under the criteria specified elsewhere in this report.

14 SOLAR ENERGY↗
DOE OSTI2017

Side-by-Side Testing of Water Heating Systems: Results from the 2013–2014 Evaluation

The Florida Solar Energy Center (FSEC) completed a fourth year-long evaluation on residential hot water heating systems in a laboratory environment (east central Florida, hot-humid climate). The evaluation studied the performance of five hot water systems (HWS) plus a reference baseline system for each fuel, (i.e., electric and natural gas). Electric HWS consisted of two residential electric heat pump water heaters (HPWHs, 60 and 80 gallons), a solar thermal system using a polymer absorber (glazed) collector with 80-gallon storage and a duplicate 50-gallon standard electric water heater with added cap and wrap insulation. Baseline performance data were collected from a standard 50-gallon electric water heater of minimum code efficiency to compare energy savings. Similarly, a standard 40-gallon upright vented natural gas water heater served as baseline for the natural gas fuel category. The latter, having a larger jacket diameter [18 in., with an energy factor (EF) of 0.62] with increased insulation, replaced a former baseline (17 in. diameter, EF = 0.59) that served during three previous testing rotations (2009–2013). A high-efficiency, condensing natural gas hybrid water heater with 27-gallon buffered tank was also tested and compared against the gas baseline. All systems underwent testing simultaneously side-by-side under the criteria specified elsewhere in this report.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗