SEARCH · Search NASA

Results for “configuration management”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 541 records · Page 30

All sources NASA NTRS DOE OSTI

DOE OSTI2021

A New Era of Nuclear Criticality Experiments: The First 10 Years of Comet Operations at NCERC

Comet is a general-purpose, heavy-duty, vertical-lift assembly designed for flexibility in conducting a variety of critical experiments. It is currently located at the National Criticality Experiments Research Center (NCERC) in Nevada. In the past, Comet resided at Technical Area-18 in Los Alamos, New Mexico, as part of the Los Alamos Critical Experiments Facility (LACEF). The Comet assembly was relocated to NCERC in 2008 and became fully operational in June of 2011. The first critical experiment performed on Comet at NCERC was a verification of one of the previous configurations of the Zeus experiment series. Over the next 10 years, many additional experiments followed including other Zeus configurations as well as completely new designs. This paper discusses the Comet vertical-lift assembly, the transition from LACEF to NCERC, and a selection of experiments that have been performed on Comet during its first 10 years of operation at NCERC.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2024

Modeling the Effects of Artificial Drainage on Agriculture-dominated Watersheds using a Fully Distributed Integrated Hydrology Model: Datasets, scripts, model files

This model-data archive supports the research paper that demonstrates the integration of agricultural drainage features—specifically, narrow engineered ditches and tile drains—into a fully distributed, basin-scale integrated surface-subsurface hydrology model (ISSHM), Amanzi-ATS. The model employs innovative computational meshes aligned with agricultural ditches and incorporates the physically based Hooghoudt's drainage equation to simulate tile drainage, offering a novel strategy that enhances the accuracy of hydrological simulations.The archived dataset includes input parameters, model configurations, and select simulation outputs for the Amanzi-ATS model that successfully captured the streamflow patterns in the Portage River Watershed as validated by USGS gauge readings. Jupyter notebook for the preparation of model inputs and post-processing of outputs are also included. The model's predictive performance achieved a normalized Kling-Gupta Efficiency (KGE) of 0.81, surpassing SWAT without the necessity for site-specific calibration.The Amanzi-ATS model presented in this modeL-data archive allows for numerical experiments to explore the shifts in the flow structure under different drainage scenarios. As a tool for advancing the understanding of distributed hydrological responses and nutrient cycling, this archived model provides valuable insights for researchers, modelers, and decision-makers involved in watershed management and environmental modeling.The Watershed Workflow package is implemented in Python3. The Jupyter notebooks can be executed through multiple open-source tools, for example, Anaconda Jupyter Lab, VS Studio Code, etc. Other data files include CSV and HDF5 files, which can be read through Python scripts. The input files for the ATS model, open-source integrated hydrology, and transport model, are in XML format and can be edited in any commonly used text editors.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2021

Reburning of Animal Waste Based Biomass with Coals for NO x Reduction, Part II: Dairy Biomass (DB) and Coal–DB Blends

Concentrated animal feeding operations (both slaughter and dairy cattle) lead to land, water, and air pollution if waste storage and handling systems are not effectively managed. At the same time, cattle biomass (CB), which includes both slaughter/feedlot biomass (FB) and dairy biomass (DB), have the potential to be a source of green energy at coal-fired power plants. Part I presented results on NOx reductions with pure FB or Coal: FB blends as reburn fuels. Part II deals with results from reburning with pure DB or Coal: DB blends as reburn fuels. A mixture of NG with a small amount of NH3 was used to generate the baseline NOx of 400–420 ppm (or 185–194 g/GJ). NOx emissions were found to be reduced by as much as 96% when reburning with FB. The effects of reburn fuel type, equivalence ratio (ER RBZ ) in the reburn zone, vitiated air, several injection configurations of reburn fuel and initial NO concentrations on NOx emissions were investigated. The ER RBZ shows a significant effect on the NOx reduction. The 20% heat input by reburning was the better operating condition for the long-term operation due to its ash production. The results reveal that reburn with DB fuels is an effective technology for NOx emission control when the initial NOx emission is higher than 275 ppm (or 127 g/GJ or 0.3 lb/MMBtu).

09 BIOMASS FUELS↗

DOE OSTI2021

Thermal and Mechanical Design of a High-Voltage Power Electronics Package

This paper presents the thermal and mechanical design aspects of a power electronics package with 5-kV GaN devices. Through finite element analyses, we investigated the impact of different cooling configurations and device locations on the thermal performance and reliability of the package. We found that placing the devices closer to the direct bond copper substrate as opposed to a centered approach in the proposed double-sided cooling configuration resulted in improved heat dissipation. This approach also reduced the total number of attachment layers, thereby likely improving the reliability of the package. Furthermore, simulations revealed that the device location had a negligible impact on the thermomechanical behavior of the attachment layers, as they are more prone to the local coefficient of thermal expansion mismatch.

electronic packaging↗

DOE OSTI2020

Decontamination of urban surfaces contaminated with radioactive materials and consequent onsite recycling of the waste water

Enhancing rapid remediation strategies is paramount for recovery after a large-scale nuclear contamination event in an urban environment. Some current strategies recommend use of readily available equipment, materials, and facilities to expedite recovery. For example, applying pressurized water to contaminated surfaces may effectively remove radioactive contamination. In this study, a commercial power washer removes soluble forms of 152 Eu 3 + , 85 Sr 2 + , and 137 Cs + contamination from common porous building materials, and computer simulations characterize the recycling of the resultant contaminated wash water. Pressure washing the porous building materials under spray conditions typical with do-it-yourself units improved decontamination factors (DFs) for 152 Eu compared to low-pressure application of tap water (majority of two-tailed t-test p-values < 0.1), but pressure did not improve DFs for 137 Cs or 85 Sr. For both pressurized and low-pressure applications, adding potassium ions (K + ) to promote ion exchange reactions produced significantly higher DFs for tested radionuclides on asphalt, brick, and concrete. The resultant contaminated wash water can be processed through self-prepared chemical filtration beds of clay and sand. Modeled in a prior study, the beds yielded linear trends (R 2 > 0.98) in sensitivity analyses between most bed configuration variables and bed performance variables, permitting flexible ad-hoc bed design. The experimental and simulation results led to estimates of the remediation rate and waste generated after cleaning 250 m 2 of cesium-contaminated concrete from the combined deployment of a power washer and two different mobile treatment beds. Furthermore, the first treatment bed was designed to reduce treatment time and processed 1900 L of wash solution in 70 min using 880 kg of clay/sand infill material. Designed to reduce the solid waste generated, the second bed processed the same solution volume in 1040 min (17 h) using 170 kg of clay/sand infill material. The results of this analysis warrant further investigation of power washing with recycled salt solution as an effective rapid decontamination method with manageable waste.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2023

An Integrated Traction Drive with a High Speed Surface Permanent Magnet External Rotor Motor for Electric Vehicles

This work presents the detailed design of an outer rotor motor and integrated drive for passenger electric vehicles. The motor uses non-heavy rare earth permanent magnets and operates at a top speed of 20,000rpm. In-slot ceramic heat exchangers are utilized for the cooling of the windings. The external rotor configuration of the motor leaves a significant amount of space in the stator bore, which is leveraged for the integration of the power electronics. A high speed external rotor motor creates a number of mechanical, thermal and assembly challenges and this work presents detailed thermal, mechanical and rotordynamics analysis.

ADVANCED PROPULSION SYSTEMS↗

DOE OSTI2020

Crystalline Silicotitanate Batch Contact Testing with Ba, Ca, Pb, and Sr

Washington River Protection Solutions is working to support initial production of immobilized low-activity waste (LAW) by feeding Hanford tank supernate from tank farms to the Hanford Waste Treatment and Immobilization Plant (WTP) LAW Facility. This goal incorporates the design of a Tank Side Cesium Removal system, which filters tank waste supernate to remove suspended solids and then removes Cs by processing it through crystalline silicotitanate (CST) ion exchange media manufactured by Honeywell UOP, LLC. The 137 Cs-depleted product is intended to be sent to the WTP for vitrification. Processing of actual tank waste supernate showed effectively complete uptake of Sr and Ba by CST and significant uptake of Ca. Further, Campbell et al. (2019) analyzed CST post-column testing and found significant (>1E-2 mmoles/g) uptake of Ca and Pb along with some Ba, Cd, Fe, Sr, and U. This led to concern that selected metals, particularly the +2 cations, Ca, Sr, Ba, and Pb may be consuming Cs exchange sites and possibly reducing CST capacity for Cs. Exchange of +2 cations was assumed to be associated with the M(OH) + species for the metal (M) ion in the caustic solution. A series of batch contact testing was conducted to evaluate the exchange behavior of Ba, Ca, Pb, and Sr onto CST. The Cs exchange behavior was also tested as a benchmark for direct comparisons. The CST was provided in the sodium form by Honeywell UOP, as IONSIV TM R9140-B, Lot 2002009604, 18 x 50 mesh. A <30-mesh aliquot was collected to match the sieve fraction expected for use in upcoming small column test configurations. Kinetic exchange rate and isotherms were measured at metal concentrations benchmarked from the AP-107 tank waste feed composition and as limited by the metal solubility in the alkaline solution. Two simplified matrices were tested: 1) 1.0 M NaOH/4.6 M NaNO 3 and 2) 0.1 M NaOH/5.5 M NaNO 3 ; these matrices represented the expected 5.6 M Na concentration of process feed and served to address the hydroxide concentration effect on exchange behavior.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2022

Feasibility Study of Millimeter Wave Radars For Safeguards Applications

Containment and surveillance are fundamental measures in nuclear safeguards. Techniques such as video surveillance and laser curtain for containment provide effective monitoring in areas where maintaining continuity of knowledge is required. These systems, however, can be susceptible to loss of monitoring capabilities under certain environmental conditions such as poor visibility (i.e. low light conditions, smoke, fog, etc.) or extended power loss past the duration that the backup power system is designed for. Brookhaven National Laboratory has been investigating the feasibility of millimeter waves (mmWave) as a new perimeter seal in which radio frequency waves in the range of 60-64 GHz are used to detect and monitor objects of interest. Signals in this frequency range are not susceptible to environmental conditions. For proof-of-concept tests, mmWave sensors from Texas Instruments (TI), specificallyIWR6843, are used in a test bed at BNL's Waste Management facility to simulate the operations at nuclear facilities. The unique design of TI mmWave sensors requires less memory and power consumption compared to counterpart systems. These devices are capable of exporting 3D point-cloud data, which is visualized graphically and compared to videos recorded at the same time to validate the performance of the mmWave sensor. A set of experiments were planned to test the feasibility of the mmWave in this application, including monitoring static containers in a storage area and detecting intrusions at the boundaries of the area. In addition, the experiments also identify potential blind spots relative to sensor position and utilize multiple operating sensors simultaneously to reduce or eliminate such blind spots. The optimal positioning of multiple sensors was determined for the experimental room configuration. In this paper, we will discuss the details of this novel perimeter sealing concept and present the test results.

98 NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL P↗

DOE OSTI2025

LCLS Big Data Handling – How I Learned to Stop Worrying and Love the Data Deluge

Advanced data and computing systems are vital to Linac Coherent Light Source (LCLS) operations, data interpretation and overall scientific productivity. The transition to MHz-era operation marks a fundamental change in scale that requires new infrastructure and architectures to link LCLS to the required scale of computing needed for scientific interpretation. The LCLS-II Data System meets big data challenges by implementing configurable data reduction that can adapt to multiple science areas, real-time analysis frameworks to provide visualization and fast feedback, and the ability to transfer data to local and remote computational facilities for near real time analysis at the appropriate scale. Feature extracted information generated in the data analysis pipeline - at the edge, local compute, or remote High-Performance Computing (HPC) resources - can be used to steer experiments and inform user decisions during beam time. Artificial Intelligence and Machine Learning (AI/ML) techniques present new opportunities to rapidly analyse large datasets and direct experiments, but create new challenges in scaling, adaptability, complexity, and trustworthiness. We describe how the LCLS-II Data System architecture addresses its data-driven challenges in the areas of data acquisition, data processing, data management, and workflow orchestration to decrease the overall time-to-science and provide a vision for future developments.

artificial intelligence↗

DOE OSTI2024

UMap: An application-oriented user level memory mapping library

Exploiting the prominent role of complex memories in exascale node architecture, the UMap page fault handler offers new capabilities to access large memory-mapped data sets directly. UMap provides flexible configuration options to customize page handling to each application, including analysis of massive observational and simulation data sets. The high-performance design features I/O decoupling, dynamic load balancing, and application-level controls. Page faults triggered by application threads and processes accessing data mapped to a UMapp’ed region are handled via the Linux userfaultfd protocol, an asynchronous message-oriented kernel-user communication mechanism that avoids the context switch penalty of traditional signal fault handlers. UMap is fully open source. In this paper, we give an overview of the UMap library architecture, its extensible plugin architecture, and the use/performance of UMap in emerging heterogeneous memory hierarchies such as near-node Non-volatile Memory (NVM) and network attached memories. We highlight new capabilities in two pagefault management plugins, the NetworkStore and SparseStore. We demonstrate the integration between UMap and multiple ECP products including Caliper, Metall, ZFP, Mochi, and Ripples.

97 MATHEMATICS AND COMPUTING↗

DOE OSTI2021

Critical and Californium Source-Driven Noise Analysis Subcritical Measurements with an Unreflected Cylindrical Tank of Mixed Uranium-Plutonium Nitrate Solution

The reported experiments were performed by Oak Ridge National Laboratory (ORNL) at the Battelle Northwest Laboratory’s (now Pacific Northwest National Laboratory) critical experiments facility at Hanford, Washington in 1981 and used 16 days of critical facility time, not including 10 days for setup and removal of ORNL equipment. These measurements were to assess the capability of the Cf source-driven noise analysis (CSDNA) method to measure the subcriticality (keff) of mixed U-Pu nitrate solutions. In addition to the CSNDA measurements, measurements were also performed near delayed criticality where CSDNA measurements cannot be performed. This report documents the experiments that were not reported at that time by presenting the ORNL experimental results and any online analysis performed during and shortly after the measurements. The mixed nitrate solution had a U concentration at 188 grams per liter (g/L), a Pu concentration of 280 g/L, free acid normality of 2.80, H ion molarity of 5.9 and a specific gravity of 1.754 g/cm 3 , a 240 Pu isotopic content of 7.981 wt. %, and a 235 U isotopic content of 0.724 wt. %. The stainless-steel tank for the solution had an inside diameter of 35.38 cm, an outside diameter of 35.53 cm, a height of 56.72 cm, and bottom thickness of 0.9525 cm. A Zircaloy pipe with a 3.1496 cm outside diameter, a 2.7788 cm inside diameter, and bottom thickness of 0.635 cm was available for insertion of the Cf source in the center of the fissile solution. The Cf source was also located at the outside surface of the tank (solution height varied from 10 to 53 cm) and in the center of the solution (solution height varied from 10 to 60.7 cm). The CSDNA measurements were not analyzed online to determine the subcritical neutron multiplication factors. At all subcritical states, the break frequency noise analysis data was fitted to obtain the prompt neutron decay constant. The neutron multiplication factors were determined for the two configurations of the measurements near delayed criticality. The subcritical neutron multiplication factors from the CNSDA measurements can be obtained with further analysis. However, the near delayed critical configuration, the prompt neutron decay constants, the count rates, and the measured cross and auto power spectral densities can be calculated directly for benchmarking. Much of data presented in this report are from ORNL notes—not in the ORNL logbooks. For the final benchmark analysis, the data from the Battelle Northwest Laboratory (which operated the critical facility in 1981) critical facility logbook should be consulted and be incorporated where appropriate. The purpose of this report is to document the experimental information for the measurements performed so that at a later date researchers could perform the required uncertainty and calculational analyses and documentation to use these data for an International Criticality Safety Benchmark Program (ICSBEP) or Nuclear Energy Agency benchmark. The data from these measurements are available from the ORNL Records Management Services Department, and the logbook is available from ICSBEP at Idaho National Laboratory. Preparation of the present report is part of a larger cooperative effort between Idaho National Laboratory (INL) and Oak Ridge National Laboratory (ORNL) to document more than 15 undocumented critical and subcritical experiments enumerated in ORNL/TM-2019/18 and performed by ORNL at ORCEF and other USDOE critical experiments facilities using more than 500 operational days of critical facility time.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2023

An Alternating, Current-Induced Electromagnetic Field for Membrane Fouling and Scaling Control during Desalination of Secondary Effluent from Municipal Wastewater

Membrane treatment of secondary effluent for reuse applications is a promising approach to expand water supplies and provide flexibility to water resources management. However, effective control of membrane fouling and scaling is crucial for cost-effective treatment and system resilience. This study compared the performance of antiscalants to an alternating, current-induced electromagnetic field (EMF) as an alternative pretreatment method to reverse osmosis. Compared to the no-EMF control experiments, the EMF device resulted in 13% higher water recovery and 366% lower flux decline at 60% of water recovery, along with 2–8 times lower precipitation of fouling and scaling, as evidenced by scanning electron microscope, energy dispersive X-ray spectroscopy, and chemical extraction analysis. The combination of the EMF with antiscalant was more effective for reducing membrane fouling and scaling, increasing water recoveries up to 89.3%, as compared to the EMF (67.5%) and antiscalant-only (73.6%) configurations. This is the first study to demonstrate synergistic effects of using an EMF in combination with antiscalants and could lead to lower pretreatment costs. Additional research is required to quantify the economics of this approach and to fully understand the fundamental mechanisms governing fouling and scaling control by an EMF.

42 ENGINEERING↗

DOE OSTI2023

Stand validation of lidar forest inventory modeling for a managed southern pine forest

We evaluated area-based approaches (ABAs) to light detection and ranging (lidar) predictions of plot- and stand-level forest attributes (tree count, height, basal area, volume, aboveground biomass, broadleaf/conifer, and diameter at breast height — “diameter”). ABA methods included post-stratification (PS), ordinary least squares (OLSs) regression, k nearest neighbors ( kNN), and random forest (RF). This study was conducted on the Savannah River Site in South Carolina, USA. Plot- and stand-level predictions were validated against fixed-radius 0.04 ha (0.1 acre) plots in 49 ≈2.0 ha (5 acre) stands. Our findings demonstrate that lidar can be incorporated operationally into forest inventory systems to provide stand-level inferences for a wide range of forest attributes. Volume predictions for specific diameter classes, however, often fared poorly (root mean squared error (RMSE) > 100%) for the methods we explored, especially for larger (less common) diameter trees. Stand-level results were consistently better than pixel-level results (10–200+ percentage points). kNN and RF performed similarly and better than OLS and PS, but RF was the most robust to model configurations, while kNN has practical advantages such as simultaneous predictions of many attributes.

Forestry↗

DOE OSTI2022

Economic viability of using thermal energy storage for flexible carbon capture on natural gas power plants

Fossil fuel-based power plants generate 80% of the electricity in the United States and provide a reliable generation source for both base and peak power demands. These plants are expected to adapt to changes in environmental policies that will require carbon management with carbon capture and storage (CCS) representing a possible solution. Current solvent-based CCS has a detrimental impact on a power plant's performance due to large heat loads required for carbon capture solvent regeneration. This parasitic load restricts the power plant's output and operation flexibility. Therefore, this study evaluates the feasibility of using thermal storage technologies for natural gas combined cycle (NGCC) power plants coupled with CCS to minimize the impact of solvent regeneration and enable the plant to operate at peak power output. Thermal storage can minimize the impact of CCS on the power plant by providing the heat load required for solvent regeneration during times of peak demand which will allow the plant to operate unrestricted and at full power. In total, fifteen unique thermal storage configurations were evaluated from three thermal storage categories: Brayton cycle heat pump, vapor compression heat pump, and heat recovery steam generator steam extraction for storage. The viability of these systems was determined by evaluating each configuration on thousands of real-world Locational Marginal Pricing (LMP) profiles from the New York Independent System Operator and California Independent System Operator electricity markets using a techno-economic analysis. Afterwards, results were compared to the performance of a base power plant (NGCC with CCS and no thermal storage) to determine the impact of thermal storage on power plant economics. Overall, six of the thermal storage configurations performed better than base CCS enabled power plant on between 11.5% and 38.7% of the LMP signals evaluated. The best performing configuration was a vapor compression heat pump that used flue gas as the working fluid and had both hot and cold thermal storage units. This configuration performed better than the base CCS power plant on 38.7% of the LMP profiles. The results of this study show thermal storage can mitigate the economic impact of carbon capture solvent regeneration on NGCC power plants. Discussion focuses on the impact of electricity pricing on the optimal thermal storage system, the advantages and disadvantages of the systems evaluated, and identifies limitations with the study.

25 ENERGY STORAGE↗

DOE OSTI2025

Corral Summit Pumped Storage Hydropower Hybrid: Site Suitability Assessment

In 2024, Idaho National Laboratory (INL) and Pacific Northwest National Laboratory (PNNL) initiated a technical-assistance project to support Cat Creek Energy, LLC, (CCE) in evaluating site suitability for the proposed Corral Summit Pumped Storage Hydropower (PSH) project in south-central Idaho near Mackay Reservoir. The Corral Summit facility incorporates battery storage and photovoltaic (PV) solar arrays in a Trybrid configuration to deliver large-volume long-duration (LVLD) storage solutions for rural electric cooperatives in eastern Idaho. The evaluation process focused on determining the most-suitable location for the upper reservoir of the PSH system, guided by a comprehensive assessment framework spanning multiple categories, including physical characteristics, environmental constraints, building infrastructure, regulatory constraints, cultural resources and sensitivity, social factors, and power market and grid integration. Each site was analyzed based on a ranking scale (0–1), which scores ranging from “severely disfavored” to “highly favored,” allowing detailed comparisons of site-specific conditions. Categories such as hydraulic head, utilities corridor, land ownership, and transmission-grid limitations emerged as key contributors to the overall assessment. Site 2 (Idaho Trust) demonstrated a slight advantage over Site 1 (Bureau of Land Management) primarily due to favorable outcomes in regulatory constraints, building infrastructure, and power-market integration. However, Site 1 outperformed Site 2 in factors related to physical characteristics and social factors. The report emphasizes the need for further evaluation of both sites before clear determination of which site is preferred, due to the limited information available on either site at the time of this report. Key areas of evaluation to clearly define the preferred site are ecological impacts, cultural-resource surveys, and economic-feasibility assessments. For successful project execution, recommended follow-up actions include seismic and geotechnical surveys, groundwater and habitat monitoring, regulatory reviews of water rights and right-of-way agreements, cultural engagement with local tribal governments, and enhanced stakeholder strategies. These efforts will ensure the Corral Summit Trybrid facility meets local energy needs while balancing environmental, social, and regulatory responsibilities.

13 - HYDRO ENERGY↗

DOE OSTI2025

Corral Summit Pumped Storage Hydropower Hybrid Site Suitability Assessment (Rev.1)

13 - HYDRO ENERGY↗

DOE OSTI2023

A Threshold Helium Leakage Detection Switch with Ultra Low Power Operation

Detecting helium leakage is important in many applications, such as in dry cask nuclear waste storage systems. This work develops a helium detection system based on the relative permittivity (dielectric constant) difference between air and helium. This difference changes the status of an electrostatic microelectromechanical system (MEMS) switch. The switch is a capacitive-based device and requires a very negligible amount of power. Exciting the switch’s electrical resonance enhances the MEMS switch sensitivity to detect low helium concentration. This work simulates two different MEMS switch configurations: a cantilever-based MEMS modeled as a single-degree-freedom model and a clamped-clamped beam MEMS molded using the COMSOL Multiphysics finite-element software. While both configurations demonstrate the switch’s simple operation concept, the clamped-clamped beam was selected for detailed parametric characterization due to its comprehensive modeling approach. The beam detects at least 5% helium concentration levels when excited at 3.8 MHz, near electrical resonance. The switch performance decreases at lower excitation frequencies or increases the circuit resistance. The MEMS sensor detection level was relatively immune to beam thickness and parasitic capacitance changes. However, higher parasitic capacitance increases the switch’s susceptibility to errors, fluctuations, and uncertainties.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2023

RAPIDS: Reconciling Availability, Accuracy, and Performance in Managing Geo-Distributed Scientific Data

In modern science, big data plays an increasingly important role. Many scientific applications, such as running simulations on supercomputers or conducting experiments on advanced instruments, produce huge amount of data at unprecedented speed. Analyzing and understanding such big data is the key for scientists to make scientific breakthroughs. However, data might become unavailable for scientists to access when outages or maintenance of the storage system occur, which severely hinders scientific discovery. To improve the data availability, data duplication and erasure coding (EC) are often used. But as the scientific data gets larger, using these two methods can cause considerable storage and network overhead.In this paper, we propose RAPIDS, a hybrid approach that combines the multigrid-based error-bounded lossy compression with erasure coding, to significantly reduce the storage and network overhead required for maintaining high data availability. Our experiments show that RAPIDS reduces the storage overhead by up to 7.5x and network overhead by up to 3x to achieve the same level of availability compared to the regular EC method. We improve RAPIDS by building two models to optimize the fault tolerance configurations and data gathering strategy. We demonstrate that RAPIDS significantly improves performance when running on many CPU cores in parallel or on GPUs.

Wan, Lipeng↗