Search NASA⌕ Search

SEARCH · Search NASA

Results for “simulation datasets”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,423 records · Page 79

A system for EPID-based real-time treatment delivery verification during dynamic IMRT treatment

Purpose: To design and develop a real-time electronic portal imaging device (EPID)-based delivery verification system for dynamic intensity modulated radiation therapy (IMRT) which enables detection of gross treatment delivery errors before delivery of substantial radiation to the patient.Methods: The system utilizes a comprehensive physics-based model to generate a series of predicted transit EPID image frames as a reference dataset and compares these to measured EPID frames acquired during treatment. The two datasets are using MLC aperture comparison and cumulative signal checking techniques. The system operation in real-time was simulated offline using previously acquired images for 19 IMRT patient deliveries with both frame-by-frame comparison and cumulative frame comparison. Simulated error case studies were used to demonstrate the system sensitivity and performance.Results: The accuracy of the synchronization method was shown to agree within two control points which corresponds to approximately ∼1% of the total MU to be delivered for dynamic IMRT. The system achieved mean real-time gamma results for frame-by-frame analysis of 86.6% and 89.0% for 3%, 3 mm and 4%, 4 mm criteria, respectively, and 97.9% and 98.6% for cumulative gamma analysis. The system can detect a 10% MU error using 3%, 3 mm criteria within approximately 10 s. The EPID-based real-time delivery verification system successfully detected simulated gross errors introduced into patient plan deliveries in near real-time (within 0.1 s).Conclusions: A real-time radiation delivery verification system for dynamic IMRT has been demonstrated that is designed to prevent major mistreatments in modern radiation therapy.

61 RADIATION PROTECTION AND DOSIMETRY↗

Mechanistic insights into mode of action of potent natural antagonists of BACE-1 for checking Alzheimer’s plaque pathology

Highlights: •Accumulation of Aβ plaques is one of the major pathology associated with Alzheimer’s disease. •Inhibition of β-Secretase or BACE-1 offers a viable prospect to check the growth of these plaques. •A large virtual dataset of natural compounds was screened against BACE-1. •Top two hits were analyzed for thermodynamic and structural stability using MD simulations. •Their detailed binding mode of actions were elucidated. -- Abstract: Alzheimer’s is a neurodegenerative disorder resulting in memory loss and decline in cognitive abilities. Accumulation of extracellular beta amyloidal plaques is one of the major pathology associated with this disease. β-Secretase or BACE-1 performs the initial and rate limiting step of amyloidic pathway in which 37–43 amino acid long peptides are generated which aggregate to form plaques. Inhibition of this enzyme offers a viable prospect to check the growth of these plaques. Numerous efforts have been made in recent years for the generation of BACE-1 inhibitors but many of them failed during the preclinical or clinical trials due to drug related or drug induced toxicity. In the present work, we have used computational methods to screen a large dataset of natural compounds to search for small molecules having BACE-1 inhibitory activity with low toxicity to normal cells. Molecular dynamics simulations were performed to analyze molecular interactions between the screened compounds and the active residues of the enzyme. Herein, we report two natural compounds of inhibitory nature active against β-secretase enzyme of amyloidic pathway and are potent lead molecules against Alzheimer’s disease.

60 APPLIED LIFE SCIENCES↗

Towards an Introspective Dynamic Model of Globally Distributed Computing Infrastructures

Large-scale scientific collaborations like ATLAS, Belle II, CMS, DUNE, and others involve hundreds of research institutes and thousands of researchers spread across the globe. These experiments generate petabytes of data, with volumes soon expected to reach exabytes. Consequently, there is a growing need for computation, including structured data processing from raw data to consumer-ready derived data, extensive Monte Carlo simulation campaigns, and a wide range of end-user analysis. To manage these computational and storage demands, centralized workflow and data management systems are implemented. However, decisions regarding data placement and payload allocation are often made disjointly and via heuristic means. A significant obstacle in adopting more effective heuristic or AI-driven solutions is the absence of a quick and reliable introspective dynamic model to evaluate and refine alternative approaches. In this study, we aim to develop such an interactive system using real-world data. By examining job execution records from the PanDA workflow management system, we have pinpointed key performance indicators such as queuing time, error rate, and the extent of remote data access. The dataset includes five months of activity. Additionally, we are creating a generative AI model to simulate time series of payloads, which incorporate visible features like category, event count, and submitting group, as well as hidden features like the total computational load—derived from existing PanDA records and computing site capabilities. These hidden features, which are not visible to job allocators, whether heuristic or AI-driven, influence factors such as queuing times and data movement.

kilic, Ozgur Ozan [Brookhaven National Laboratory ↗

Data and Code for: Observation-constrained agroecosystem model inversion reveals continental-scale variation of winter wheat traits

This repository contains the simulation outputs and processing scripts associated with the study of winter wheat traits across the United States, utilizing the Ecosys agroecosystem model. The dataset includes model results for both rainfed and irrigated winter wheat systems, supporting the findings presented in the manuscript titled "Observation-constrained agroecosystem model inversion reveals continental-scale variation of winter wheat traits." Data includes the original Ecosys simulation outputs (archived in .db format within the compressed .zip files) and extracted analysis data (stored in .pkl files for efficient processing). Python code for data processing and figure generation is provided in a Jupyter notebook. External Observational Datasets should refer to the following official repositories for the input and validation data used in this study. The eddy covariance data from the AmeriFlux network (https://ameriflux.lbl.gov/). Climate-forcing data of NLDAS-2 from NASA LDAS (https://ldas.gsfc.nasa.gov/nldas/nldas-2-forcing-data). Soil data from the Gridded Soil Survey Geographic Database (gSSURGO), available at (https://www.nrcs.usda.gov/resources/data-and-reports/gridded-soil-survey-geographic-gssurgo-database). Crop yields, planting and harvest dates from the USDA public databases (https://quickstats.nass.usda.gov/; https://webapp.rma.usda.gov/apps/actuarialinformationbrowser/CropCriteria.aspx). Satellite-derived SLOPE GPP data from ORNL DAAC (https://daac.ornl.gov/cgi-bin/dsviewer.pl?ds_id=1786). Land use and crop progress information from the USDA Crop Data Layer and Crop Progress and Condition Gridded Layers (https://www.nass.usda.gov/Research_and_Science/). The Ecosys model code is available online at https://github.com/jinyun1tang/ECOSYS.

Wheat↗

Estimating soil thermal inertia profiles from the passive equilibration of a temperature probe: Supporting data and code

This dataset contains measurements of soil thermal properties and organic layer thickness at three field sites on the Seward Peninsula near Nome, Alaska, USA. Code is also provided for estimation of soil thermal properties from temperature data, including a forward heat transfer mode, a library of simulated temperature data, and an inversion scheme to retrieve thermal properties from temperature measurements (Lamb et al., 2025). This dataset and code is published in support of the manuscript “Estimating Soil Thermal Inertia Profiles from the Passive Equilibration of a Temperature Probe” by Lamb et al., 2025, in which an experimental thermal measurements technique is developed. The dataset contains measurements of soil thermal properties and organic layer thickness obtained with an industry standard thermal properties analyzer and direct observations, as well as data generated using the experimental technique to estimate thermal properties and soil structure from temperature observations. All data is reported in .csv format, and all code is written in Python 3.7 using standard packages. This research was performed as a part of the NGEE Arctic project, which aims to advance model predictions of arctic carbon cycle responses to a changing climate over the 21st Century.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research. The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska. Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

End-of-Century Changes in Orographic Precipitation with the Intermediate Complexity Atmospheric Research Model over the Western United States

Abstract Downscaled precipitation projections were created using the Intermediate Complexity Atmospheric Research (ICAR) model over the western United States to increase the physical realism in orographic precipitation changes. End-of-century simulations from eight models in phase 5 of the Coupled Model Intercomparison Project (CMIP5) were downscaled with ICAR and compared to the widely utilized statistically downscaled dataset, localized constructed analogs (LOCAs), to understand where and why projections of cool-season (September–May) precipitation differed. ICAR and LOCA precipitation projections were similar, but their sign differed in hydrologically relevant regions likely due to ICAR’s simulation of microphysics and mesoscale dynamics with high-resolution topography (6 km). In the Pacific Northwest, cool-season precipitation projections from ICAR showed an increase on the windward side of the Cascades and no significant change within the lee. This difference between the windward and leeward side was attributed to reduced zonal wind speeds, allowing more time for microphysical processes within ICAR. This contrast is enhanced by rain’s faster fall speed compared to snow, limiting transport into the lee. Meanwhile, LOCA projected an increase in precipitation across the Cascades. In the Upper Colorado River basin, LOCA projected an increase in precipitation in high elevation regions (>3000 m), but ICAR projected no significant change or a decrease in precipitation. High elevation differences were most evident in the spring and fall and were also attributed to a snow-to-rain transition and dynamical processes that impacted orographic enhancement within ICAR. Idealized, controlled studies are needed to better isolate individual processes, but these results underscore the importance of including microphysics and mesoscale dynamics within regional-scale precipitation projections. Significance Statement A set of global climate model simulations was downscaled using an atmospheric model that contains key physical equations, referred to as Intermediate Complexity Atmospheric Research (ICAR). ICAR was used to examine projected changes in end-of-century cool-season precipitation over mountains in the western United States. Precipitation projections from ICAR were similar to projections that used statistical relationships to downscale climate projections. However, projections differed between ICAR and statistically downscaled datasets in whether they increased, decreased, or stayed the same in specific, hydrologically relevant regions such as the eastern Cascades and high elevation areas of the Upper Colorado River basin. These differences were attributed to the simulation of physical processes in ICAR. The results highlight the importance of kilometer-scale atmospheric processes in regional climate projections.

Currier, William Ryan [NOAA/Physical Sciences Labo↗

SU-F-T-384: Step and Shoot IMRT, VMAT and Autoplan VMAT Nasopharnyx Plan Robustness to Linear Accelerator Delivery Errors

Purpose: To identify the robustness of different treatment techniques in respect to simulated linac errors on the dose distribution to the target volume and organs at risk for step and shoot IMRT (ssIMRT), VMAT and Autoplan generated VMAT nasopharynx plans. Methods: A nasopharynx patient dataset was retrospectively replanned with three different techniques: 7 beam ssIMRT, one arc manual generated VMAT and one arc automatically generated VMAT. Treatment simulated uncertainties: gantry, collimator, MLC field size and MLC shifts, were introduced into these plans at increments of 5,2,1,−1,−2 and −5 (degrees or mm) and recalculated in Pinnacle. The mean and maximum doses were calculated for the high dose PTV, parotids, brainstem, and spinal cord and then compared to the original baseline plan. Results: Simulated gantry angle errors have <1% effect on the PTV, ssIMRT is most sensitive. The small collimator errors (±1 and ±2 degrees) impacted the mean PTV dose by <2% for all techniques, however for the ±5 degree errors mean target varied by up to 7% for the Autoplan VMAT and 10% for the max dose to the spinal cord and brain stem, seen in all techniques. The simulated MLC shifts introduced the largest errors for the Autoplan VMAT, with the larger MLC modulation presumably being the cause. The most critical error observed, was the MLC field size error, where even small errors of 1 mm, caused significant changes to both the PTV and the OAR. The ssIMRT is the least sensitive and the Autoplan the most sensitive, with target errors of up to 20% over and under dosages observed. Conclusion: For a nasopharynx patient the plan robustness observed is highest for the ssIMRT plan and lowest for the Autoplan generated VMAT plan. This could be caused by the more complex MLC modulation seen for the VMAT plans. This project is supported by a grant from NSW Cancer Council.

60 APPLIED LIFE SCIENCES↗

Cognitive simulation models for inertial confinement fusion: Combining simulation and experimental data

The design space for inertial confinement fusion (ICF) experiments is vast, and experiments are extremely expensive. Researchers rely heavily on computer simulations to explore the design space in search of high-performing implosions. However, ICF multiphysics codes must make simplifying assumptions, and thus deviate from experimental measurements for complex implosions. For more effective design and investigation, simulations require input from past experimental data to better predict future performance. In this work, we describe a cognitive simulation method for combining simulation and experimental data into a common, predictive model. This method leverages a machine learning technique called “transfer learning,” the process of taking a model trained to solve one task, and partially retraining it on a sparse dataset to solve a different, but related task. In the context of ICF design, neural network models are trained on large simulation databases and partially retrained on experimental data, producing models that are far more accurate than simulations alone. Here, we demonstrate improved model performance for a range of ICF experiments at the National Ignition Facility and predict the outcome of recent experiments with less than 10% error for several key observables. We discuss how the methods might be used to carry out a data-driven experimental campaign to optimize performance, illustrating the key product—models that become increasingly accurate as data are acquired.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Non-negative Tensor Factorization for Robust Exploratory Big-Data Analytics

Currently, large multidimensional datasets are being accumulated in almost every field. Data are: (1) collected by distributed sensor networks in real-time all over the globe, (2) produced by large-scale experimental measurements or engineering activities, (3) generated by high-performance simulations, and (4) gathered by electronic communications and socialnetwork activities, etc. Simultaneous analysis of these ultra-large heterogeneous multidimensional datasets is often critical for scientific discoveries, decision-making, emergency response, and national and global security. The importance of such analyses mandates the development of the next-generation of robust machine learning (ML) methods and tools for bigdata exploratory analysis.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Dynamical Downscaling of Earth System Model Data for Energy System Analysis

Assessing energy resources (e.g., solar, wind, and hydro) under future scenarios requires datasets with sufficient spatial and temporal detail to capture variability and extreme events. While global-scale Earth System Model (ESM) projections are widely used, their coarse resolution limits direct application to regional energy system analyses. Dynamical downscaling offers a robust approach to generate physically consistent, fine-scale datasets that better represent local atmospheric processes impacting energy resources. In this work, we present a two-stage approach for producing high-resolution historical and future projections over the contiguous United States (CONUS). First, we optimize the Weather Research and Forecasting (WRF) model configuration for energy-relevant variables - solar irradiance, wind speed, and precipitation - by conducting ERA5-driven simulations at 8-km and 28-km resolution. Multiple physics schemes and model configurations within the WRF are evaluated against observational datasets including the National Solar Radiation Database (NSRDB), the Parameter-elevation Regressions on Independent Slopes Model (PRISM), and the Stage IV multi-radar/multi-sensor precipitation product for the CONUS domain. Using the best-performing configuration, we dynamically downscale MPI-ESM1-2-HR simulations for 2000-2060 under SSP2-4.5 and SSP5-8.5 scenarios at 4-km spatial and hourly temporal resolution. This presentation will provide a comprehensive analysis of the results from multiple numerical experiments and high-resolution ESM projections. In addition, we will discuss potential applications of our high-resolution datasets within the energy sector and outline future research avenues dedicated to evaluating how extreme weather events influence system performance and resilience.

24 POWER TRANSMISSION AND DISTRIBUTION↗

How May the Choice of Downscaling Techniques and Meteorological Reference Observations Affect Future Hydroclimate Projections?

Abstract We present an intercomparison of a suite of high‐resolution downscaled climate projections based on a six‐member General Circulation Model (GCM) ensemble from Coupled Models Intercomparison Project (CMIP6). The CMIP6 GCMs have been downscaled using dynamical and statistical downscaling techniques based on two meteorological reference observations over the conterminous United States. We use the regional climate model, RegCM4, for dynamical downscaling, double bias correction constructed analogs method for statistical downscaling, and Daymet and Livneh datasets as the reference observations for statistical training and bias‐correction. We evaluate the performances of downscaled data in both historical and future periods under the SSP585 scenario. While dynamical downscaling improves the simulation of some performance evaluation indices, it adds an extra bias in others, highlighting the need for statistical correction before its use in impact assessments. Downscaled datasets after bias‐correction compare exceptionally well with observations. However, the choice of downscaling techniques and the underlying reference observations influence the hydroclimate characteristics of downscaled data. For instance, the statistical downscaling generally preserves the GCMs climate change signal but overestimates the frequency of hot extremes. Similarly, simulated future changes are sensitive to the choice of reference observations, particularly for precipitation extremes that exhibit a higher projected increase in the ensembles trained and/or corrected by Daymet than Livneh. Overall, these results demonstrate that multiple factors, including downscaling techniques and reference observations, can substantially influence the outcome of downscaled climate projections and stress the need for a comprehensive understanding of such method‐based uncertainties.

54 ENVIRONMENTAL SCIENCES↗

M3MS-19IN0101201 Modeling Intra-granular Fission Gas Bubble Evolution in Uranium Dioxide during Transients

The SciDAC milestone M3MS-19IN0101201 on Modeling Intra-granular Fission Gas Bubble Evolution in Uranium Dioxide during Transients was completed with the development, testing and implementation in the Bison code of an engineering model for intra-granular fission gas bubble evolution that is applicable to transient conditions. The model extends the previous work on modeling bubble evolution during normal operating conditions to include additional complexity that arises during transient situations, in particular, the so-called intra-granular bubble coarsening. This phenomenon refers to the appearance of a second population of coarsened bubbles with diameters of tens to hundreds of nm, which co-exist with nanometric bubbles present also during normal operating conditions and are responsible for a large portion of transient fuel swelling. The developed model considers growth of bubbles along dislocations, associated with the absorption of vacancies available in the dislocation core region, and favored by pipe diffusion of fission gas atoms. The model also utilizes the formulation for the gas atom resolution rate developed in the present SciDAC project through molecular dynamics simulations by other institutions. The present work therefore demonstrates an operational multiscale approach, based on collaboration within the project. The model is validated to an extensive experimental dataset of ramp tested uranium dioxide fuel samples. Additionally, we present new applications of the base model for bubble evolution during normal operating conditions to Bison engineering simulations of Cr2O3-doped fuel rods irradiated in the Halden reactor. Simulation results in terms of integral fission gas release in the fuel rods are compared to the available experimental data, pointing out significant improvements in the engineering predictions compared to application of the empirical model that was available in Bison before the SciDAC developments.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

A robust synthetic data generation framework for machine learning in high-resolution transmission electron microscopy (HRTEM)

Machine learning techniques are attractive options for developing highly-accurate analysis tools for nanomaterials characterization, including high-resolution transmission electron microscopy (HRTEM). However, successfully implementing such machine learning tools can be difficult due to the challenges in procuring sufficiently large, high-quality training datasets from experiments. In this work, we introduce Construction Zone, a Python package for rapid generation of complex nanoscale atomic structures which enables fast, systematic sampling of realistic nanomaterial structures and can be used as a random structure generator for large, diverse synthetic datasets. Using Construction Zone, we develop an end-to-end machine learning workflow for training neural network models to analyze experimental atomic resolution HRTEM images on the task of nanoparticle image segmentation purely with simulated databases. Further, we study the data curation process to understand how various aspects of the curated simulated data—including simulation fidelity, the distribution of atomic structures, and the distribution of imaging conditions—affect model performance across three benchmark experimental HRTEM image datasets. Using our workflow, we are able to achieve state-of-the-art segmentation performance on these experimental benchmarks and, further, we discuss robust strategies for consistently achieving high performance with machine learning in experimental settings using purely synthetic data. Construction Zone and its documentation are available at https://github.com/lerandc/construction_zone.

36 MATERIALS SCIENCE↗

Spatial heterogeneity effects on land surface modeling of water and energy partitioning

Abstract. Understanding the influence of land surface heterogeneity on surface water and energy fluxes is crucial for modeling earth system variability and change. This study investigates the effects of four dominant heterogeneity sources on land surface modeling, including atmospheric forcing (ATM), soil properties (SOIL), land use and land cover (LULC), and topography (TOPO). Our analysis focused on their impacts on the partitioning of precipitation (P) into evapotranspiration (ET) and runoff (R), partitioning of net radiation into sensible heat and latent heat, and corresponding water and energy fluxes. An initial set of 16 experiments were performed over the continental US (CONUS) using the E3SM land model (ELMv1) with different combinations of heterogeneous and homogeneous datasets. The Sobol' total and first-order sensitivity indices were utilized to quantify the relative importance of the four heterogeneity sources. Sobol' total sensitivity index measures the total heterogeneity effects induced by a given heterogeneity source, consisting of the contribution from its own heterogeneity (i.e., the first-order index) and its interactions with other heterogeneity sources. ATM and LULC are the most dominant heterogeneity sources in determining spatial variability of water and energy partitioning, mainly contributed by their own heterogeneity and slightly contributed by their interactions with other heterogeneity sources. Their heterogeneity effects are complementary, both spatially and temporally. The overall impacts of SOIL and TOPO are negligible, except TOPO dominates the spatial variability of R/P across the transitional climate zone between the arid western and humid eastern CONUS. Accounting for more heterogeneity sources improves the simulated spatial variability of water and energy fluxes when compared with ERA5-Land reanalysis dataset. An additional set of 13 experiments identified the most critical components within each heterogeneity source, which are precipitation, temperature, and longwave radiation for ATM, soil texture, and soil color for SOIL and maximum fractional saturated area parameter for TOPO.

54 ENVIRONMENTAL SCIENCES↗

Building Datasets and Training Methods for ML Based Magnet Quench Detection

Detecting quenches in superconducting (SC) magnets during training is a challenging process that involves capturing physical events that occur at different frequencies and appear as various signal features. These events may be correlated across instrumentation type, thermal cycle, and ramp. These events together build a more complete picture of continuous processes occurring in the magnet, and may allow us to flag potential precursors for quench detection. We present our work on building an automatic machine learning (ML) based quench detection system. We build upon our existing work on unsupervised auto-encoders for acoustic sensors and quench antenna (QA) by first establishing a supervised ML training pipeline. We show the results of an event tagging, analysis, and simulation framework on our QA and acoustic data which are used concurrently to build a training dataset for a supervised implementation. We then show how this supervised training can be used as a prior in a semi-supervised framework and compare this to the unsupervised neural network auto-encoder performance.This allows us to have a more concrete understanding of the performance of our algorithms relative to physical events occurring in the magnet, and also provides a baseline software tool to generically evaluate our quench prediction autoencoders under completely unsupervised, supervised, and semi-supervised training conditions.

Khan, Maira [Fermilab]↗

Uncertainty Quantification and Sensitivity Analysis for Simulation of Hostile Blast Events [Slides]

A mesh convergence study shows that shock arrival times converge for basic flow shots. Sensitivity study reveals a surprisingly high sensitivity to the placement of the HE package. Bayesian inference tools used on the experimental datasets suggest this is a real effect. Sensitivity analysis of ambient conditions such as temperature and pressure reveal minimal effect. Porting of pressure data from QUINOA simulations to perform structural analysis on the aeroshell geometry was successful, and shows a strong dependence of payload response to the angle of attack.

42 ENGINEERING↗

Representation and evolution of urban weather boundary conditions in downtown Chicago

Our study presents a novel computing technique for data exchange and coupling between a high-resolution weather simulation model and a building energy model, with a goal of evaluating the impact of urban weather boundary conditions on energy performance of urban buildings. The Weather Research and Forecasting (WRF) model is initialized with the operational High-Resolution Rapid Refresh (HRRR) dataset to provide hourly weather conditions over the Chicago region. We utilize the building footprint, land use, and building stock datasets to generate building energy models using EnergyPlus. We mapped the building exterior surfaces to local air nodes to import simulated microclimate data and to export buildings' heat emissions to their local environment. Preliminary experiments for a test area in Chicago show that predicted building cooling energy use differs by about 4.7% for the selected date when compared with simulations using TMY weather data and without considering the urban microclimate boundary conditions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

SIRIUS: Enabling Progressive Data Exploration for Extreme-Scale Scientific Data

Scientific simulations on high performance computing (HPC) platforms generate large quantities of data. To bridge the widening gap between compute and I/O, and enable data to be more efficiently stored and analyzed, simulation outputs need to be refactored, reduced, and appropriately mapped to storage tiers. However, a systematic solution to support these steps has been lacking in the current HPC software ecosystem. To that end, this paper develops SIRIUS, a progressive JPEG-like data management scheme for storing and analyzing big scientific data. It co-designs data decimation, compression, and data storage, taking the hardware characteristics of each storage tier into considerations. With reasonably low overhead, our approach refactors simulation data, using either topological or uniform decimation, into a much smaller, reduced-accuracy base dataset, and a series of deltas that is used to augment the accuracy if needed. The base dataset and deltas are compressed and written to multiple storage tiers. Data saved on different tiers can then be selectively retrieved to restore the level of accuracy that satisfies data analytics. Thus, SIRIUS provides a paradigm shift towards elastic data analytics and enables end users to make trade-offs between analysis speed and accuracy on-the-fly. This paper further develops algorithms to preserve statistics for data decimation, a common requirement for reducing data. Here, we assess the impact of SIRIUS on unstructured triangular meshes, a pervasive data model used in scientific simulations. In particular, we evaluate two realistic use cases: the blob detection in fusion and high-pressure area extraction in computational fluid dynamics.

97 MATHEMATICS AND COMPUTING↗