Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,099 records · Page 61
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Improving and testing machine learning methods for benchmarking soil carbon dynamics representation of land surface models

Representation of soil organic carbon (SOC) dynamics in Earth system models (ESMs) is a key source of uncertainty in predicting carbon climate feedbacks. The magnitude of this uncertainty can be reduced by accurate representation of environmental controllers of SOC stocks in ESMs. In this study, we used data of environmental factors, field SOC observations, ESM projections and machine learning approaches to identify dominant environmental controllers of SOC stocks and derive functional relationships between environmental factors and SOC stocks. Our derived functional relationships predicted SOC stocks with similar accuracy as the machine learning approach. We used the derived relationships to benchmark the coupled model intercomparison project phase six ESM representation of SOC stocks. We found divergent environmental control representation in ESMs in comparison to field observations. Representation of SOC in ESMs can be improved by including additional environmental factors and representing their functional relationships with SOC consistent with observations.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

U.S. Solar Photovoltaic System and Energy Storage Cost Benchmarks, With Minimum Sustainable Price Analysis: Q1 2022

NREL's bottom-up cost models can be used to assess the minimum sustainable price (MSP) and modeled market price (MMP) of PV and storage systems having various configurations. MSP can be used to estimate future potential cost-reduction opportunities for PV and PV-plus-storage systems, thus helping guide research and development aimed at advancing cost-effective system configurations. MMP can be used to understand the cost of systems under recent market conditions. The MSP data in this annual benchmarking report will be used to inform the formulation of, and track progress toward, the Solar Energy Technologies Office's Government Performance and Reporting Act cost targets.

14 SOLAR ENERGY↗
DOE OSTI2022

Scalable Gaussian Processes, GPyTorch Application Benchmarking, and Targeted Adaptive Design (TAD) on ThetaGPU

We aim at showcasing the scalability of Gaussian Process (GP). The naive GP implementation scales cubically with data size, which can be prohibitive, so GP has not heretofore been considered suitable for very large-scale problem settings. We take advantage of GPyTorch, a library for scalable GPs built on top of PyTorch that incorporates GPU acceleration. With GPyTorch, one can achieve nearly linear scaling with structured kernel interpolation (SKI) and constant-time predictive covariances computation with LanczOs Variance Estimates (LOVE) while preserving accuracy. We also take advantage of the computational power of ThetaGPU, a supercomputer of Argonne Leadership Computing Facility (ALCF). In addition, we implement a scalable, GPU-ready version of Targeted Adaptive Design (TAD), a GP-based data-driven algorithm that efficiently searches the control space of an advanced manufacturing experiment for settings capable of producing a required design within a specified tolerance, despite the poorly known mapping from control settings to design. We finally show our benchmarking for GPyTorch and TAD performance on CPU vs. ThetaGPU and discuss the results and implications.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2021

Flattop Benchmark CED-2 (IER-423)

The final design for the Flattop HEU Core Benchmark is presented in this report. Physical measurements will be performed on Flattop components using modern instruments and techniques to define uncertainties necessary to meet contemporary ICSBEP standards. A preliminary re-evaluation of ICSBEP HMF-0028 (Flattop HEU Core) led to the development of IER-423 CED-1, wherein total relative uncertainties in $k_{eff}$ were provided based on a new detailed model. The estimated uncertainty on the total uncertainty is +100%/-0%, given a lack of documented uncertainties on mass, dimensions, and densities of predominantly the natural uranium (NU) and highly enriched uranium (HEU) components.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Uncertainty Analysis for a Criticality Benchmark [Slides]

This presentation discusses how KRUSTY was analyzed with LANL’s MCNP neutron transport code, Monte Carlo N-Particle. In the Monte Carlo method, neutrons are born in fission events and fly around the geometry having events (e.g., scatter, capture, fission, escape) according to the natural probabilities. “Natural probabilities” are given as neutron cross sections , which are measured and evaluated nuclear data. One of the goals of the benchmark project is to improve the nuclear cross sections (and the codes that use them). In this work, the response of interest is called k eff , which describes the neutron multiplication in a near critical system. Like most quantities, k eff is not measured directly; it is only inferred. In summation, MCNP results for k eff matched the measurements extremely well. The evaluated uncertainties are asymmetric and smaller than 0.00100 (e.g., + 0.00080 / – 0.00062). The largest uncertainties are due to alignment.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

IER - 551: Experiment for Unresolved Resonance of Plutonium Actinides (EUROPA): CED-1 for True Intermediate Pu Benchmark

IER-551 or the Experiment for Unresolved Resonance Of Plutonium Actinides (EUROPA), is an experiment campaign to design and execute a series of true intermediate energy plutonium critical configurations. EUROPA is a collaboration between Los Alamos National Laboratory (LANL), Oak Ridge National Laboratory (ORNL), and the French Institut de Radioprotection et de Sûreté Nucléaire (IRSN) to validate current and future evaluations of 239,240 Pu isotopes. Preliminary designs were optimized using Particle Swarm Optimization to save on computation time and the configuration with highest percentage of intermediate neutrons causing fission found was BeO-Cd. This combination has over 70% of fissions occurring from intermediate energy neutrons. Secondary considerations regarding end benchmark uncertainties and quality of nuclear data resulted in a beryllium metal only experiment proposed as the final system. This final preliminary design achieves 68% intermediate fissions, sensitivities of fission and capture cross sections of 239 Pu of 0.3021 and -0.1610 respectively.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Benchmarking the PCMCI Causal Discovery Algorithm for Spatiotemporal Systems

Causal discovery algorithms construct hypothesized causal graphs that depict causal dependencies among variables in observational data. While powerful, the accuracy of these algorithms is highly sensitive to the underlying dynamics of the system in ways that have not been fully characterized in the literature. In this report, we benchmark the PCMCI causal discovery algorithm in its application to gridded spatiotemporal systems. Effectively computing grid-level causal graphs on large grids will enable analysis of the causal impacts of transient and mobile spatial phenomena in large systems, such as the Earth’s climate. We evaluate the performance of PCMCI with a set of structural causal models, using simulated spatial vector autoregressive processes in one- and two-dimensions. We develop computational and analytical tools for characterizing these processes and their associated causal graphs. Our findings suggest that direct application of PCMCI is not suitable for the analysis of dynamical spatiotemporal gridded systems, such as climatological data, without significant preprocessing and downscaling of the data. PCMCI requires unrealistic sample sizes to achieve acceptable performance on even modestly sized problems and suffers from a notable curse of dimensionality. This work suggests that, even under generous structural assumptions, significant additional algorithmic improvements are needed before causal discovery algorithms can be reliably applied to grid-level outputs of earth system models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Generalized Bayesian Framework for Evaluation of Integral Benchmark Experiments [Slides]

This presentation is on generalized Bayesian framework for evaluation of integral benchmark experiments. This presentation starts off with assumptions and approximations used with Bayes Theorem. then an overview of approximations used by ORNL codes, and Generalized Bayesian Monto Carlo (GBMC). The presentation then details out a precise framework. This presentation then concludes with considerations.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Reservoir Thermal Energy Storage Benchmarking (Rev. 3)

A benchmarking analysis of RTES research funded by GTO through the Beyond Batteries projects was conducted against the ESGC to see where they fit within the identified ESGC Use Cases. The projects were found to advance knowledge in multiple ESGC use cases, either directly or in some cases, indirectly as enabling technologies. This analysis is helpful to understand where RTES and associated research fits into the larger discussion around energy storage technologies. Also, a retrospective analysis of the Beyond Batteries projects was conducted to evaluate what the projects learned and how the results can be applied to advance the value of RTES. Major results of each of the studies are summarized in Table 2. Additionally, a comparative metrics analysis for RTES was completed to understand where RTES lies within the energy storage industry. Metrics for evaluation of RTES and its comparison to other storage technologies were selected and ranges of their values compiled. The selected metrics – LCOE (levelized cost of energy), capital costs, roundtrip efficiency, energy storage capacity, and storage time – were chosen based on data availability and have a particularly strong influence on the potential deployment of a storage technology. Charts which compare the metrics are presented in section 4.3 and show ranges for each of the 10 selected technologies. However, due to a lack of domestic operational facilities, values for RTES and for portions of the remaining technologies are based on theoretical modeling and studies of best-case scenarios. LCOE estimates for RTES fall within the lower reaches of Figure 15, but nevertheless amount to 2 – 5 times the ESGC Roadmap goal for LCOE, for example in the Facilitating and Evolving Grid Use Case. Capital costs for RTES sit on the higher end (Figure 16) but are expected to decrease as new projects are developed and the technology is refined. The theoretical roundtrip efficiency reported for RTES varies from mid to high percentages (Figure 17) with efficiencies upwards of 93% in modeled scenarios in the Portland Basin (Bershaw et al.,2020). RTES is also expected to have the largest energy storage capacities and longest storage times, likely matched only by lower efficiency hydrogen storage. To better assess the role that RTES could play in energy storage we examined it’s potential in the U.S. The potential depends on many factors. Recently, many researchers have started looking at deep sedimentary basins, depleted oil and gas fields, and basalt formations as potential targets for RTES development. The United States Geological Survey (USGS) has analyzed various cities and shown substantial RTES potential in the cooling sector (Pepin et al., 2021). By modeling RTES in low-quality groundwater (e.g., brackish), it is shown to be favorable across the U.S. with particular suitability in the Illinois Basin, Coastal Plains, and Basin and Range regions. Seasonal RTES operations have also been modeled in the Portland Basin by those at the USGS and Portland State University to simulate an RTES system supplying heating loads needed for the Oregon Health and Science University. Simulations suggest that high conductive heat loss in the initial years exists but tends to decrease with increasing time and development of the resource due to self-insulating nature of the basalts (Burns et al., 2020). Other national laboratory efforts are taking a close look at many of the technical issues involved with RTES (McLing et al., 2019, McLing et al., 2022). These include difficulties in understanding geochemical, hydrogeological, mechanical, and microbiological changes at such elevated temperatures and operational scenarios. Major gaps in research are identified and suggested for future work. With this increased focus to understand how to make RTES successful in the U.S., this technology could be a potential solution to many of the nation’s energy storage problems. For the energy independence of this country, the DOE should prioritize de-risking this technology by making future investments in pilot-scale demonstrations to attract potential investors.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

An Inventory of AI-ready Benchmark Data for US Fires, Heatwaves, and Droughts

Extreme weather events, including fires, heatwaves, and droughts, have significant impacts on earth, environmental, and energy systems. Mechanistic and predictive understanding, as well as probabilistic risk assessment of these extreme weather events, are crucial for detecting, planning for, and responding to these extremes. Records of extreme weather events provide an important data source for understanding present and future extremes, but the existing data needs preprocessing before it can be used for analysis. Moreover, there are many nonstandard metrics defining the levels of severity or impacts of extremes. In this study, we compile a comprehensive benchmark data inventory of extreme weather events, including fires, heatwaves, and droughts. The dataset covers the period from 2001 to 2020 with a daily temporal resolution and a spatial resolution of 0.5°×0.5° (~55km×55km) over the continental United States (CONUS), and a spatial resolution of 1km × 1km over the Pacific Northwest (PNW) region, together with the co-located and relevant meteorological variables. By exploring and summarizing the spatial and temporal patterns of these extremes in various forms of marginal, conditional, and joint probability distributions, we gain a better understanding of the characteristics of climate extremes. The resulting AI/ML-ready data products can be readily applied to ML-based research, fostering and encouraging AI/ML research in the field of extreme weather. This study can contribute significantly to the advancement of extreme weather research, aiding researchers, policymakers, and practitioners in developing improved preparedness and response strategies to protect communities and ecosystems from the adverse impacts of extreme weather events.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Validation of New MCNP6.3 Features for Critical and Subcritical Benchmark Simulations [Slides]

A separate MCNP6.3 V&V document reports on all the default calculations for all test suites was introduced. Here, we look at new optional features of the MCNP6.3 code. Then, Fission-matrix-based options with Convergence and population testing and Automated acceleration of the fission source and Doppler Broadening Resonance Correction and HDF5 Particle Track Output (PTRAC) and Validation benchmarks: criticality, Rossi-α, and subcritical multiplication. All the new MCNP6.3 options give statistically equivalent results and should be considered for default usage in criticality and subcritical multiplication calculation work. Discussed are two separate papers at the upcoming 2023 International Conference on Nuclear Criticality Safety (ICNC 2023) that give more details on the results presented here.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Using Likwid and Byfl to Benchmark Hardware Performance [Poster]

Benchmark Study conducted focusing on CPU and program performance analysis. Performance data gathered using 2 different programs and comparisons made based on performance. After comparisons are made, conclusions can be drawn and improvements are made upon hardware and software.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Assessing and benchmarking the fidelity of posterior inference methods for astrophysics data analysis

In this era of large and complex astronomical survey data, interpreting, validating, and comparing inference techniques becomes increasingly difficult. This is particularly critical for emerging inference methods like Simulation-Based Inference (SBI), which offer significant speedup potential and posterior modeling flexibility, especially when deep learning is incorporated. We present a study to assess and compare the performance and uncertainty prediction capability of Bayesian inference algorithms – from traditional MCMC sampling of analytic functions to deep learning-enabled SBI. We focus on testing the capacity of hierarchical inference modeling in those scenarios. Before we extend this study to cosmology, we first use astrophysical simulation data to ensure interpretability. We demonstrate a probabilistic programming implementation of hierarchical and non-hierarchical Bayesian inference using simulations derived from the DeepBench software library, a benchmarking tool developed by our group that generates simple and controllable astrophysical objects from first principles. This study will enable astronomers and physicists to harness the inference potential of these methods with confidence.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Benchmark Analysis for the Optical Dilatometry Method Using Silicon Carbide Temperature Monitors

The main objective of this project was to conduct a benchmark analysis for the optical dilatometry method by using NSUF’s SiC temperature monitors: two (2) SiC temperature monitors provided by NSUF’s BSU-8242 experiment and two (2) SiC temperature monitors provided by NSUF’s GE-Hitachi experiment. Per the BSU-8242 experiment, KGT-3597 and KGT-3591 had a design temperature of 400°C and an exposure of 3 dpa. Per the GE-Hitachi experiment, KGT-3341 and KGT-3336 had a design temperature of 290°C +/- 50°C and an exposure of 0.5–1 dpa. The KGT-3336 SiC monitor was split into two pieces during the decontamination process, making the dilatometry method the only way to analyze both those pieces. The BSU-8242 monitors revealed peak irradiation temperatures under the design temperature, and the GE-Hitachi monitors revealed peak irradiation temperatures within the design temperature range. The optical dilatometry method measured the peak irradiation temperature of BSU-8242 KGT-3597 to be 330°C, while the resistivity method measured the peak irradiation temperature of BSU-8242 KGT-3591 to be 320°C +/- 20°C. The optical dilatometry method measured the peak irradiation temperatures of the pieces of GE-Hitachi KGT-3336 to be 260°C (for the larger piece) and 220°C (for the smaller piece), while the resistivity method measured the peak irradiation temperature of GE-Hitachi KGT-3341 to be 300°C, with an accuracy range of -50°C to +20°C. Both methods of SiC temperature monitor analysis produced very similar peak irradiation temperatures for each pair of SiC passive monitors from the two experiments, BSU-8242 and GE-Hitachi. The results show dilatometry method to be a reliable and less time-intensive process for determining irradiation temperatures from passive SiC thermometry.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2024

Godiva 4 CAAS Shielding Benchmark: Pre CED-3A Overview [Slides]

Thiis presentation shows the character of an integral experiment. It covers the question why Godiva 4? Additionally, this presentation touches on the IER 498 benchmark quantity, activities facilitated by IER 498, proposed CED-2 test matrix, and what happened during the IER 498’s gap year.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

LLNL IPD Tasks: ICSBEP and Pulsed Spheres Benchmark [Slides]

This lecture presented benchmark of the blank and polyethylene spheres at SINBAD in April 2023. In responding to comments about the building, we uncovered a large amount of additional detailed information about the building and experimental set up. We incorporated this new information into an even higher fidelity model and are working on uncertainty analysis at a low level. Scheduled to present at SINBAD meeting in April 2024.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗