Search NASA⌕ Search

SEARCH · Search NASA

Results for “model benchmark”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6

Importance of Internal Variability for Climate Model Assessment

Benchmarking climate model simulations against observations of the climate is core to the process of building realistic climate models and developing accurate future projections. However, in many cases, models do not match historical observations, particularly on regional scales. If there is a mismatch between modeled and observed climate features, should we necessarily conclude that our models are deficient? Using several illustrative examples, we emphasize that internal variability can easily lead to marked differences between the basic features of the model and observed climate, even when decades of model and observed data are available. This can appear as an apparent failure of models to capture regional trends or changes in global teleconnections, or simulation of extreme events. Despite a large body of literature on the impact of internal variability on climate, this acknowledgment has not yet penetrated many model evaluation activities, particularly for regional climate. We emphasize that using a single or small ensemble of simulations to conclude that a climate model is in error can lead to premature conclusions on model fidelity. A large ensemble of multidecadal simulations is therefore needed to properly sample internal climate variability in order to robustly identify model deficiencies and convincingly demonstrate progress between generations of climate models.

Climate and Earth system modelling↗

Search for single production of vector-like quarks decaying into W(ℓν)b in pp collisions at $\sqrt{s}=13$ TeV with the ATLAS detector

A search for single production of a vector-like quark Q, which could be either a singlet T, with charge $\frac{2}{3}$, or a Y from a (T, B, Y) triplet, with charge $-\frac{4}{3}$, is performed using data from proton-proton collisions at a centre-of-mass energy of 13 TeV. The data correspond to the full integrated luminosity of 140 fb −1 recorded with the ATLAS detector during Run 2 of the Large Hadron Collider. The analysis targets Q → Wb decays where the W boson decays leptonically. The data are found to be consistent with the expected Standard Model background, so upper limits are set on the cross-section times branching ratio, and on the coupling of the Q to the Standard Model sector for these two benchmark models. Effects of interference with the Standard Model background are taken into account. For the singlet T, the 95% confidence level limit on the coupling strength κ ranges between 0.22 and 0.52 for masses from 1150 to 2300 GeV. For the (T, B, Y) triplet, the limits on κ vary from 0.14 to 0.46 for masses from 1150 to 2600 GeV.

beyond Standard Model↗

Evaluation of Industry Standard Turbulence Models on an Axisymmetric Supersonic Compression Corner

Reynolds-averaged Navier-Stokes computations of a shock-wave/boundary-layer interaction (SWBLI) created by a Mach 2.85 flow over an axisymmetric 30-degree compression corner were carried out. The objectives were to evaluate four turbulence models commonly used in industry, for SWBLIs, and to evaluate the suitability of this test case for use in further turbulence model benchmarking. The Spalart-Allmaras model, Menter's Baseline and Shear Stress Transport models, and a low-Reynolds number k- model were evaluated. Results indicate that the models do not accurately predict the separation location; with the SST model predicting the separation onset too early and the other models predicting the onset too late. Overall the Spalart-Allmaras model did the best job in matching the experimental data. However there is significant room for improvement, most notably in the prediction of the turbulent shear stress. Density data showed that the simulations did not accurately predict the thermal boundary layer upstream of the SWBLI. The effect of turbulent Prandtl number and wall temperature were studied in an attempt to improve this prediction and understand their effects on the interaction. The data showed that both parameters can significantly affect the separation size and location, but did not improve the agreement with the experiment. This case proved challenging to compute and should provide a good test for future turbulence modeling work.

Turbulent Boundary Layer↗

The Benchmark Active Controls Technology Model Aerodynamic Data

The Benchmark Active Controls Technology (BACT) model is a part of the Benchmark Models Program (BMP). The BMP is a NASA Langley Research Center program that includes a series of models which were used to study different aeroelastic phenomena and to validate computational fluid dynamics codes. The primary objective of BACT testing was to obtain steady and unsteady loads, accelerations, and aerodynamic pressures due to control surface activity in order to calibrate unsteady CFD codes and active control design tools. Three wind-tunnel tests in the Transonic Dynamics Tunnel (TDT) have been completed. The first and parts of the second and third tests focused on collecting open-loop data to define the model's aeroservoelastic characteristics, including the flutter boundary across the Mach range. It is this data that is being presented in this paper. An extensive database of over 3000 data sets was obtained. This database includes steady and unsteady control surface effectiveness data, including pressure distributions, control surface hinge moments, and overall model loads due to deflections of a trailing edge control surface and upper and lower surface

Scott, Robert C.↗

Verifying MCNP Models of the TEX High 240 Plutonium Benchmark

Computational modeling programs are invaluable tools that allow us to understand systems, safely develop new processes, and make reliable predictions about future designs. However, the effectiveness of these codes is limited by the degree to which their parameters match the real world. In the field of nuclear engineering, cross section data is one of these vital parameters. Accurate cross section data on important fissile and fissionable isotopes promotes the design of safer and more efficient fabrication, transportation, storage, and stockpiling of nuclear fuel. Unfortunately, there are knowledge gaps in data on key isotopes. In 2011, a multinational meeting hosted by the US Department of Energy Nuclear Criticality Safety Program ranked the priority of certain cross section data needs. In response, Lawrence Livermore National Lab (LLNL) designed the Thermal and Epithermal eXperiment (TEX) series of benchmark experiments. Benchmark experiments are used to validate current cross section data. They validate data by comparing the results of an actual experiment to the predicted results from a computational model. The data a benchmark applies to depends on the isotope and energy range the experiment’s neutron multiplication factor ( k eff ) is most sensitive to. The development and testing of the TEX High 240 Plutonium Benchmark will help validate 240 Pu cross section data. The configuration and materials of this benchmark are designed to be most sensitive to 240 Pu's intermediate energy range (from 0.625 ev to 100 keV ). MCNP® models of the assembly have been developed by LLNL and the results have been written in the final design report. In order for the discrepancies between benchmark models and experiments to be attributed to cross section inaccuracies, the accuracy of the models needs to be verified. The goal of this project is to verify of the results of LLNL's modeling by creating a new set of MCNP models and comparing the results.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

HELIUM LEAK TEST MODELING OF A SPENT NUCLEAR FUEL CANISTER

The U.S. Department of Energy (DOE) is considering the development of one or more federal consolidated interim storage facilities (CISFs) to be used to store commercial spent nuclear fuel (SNF) at locations in the U.S. One of the first technical challenges of a CISF is performing an inspection of SNF canisters upon their receipt to confirm they can be placed into the CISF’s licensed storage configuration. The canister receipt inspection is critical to CISF site operations. The test is conceived as being a helium (He) leak check, intended to confirm that the confinement boundary of a SNF canister is intact. SNF canisters are filled with He when they are sealed, so detection of a He leak indicates that a through-wall flaw has occurred in the canister confinement boundary. Other measurements are planned to occur upon canister receipt in addition to the He leak check such as krypton-85 measurements, which would indicate confinement breaches of one or more fuel rods in addition to a breach of the SNF canister. However, the He leak check has been identified as one of such high importance and has such significant technical challenges that a full-scale demonstration is needed to confirm the He leak test’s viability and to assist in planning relative to its operational requirements. A modeling methodology for simulating the He detection test was developed to help inform the test plan and the design of the test vessels. To develop the modeling methodology a detailed computational fluid dynamics (CFD) benchmark model was constructed to compare against leak rate test data from a transportation package for radioactive material. This report is focused on modeling efforts to simulate the benchmark leak test.

Suffield, Sarah R.↗

Interplay of freeze-in and freeze-out: Lepton-flavored dark matter and muon colliders

We study a lepton-flavored dark matter model and its signatures at a future muon collider. We focus on the less-explored regime of feeble dark matter interactions, which suppresses the dangerous lepton-flavor-violating processes, gives rise to dark matter freeze-in production, and leads to long-lived particle signatures at colliders. We find that the interplay of dark matter freeze-in and its mediator freeze-out gives rise to an upper bound of around TeV scales on the dark matter mass. The signatures of this model depend on the lifetime of the mediator and can range from generic prompt decays to more exotic long-lived particle signals. In the prompt region, we calculate the signal yield, study useful kinematics cuts, and report tolerable systematics that would allow for a 5 σ discovery. In the long-lived region, we calculate the number of charged tracks and displaced lepton signals of our model in different parts of the detector and uncover kinematic features that can be used for background rejection. We show that, unlike in hadron colliders, multiple production channels contribute significantly, which leads to sharply distinct kinematics for electroweakly charged long-lived particle signals. Ultimately, the collider signatures of this lepton-flavored dark matter model are common among models of electroweak-charged new physics, rendering this model a useful and broadly applicable benchmark model for future muon collider studies that can help inform work on detector design and studies of systematics. Published by the American Physical Society 2024

Astronomy & Astrophysics↗

Reduce, reuse, reinterpret: An end-to-end pipeline for recycling particle physics results

Searches for new physics at the Large Hadron Collider have constrained many models of physics beyond the Standard Model. Many searches also provide resources that allow them to be reinterpreted in the context of other models. We describe a reinterpretation pipeline that examines previously untested models of new physics using supplementary information from ATLAS Supersymmetry (SUSY) searches in a way that provides accurate constraints even for models that differ meaningfully from the benchmark models of the original analysis. The public analysis information, such as public analysis routines and serialized probability models, is combined with common event generation and simulation toolkits MADGRAPH, PYTHIA8, and DELPHES into workflows steered by TOML configuration files, and bundled into the mapyde Python package. The use of mapyde is demonstrated by constraining previously untested SUSY models with compressed sleptons and electroweakinos using ATLAS results.

Stark, Giordon↗

Codebase release 0.5 for mapyde

Searches for new physics at the Large Hadron Collider have constrained many models of physics beyond the Standard Model. Many searches also provide resources that allow them to be reinterpreted in the context of other models. We describe a reinterpretation pipeline that examines previously untested models of new physics using supplementary information from ATLAS Supersymmetry (SUSY) searches in a way that provides accurate constraints even for models that differ meaningfully from the benchmark models of the original analysis. The public analysis information, such as public analysis routines and serialized probability models, is combined with common event generation and simulation toolkits MADGRAPH, PYTHIA8, and DELPHES into workflows steered by TOML configuration files, and bundled into the mapyde Python package. The use of mapyde is demonstrated by constraining previously untested SUSY models with compressed sleptons and electroweakinos using ATLAS results.

Stark, Giordon↗

Modelling the Huge, Herschel-Resolved Debris Ring around HD 207129

Debris disks, which are inferred from the observed infrared excess to be ensembles of dust, rocks. and probably planetesimals, arc common features of stellar systems. As the mechanisms of their fonnation and evolution are linked to those of planetary bodies, they provide valuable infonnation. The few well-resolved debris disks are even more valuable because they can serve as modelling benchmarks and help resolve degeneracies in modelling aspects such as typical grain sizes and distances. Here, we present an analysis of the HD 207129 debris disk, based on its well-covered spectral energy distribution and Herschel/PACS images obtained in the framework of the DUNES (DUst around NEarby Stars) programme. We use an empirical power-law approach to the distribution of dust and we then model the production and removal of dust by means of collisions, direct radiation pressure, and drag forces. The resulting best-fit model contains a total of nearly 10(sup -2) Earth masses in dust, with typical grain sizes in the planetesimal beh ranging from 4 to 7 micrometers. We constrain the dynamical excitation to be low, which results in very long collisional lifetimes and a drag that notably fills the inner gap, especially at 70 micrometers. The radial distribution stretches from well within 100 AU in an unusual, outward-riSing slope towards a rather sharp outer edge at about 170-190 AU. The inner edge is therefore smoother than that reported for Fomalhaut, but the contribution from the extended halo of barely bound grains is similarly small. Both slowly self-stirring and planetary perturbations could potentially have formed and shaped this disk.

Lohne, T.↗

Goldsim Modeling of Vadose Zone Transport for E-Area Naval Reactor Component Disposal Areas: Model Description and Benchmarking

This report documents the development of a GoldSim® model of flow and radionuclide transport to the water table through the Naval Reactor Components Disposal Area (NRCDA) waste disposal sites and underlying vadose zones. The model is designed to be used for Monte Carlo uncertainty analysis in support of the E-Area Performance Assessment (PA). This report describes the model and shows results obtained from benchmarking the model to best-estimate deterministic results obtained using a PORFLOW model of NRCDA vadose zone transport. The PORFLOW model is three-dimensional while the GoldSim model is a simplified one-dimensional treatment. Nevertheless, the GoldSim model was able to accurately reproduce PORFLOW results with some adjustment to the nominal dispersion coefficient and vadose zone flow area used as “tuning” parameters. An example of the results obtained comparing GoldSim and PORFLOW calculation of releases of I-129, Tc-99, C-14 and Ni-59 from waste disposal containers at the 643-26E site is shown in Figure 1 below. For all of the test cases evaluated, GoldSim predicted peak concentrations within 6% of the PORFLOW values and peak times agreed within 8% with the majority of the results in better agreement. The close agreement between the two models provides confidence that GoldSim will give results accurately reflecting the behavior of releases from the NRCDA under off-normal operating conditions for sensitivity and uncertainty analysis

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Observational Constraints Reduce Model Spread but Not Uncertainty in Global Wetland Methane Emission Estimates

The recent rise in atmospheric methane (CH 4 ) concentrations accelerates climate change and offsets mitigation efforts. Although wetlands are the largest natural CH 4 source, estimates of global wetland CH 4 emissions vary widely among approaches taken by bottom-up (BU) process-based biogeochemical models and top-down (TD) atmospheric inversion methods. Here, we integrate in situ measurements, multi-model ensembles, and a machine learning upscaling product into the International Land Model Benchmarking system to examine the relationship between wetland CH 4 emission estimates and model performance. We find that using better-performing models identified by observational constraints reduces the spread of wetland CH 4 emission estimates by 62% and 39% for BU- and TD-based approaches, respectively. However, global BU and TD CH 4 emission estimate discrepancies increased by about 15% (from 31 to 36 TgCH 4 year −1 ) when the top 20% models were used, although we consider this result moderately uncertain given the unevenly distributed global observations. Our analyses demonstrate that model performance ranking is subject to benchmark selection due to large inter-site variability, highlighting the importance of expanding coverage of benchmark sites to diverse environmental conditions. We encourage future development of wetland CH 4 models to move beyond static benchmarking and focus on evaluating site-specific and ecosystem-specific variabilities inferred from observations.

Kuang-Yu Chang↗

Comparison of Modeled to Measured Spectra using MCNP and GADRAS to Benchmark and Contrast Modeling Limitations

The desire to improve nuclear material detection through portal monitors and other low resolution detectors has led to interest in benchmarking the performance of radiation transport codes. These codes can be used to generate a variety and quantity of spectra that may be cost prohibitive to measure directly. Particular characteristics of typical detection scenarios were isolated to compare the performance of radiation transport codes to laboratory experiments. These benchmark experiments were performed to validate simulations in a number of key areas. These experiments included high areal density configurations in both one- and three-dimensional configurations. In addition, off-angle measurements were conducted to investigate the impact of detector response assumptions on complex geometries, for example where an unknown source is not physically collocated with an apparent hot spot. Furthermore, by examining both simple shielding and backscatter configurations, isolation of the scatter emanating from the object and environmental background scatter contributions to the spectra could be elucidated. A series of experiments were performed in which the order of shielding layers and the thickness were evaluated to examine the ability of the simulations to address these variables. Experiments with increasing thickness of polyethylene around a neutron source also allows for examinations of the ability of the transport simulations to properly model n,γ reactions. Finally, depleted uranium (DU) measurements have more complicated spectra than the cobalt-60 and consequently allow for investigations into the ability of transport simulations to model self-shielding. In each case, the results of the experiments were compared to MCNP simulations to judge the performance of the code, as was done previously with GADRAS 18.7.9 simulations [7]. All of the benchmark measurements were taken with a liquid nitrogen cooled 140% high purity germanium (HPGe) detector with a bismuth collimator that had the front tin filter removed. The measurement location and detector configuration were subsequently used for all experimental configurations. The remainder of this report details the experimental measurements, Section 2 and the MCNP model, Section 3. Comparisons between the experimental results and MCNP-generated spectra are performed, Section 4. Section 5 compares the results obtained with MCNP with the previously reported results obtained with GADRAS [2]. Finally, Section 6 presents the conclusions.

61 RADIATION PROTECTION AND DOSIMETRY↗

Infrared cooling rate calculations in operational general circulation models - Comparisons with benchmark computations

The performance of several parameterized models is described with respect to numerical prediction and climate research at GFDL, NCAR, and GISS. The radiation codes of the models were compared to benchmark calculations and other codes for the intercomparison of radiation codes in climate models (ICRCCM). Cooling rates and fluxes calculated from the models are examined in terms of their application to established general circulation models (GCMs) from the three research institutions. The newest radiation parameterization techniques show the most significant agreement with the benchmark line-by-line (LBL) results. The LBL cooling rates correspond to cooling rate profiles from the models, but the parameterization of the water vapor continuum demonstrates uncertain results. These uncertainties affect the understanding of some lower tropospheric cooling, and therefore more accurate parameterization of the water vapor continuum, as well as the weaker absorption bands of CO2 and O3 is recommended.

Kiehl, J. T.↗

Multimodel validation of single wakes in neutral and stratified atmospheric conditions

Previous research has revealed the need for a validation study that considers several wake quantities and code types so that decisions on the trade-off between accuracy and computational cost can be well informed and appropriate to the intended application. In addition to guiding code choice and setup, rigorous model validation exercises are needed to identify weaknesses and strengths of specific models and guide future improvements. Here, we consider 13 approaches to simulating wakes observed with a nacelle-mounted lidar at the Scaled Wind Technology Facility (SWiFT) under varying atmospheric conditions. We find that some of the main challenges in wind turbine wake modeling are related to simulating the inflow. In the neutral benchmark, model performance tracked as expected with model fidelity, with large-eddy simulations performing the best. In the more challenging stable case, steady-state Reynolds-averaged Navier–Stokes simulations were found to outperform other model alternatives because they provide the ability to more easily prescribe noncanonical inflows and their low cost allows for simulations to be repeated as needed. Dynamic measurements were only available for the unstable benchmark at a single downstream distance. These dynamic analyses revealed that differences in the performance of time-stepping models come largely from differences in wake meandering. This highlights the need for more validation exercises that take into account wake dynamics and are able to identify where these differences come from: mesh setup, inflow, turbulence models, or wake-meandering parameterizations. In addition to model validation findings, we summarize lessons learned and provide recommendations for future benchmark exercises.

17 WIND ENERGY↗

Initial Efforts Organizing WPNCS SG-8: Preservation of Expert Knowledge and Judgement Applied to Criticality Benchmarks

The Working Party on Nuclear Criticality Safety (WPNCS) under the guidance of the Organization for Economic Co-operation and Development (OECD) Nuclear Energy Agency (NEA) has over 20 years of experience addressing concerns related to static and transient configurations encountered within the nuclear fuel cycle: fuel fabrication, transportation, reprocessing, storage, and geological disposal. One of the cornerstone activities of the WPNCS is the International Criticality Safety Benchmark Evaluation Project (ICSBEP), which was established to identify a comprehensive set of criticality benchmark data, evaluate the data, including quantification of overall uncertainties; compile the data into a standardized format, perform sample calculations utilizing modern nuclear data sets and codes utilized in nuclear criticality safety, and formally document the work into a single source of verified benchmark data. Annually, members of the ICSBEP Technical Review Group (TRG) contribute evaluated benchmark data that undergoes comprehensive technical review prior to publication in the ICSBEP Handbook. In the years since the ICSBEP was established, there has been much work to prepare benchmark data to support validation activities in nuclear criticality safety. The 2020 edition of the ICSBEP Handbook contains acceptable benchmark specifications for 5,053 critical, subcritical, or near-critical configurations in 582 benchmark evaluations. Modern benchmark development benefits from decades of experienced international participants, a well-established handbook format, supplementary guides to deal with uncertainty quantification, and a comprehensive review process based upon independent reviews from international experts. The ICSBEP Handbook also contains 838 configurations deemed unacceptable to support criticality safety efforts. They are recorded, with the reasoning for their rejection, to preserve the experimental data, prevent reevaluation of data that are incomplete or contain known errors, and/or to potentially allow future reevaluation of the experiment pending the identification of sufficient data to resolve identified inconsistencies and errors. Users of the ICSBEP Handbook today might notice that the rigor and quality of modern criticality safety benchmarks is much greater than those prepared within the initial decade of the project. Benchmarks with 1s uncertainties in k eff greater than 1% were traditionally rejected unless they were identified as unique experiment types that encompassed materials, fuels, or designs not available from other benchmark experiments. However, benchmarks developed using modern experimental techniques and practices typically have uncertainties on the order of a few tenths of a percent. There have been ongoing efforts to improve the overall quality of previously published benchmark evaluations. Seventy-eight evaluations, containing approximately 600 configurations, have been revised just within the past decade. An additional eleven benchmarks are under revision for updated release in the 2020 edition of the ICSBEP Handbook. If some of the historic benchmarks were resubmitted in their current form to the TRG today, they would be rejected due to lack of data, missing components in the uncertainty analysis, or incomplete benchmark model development. The use of historic criticality safety benchmarks that underestimate the total uncertainty, lack properly quantified biases, or provide inadequate benchmark specifications do not sufficiently support modern criticality safety and nuclear data efforts. Although the ICSBEP Handbook is recognized by regulating bodies to support criticality safety, users are required to justify their reasons to ignore historic benchmark data and include additional safety margins within their designs. Discussions were held at the WPNCS 23rd Annual Meeting in September 2019 regarding the aforementioned issues. The resultant decision was to establish Subgroup 8 (SG-8): Preservation of Expert Knowledge and Judgement Applied to Criticality Benchmarks. The current activities of SG-8 are discussed herein.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Particle Swarm Optimization Algorithm for Critical Experiment Design

Nuclear criticality experiments are used to validate nuclear cross section data used by simulation software. This is typically achieved by designing a critical system with a high sensitivity to a certain material’s cross section. Once the experiment has been carried out, a high fidelity model of the system is developed into a benchmark. When this benchmark model is simulated by a transport code, some of the difference between the experimental and computational effective neutron multiplication factor can be attributed to inaccurate nuclear data. Nuclear data evaluators then can make adjustments accordingly to improve cross section data.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Phase field benchmark problems for nucleation

In this work, we present nucleation phase field model benchmark problems, expanding on our previous benchmark problems on diffusion, precipitation, dendritic growth, linear elasticity, fluid flow and electrochemistry. Nucleation is the process in which either a new thermodynamic phase or a new structure is created, such as solidification from the melt, or self-assembly of particulates. Based on where the nucleation occurs, it can be divided into two main categories: homogeneous nucleation and heterogeneous nucleation. In the first nucleation benchmark problem, we focus on homogeneous nucleation for both single seed under different initial conditions and multiple seeds. The second nucleation benchmark problem focuses on athermal heterogeneous nucleation and nucleation behavior near the free growth limit with different undercooling driving forces.

36 MATERIALS SCIENCE↗