Search NASA⌕ Search

SEARCH · Search NASA

Results for “Benchmark testing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,297 records · Page 72

Benchmarking characterization methods for noisy quantum circuits

Effective methods for characterizing the noise in quantum computing devices are essential for programming and debugging circuit performance. Existing approaches vary in the information obtained as well as the amount of quantum and classical resources required, with more information generally requiring more resources. Here we benchmark the characterization methods of gate set tomography, Pauli channel noise reconstruction, and empirical direct characterization for developing models that describe noisy quantum circuit performance on a 27-qubit superconducting transmon device. We evaluate these models by comparing the accuracy of noisy circuit simulations with the corresponding experimental observations. Here, we find that the agreement of noise model to experiment does not correlate with the information gained by characterization and that the underlying circuit strongly influences the best choice of characterization approach. Empirical direct characterization scales best of the methods we tested and produced the most accurate characterizations across our benchmarks.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

IMPLEMENTATION OF THE IMPROVED QUASI-STATIC METHOD IN RATTLESNAKE/MOOSE FOR TIME-DEPENDENT RADIATION TRANSPORT MODELLING

Because of the recent interest in reactor transient modeling and the restart of the Transient Reactor (TREAT) Facility, there has been a need for more efficient, robust methods in computation frameworks. This is the impetus of implementing the Improved Quasi-Static method (IQS) in the RATTLESNAKE/MOOSE framework. IQS has implemented with CFEM diffusion by factorizing flux into time-dependent amplitude and spacial- and weakly time-dependent shape. The shape evaluation is very similar to a flux diffusion solve and is computed at large (macro) time steps. While the amplitude evaluation is a PRKE solve where the parameters are dependent on the shape and is computed at small (micro) time steps. IQS has been tested with a custom one-dimensional example and the TWIGL ramp benchmark. These examples prove it to be a viable and effective method for highly transient cases. More complex cases are intended to be applied to further test the method and its implementation.

IQS↗

Improved Verification and Validation Testing and Tools including Nuclear Criticality Safety Applications with the MCNP6.3® Code [Abstract]

A new Python-based framework has been developed to enable a more consistent layout with automatable setup, execution, and documentation of all verification and validation (V&V) test suites previously established for use with the MCNP code. In this paper, the new general framework for the V&V test suites is discussed, including information on all of the current capabilities and plans for future capabilities. For nuclear criticality safety applications, the existing V&V benchmark problems within the criticality, extended criticality, and analytic k-effective test suites have been ported into this new framework. In addition, the status and updates to the Rossi-alpha and subcritical multiplicationtest suites will be discussed. Some V&V results exercising new MCNP6.3 capabilities will be demonstrated.

97 MATHEMATICS AND COMPUTING↗

Roadmap for Molecular Benchmarks in Nonadiabatic Dynamics

Simulating the coupled electronic and nuclear response of a molecule to light excitation requires the application of nonadiabatic molecular dynamics. However, when faced with a specific photophysical or photochemical problem, selecting the most suitable theoretical approach from the wide array of available techniques is not a trivial task. The challenge is further complicated by the lack of systematic method comparisons and rigorous testing on realistic molecular systems. This absence of comprehensive molecular benchmarks remains a major obstacle to advances within the field of nonadiabatic molecular dynamics. A CECAM workshop, Standardizing Nonadiabatic Dynamics: Towards Common Benchmarks, was held in May 2024 to address this issue. This Perspective highlights the key challenges identified during the workshop in defining molecular benchmarks for nonadiabatic dynamics. Specifically, this work outlines some preliminary observations on essential components needed for simulations and proposes a roadmap aiming to establish, as an ultimate goal, a community-driven, standardized molecular benchmark set.

36 MATERIALS SCIENCE↗

Matrix-Model Simulations Using Quantum Computing, Deep Learning, and Lattice Monte Carlo

Matrix quantum mechanics plays various important roles in theoretical physics, such as a holographic description of quantum black holes, and it underpins the only practical numerical approach to the study of complex high-dimensional supergravity theories. Understanding quantum black holes and the role of entanglement in a holographic setup is of paramount importance for the realization of a quantum theory of gravity. Moreover, a complete numerical understanding of the holographic duality and the emergence of geometric space-time features from microscopic degrees of freedom could pave the way for new discoveries in quantum information science. Euclidean lattice Monte Carlo simulations are the de facto numerical tool for understanding the spectrum of large matrix models and have been used to test the holographic duality. However, they are not tailored to extract dynamical properties or even the quantum wave function of the ground state of matrix models. Quantum computing and deep learning provide potentially useful approaches to study the dynamics of matrix quantum mechanics. If successful in the context of matrix models, these rapidly improving numerical techniques could become the new Swiss army knife of quantum gravity practitioners. In this paper, we perform the first systematic survey for quantum computing and deep-learning approaches to matrix quantum mechanics, comparing them to lattice Monte Carlo simulations. These provide baseline benchmarks before addressing more complicated problems. In particular, we test the performance of each method by calculating the low-energy spectrum.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DSMC Simulations of Hypersonic Flows and Comparison With Experiments

This paper presents computational results obtained with the direct simulation Monte Carlo (DSMC) method for several biconic test cases in which shock interactions and flow separation-reattachment are key features of the flow. Recent ground-based experiments have been performed for several biconic configurations, and surface heating rate and pressure measurements have been proposed for code validation studies. The present focus is to expand on the current validating activities for a relatively new DSMC code called DS2V that Bird (second author) has developed. Comparisons with experiments and other computations help clarify the agreement currently being achieved between computations and experiments and to identify the range of measurement variability of the proposed validation data when benchmarked with respect to the current computations. For the test cases with significant vibrational nonequilibrium, the effect of the vibrational energy surface accommodation on heating and other quantities is demonstrated.

Moss, James N.↗

Maxine Onsite: Estimating Dose from Chronic Atmospheric Radioactive Releases

Maxine Onsite: Developed to estimate dose to individuals who are working onsite but are not site employees. Follows the methodologies discussed in US NRC Regulatory Guides 1.109 and 1.111 to determine dose from a chronic atmospheric release. Determines the dose given relative air concentrations and deposition concentrations as input. We look at four locations onsite that may be accessed by members of the public. To show compliance with the EPA and DOE public dose limit of 10 mrem/yr, determining who is a member of the public or a worker onsite is necessary. SRS has defined a member of the public for NESHAPS and DOE O 458.1. Maxine Onsite runs on Microsoft Windows Operating System through a Microsoft Excel spreadsheet. Input from the user includes parameters that are variable such as air concentrations, and fixed, such as distance to the receptor and elemental iodine fraction. Maxine Onsite was developed from Maxine which is a program meant to calculate dose to the MEI via three pathways, external, inhalation, and ingestion. Maxine Onsite is meant for looking at the exposure through the external and inhalation pathways only, no local food production onsite. The ways that Maxine Onsite is used to calculate the dose to the MEI includes the following: - Determining the respective air concentration values at each location using distances from the release stack; - Breathing rates for the MEI; - Determining the amount of time the MEI is outside; - Determining the plume travel time. The following relative air and ground deposition values for proper execution of the spreadsheet come from MAXDOSE-SR and are needed to accurately determine the dose to the MEI: relative air concentration, relative air concentration decayed by 2.26 days, relative air concentration decayed and depleted for 8 days, and relative deposition. Test cases were designed, developed, and executed to determine the accuracy of Maxine Onsite, which were verified additionally through hand calculations. Several changes were made for testing, including varying air concentrations, distances, breathing rates, and shielding factors. Each test case was then verified through hand calculations to assure the accuracy of the calculated doses. The results from the test cases, as well as the default parameters, can be used for benchmarking Maxine Onsite in the future. Maxine Onsite performs as expected and produces correct and accurate results at each of the different locations onsite (TRL, B-Area Barracks, SRELCC, and Gunsite 51) for varying test cases. The input required by the user is minimal and includes the results of the dose to the MEI on the first page for ease of access. Future plans for Maxine Onsite include adding in the ingestion pathway and adding in atmospheric dispersion estimates to the spreadsheet.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Verification of High-Fidelity Neutronics Code PROTEUS for C5G7 Benchmark Problems

Under the DOE Nuclear Energy Advanced Modeling and Simulation (NEAMS) program, the 3D whole core transport code PROTEUS has been developed at Argonne National Laboratory to perform the detailed 3D transport calculation for various reactor types including irregular or complex geometric reactors. PROTEUS includes two distinct transport solvers: SN and MOC. Among them, the MOC solver based on the extruded geometry, thus named MOCEX, can produce the faithful 3-D transport solutions by employing the rigorous and robust 2D/3D MOC formulation. This formulation overcomes the drawbacks of a full 3D MOC method requiring enormous memory and computational resources and resolves the instability issues of the 2D/1D method as well. The solution of MOCEX converges to that of a Monte Carlo code as the discretization of angle, space and energy is refined, unlike the 2D/1D method. As a verification and validation (V and V) effort of PROTEUS-MOCEX, this paper presents verification test results using the well-known 3D C5G7 benchmark problems by solving and comparing eigenvalues and pin powers with MCNP solutions.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Global Gridded Crop Model Evaluation: Benchmarking, Skills, Deficiencies and Implications.

Crop models are increasingly used to simulate crop yields at the global scale, but so far there is no general framework on how to assess model performance. Here we evaluate the simulation results of 14 global gridded crop modeling groups that have contributed historic crop yield simulations for maize, wheat, rice and soybean to the Global Gridded Crop Model Intercomparison (GGCMI) of the Agricultural Model Intercomparison and Improvement Project (AgMIP). Simulation results are compared to reference data at global, national and grid cell scales and we evaluate model performance with respect to time series correlation, spatial correlation and mean bias. We find that global gridded crop models (GGCMs) show mixed skill in reproducing time series correlations or spatial patterns at the different spatial scales. Generally, maize, wheat and soybean simulations of many GGCMs are capable of reproducing larger parts of observed temporal variability (time series correlation coefficients (r) of up to 0.888 for maize, 0.673 for wheat and 0.643 for soybean at the global scale) but rice yield variability cannot be well reproduced by most models. Yield variability can be well reproduced for most major producing countries by many GGCMs and for all countries by at least some. A comparison with gridded yield data and a statistical analysis of the effects of weather variability on yield variability shows that the ensemble of GGCMs can explain more of the yield variability than an ensemble of regression models for maize and soybean, but not for wheat and rice. We identify future research needs in global gridded crop modeling and for all individual crop modeling groups. In the absence of a purely observation-based benchmark for model evaluation, we propose that the best performing crop model per crop and region establishes the benchmark for all others, and modelers are encouraged to investigate how crop model performance can be increased. We make our evaluation system accessible to all crop modelers so that other modeling groups can also test their model performance against the reference data and the GGCMI benchmark.

wheat↗

DeepSurveySim: Simulation Software and Benchmark Challenges for Astronomical Observation Scheduling

Modern astronomical surveys have multiple competing scientific goals. Optimizing the observation schedule for these goals presents significant computational and theoretical challenges, and state-of-the-art methods rely on expensive human inspection of simulated telescope schedules. Automated methods, such as reinforcement learning, have recently been explored to accelerate scheduling. However, there do not yet exist benchmark data sets or user-friendly software frameworks for testing and comparing these methods. We present DeepSurveySim -- a high-fidelity and flexible simulation tool for use in telescope scheduling. DeepSurveySim provides methods for tracking and approximating sky conditions for a set of observations from a user-supplied telescope configuration. We envision this tool being used to produce benchmark data sets and for evaluating the efficacy of ground-based telescope scheduling algorithms, particularly for machine learning algorithms that would suffer in efficacy if limited to real data for training.We introduce three example survey configurations and related code implementations as benchmark problems that can be simulated with DeepSurveySim.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Stress Testing of Organic Light- Emitting Diode Panels and Luminaires

This report builds on previous DOE efforts with OLED technology by updating information on a previously benchmarked OLED product (the Chalina luminaire from Acuity Brands) and provides new benchmarks on the performance of Brite 2 and Brite Amber OLED panels from OLEDWorks. During the tests described here, samples of these devices were subjected to continuous operation in stress tests at elevated ambient temperature environments of 35°C or 45°C. In addition, samples were also operated continuously at room temperature in a room temperature operational life test (RTOL). One goal of this study was to investigate whether these test conditions can accelerate failure of OLED panels, either through panel shorting or an open circuit in the panel. These stress tests are shown to provide meaningful acceleration of OLED failure modes, and an acceleration factor of 2.6 was calculated at 45°C for some test conditions. In addition, changes in the photometric properties of the emitted light (e.g., luminous flux and chromaticity maintenance) was also evaluated for insights into the long-term stability of these products compared to earlier generations. Because OLEDs are a lighting system, electrical testing was also performed on the panel-driver pairs to provide insights into the impact of the driver on long-term panel performance.

29 ENERGY PLANNING, POLICY, AND ECONOMY↗

Single pin BWR benchmark problem for coupled Monte Carlo - Thermal hydraulics analysis

As part of the European NURISP research project, a single pin BWR benchmark problem was defined. The aim of this initiative is to test the coupling strategies between Monte Carlo and subchannel codes developed by different project participants. In this paper the results obtained by the Delft Univ. of Technology and Karlsruhe Inst. of Technology will be presented. The benchmark problem was simulated with the following coupled codes: TRIPOLI-SUBCHANFLOW, MCNP-FLICA, MCNP-SUBCHANFLOW, and KENO-SUBCHANFLOW. (authors)

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

The Electrolyte Genome project: A big data approach in battery materials discovery

We present a high-throughput infrastructure for the automated calculation of molecular properties with a focus on battery electrolytes. The infrastructure is largely open-source and handles both practical aspects (input file generation, output file parsing, and information management) as well as more complex problems (structure matching, salt complex generation, and failure recovery). Using this infrastructure, we have computed the ionization potential (IP) and electron affinities (EA) of 4830 molecules relevant to battery electrolytes (encompassing almost 55,000 quantum mechanics calculations) at the B3LYP/6-31+G(*) level. We describe automated workflows for computing redox potential, dissociation constant, and salt-molecule binding complex structure generation. We present routines for automatic recovery from calculation errors, which brings the failure rate from 9.2% to 0.8% for the QChem DFT code. Automated algorithms to check duplication between two arbitrary molecules and structures are described. We present benchmark data on basis sets and functionals on the G2-97 test set; one finding is that a IP/EA calculation method that combines PBE geometry optimization and B3LYP energy evaluation requires less computational cost and yields nearly identical results as compared to a full B3LYP calculation, and could be suitable for the calculation of large molecules. Our data indicates that among the 8 functionals tested, XYGJ-OS and B3LYP are the two best functionals to predict IP/EA with an RMSE of 0.12 and 0.27 eV, respectively. Application of our automated workflow to a large set of quinoxaline derivative molecules shows that functional group effect and substitution position effect can be separated for IP/EA of quinoxaline derivatives, and the most sensitive position is different for IP and EA. Published by Elsevier B.V

Battery↗

‘Wasteaware’ benchmark indicators for integrated sustainable waste management in cities

Highlights: • Solid waste management (SWM) is a key utility service, but data is often lacking. • Measuring their SWM performance helps a city establish priorities for action. • The Wasteaware benchmark indicators: measure both technical and governance aspects. • Have been developed over 5 years and tested in more than 50 cities on 6 continents. • Enable consistent comparison between cities and countries and monitoring progress. - Abstract: This paper addresses a major problem in international solid waste management, which is twofold: a lack of data, and a lack of consistent data to allow comparison between cities. The paper presents an indicator set for integrated sustainable waste management (ISWM) in cities both North and South, to allow benchmarking of a city’s performance, comparing cities and monitoring developments over time. It builds on pioneering work for UN-Habitat’s solid waste management in the World’s cities. The comprehensive analytical framework of a city’s solid waste management system is divided into two overlapping ‘triangles’ – one comprising the three physical components, i.e. collection, recycling, and disposal, and the other comprising three governance aspects, i.e. inclusivity; financial sustainability; and sound institutions and proactive policies. The indicator set includes essential quantitative indicators as well as qualitative composite indicators. This updated and revised ‘Wasteaware’ set of ISWM benchmark indicators is the cumulative result of testing various prototypes in more than 50 cities around the world. This experience confirms the utility of indicators in allowing comprehensive performance measurement and comparison of both ‘hard’ physical components and ‘soft’ governance aspects; and in prioritising ‘next steps’ in developing a city’s solid waste management system, by identifying both local strengths that can be built on and weak points to be addressed. The Wasteaware ISWM indicators are applicable to a broad range of cities with very different levels of income and solid waste management practices. Their wide application as a standard methodology will help to fill the historical data gap.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Contra-Rotating Open Rotor Tone Noise Prediction

Reliable prediction of contra-rotating open rotor (CROR) noise is an essential element of any strategy for the development of low-noise open rotor propulsion systems that can meet both the community noise regulations and the cabin noise limits. Since CROR noise spectra typically exhibits a preponderance of tones, significant efforts have been directed towards predicting their tone spectra. To that end, there has been an ongoing effort at NASA to assess various in-house open rotor tone noise prediction tools using a benchmark CROR blade set for which significant aerodynamic and acoustic data had been acquired in wind tunnel tests. In the work presented here, the focus is on the near-field noise of the benchmark open rotor blade set at the cruise condition. Using an analytical CROR tone noise model with input from high-fidelity aerodynamic simulations, detailed tone noise spectral predictions have been generated and compared with the experimental data. Comparisons indicate that the theoretical predictions are in good agreement with the data, especially for the dominant CROR tones and their overall sound pressure level. The results also indicate that, whereas individual rotor tones are well predicted by the linear sources (i.e., thickness and loading), for the interaction tones it is essential that the quadrupole sources be included in the analysis.

Propeller Noise↗

RELAP-7 and PRONGHORN Initial Integration Plan

Modern nuclear reactor safety codes require the ability to solve detailed coupled neutronicthermal fluids problems. For larger cores, this implies fully coupled 3-D spatial dynamics with appropriate feedback models that can provide enough resolution to accurately compute core heat generation and removal during steady and unsteady conditions. The reactor analyis code PRONGHORN is being coupled to RELAP-7 as a first step to extend RELAP's current capabilities. This report details the mathematical models, the type of coupling, and the testing that will be used to produce an integrated system. RELAP-7 is a MOOSE-based application that solves the continuity, momentum, and energy equations in 1-D for a compressible fluid. The pipe and joint capabilities enable it to model parts of the PCU system. The PRONGHORN application, also developed on the MOOSE infrastructure, solves the coupled equations that define the neutron diffusion, fluid flow, and heat transfer in a 3-D core model. Initially, the two systems will be loosely coupled to simplify the transition towards a more complex infrastructure. The integration will be tested with the OECD/NEA MHTGR-350 Coupled Neutronics-Thermal Fluids benchmark model.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Robust validation of approximate 1-matrix functionals with few-electron harmonium atoms

A simple comparison between the exact and approximate correlation components U of the electron-electron repulsion energy of several states of few-electron harmonium atoms with varying confinement strengths provides a stringent validation tool for 1-matrix functionals. The robustness of this tool is clearly demonstrated in a survey of 14 known functionals, which reveals their substandard performance within different electron correlation regimes. Unlike spot-testing that employs dissociation curves of diatomic molecules or more extensive benchmarking against experimental atomization energies of molecules comprising some standard set, the present approach not only uncovers the flaws and patent failures of the functionals but, even more importantly, also allows for pinpointing their root causes. Since the approximate values of U are computed at exact 1-densities, the testing requires minimal programming and thus is particularly suitable for rapid screening of new functionals.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

SAS4A/SASSYS-1 Validation with EBR-II Tests Performed During the SHRT Testing Program (Rev. 2)

The EBR-II sodium-cooled fast reactor supported a wide range of fast reactor research activities. The final fifteen years of operations at EBR-II were used for experiments and tests to demonstrate the importance of passive safety in liquid metal reactors and the capability of such systems to provide strong passive responses during off-normal conditions. The Shutdown Heat Removal Test program at EBR-II provided test data supporting the validation of computer codes for design, licensing, and operation of LMRs, among other objectives. The program included nearly sixty tests, most of which were protected and unprotected loss of flow and loss of heat sink tests run at various initial powers and flow rates. This report documents the results of modeling and simulation of tests from the Shutdown Heat Removal Test program to support validation of Argonne’s SAS4A/SASSYS-1 fast reactor safety analysis code. SHRT-17 and SHRT-6 were protected loss of flow tests where a loss of electrical power to all sodium coolant pumps was simulated to demonstrate the effectiveness of natural circulation cooling characteristics. SHRT-45R, SHRT-39, and SHRT-43R were unprotected loss of flow tests where the control rod scram function of the plant protection system was disabled to demonstrate the effectiveness of EBR-II’s passive reactivity feedbacks. BOP-301 and BOP-302R were unprotected loss of heat sink tests that demonstrated how reactivity feedback effects driven by changes to the core inlet temperature can shut down the fission process. PICT-5/6 demonstrated how core power could be increased and decreased via changes in the intermediate sodium and feedwater flow rates. This validation activity leverages modeling and simulation efforts that originated under an International Atomic Energy Agency Coordinated Research Project for benchmark analysis of the EBR-II Shutdown Heat Removal Tests. The SAS4A/SASSYS-1 models originally developed for the CRP have been modernized to utilize new code features and modeling practices. Differences between the predicted and measured test data were explored and the causes of these differences were identified as being due to modeling approximations, uncertainty in transient component behavior, and instrumentation uncertainties. Overall, it was concluded that for all tests simulated there was good agreement between the flow, power, and temperature predictions and the measured data.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗