Results for “reproducibility of results” — Page 2

DOE OSTI2023

Entropy-driven Optimal Sub-sampling of Fluid Dynamics for Developing Machine-learned Surrogates

Optimal sub-sampling of large datasets from fluid dynamics simulations is essential for training reduced-order machine learned models. A method using Shannon entropy was developed to weight flow features according to their level of information content, such that the most informative features can be extracted and used for training a surrogate model. The method is demonstrated in the canonical flow over a cylinder problem simulated with OpenFOAM. Both time-independent predictions and temporal forecasting were investigated as well as two types of prediction targets: local per-grid-point predictions and global per-time-step predictions. When tested on training a surrogate model, results indicate that our entropy-based sampling method typically outperforms random sampling and yields more reproducible results in less iterations. Finally, the method was used to train a surrogate model for modeling turbulence in magnetohydrodynamic flows, which revealed various challenges and opportunities for future research.

Brewer, Wes↗

DOE OSTI2026

DEM Modeling and Validation of Pebble Bed Packing Using Chrono::GPU

Accurate prediction of pebble packing structure is important for pebble bed reactors because the spatial distribution of void fraction directly affects coolant flow, pressure drop, heat transfer, and neutronic behavior. However, experimentally validated DEM studies that directly evaluate local void-fraction structure in reactor-relevant pebble beds remain limited. In this work, the pebble bed experiment conducted at Missouri University of Science and Technology is simulated using the graphics processing unit (GPU)-based discrete element method (DEM) code Chrono::GPU. The study focuses on evaluating the ability of Chrono::GPU to reproduce the packing arrangement and void-fraction distribution of a randomly packed spherical pebble bed. The DEM results are first verified against established radial void-fraction correlations, including the Mueller and Vortmeyer-Schuster models, to assess the predicted bulk porosity, near-wall behavior, and oscillatory packing structure. The simulation is then verified against reference DEM data and validated against gamma-ray computed tomography (CT) experimental data at three axial locations. The Chrono::GPU results reproduce the main features of the experimental packing, including the high void fraction near the wall, the first near-wall trough, and the damped oscillatory radial profile caused by wall-induced ordering. Quantitative comparison with DEM data and the CT-based radial profiles shows good agreement, with mean absolute errors on the order of 0.07 and root-mean-square errors below 0.09 for the averaged profiles. These results demonstrate that Chrono::GPU can accurately capture the void-fraction structure of spherical pebble beds and provides a reliable DEM framework for future pebble bed reactor packing, recycling, and thermal-hydraulic studies.

97 - MATHEMATICS AND COMPUTING↗

NASA NTRSDate not supplied

INCREASING THE TRANSPARENCY AND REPRODUCIBILITY OF SPACE RADIATION SCIENCE: THE RADIATION BIOLOGY ONTOLOGY

Among the primary objectives of the Open/Open-Source Science paradigm are making scientific investigation data transparent and results reproducible [1], objectives shared by the FAIR principles [2]. To accomplish this, the conceptual framework that includes all the investigation objects needs to be accurately captured and communicated to all data consumers. A large part of this requires using metadata standards to annotate data collected. These standards should be readily accessible, informed by scientific community consensus and sufficiently specific to encompass all of the important aspects of the investigation. Starting in 2020 we have been co-leading an open consortium to develop a new metadata standard, the Radiation Biology Ontology (RBO), through the Open Biological and Biomedical Ontologies (OBO) Foundry [3]. We began by transforming many of the terms from the National Council on Radiation Protection and Measurement into concepts that can be formally related to existing OBO Foundry classes or attributes. We then identified and imported into the RBO existing OBO Foundry classes that have obvious relevance for radiation biomedicine (for example, concepts from the Environment Ontology that describe radiative processes, and concepts from the Gene Ontology dealing with molecular and cellular responses to radiation). Finally, we scrutinized datasets from investigations of radiation effects held in NASA GeneLab and LSDA repositories and added additional classes, instances, and attributes into the RBO that should be used to annotate these data. We developed the RBO using the open-source tools of GitHub and publish the RBO periodically through the NIH/NCBI BioPortal website, so systems worldwide can leverage the knowledge it contains [4]. This initial phase of concept modeling has yielded an RBO that at present has more than 300 declared concepts, with more than 3500 additional concepts imported from other OBO Foundry ontologies. While this first phase has focused on concepts for annotating samples, environments, exposures, and measurements, the next phase will center on supporting annotation of results and findings, such as concept models of molecular, cellular and tissue effects. The value of the RBO will be determined in part by our ability to engage the community in its development, and we have established a Radiobiology Informatics Consortium with unrestricted membership as the owner of the RBO in order to encourage investigators, system owners and other to join in this effort. Anyone can report issues or request new concept modeling or other features directly on GitHub. By using the BioPortal application programming interface, systems can pose dynamic queries to the latest version of the RBO for information on individual classes or entire hierarchies; this design eliminates the need for systems to be updated in order to use newer versions of the RBO. We hope to contribute to the advancement of open radiobiological science through the continued, open development of the RBO, that will provide more precise, machine-interpretable descriptions of investigations, as well as support data meta-analysis through machine learning or other artificial intelligence methods. REFERENCES [1] Open science in space. Nature Medicine, 2021. 27(9): p. 1485-1485. [2] Wilkinson, M.D., et al., The FAIR Guiding Principles for scientific data management and stewardship. Sci Data, 2016. 3: p. 160018. [3] Smith, B., et al., The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol, 2007. 25(11): p. 1251-5. [4] Whetzel, P.L., et al., BioPortal: enhanced functionality via new Web services from the National Center for Biomedical Ontology to access and use ontologies in software applications. Nucleic Acids Res, 2011. 39(Web Server issue): p. W541-5.

informatics↗

NASA NTRSDate not supplied

INCREASING THE TRANSPARENCY AND REPRODUCIBILITY OF SPACE RADIATION SCIENCE: THE RADIATION BIOLOGY ONTOLOGY

Among the primary objectives of the Open/Open-Source Science paradigm are making scientific investigation data transparent and results reproducible [1], objectives shared by the FAIR principles [2]. To accomplish this, the conceptual framework that includes all the investigation objects needs to be accurately captured and communicated to all data consumers. A large part of this requires using metadata standards to annotate data collected. These standards should be readily accessible, informed by scientific community consensus and sufficiently specific to encompass all of the important aspects of the investigation. Starting in 2020 we have been co-leading an open consortium to develop a new metadata standard, the Radiation Biology Ontology (RBO), through the Open Biological and Biomedical Ontologies (OBO) Foundry [3]. We began by transforming many of the terms from the National Council on Radiation Protection and Measurement into concepts that can be formally related to existing OBO Foundry classes or attributes. We then identified and imported into the RBO existing OBO Foundry classes that have obvious relevance for radiation biomedicine (for example, concepts from the Environment Ontology that describe radiative processes, and concepts from the Gene Ontology dealing with molecular and cellular responses to radiation). Finally, we scrutinized datasets from investigations of radiation effects held in NASA GeneLab and LSDA repositories and added additional classes, instances, and attributes into the RBO that should be used to annotate these data. We developed the RBO using the open-source tools of GitHub and publish the RBO periodically through the NIH/NCBI BioPortal website, so systems worldwide can leverage the knowledge it contains [4]. This initial phase of concept modeling has yielded an RBO that at present has more than 300 declared concepts, with more than 3500 additional concepts imported from other OBO Foundry ontologies. While this first phase has focused on concepts for annotating samples, environments, exposures, and measurements, the next phase will center on supporting annotation of results and findings, such as concept models of molecular, cellular and tissue effects. The value of the RBO will be determined in part by our ability to engage the community in its development, and we have established a Radiobiology Informatics Consortium with unrestricted membership as the owner of the RBO in order to encourage investigators, system owners and other to join in this effort. Anyone can report issues or request new concept modeling or other features directly on GitHub. By using the BioPortal application programming interface, systems can pose dynamic queries to the latest version of the RBO for information on individual classes or entire hierarchies; this design eliminates the need for systems to be updated in order to use newer versions of the RBO. We hope to contribute to the advancement of open radiobiological science through the continued, open development of the RBO, that will provide more precise, machine-interpretable descriptions of investigations, as well as support data meta-analysis through machine learning or other artificial intelligence methods.

knowledge↗

DOE OSTI2022

Pulse shaping in strong-field ionization: Theory and experiments

Intense ultrafast pulses cause dissociative ionization and shaping the pulses may allow control of both electronic and nuclear dynamics that determine ion yields. We report on a combined experimental and theoretical effort to determine how shaped laser pulses affect tunnel ionization, the process that precedes many strong-field phenomena. We carried out experiments on Ar, N 2 , H 2 O, and O 2 using a phase-step function of amplitude 3/4π that is scanned across the spectrum of the pulse. In addition, we changed the amount of chirp in the pulses. Semiclassical as well as fully quantum mechanical time-dependent Schrödinger equation calculations are found to be in excellent agreement with experimental results. We find that precise knowledge of the field parameters in the time and frequency domains is essential to afford reproducible results and quantitative theory and experiment comparisons.

74 ATOMIC AND MOLECULAR PHYSICS↗

DOE OSTI2024

Code for BALDR Study 07.04

SAND2024-11256O The Code for BALDR Study 07.04 software reproduces results from the BALDR study concerning "Multilabel Proportion Prediction and Out-of-Distribution Detection on Gamma Spectra of Short-Lived Fission Products." This code can reproduce a scientific study following the step numbers present in the file names. The scientific study uses synthetic and measured data to find the best model for the radioisotope proportion estimation task of interest and generates results. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Morrow, Tyler↗

NASA NTRS1998

A Summary of Validation Results for LEWICE 2.0

A research project is underway at NASA Lewis to produce a computer code which can accurately predict ice growth under any meteorological conditions for any aircraft surface. This report will present results from version 2.0 of this code, which is called LEWICE. This version differs from previous releases due to its robustness and its ability to reproduce results accurately for different point spacing, and time step criteria across general computing platforms. It also differs in the extensive amount of effort undertaken to compare the results in a quantifiable manner against the database of ice shapes which have been generated in the NASA Lewis Icing, Research Tunnel (IRT), The complete set of data used for this comparison is available in a recent contractor report . The result of this comparison shows that the difference between the predicted ice shape from LEWICE 2.0 and the average of the experimental data is 7.2% while the variability of the experimental data is 2.5%.

Wright, William B.↗

NASA NTRS1999

Validation Results for LEWICE 2.0

A research project is underway at NASA Lewis to produce a computer code which can accurately predict ice growth under any meteorological conditions for any aircraft surface. This report will present results from version 2.0 of this code, which is called LEWICE. This version differs from previous releases due to its robustness and its ability to reproduce results accurately for different spacing and time step criteria across computing platform. It also differs in the extensive amount of effort undertaken to compare the results in a quantified manner against the database of ice shapes which have been generated in the NASA Lewis Icing Research Tunnel (IRT). The results of the shape comparisons are analyzed to determine the range of meteorological conditions under which LEWICE 2.0 is within the experimental repeatability. This comparison shows that the average variation of LEWICE 2.0 from the experimental data is 7.2% while the overall variability of the experimental data is 2.5%.

Wright, William B.↗

DOE OSTI2023

Code for BALDR Study 07.05 v.1.0.0

SAND2024-01293O The Code for BALDR Study 07.05 software reproduces results from the BALDR study concerning "A Semi-Supervised Learning Method to Produce Explainable Radioisotope Proportion Estimates for NaI-based Synthetic and Measured Gamma Spectra." The code for BALDR Study 07.05 can reproduce a scientific study following the step numbers within the file names. Using synthetic and measured data to find the best model for the radioisotope proportion estimation task of interest, the study generated results for inclusion in a paper. The high-level methods involved are neural networks, semi-supervised learning, and out-of-distribution detection. Sandia National Laboratories is a multimission laboratory managed and operated by National Technology & Engineering Solutions of Sandia, LLC, a wholly owned subsidiary of Honeywell International Inc., for the U.S. Department of Energy’s National Nuclear Security Administration under contract DE-NA0003525.

Morrow, Tyler↗

DOE OSTI2024

The Dual Nature of Entrainment-Mixing Signatures Revealed through Large-Eddy Simulations of a Convection-Cloud Chamber

Abstract Entrainment of subsaturated air into a cloud can influence its optical and microphysical properties in various ways, depending on the droplet evaporation and turbulent mixing time scales. Previous experiments in the Pi convection-cloud chamber have revealed that, given a fixed entrained air property, the mixing of entrained subsaturated air results in complete evaporation of some cloud droplets, with the rest remaining unchanged. This is a signature of inhomogeneous mixing. While comparing the results of entrainment with varying air properties, the mixing signature appears as if the subsaturated air is well mixed with the cloud to evenly reduce the droplets’ size. In other words, taken together, the experiments appear to have the signature of homogeneous mixing. To explore these results in a greater depth, we conduct large-eddy simulations combined with a bin microphysics scheme. Our results reproduce the similar signatures of inhomogeneous and homogeneous mixing, implying that LES can resolve the inhomogeneous mixing when the grid spacing is smaller than the entrained air parcel. Additionally, we observe that increasing the aerosol injection rate enhances the signature of inhomogeneous mixing, while coarser grid spacing diminishes it. Finally, the change in wall fluxes in response to various entrained air properties confirms that the homogeneous signature seen in the analysis of an ensemble of simulations is the result of various equilibrium states. This further strengthens the suggestion that the homogeneous mixing signature found in aircraft observations near the cloud top may result from combining entrainment events of different intensities, possibly caused by various-sized eddies. Significance Statement Large-eddy simulation and size-resolved microphysics can resolve time scales for turbulent mixing and evaporation and, therefore, are well suited for reproducing, extending, and interpreting the entrainment experiment in the Pi convection-cloud chamber. Our simulation results confirm (i) the inhomogeneous mixing signature for an individual entrainment event and (ii) the appearance of homogeneous mixing in an ensemble of entrainment episodes. Furthermore, we demonstrate that the inhomogeneous mixing signature is more pronounced in a polluted cloud, but coarser grid spacing in simulations may compromise the accuracy of this signature. Last, the homogeneous mixing signature results from various equilibrium states established for different entrainment intensities and adjusted wall fluxes, which are challenging to measure experimentally but can be easily analyzed in the simulations.

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2026

Experimental Benchmarking for High-Reproducibility, Cross-Institutional Evaluation of Iron Redox Electrochemistry

We present a practical case study standardizing experimental protocols between collaborators with the goal of understanding ferrous iron (Fe 2+ ) chemistry and improving the iron deposition reaction for energy-efficient, electrochemical iron production. The study of iron reactions can be difficult, as aqueous iron electrolytes exhibit complex behaviors that can lead to differing interpretation of ostensibly similar experiments. The question we want to answer: are we studying the same chemistry? Our protocols address inherent challenges such as the tendency for Fe 2+ to spontaneously oxidize to ferric iron (Fe 3+ ) and the production of hydrogen at the potentials of interest. Our standardized protocol, executed by four collaborators in different labs and institutions, yields high-reproducibility results, and identified glassy carbon electrode surface quality and Fe 3+ impurities in the salt as key factors with outsized effects on cyclic voltammetry measurements. The process of developing the protocols helped to troubleshoot underlying issues that created poorer reversibility and reproducibility. This study highlights the fact that even nominally straightforward electrochemical systems can yield vastly different outcomes due to small differences in experimental preparation and serves as a useful example for creating transparent and achievable standards for the generation of reliable datasets that can be widely used and shared.

Ketter, Benjamin [Argonne National Laboratory (ANL↗

DOE OSTI2025

SCALE Input and Result Files Supporting SCALE Inventory and Reactivity Analysis of the gFHR

This dataset contains input and result files of computational simulations with the SCALE code system. The simulations cover radionuclide inventory and reactivity analyses of a fluoride salt-cooled high temperature pebble-bed reactor (PB-FHR), specifically the generic FHR benchmark. Users wanting to reproduce results from this dataset are required to obtain a license to the SCALE code system for which details on the distribution can be found here: https://www.ornl.gov/scale/releases

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DOE OSTI2022

Best practices for photovoltaic performance loss rate calculations

Abstract The performance loss rate (PLR) is a vital parameter for the time-dependent assessment of photovoltaic (PV) system performance and health state. Although this metric can be calculated in a relatively straightforward manner, it is challenging to achieve accurate and reproducible results with low uncertainty. Furthermore, the temporal evolution of PV system performance is usually nonlinear, but in many cases a linear evaluation is preferred as it simplifies the assessment and it is easier to evaluate. As such, the search for a robust and reproducible calculation methodology providing reliable linear PLR values across different types of systems and conditions has been the focus of many research activities in recent years. In this paper, the determination of PV system PLR using different pipelines and approaches is critically evaluated and recommendations for best practices are given. As nonlinear PLR assessments are fairly new, there is no consent on how to calculate reliable values. Several promising nonlinear approaches have been developed recently and are presented as tools to evaluate the PV system performance in great detail. Furthermore, challenges are discussed with respect to the PLR calculation but also opportunities for differentiating individual performance losses from a generic PLR value having the potential of enabling actionable insights for maintenance.

14 SOLAR ENERGY↗

DOE OSTI2024

Effect of detachment on Magnum-PSI ELM-like pulses: direct observations and qualitative results

Abstract Conditions similar to those at the end of the divertor leg in a tokamak were replicated in the linear plasma machine Magnum-PSI. The neutral pressure in the target chamber is then increased to cause the target to transition from an attached to a detached state. Superimposed to this steady state regime, edge localised mode (ELM)-like pulses are reproduced, resulting in a sudden increase in plasma temperature and density, such that the heat flux increases transiently by half an order of magnitude. Visible light emission, target thermography, and Thomson scattering are used to demonstrate that the higher the neutral pressure the more energy is removed from the ELM-like pulse in the volume. If the neutral pressure is sufficiently high, the ELM-like pulse can be prevented from affecting the target and the plasma energy is fully dissipated in the volume instead (ID 4 in table 1). The visible light images allow the division of the ELM-plasma interaction process of ELM energy dissipation into 3 ‘stages’ ranging from no dissipation to full dissipation (the target plasma is detached). In the second publication related to this study, spectroscopic data is analysed with a Bayesian approach, to acquire insights into the significance of molecular processes in dissipating the plasma energy and particles.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2024

SCALE Input and Result Files Supporting SCALE Inventory and Reactivity Analysis as Part of the Hermes 2021 PSAR Review

This dataset contains input and result files of computational simulations with the SCALE code system. The simulations cover radionuclide inventory and reactivity analyses of a fluoride salt-cooled high temperature pebble-bed reactor (PB-FHR), specifically the Hermes low-power PB-FHR demonstration reactor. Users wanting to reproduce results from this dataset are required to obtain a license to the SCALE code system for which details on the distribution can be found here: https://www.ornl.gov/scale/releases

equilibrium core↗

NASA NTRS1994

Theoretical investigation of wave-vector-dependent analytical and numerical formulations of the interband impact-ionization transition rate for electrons in bulk silicon and GaAs

The electron interband impact-ionization rate for both silicon and gallium arsenide is calculated using an ensemble Monte Carlo simulation with the expressed purpose of comparing different formulations of the interband ionization transition rate. Specifically, three different treatments of the transition rate are examined: the traditional Keldysh formula, a new k-dependent analytical formulation first derived by W. Quade, E. Scholl, and M. Rudan (1993), and a more exact, numerical method of Y. Wang and K. F. Brennan (1994). Although the completely numerical formulation contains no adjustable parameters and as such provides a very reliable result, it is highly computationally intensive. Alternatively, the Keldysh formular, although inherently simple and computationally efficient, fails to include the k dependence as well as the details of the energy band structure. The k-dependent analytical formulation of Quade and co-workers overcomes the limitations of both of these models but at the expense of some new parameterization. It is found that the k-dependent analytical method of Quade and co-workers produces very similar results to those obtained with the completely numerical model for some quantities. Specifically, both models predict that the effective threshold for impact ionization in GaAs and silicon is quite soft, that the majority of ionization events originate from the second conduction band in both materials, and that the transition rate is k dependent. Therefore, it is concluded that the k-dependent analytical model can qualitatively reproduce results similar to those obtained with the numerical model yet with far greater computational efficiency. Nevertheless, there exist some important drawbacks to the k-dependent analytical model of Quade and co-workers: These are that it does not accurately reproduce the quantum yield data for bulk silicon, it requires determination of a new parameter, related physically to the overlap intergrals of the Bloch state which can only be adjusted by comparison to experiment, and fails to account for any wave-vector dependence of the overlap integrals. As such the transition rate may be overestimated at those points for which 'near vertical,' small change in k, transitions occur.

Kolnik, Jan↗

NASA NTRS1994

Theoretical Investigation of Wave-Vector-Dependent Analytical and Numerical Formulations of the Interband Impact-Ionization Transition Rate for Electron in Bulk Silicon and GaAs

The electron interband impact-ionization rate for both silicon and gallium arsenide is calculated using an ensemble Monte Carlo simulation with the expressed purpose of comparing different formulations of the interband ionization transition rate. Specifically, three different treatments of the transition rate are examined: the traditional Keldysh formula, a new k-dependent analytical formulation first derived by W. Quade, E Scholl, and M. Rudan, and a more exact, numerical method of Y. Wang and K. F. Brennan. Although the completely numerical formulation contains no adjustable parameters and as such provides a very reliable result, it is highly computationally intensive. Alternatively, the Keldysh formula, although inherently simple and computationally efficient, fails to include the k dependence as well as the details of the energy band structure. The k-dependent analytical formulation of Quade and co-workers overcomes the limitations of both of these models but at the expense of some new parameterization. It is found that the k-dependent analytical method of Quade and co-workers produces very similar results to those obtained with (he completely numerical model for some quantities. Specifically, both models predict that the effective threshold for impact ionization in GaAs and silicon is quite soft, that the majority of ionization events originate from the second conduction band in both materials, and that the transition rate is k dependent. Therefore, it is concluded that the k-dependent analytical model can qualitatively reproduce results similar to those obtained with the numerical model yet with far greater computational efficiency. Nevertheless, there exist some important drawbacks to the k-dependent analytical model of Quade and co-workers: These are that it does not accurately reproduce the quantum yield data for bulk silicon, it requires determination of a new parameter, related physically to (he overlap integrals of the Bloch state which can only be adjusted by comparison to experiment, and fails to account for any wave-vector dependence of the overlap integrals. As such [he transition rate may be overestimated at those points for which "near vertical," small change in k, transitions occur.

Kolnik, Jan↗

DOE OSTI2020

The LPM effect in sequential bremsstrahlung: nearly complete results for QCD

The splitting processes of bremsstrahlung and pair production in a medium are coherent over large distances in the very high energy limit, which leads to a suppression known as the Landau-Pomeranchuk-Migdal (LPM) effect. We continue study of the case when the coherence lengths of two consecutive splitting processes overlap (which is important for understanding corrections to standard treatments of the LPM effect in QCD), avoiding soft-emission approximations. Previous work has computed overlap effects for double splitting g → gg → ggg. To make use of those results, one also needs calculations of related virtual loop corrections to single splitting g → gg in order to cancel severe (power-law) infrared (IR) divergences. This paper provides calculations of nearly all such processes involving gluons and discusses how to organize the results to demonstrate the cancellation. In the soft emission limit, our results reproduce the known double-log behavior of earlier authors who worked in leading-log approximation. We also present a first (albeit numerical and not yet analytic) investigation of sub-leading, single IR logarithms. Ultraviolet divergences appearing in our calculations correctly renormalize the coupling αs in the usual LPM result for leading-order g → gg.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗