Search NASA⌕ Search

SEARCH · Search NASA

Results for “reproducibility of results”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

A reproducibility study of "Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space"

Nigam et al. reported a genetic algorithm (GA) utilizing the SELFIES representation and also propose an adaptive, neural network-based penalty that is supposed to improve the diversity of the generated molecules. The main claims of the paper are that this GA outperforms other generative techniques (as measured by the penalized logP) and that a neural network-based adaptive penalty increases the diversity of the generated molecules. In this work, we investigated the reproducibility of their claims. Overall, we were able to reproduce comparable results using the SELFIES-based GA, but mostly by exploiting deficiencies of the (easily optimizable) fitness function (i.e., generating long, sulfur containing chains). In addition, we reproduce results showing that the discriminator can be used to bias the generation of molecules to ones that are similar to the reference set. Lastly, we attempted to quantify the evolution of the diversity, understand the influence of some hyperparameters, and propose improvements to the adaptive penalty.

Jablonka, Kevin Maik↗
DOE OSTI2021

Temperature dependence of nitrate-reducing Fe(II) oxidation by Acidovorax strain BoFeN1 – evaluating the role of enzymatic vs. abiotic Fe(II) oxidation by nitrite

ABSTRACT Fe(II) oxidation coupled to nitrate reduction is a widely observed metabolism. However, to what extent the observed Fe(II) oxidation is driven enzymatically or abiotically by metabolically produced nitrite remains puzzling. To distinguish between biotic and abiotic reactions, we cultivated the mixotrophic nitrate-reducing Fe(II)-oxidizing Acidovorax strain BoFeN1 over a wide range of temperatures and compared it to abiotic Fe(II) oxidation by nitrite at temperatures up to 60°C. The collected experimental data were subsequently analyzed through biogeochemical modeling. At 5°C, BoFeN1 cultures consumed acetate and reduced nitrate but did not significantly oxidize Fe(II). Abiotic Fe(II) oxidation by nitrite at different temperatures showed an Arrhenius-type behavior with an activation energy of 80±7 kJ/mol. Above 40°C, the kinetics of Fe(II) oxidation were abiotically driven, whereas at 30°C, where BoFeN1 can actively metabolize, the model-based interpretation strongly suggested that an enzymatic pathway was responsible for a large fraction (ca. 62%) of the oxidation. This result was reproduced even when no additional carbon source was present. Our results show that at below 30°C, i.e. at temperatures representing most natural environments, biological Fe(II) oxidation was largely responsible for overall Fe(II) oxidation, while abiotic Fe(II) oxidation by nitrite played a less important role.

Microbiology↗
DOE OSTI2024

ExaCA grain structure predictions for laser powder bed fusion processing

This dataset contains simulated cross-sections of laser powder bed fusion grain structures produced using the microstructure model ExaCA, in turn using time-temperature history data produced using the heat transport model AdditiveFOAM. These predictions show the grain structure using various permutations of hatch spacing and nucleation density. Also contained in the dataset are the input files necessary to reproduce the results. AdditiveFOAM (https://github.com/ORNL/AdditiveFOAM) and ExaCA (https://github.com/LLNL/ExaCA), both open source software, are required to reproduce the results in this dataset using the given input files. This DOI was updated 12/09/2025.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Impact of mixed grain size sediment input on braided river Morphodynamics: Insights from experimental and numerical modeling

The rate of sediment supply has significant impacts on river morphology, making it crucial to understand the geomorphic changes and grain size distribution dynamics in rivers. However, the effects of varying grain size sediment input on morphological changes in braided channels remain poorly understood. This study is the first to investigate the bar development and sediment sorting processes in braided channels with non-uniform sediment inputs using both numerical and experimental approaches. We applied a two-dimensional numerical model, Nays2DH to confirm and generalize experimental results. The model reproduced key experiment results, including 1) stream elevation changes, and 2) grain size distribution. Using this validated model, we explored the morphological changes and sorting process in a braided river with sediment inputs. The numerical experiments demonstrate that sediment input controls the elevation of the stream bed and the grain size distribution. Notably, both the elevation and grain-size distribution become relatively stable in downstream of the channel. Additionally, the simulation results suggest that an increased sediment supply leads to greater channel complexity, with bed surface armoring decreasing.

Nays2DH↗
DOE OSTI2023

HPC-driven computational reproducibility in numerical relativity codes: a use case study with IllinoisGRMHD

Abstract Reproducibility of results is a cornerstone of the scientific method. Scientific computing encounters two challenges when aiming for this goal. Firstly, reproducibility should not depend on details of the runtime environment, such as the compiler version or computing environment, so results are verifiable by third-parties. Secondly, different versions of software code executed in the same runtime environment should produceconsistent numerical results for physical quantities. In this manuscript, we test the feasibility of reproducing scientific results obtained using theIllinoisGRMHDcode that is part of an open-source community software for simulation in relativistic astrophysics, theEinstein Toolkit. We verify that numerical results of simulating a single isolated neutron star withIllinoisGRMHDcan be reproduced, and compare them to results reported by the code authors in 2015. We use two different supercomputers: Expanse at SDSC, and Stampede2 at TACC. By compiling the source code archived along with the paper on both Expanse and Stampede2, we find thatIllinoisGRMHDreproduces results published in its announcement paper up to errors comparable to round-off level changes in initial data parameters. We also verify that a current version ofIllinoisGRMHDreproduces these results once we account for bug fixes which have occurred since the original publication.

Astronomy & Astrophysics↗
DOE OSTI2021

Learning Universal Authorship Representations

This code contains all the utilities required to reproduce the results of our EMNLP 2021 paper "Learning Universal Authorship Representations". It contains the utilities required for download the datasets, training our model, and performing all evaluations necessary for reproducing the results in the paper. Here's the abstract of our work: Determining whether two documents were composed by the same author, also known as authorship verification, has traditionally been tackled using statistical methods. Recently, authorship representations learned using neural networks have been found to outperform alternatives, particularly in large-scale settings involving hundreds of thousands of authors. But do such representations learned in a particular domain transfer to other domains? Or are these representations inherently entangled with domain-specific features? To study these questions, we conduct the first large-scale study of cross-domain transfer for authorship verification considering zero-shot transfers involving three disparate domains: Amazon reviews, fanfiction short stories, and Reddit comments. We find that although a surprising degree of transfer is possible between certain do- mains, it is not so successful between others. We examine properties of these domains that influence generalization and propose simple but effective methods to improve transfer.

Rivera Soto, RafaelA↗
DOE OSTI2021

Predicting nepheline precipitation in waste glasses using ternary submixture model and machine learning

Nepheline precipitation in nuclear waste glasses during vitrification can be detrimental due to its negative effect on chemical durability. Developing models to accurately predict nepheline precipitation from compositions is important to increase waste loading since existing models can be overly conservative. In this study, an expanded dataset containing 955 glasses was compiled from literature data, where 355 glasses are for high-level waste (HLW). Previously developed submixture models were refitted using the new dataset, where a misclassification rate of 7.8% was achieved. Nine machine learning (ML) algorithms (e.g., k-nearest neighbor, Gaussian process regression, artificial neural network, support vector machine, decision tree, etc.) were applied to evaluate their ability of predicting nepheline precipitation from compositions. Model accuracy, precision, recall/sensitivity, and F1 score were systemically compared between different ML algorithms and modeling protocols. Good model prediction with an accuracy ~0.9 (misclassification rate of ~10%) was observed with different algorithms under certain protocol. This study evaluated various ML models to predict nepheline precipitations in waste glasses, highlighting the importance of data preparation, modeling protocol, and their effect on model stability and reproducibility. The results provide insights into applying ML to predict glass properties and suggest areas for future research on modeling nepheline precipitations.

Lu, Xiaonan↗
DOE OSTI2020

Reproducibility and Replication of Experimental Particle Physics Results

Recently, much attention has been focused on the replicability of scientific results, causing scientists, statisticians, and journal editors to examine closely their methodologies and publishing criteria. Experimental particle physicists have been aware of the precursors of non-replicable research for many decades and have many safeguards to ensure that the published results are as reliable as possible. The experiments require large investments of time and effort to design, construct, and operate. Large collaborations produce and check the results, and many papers are signed by more than three thousand authors. This paper gives an introduction to what experimental particle physics is and to some of the tools that are used to analyze the data. It describes the procedures used to ensure that results can be computationally reproduced, both by collaborators and by non-collaborators. It describes the status of publicly available data sets and analysis tools that aid in reproduction and recasting of experimental results. It also describes methods particle physicists use to maximize the reliability of the results, which increases the probability that they can be replicated by other collaborations or even the same collaborations with more data and new personnel. Examples of results that were later found to be false are given, both with failed replication attempts and one with alarmingly successful replications. While some of the characteristics of particle physics experiments are unique, many of the procedures and techniques can be and are used in other fields.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

Microphysical Modeling of Fault Slip and Stability Transition in Hydrothermal Conditions

Abstract Field and laboratory observations indicate that the frictional behaviors of faults depend on hydrothermal conditions. We extend the microphysical Chen‐Niemeijer‐Spiers (CNS) model to hydrothermal conditions by using the observed temperature variation of indentation hardness to infer the temperature dependence of a microphysical parameter . This parameter is assumed constant in previous versions of the CNS model. A simple spring‐slider system is used to simulate the fault system and investigate the steady‐state frictional behaviors of wet granite gouges. Our numerical results quantitatively reproduce experimental data showing the frictional‐plastic transition. The results also describe the transition from velocity‐strengthening at low temperatures (<160°C), to velocity‐weakening at intermediate temperatures (160°C–370°C), then back to velocity‐strengthening at high temperatures (>370°C). In our extended CNS model, these results suggest that the dominant shear deformation mechanism does transition from frictional granular flow to fully plastic creep with increasing temperature.

58 GEOSCIENCES↗
DOE OSTI2021

Evolution of microstructures in radiation fields using a coupled binary-collision Monte Carlo phase field approach

The simulation of radiation effects in materials broadly falls into two categories. At short time and length scales lies the modeling of primary radiation damage, such as point defect creation, energy deposition, and ballistic mixing. This is followed by the modeling at longer time scales of thermally activated microstructure evolution and defect reactions, such as recombination, clustering, and coarsening. The binary collision Monte Carlo method is an established, numerically efficient method for the computation of primary radiation damage. Conversely, the phase field method is a state of the art method for microstructure evolution on longer time and length scales. Here we present a concurrent coupling of these two methods, overcoming the difference between the discrete object Monte Carlo paradigm for primary radiation damage and the continuum field variable approach for microstructure evolution. The coupling is bidirectional, in which the microstructure evolution in the MOOSE finite element frame- work provides the spatial scattering data set for the charged particle trans- port and receives point defect, mass transport, and heat source terms from the simulated collision cascades that contribute to the field variable evolution. The concurrent coupling scheme is implemented in the code Magpie and demonstrated by investigating patterning for a model irradiated immiscible binary alloy. The results from the coupled binary collision Monte Carlo/phase field simulations reproduce the results of analytical models for phase separation, phase mixing, and patterning, supporting the approach and indicating its utility for modeling real materials systems.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Full versus quasiparticle self-consistency in vertex-corrected GW approaches

Here, using seven semiconductors/insulators with band gaps covering the range from 1 eV to 10 eV we systematically explore the performance of two different variants of self-consistency associated with famous Hedin's system of equations: the full self-consistency and the so called quasi-particle approximation to it. The pros and cons of these two variants of self-consistency are sufficiently well documented in literature for the simplest GW approximation to the Hedin's equations. Our study, therefore, aims primarily at the level of theory beyond GW approximation, i.e. at the level of theory which includes vertex corrections. Whereas quasi-particle self-consistency has certain advantages at GW level (well known fact), the situation becomes quite different when vertex corrections are included. In the variant with full self-consistency, vertex corrections (both for polarizability and for self energy) systematically reduce the calculated band gaps making them closer to the experimental values. In the variant with quasi-particle self-consistency, however, an inclusion of the same diagrams has considerably larger effect and calculated band gaps become severely underestimated. Different effect of vertex corrections in two variants of self-consistency can be related to the Z-factor cancellation which plays positive role in quasi-particle self-consistency at GW level of theory but appears to be destructive for the quasi-particle approximation when higher order diagrams are included. Second result of our study is that we were able to reproduce the results obtained with the Questaal code using our FlapwMBPT code when the same variant of self-consistency (quasi-particle) and the same level of vertex corrections (for polarizability only, static approximation for screened interaction, and Tamm-Dancoff approximation for the Bethe-Salpeter equation) are used.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗
DOE OSTI2023

Dataset for 'Ombadi et al. (2023). A warming-induced reduction in snow fraction amplifies rainfall extremes, Nature'

This package contains the main codes, sample input data and main result files to reproduce the analysis and results presented in the article: “Ombadi et al. (2023), A warming-induced reduction in snow fraction amplifies rainfall extremes, Nature”. The folder consists of the following: (1) “Raw data”: a folder that contains sample input data which is used in some of the codes for demonstration purposes. It also contains data that was not pre-processed such as Elevation data; (2) “Results”: this folder contains files of the main results presented in the paper including: “Annual-Max-Series”, “Change-rainfall-extremes”, “Change-snow-fraction”, “Warming levels_By scenario_model_year” and “Masks”. Description of these folders is detailed in the "Readme.rtf" file; (3) Python jupyter notebooks (Extract_Annual Max Series (AMS).ipynb, Elevation-dependent amplification of rainfall extremes.ipynb, Sensitivity_to_global_warming.ipynb) demonstrate the main steps of analysis. Further description of those notebooks is provided in the "Readme.rtf" file; (4) R code for extreme value analysis (Extreme_Value_Analysis.R). The sample and pre-processed dataset in "Raw data" is obtained from publicly available repositories of CMIP6 and ERA5 datasets; see Methods for more detail. This research was supported by Office of Science, Office of Biological and Environmental Research of the US Department of Energy under contract no. DE-AC02-05CH11231 for the CASCADE Scientific Focus (funded by the Regional and Global Model Analysis Program area within the Earth and Environmental Systems Modeling Program) and the iNAIADS Early Career Research Project (funded by the Environmental Systems Science program).

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Towards Redefining the Reproducibility in Quantum Computing: A Data Analysis Approach on NISQ Devices

Although the building of quantum computers has kept making rapid progress in recent years, noise is still the main challenge for any application to leverage the power of quantum computing. Existing works addressing noise in quantum devices proposed noise reduction when deploying a quantum algorithm to a specified quantum computer. The reproducibility issue of quantum algorithms has been raised since the noise levels vary on different quantum computers. Importantly, existing works largely ignore the fact that the noise of quantum devices varies as time goes by. Therefore, reproducing the results on the same hardware will even become a problem. We analyze the reproducibility of quantum machine learning (QML) algorithms based on daily model training and execution data collection. Our analysis shows a correlation between our QML models’ test accuracy and quantum computer hardware’s calibration features. We also demonstrate that noisy simulators for quantum computers are not a reliable tool for quantum machine learning applications.

Senapati, Priyabrata↗
DOE OSTI2024

Gupta-et-al_2024_EarthsFuture

Results from Gupta et al. submitted to Earth's Future. All code to reproduce the experiment and make the figures can be found here: https://github.com/rg727/Gupta-etal_2024_EarthsFuture The data provided in this repository are (1) Weather Regime Data , (2) Hydroclimate Data, and (3) CALFEWS output. In (1), there are Markov chains of daily weather regimes generated over the 600-year paleo-period. In (2), there are three sets of data: Historical daily CDEC data for 12 input locations into CALFEWS, 600-year long daily paleo data (streamflow and snow) for each input location, and (3) 600-year long daily climate-change data (4 degree temperature increase + 7% precipitation scaling applied to (2)) which serves as the "climate change scenario" in the study. Please reference the GitHub repository on how to use these data to reproduce the results. The CALFEWS output for the Paleo and Climate Change scenarios is stored in (3). More information can be found in the Gupta-et-al_2024_EarthsFuture-README file.

Gupta, Rohini↗
DOE OSTI2021

Toward Complete Leading-Order Predictions for Neutrinoless Double β Decay

The amplitude for the neutrinoless double β (0νββ) decay of the two-neutron system nn → ppe – e – constitutes a key building block for nuclear-structure calculations of heavy nuclei employed in large-scale 0νββ searches. Assuming that the 0νββ process is mediated by a light-Majorana-neutrino exchange, a systematic analysis in chiral effective field theory shows that already at leading order a contact operator is required to ensure renormalizability. Here, we develop a method to estimate the numerical value of its coefficient (in analogy to the Cottingham formula for electromagnetic contributions to hadron masses) and validate the result by reproducing the charge-independence-breaking contribution to the nucleon-nucleon scattering lengths. Our central result, while derived in dimensional regularization, is given in terms of the renormalized amplitude Aν(|p|, |p'|) , matching to which will allow one to determine the contact-term contribution in regularization schemes employed in nuclear-structure calculations. Our results thus greatly reduce a crucial uncertainty in the interpretation of searches for 0νββ decay.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) peak intensity normalization for complex mixture analyses

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) is a preferred technique for analyzing complex organic mixtures. Currently, there is no consensus normalization approach, nor an objective method for selecting one, for quantitative analyses of FT-ICR-MS data. We investigate a method to evaluate and score the amount of bias various normalization approaches introduce into the data. We evaluate the ability of the Statistical Procedure for the Analysis of Normalization Strategies (SPANS) to guide selection of appropriate normalization approaches for two different FT-ICR-MS datasets. Further, we test the robustness of SPANS results to changes in SPANS parameter values and assess the impact of using various normalization approaches on downstream statistical analyses. The normalization approach identified by SPANS differed for the two datasets. Normalization methods impacted the statistical significance of peaks differently, underscoring the importance of carefully evaluating potential methods. More consistent SPANS scores resulted when at least 120 significant peaks are used, where larger sets of peaks were obtained by increasing the p-value threshold. Interestingly, we show that Total Sum Scaling and highest peak normalization, used in previous studies, underperformed relative to SPANS-recommended normalization approaches. While there is no single, best normalization method for all datasets, SPANS provides a mechanism to identify an appropriate normalization method for analyzing FT-ICR-MS data quantitatively. As a result, the number of peaks used in the background distributions of SPANS contributes more significantly to the reproducibility of results than the p-value thresholds used to obtain those peaks.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Mott vs Kondo: Influence of various density functional based methods on the Ce isostructural phase transition mechanism

The cerium iso-structural phase transition (γ–α) is dominated by f-electron localization changes that results in a magnetic ordering change and a volume collapse. Generally, these physics are difficult to capture with ab initio and first principles methods. However, previous works have shown various methods to be successful in predicting at least some of the physics of the γ to α phase transition. Therefore, here, we perform a broad survey of density functional based methods across three levels of theory and types of functions [GGA (generalized gradient approximation), MetaGGA, and hybrid functionals] and compare the results, focusing on hydrostatic compression across the phase boundary at zero kelvin. For the methods that best reproduce experimental results, we directly probe the predicted mechanisms and frame the results in the Mott/Kondo debate, assessing how the underlying methods and assumptions of different functionals can assess the physical drivers in the phase transition, providing insight into the governing processes of this unique phase transition.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Generation of meter-scale hydrogen plasmas and efficient, pump-depletion-limited wakefield excitation using 10 GeV electron bunches

High repetition rates and efficient energy transfer to the accelerating beam are important for a future linear collider based on the beam-driven plasma wakefield acceleration scheme (PWFA-LC). This paper reports the first results from the Plasma Wakefield Acceleration Collaboration (E300) that are beginning to address both of these issues using the recently commissioned FACET-II facility at SLAC national accelerator laboratory. Here we have generated meter-scale hydrogen plasmas using time-structured 10 GeV electron bunches from FACET-II, which hold the promise of dramatically increasing the repetition rate of PWFA by rapidly replenishing the gas between each shot compared to the hitherto used lithium plasmas that operate at 1–10 Hz. Furthermore, we have excited wakes in such plasmas that are suitable for high gradient particle acceleration with high drive-bunch to wake energy transfer efficiency- a first step in achieving a high overall energy transfer efficiency. We have done this by using time-structured electron drive bunches that typically have one or more ultra-high current (> 30 kA) femtosecond spike(s) superimposed on a longer (~0.4 ps) lower current (< 10 kA) bunch structure. The first spike effectively field-ionizes the gas and produces a meter-scale (30–160 cm) plasma, whereas the subsequent beam charge creates a wake. The length and amplitude of the wake depends on the longitudinal current profile of the bunch and plasma density. We find that the onset of pump depletion, when some of the drive beam electrons are nearly fully depleted of their energy, occurs for hydrogen pressure ≥ 1.5Torr. We also show that some electrons in the rear of the bunch can gain several GeV energies from the wake. These results are reproduced by particle-in-cell simulations using the QPAD code. At a pressure of ~2Torr, simulation results and experimental data show that the beam transfers about 60% of its energy to the wake.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗