SEARCH · Search NASA

Results for “reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

All sources NASA NTRS DOE OSTI

DOE OSTI2024

MISIP: a data standard for the reuse and reproducibility of any stable isotope probing-derived nucleic acid sequence and experiment

DNA/RNA-stable isotope probing (SIP) is a powerful tool to link in situ microbial activity to sequencing data. Every SIP dataset captures distinct information about microbial community metabolism, process rates, and population dynamics, offering valuable insights for a wide range of research questions. Data reuse maximizes the information derived from the labor and resource-intensive SIP approaches. Yet, a review of publicly available SIP sequencing metadata showed that critical information necessary for reproducibility and reuse was often missing. Here, we outline the Minimum Information for any Stable Isotope Probing Sequence (MISIP) according to the Minimum Information for any (x) Sequence (MIxS) framework and include examples of MISIP reporting for common SIP experiments. Our objectives are to expand the capacity of MIxS to accommodate SIP-specific metadata and guide SIP users in metadata collection when planning and reporting an experiment. The MISIP standard requires 5 metadata fields—isotope, isotopolog, isotopolog label, labeling approach, and gradient position—and recommends several fields that represent best practices in acquiring and reporting SIP sequencing data (e.g., gradient density and nucleic acid amount). The standard is intended to be used in concert with other MIxS checklists to comprehensively describe the origin of sequence data, such as for marker genes (MISIP-MIMARKS) or metagenomes (MISIP-MIMS), in combination with metadata required by an environmental extension (e.g., soil). The adoption of the proposed data standard will improve the reuse of any sequence derived from a SIP experiment and, by extension, deepen understanding of in situ biogeochemical processes and microbial ecology.

Simpson, Abigayle↗

DOE OSTI2026

Reproducibility, Replicability, and Research Quality in Homogeneous Catalysis

Researchers from all sectors of homogeneous catalysis convened in response to concerns regarding reproducibility in science to analyze the issue and provide recommendations. In addition to an in- person workshop, the group engaged the broader homogeneous catalysis community through a webinar series and virtually during the workshop. Results of the project affirm that homogeneous catalysis is not in a reproducibility crisis, as evidenced by the field’s current and past contributions to society that have led to economic growth and advances in a range of industries from agriculture to consumer goods to human health. However, it is not uncommon for researchers to encounter obstacles related to reproducibility. Ensuring reproducibility remains the responsibility of the community, both in current work and in training future researchers. This report is intended to engage key stakeholders, including disciplinary societies, publishers, employers, research leaders, and researchers, in practices that maximize reproducible homogeneous catalysis and ensure continued innovation and translatable discoveries. Recommendations made herein are also framed to be applicable beyond homogeneous catalysis, empowering the broader chemical if not scientific community.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2025

Reproducibility of fixed-node diffusion Monte Carlo across diverse community codes: The case of water–methane dimer

Fixed-node diffusion quantum Monte Carlo (FN-DMC) is a widely trusted many-body method for solving the Schrödinger equation, known for its reliable predictions of material and molecular properties. Furthermore, its excellent scalability with system complexity and near-perfect utilization of computational power make FN-DMC ideally positioned to leverage new advances in computing to address increasingly complex scientific problems. Even though the method is widely used as a computational gold standard, reproducibility across the numerous FN-DMC code implementations has yet to be demonstrated. This difficulty stems from the diverse array of DMC algorithms and trial wave functions, compounded by the method’s inherent stochastic nature. Here, this study represents a community-wide effort to assess the reproducibility of the method, affirming that yes, FN-DMC is reproducible (when handled with care). Using the water–methane dimer as the canonical test case, we compare results from eleven different FN-DMC codes and show that the approximations to treat the non-locality of pseudopotentials are the primary source of the discrepancies between them. In particular, we demonstrate that, for the same choice of determinantal component in the trial wave function, reliable and reproducible predictions can be achieved by employing the T-move, the determinant locality approximation, or the determinant T-move schemes, while the older locality approximation leads to considerable variability in results. These findings demonstrate that, with appropriate choices of algorithmic details, fixed-node DMC is reproducible across diverse community codes—highlighting the maturity and robustness of the method as a tool for open and reliable computational science.

Della Pia, Flaviano [Univ. of Cambridge (United Ki↗

DOE OSTI2026

Triboelectric Nanogenerator Repeatability and Reproducibility Study

The development of triboelectric nanogenerators (TENGs) has largely focused on optimizing output performance, often at the expense of other critical research considerations such as the development of reliable technical procedures. In particular, the reliability of reported results—specifically repeatability and reproducibility—remains underexplored and is frequently limited to brief discussion within available literature. Without rigorous validation through repeatability and reproducibility studies, the credibility and broader applicability of reported findings remain uncertain. This study addresses this gap by systematically evaluating the repeatability and reproducibility of TENG performance data. Five polymer materials—Kapton, polyethylene (PE), polyethylene terephthalate (PET), polytetrafluoroethylene (PTFE), and polyvinylidene fluoride (PVDF)—were investigated across all pairwise combinations of 25 total combinations for the reproducibility study and three selected pairs of the 25 samples were selected for the repeatability study. For each TENG pairing, we analyzed the methodology, experimental procedures, and resulting performance data to quantify consistency and reliability. The objective of this work is to assess the validity of the collected dataset and determine whether the observed performance trends are consistent for use in future TENG design and optimization studies. Establishing reliable and reproducible data is essential for advancing the development of high-output TENG systems and ensuring their dependable implementation in practical applications.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Using AI to Reproduce Neutrino Cross Section Analysis - Prototyping the Neutrino Discovery Platform

The Neutrino Discovery Platform (NDP) aims to accelerate DUNE-era science by making the neutrino program's existing datasets analyzable through fast, reproducible, and auditable workflows. We report a working version of two of its layers, data curation and agentic orchestration, built and tested end to end on MINERvA open data. The guiding lesson throughout is that a cross section is a measurement, and not just a plotted shape, only if it carries a defensible systematic-uncertainty budget, a trustworthy unfolding, and a reproducible record. Using a single medium-energy playlist pair from the MINERvA open-data release (about $2.05\times10^{17}$ protons on target of data), we first reproduced the shapes of two published charged-current inclusive $\nu_\mu$ measurements through a complete extraction ladder: selection, background subtraction, D'Agostini unfolding, efficiency correction, and flux normalization. These shape-level reproductions ran and tracked the published results, but they lacked the systematic-uncertainty machinery that defines a MINERvA cross section. To supply it, we vendored and built the MINERvA Analysis Toolkit and developed a many-universe systematic-uncertainty tool that produces a portable covariance artifact, a parallel event-loop runner, and a per-run auditability harness. Validated against a published covariance release, the toolchain reproduces the released statistical, flux, and muon-energy-scale terms and shows that they account for roughly 63\% of the total variance, with the remainder unreleased. Using this same infrastructure, we then performed a measurement of our own design, the hadronic recoil-energy distribution of low-energy ($E_\nu<2.5$~GeV) charged-current inclusive events, and found data/simulation shape agreement of $\chi^2/\mathrm{ndf}=1.26$. Together these results show that the platform supports original physics and not only reproductions.

Breaux, Auto [Tulane U. (main)]↗

DOE OSTI2026

Best practices in software development for robust and reproducible geoscientific models based on insights from the Global Carbon Budget's dynamic vegetation models

Computational models play an increasingly vital role in scientific research by enabling the numerical simulation of complex processes. Such models are also fundamental in geosciences. For instance, they offer critical insights into the impacts of global change on the Earth system today and in the future. Beyond their value as research tools, models are also software products and should therefore adhere to certain established software engineering standards. However, scientists are rarely trained as software developers, which can lead to potential deficiencies in software quality like unreadable, inefficient, or erroneous code. The complexity of models, coupled with their integration into broader workflows, also often makes it challenging to reproduce results, evaluate processes, and build upon them. In this paper, we review the state and current practices of the development processes of the state-of-the-art land surface models used by the Global Carbon Budget. We combine the experience of modelers from the respective research groups with the expertise of software engineers from tech companies to outline key principles and tools for improving software quality in research. We explore four main areas: (1) model testing and validation, (2) scientific, technical, and user documentation, (3) version control, continuous integration, and code review, and (4) the portability and reproducibility of workflows. Our review reveals that while modeling communities are incorporating many best practices, significant room for improvement remains in areas such as automated testing, automated documentation, and reproducibility. Therefore, we here identify and promote essential software engineering practices, including numerous examples of practices from within the community that can serve as guidelines for other models and could help streamline processes across the entire community. We conclude with an open-source example implementation of these principles, demonstrating portable and reproducible data flows, a continuous integration setup, and web-based visualizations. This example may serve as a practical resource for model developers, users, and all scientists engaged in scientific programming.

Gregor, Konstantin [Technical Univ. of Munich (Ger↗

DOE OSTI2026

Rigor and Reproducibility in Electrocatalysis: Best Practices for Operando Studies

Operando measurements have rapidly expanded the scope of electrocatalysis by enabling direct observation of catalytic interfaces under working conditions and by linking structural, compositional, and spectroscopic observables to activity and selectivity. However, the growth of operando methods has outpaced the adoption of broadly shared experimental standards, creating persistent challenges in reproducibility, interpretation, and comparison across laboratories and platforms. This perspective synthesizes discussions from the 2025 National Science Foundation Workshop on Rigor and Reproducibility in Electrocatalysis and outlines a practical framework for the rigorous use of operando measurements in electrocatalysis. We highlight three recurring needs: careful implementation of complex methods to avoid overinterpretation; recognition that (subtle) differences in reactor architecture, hydrodynamics, and electrical boundary conditions can alter apparent kinetics and selectivity; and transparent reporting standards that enable meaningful cross-comparison without constraining measurement-specific cell innovation. Focusing on widely used techniques (including X-ray and vibrational spectroscopies, mass spectrometry, and electron microscopy), we discuss technique-specific pitfalls, cross-validation strategies, and recurring platform-agnostic considerations such as mass transport, current distribution, temporal-resolution mismatches, and catalyst evolution. This Perspective aims to strengthen the mechanistic inference and improve the reproducibility, comparability, and predictive value of operando electrocatalysis research.

X-ray absorption spectroscopy↗

DOE OSTI2026

From Reproducible Edge–Cloud Experimentation to Real-World Practice: The E2Clab Experience

Reproducibility is already difficult in distributed systems; on the computing continuum, it becomes substantially harder. Applications that span sensing devices, edge and fog resources, and cloud platforms must be evaluated across heterogeneous hardware, variable network conditions, cross-layer orchestration decisions, and long-running workflow lifecycles. We use E2Clab as a case study to examine these challenges and their implications for experimental methodology. We explain why reproducible experimentation is harder on the continuum, then revisit E2Clab as an initial response based on explicit modeling of infrastructure, workflow lifecycle, and artifacts. Lastly, we discuss how its evolution toward more realistic application settings can be understood through the lens of Translational Computer Science. We argue that reproducible continuum experimentation requires methods that are rigorous enough for research while remaining adaptable to real-world practice.

42 ENGINEERING↗

DOE OSTI2024

Impacts of floating-point non-associativity on reproducibility for HPC and deep learning applications

Run to run variability in parallel programs caused by floating-point non-associativity has been known to significantly affect reproducibility in iterative algorithms, due to accumulating errors. Non-reproducibility can critically affect the efficiency and effectiveness of correctness testing for stochastic programs. Recently, the sensitivity of deep learning training and inference pipelines to floating-point non-associativity has been found to sometimes be extreme. It can prevent certification for commercial applications, accurate assessment of robustness and sensitivity, and bug detection. New approaches in scientific computing applications have coupled deep learning models with high-performance computing, leading to an aggravation of debugging and testing challenges. Here we perform an investigation of the statistical properties of floating-point non-associativity within modern parallel programming models, and analyze performance and productivity impacts of replacing atomic operations with deterministic alternatives on GPUs. We examine the recently-added deterministic options in PyTorch within the context of GPU deployment for deep learning, uncovering and quantifying the impacts of input parameters triggering run to run variability and reporting on the reliability and completeness of the documentation. Finally, we evaluate the strategy of exploiting automatic determinism that could be provided by deterministic hardware, using the Groq LPUTM accelerator for inference portions of the deep learning pipeline. We demonstrate the benefits that a hardware-based strategy can provide within reproducibility and correctness efforts.

Shanmugavelu, Sanjif↗

DOE OSTI2025

Breaking the reproducibility barrier with standardized protocols for plant–microbiome research

Inter-laboratory replicability is crucial yet challenging in microbiome research. Leveraging microbiomes to promote soil health and plant growth requires understanding underlying molecular mechanisms using reproducible experimental systems. In a global collaborative effort involving five laboratories, we aimed to help advance reproducibility in microbiome studies by testing our ability to replicate synthetic community assembly experiments. Our study compared fabricated ecosystems constructed using two different synthetic bacterial communities, the model grass Brachypodium distachyon, and sterile EcoFAB 2.0 devices. All participating laboratories observed consistent inoculum-dependent changes in plant phenotype, root exudate composition, and final bacterial community structure, where Paraburkholderia sp. OAS925 could dramatically shift microbiome composition. Comparative genomics and exudate utilization linked the pH-dependent colonization ability of Paraburkholderia, which was further confirmed with motility assays. The study provides detailed protocols, benchmarking datasets, and best practices to help advance replicable science and inform future multi-laboratory reproducibility studies.

Novak, Vlastimil↗

DOE OSTI2025

A standards perspective on genomic data reusability and reproducibility

Genomic and metagenomic sequence data provides an unprecedented ability to re-examine findings, offering a transformative potential for advancing research, developing computational tools, enhancing clinical applications, and fostering scientific collaboration. However, effective and ethical reuse of genomics data is hampered by numerous technical and social challenges. The International Microbiome and Multi’Omics Standards Alliance (IMMSA, https://www.microbialstandards.org/) and the Genomic Standards Consortium (GSC, https://gensc.org) hosted a 5-part seminar series “A Year of Data Reuse” in 2024 to explore challenges and opportunities of data reuse and reproducibility across disparate domains of the genomic sciences. Addressing these challenges will require a multifaceted approach, including common metadata reporting, clear communication, standardized protocols, improved data management infrastructure, ethical guidelines, and collaborative policies that prioritize transparency and accessibility. We offer strategies to enable responsible and technically feasible data reuse, recognition of data reproducibility challenges, and emphasizing the importance of cross-disciplinary efforts in the pursuit of open science and data-driven innovation.

59 BASIC BIOLOGICAL SCIENCES↗

DOE OSTI2025

Demonstration of the Reproducibility Challenges in the Sintering Behavior of Lithium‐Stuffed Garnets in Scaling up Synthesis

Lithium-stuffed garnets, such as Li 7 La 3 Zr 2 O 12 (LLZO), are promising candidates for next-generation solid-state batteries because of their high room-temperature ionic conductivity and chemical stability against lithium metal anodes, which are crucial for achieving higher energy density. However, realizing LLZO's potential in practical devices requires synthesis methods that can be scaled reliably to large batch sizes for manufacturing. Herein, we investigate the sintering reproducibility of LLZO synthesized at larger scales using ultrasonic spray pyrolysis, a cost-effective and scalable synthesis route. Two 100 g batches of Al-doped LLZO are prepared and their sintering behavior is examined in detail. Both Al-LLZO batches contain over 90 wt.% cubic-phase LLZO, and both batches exhibit room temperature conductivities greater than 1 × 10 −4 S cm −1 at a relative density above 0.8. However, variations in secondary phases and subtle differences in Al content lead to significant differences in densification and microstructure. These results demonstrate that LLZO's sintering behavior is highly sensitive to small changes in secondary phases and Al content, creating reproducibility challenges when moving from laboratory- to manufacturing-scale synthesis.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Reproducible benchmark for the SNAP 8 experimental reactor at operating conditions

This work presents fully reproducible multiphysics benchmark models of the Systems for Nuclear Auxiliary Power (SNAP) 8 Experimental Reactor at operating conditions with coolant flow. Wet experiment (with coolant, at power) validation benchmarks are presented using both deterministic (Serpent-Griffin) and Monte-Carlo (OpenMC-Cardinal) multiphysics frameworks coupled with thermal-hydraulic solvers in MOOSE. Reactivity coefficient measurements including fuel temperature, isothermal temperature, and power coefficients show good agreement with experiments, with discrepancies within experimental uncertainty. Reactivity worth experiments for coolant, samarium, and xenon poisoning are reproduced with differences under 200 pcm. Comparison between Serpent-Griffin and OpenMC-Cardinal frameworks reveal multiphysics coupling introduces positive reactivity effects (100-200 pcm) compared to uniform temperature and density fields at nominal operating conditions. Comparison between Serpent-Griffin and reference Serpent solution shows that power distributions maintain consistent radial and axial peaking behavior. All models, assumptions, thermophysical and thermomechanical properties, and material definitions are thoroughly documented with cited references; model inputs and model generating scripts are stored in the snapReactors GitHub repository.

SNAP↗

DOE OSTI2026

How should reproducibility be approached in plastic recycling?

With the growing importance of developing new and improved methodologies for plastic recycling, conducting reproducible research and ensuring that results are transferable across labs are increasingly important. This Voices article reflects on how academia and industry view the path forward for strengthening reproducibility to advance science and enable a circular plastics economy.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

DOE OSTI2026

Molecular Additive Engineering for Process-Humidity Robustness and Reproducible Fabrication of Perovskite Solar Cells and Modules

The commercialization of perovskite solar cells (PSCs) faces significant challenges due to their sensitivity to environmental humidity, which compromises film crystallization and device stability. Here, we introduce diphenylvinylphosphine (DPVP) as a Lewis base additive that enhances the performance and reproducibility of PSCs fabricated under ambient-air conditions. DPVP suppresses moisture-induced defect formation and stabilizes crystallization within realistic process-humidity ranges (20–40% relative humidity) commonly encountered in laboratory and pilot-scale manufacturing environments. It improves film uniformity, reduces trap densities, and yields highly reproducible device performance, enabling champion PCEs of 24.2% in small-area devices and 20.5% in blade-coated 12 cm 2 mini-modules. Furthermore, DPVP-assisted modules exhibit enhanced stability, retaining over 85% of their initial efficiency after 900 h of maximum power point tracking (MPPT) at 65 °C. This study demonstrates a humidity-resilient and scalable additive strategy for ambient-air perovskite photovoltaic manufacturing.

defect passivation↗

DOE OSTI2025

Reproducibility of Radiokrypton in Deep Desert Aquifers: Insights from a Decade of Research

Great technical advances have been achieved since the first atom-trap trace analysis (ATTA) -based radiokrypton application in Egypt, where 1 Myr old groundwater was discovered. Beyond advances in ATTA measurement capabilities, including reduction in sample size, analysis duration, and analytical uncertainty, major progress has been achieved over the past two decades in the sample collection and preparation techniques. These advances paved the expansion of ATTA-based noble gas applications to many other aquifers worldwide, illuminating the nature and flow pattern of deep groundwater systems. While the potential of this new analytical technique for old groundwater dating is well recognized, another important aspect yet to be examined is the reproducibility of radiokrypton in aquifers over time, i.e., how representative is a discrete groundwater sample, collected at a specific time and location, for the natural groundwater system? The likelihood of a negative answer is increased by flow-field disturbance in aquifers following massive groundwater abstraction. Here, in this work, we present repeated 81 Kr sampling and measurements in twenty-one sites over Israel, mostly of deep (up to 1 km) wells tapping confined aquifers in the arid to hyperarid Negev desert. The results demonstrate that radiokrypton measurements are indeed reproducible, even in cases where samples were collected as long as nine years apart and from highly productive (∼1 Mm 3 /yr order) pumping wells. Furthermore, many of the repeated measurements in this study (17 out of the 21 sites) were conducted with different ATTA Instruments in two different laboratories using slightly different sampling, preparation, and analysis techniques, yet with an overall good agreement. The consistency in the ATTA-based 81 Kr-dating results over time highlights the robustness of this state-of-the-art technique as a tool to unravel groundwater flow patterns and encourages further applications to many other yet-to-be-explored deep aquifers.

atom-trap trace analysis↗

DOE OSTI2024

fluxfinder: An R Package for Reproducible Calculation and Initial Processing of Greenhouse Gas Fluxes From Static Chamber Measurements

Fluxes of greenhouse gases are a critical component of the earth's natural climate, but anthropogenic emissions have created an imbalance and resulted in global climate change. Quantifying the emission of these gases is vital to our understanding of their sources and sinks, both natural and anthropogenic. The static chamber method, in which a system of interest is enclosed, and gas concentrations are measured over time, is widely used to estimate fluxes of greenhouse gases. With the development of instruments such as infrared gas analyzers (IRGAs) supporting high-frequency concentration data, there is a growing need for open-source workflows to calculate fluxes. Here we present fluxfinder, an R package designed to support reproducible calculations and processing of greenhouse gas fluxes measured with the static chamber method. The package includes raw data file parsing from widely used IRGAs, metadata matching, unit conversion, flux estimations, and initial quality assurance/quality control (QA/QC). Diagnostic graphical plots provide a transparent way to differentiate between measurement issues and nonlinear behavior. The package is also designed to be easily integrated with the gasfluxes package for further fitting of nonlinear concentration-time models, allowing alternative or additional flux QA/QC. The fluxfinder package offers a flexible workflow that is easily adaptable to promote open and reproducible greenhouse gas flux estimations.

Wilson, Stephanie J.↗

DOE OSTI2026

Frequency reproducibility of solid-state thorium-229 nuclear clocks

Solid-state thorium-229 ( 229 Th) nuclear clocks are set to provide new opportunities for precision metrology and fundamental physics. Taking advantage of inherent low sensitivity of a nuclear transition to its environment, orders of magnitude more emitters can be hosted in a solid-state crystal compared with current optical lattice atomic clocks. Furthermore, solid-state systems needing only simple thermal control are key to the development of field-deployable compact clocks. Here we explore and characterize the frequency reproducibility of the 229 Th:CaF 2 nuclear clock transition, a key performance metric for all clocks. We measure the transition linewidth and centre frequency as a function of the doping concentration, temperature and time. We report the concentration-dependent inhomogeneous linewidth of the nuclear transition, limited by the intrinsic host crystal properties. We determine an optimal working temperature for the 229 Th:CaF 2 nuclear clock at 196(5) K, at which the first-order thermal sensitivity vanishes. This would enable in situ temperature co-sensing using different quadrupole-split lines, reducing the temperature-induced systematic shift below the 10 −18 fractional frequency uncertainty level. At 195 K, the reproducibility of the nuclear transition frequency is 220 Hz (fractionally 1.1 × 10 −13 ) for two differently doped 229 Th:CaF 2 crystals over 7 months. Furthermore, these results form the foundation for understanding, controlling and harnessing the coherent nuclear excitation of 229 Th in solid-state hosts and for their applications in constraining temporal variations of fundamental constants.

Atomic and molecular physics↗