Search NASA⌕ Search

SEARCH · Search NASA

Results for “Reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 181 records · Page 10

A reproducible study design for the MIMIC-IV in-hospital mortality task

Open, tabular electronic health record (EHR) datasets such as MIMIC-III and MIMIC-IV have become critical resources for developing machine learning (ML) models addressing clinical prediction tasks, including hospital readmission, length of stay, and in-hospital mortality (IHM). While MIMIC-III has benefited from well-established preprocessing pipelines and standardized feature sets, MIMIC-IV remains comparatively challenging to work with because there are no standardized benchmarks to support reproducibility and comparability across studies. To address this limitation, we present a rigorously curated MIMIC-IV custom feature set optimized for IHM prediction, constructed through a reproducible preprocessing pipeline and feature selection strategy.

97 MATHEMATICS AND COMPUTING↗

Plasmonic Core–Shell–Satellites with Abundant Electromagnetic Hotspots for Highly Sensitive and Reproducible SERS Detection

In this work, we develop a Ag@Al2O3@Ag plasmonic core–shell–satellite (PCSS) to achieve highly sensitive and reproducible surface-enhanced Raman spectroscopy (SERS) detection of probe molecules. To fabricate PCSS nanostructures, we employ a simple hierarchical dewetting process of Ag films coupled with an atomic layer deposition (ALD) method for the Al2O3 shell. Compared to bare Ag nanoparticles, several advantages of fabricating PCSS nanostructures are discovered, including high surface roughness, high density of nanogaps between Ag core and Ag satellites, and nanogaps between adjacent Ag satellites. Finite-difference time-domain (FDTD) simulations of the PCSS nanostructure confirm an enhancement in the electromagnetic field intensity (hotspots) in the nanogap between the Ag core and the satellite generated by the Al2O3 shell, due to the strong core–satellite plasmonic coupling. The as-prepared PCSS-based SERS substrate demonstrates an enhancement factor (EF) of 1.7 × 107 and relative standard deviation (RSD) of ~7%, endowing our SERS platform with highly sensitive and reproducible detection of R6G molecules. We think that this method provides a simple approach for the fabrication of PCSS by a solid-state technique and a basis for developing a highly SERS-active substrate for practical applications.

47 OTHER INSTRUMENTATION↗

The Gut Microbiome and Alcoholic Liver Disease: Ethanol Consumption Drives Consistent and Reproducible Alteration in Gut Microbiota in Mice

Phenotypic health effects, both positive and negative, have been well studied in association with the consumption of alcohol in humans as well as several other mammals including mice. Many studies have also associated these same health effects and phenotypes to specific members of gut microbiome communities. Here we utilized a chronic plus binge ethanol feed model (Gao-binge model) to explore microbiome community changes across three independent experiments performed in mice. We found significant and reproducible differences in microbiome community assemblies between ethanol-treated mice and control mice on the same diet absent of ethanol. We also identified significant differences in gut microbiota occurring temporally with ethanol treatment. Peak shift in communities was observed 4 days after the start of daily alcohol consumption. We quantitatively identified many of the bacterial genera indicative of these ethanol-induced shifts including 20 significant genera when comparing ethanol treatments with controls and 14 significant genera based on temporal investigation. Including overlap of treatment with temporal shifts, we identified 25 specific genera of interest in ethanol treatment microbiome shifts. Shifts coincide with observed presentation of fatty deposits in the liver tissue, i.e., Alcoholic Liver Disease-associated phenotype. The evidence presented herein, derived from three independent experiments, points to the existence of a common, reproducible, and characterizable “mouse ethanol gut microbiome”.

59 BASIC BIOLOGICAL SCIENCES↗

REPRODUCING THE CORRELATIONS OF TYPE C LOW-FREQUENCY QUASI-PERIODIC OSCILLATION PARAMETERS IN XTE J1550–564 WITH A SPIRAL STRUCTURE

While it has been observed that the parameters intrinsic to the type C low-frequency quasi-periodic oscillations are related in a nonlinear manner among themselves, there has been, up to now, no model to explain or reproduce how the frequency, the FWHM, and the rms amplitude of the type C low-frequency quasi-periodic oscillations behave with respect to one another. Here we are using a simple toy model representing the emission from a standard disk and a spiral such as that caused by the accretion–ejection instability to reproduce the overall observed behavior and shed some light on its origin. This allows us to prove the ability of such a spiral structure to be at the origin of flux modulation over more than an order of magnitude in frequency.

79 ASTRONOMY AND ASTROPHYSICS↗

Enhanced climate reproducibility testing with false discovery rate correction

Simulating the Earth's climate is an important and complex problem, thus climate models are similarly complex, comprised of millions of lines of code. In order to appropriately utilize the latest computational and software infrastructure advancements in Earth system models running on modern hybrid computing architectures to improve their performance, precision, accuracy, or all three; it is important to ensure that model simulations are repeatable and robust. This introduces the need for establishing statistical or non-bit-for-bit reproducibility, since bit-for-bit reproducibility may not always be achievable. Here, we propose a short-simulation ensemble-based test for an atmosphere model to evaluate the null hypothesis that modified model results are statistically equivalent to that of the original model. We implement this test in version 2 of the US Department of Energy's Energy Exascale Earth System Model (E3SM). The test evaluates a standard set of output variables across the two simulation ensembles and uses a false discovery rate correction to account for multiple testing. The false positive rates of the test are examined using re-sampling techniques on large simulation ensembles and are found to be lower than the currently implemented bootstrapping-based testing approach in E3SM. We also evaluate the statistical power of the test using perturbed simulation ensemble suites, each with a progressively larger magnitude of change to a tuning parameter. The new test is generally found to exhibit more statistical power than the current approach, being able to detect smaller changes in parameter values with higher confidence.

Kelleher, Michael E. [Oak Ridge National Laborator↗

Vectorization of Random Number Generation and Reproducibility of Concurrent Particle Transport Simulation

Efficient random number generation with high quality statistical properties and exact reproducibility of Monte Carlo simulations are important requirements in many areas of computational science. VecRNG is a package providing pseudo-random number generation (pRNG) in the context of a new library VecMath. This library bundles up several general-purpose mathematical utilities, data structures, and algorithms having both SIMD and SIMT (GPUs) support based on VecCore.~Several state-of-the-art RNG algorithms are implemented as kernels supporting parallel generation of random numbers in scalar, vector, and Cuda workflows. In this report, we will present design considerations, implementation details, and computing performance of parallel pRNG engines on both CPU and GPU. Reproducibility of propagating multiple particles in parallel for HEP event simulation is demonstrated, using GeantV-based examples, for both sequential and fine-grain track-level concurrent simulation wor kflows. Strategies for efficient uses of vectorized pRNG and non-overlapping streams of random number sequences in concurrent computing environments is discussed as well.

Jun, S. Y.↗

The Acoustic Emission signal acquired by the microphones placed in the CABRI test device along the fourteen last R.I.A. experiments: an example of reproducible research in nuclear science

The CABRI facility is an experimental nuclear reactor of the French Atomic Energy Commission. It is located at the Cadarache Research Centre in southern France and it is designed to act as a support to the French nuclear infrastructure. The purpose of the new testing programme termed, 'CABRI International Programme' (CIP) is to study the behaviour of PWR-type fuel rods at high burnup, equipped with an 'advanced' cladding, under Reactivity Initiated Accident (RIA) conditions (such as the scenario of a control rod ejection). Within the framework of this programme, piloted and funded by the French Institute of Nuclear Radioprotection and Safety (IRSN), ten tests are to be conducted with a frequency of two tests per year. The LPRE laboratory of the CEA which is in charge of the Preparation, realisation and breakdown of the test results studies the possibility to set up a new test analysis based on the processing of signals coming from sensors placed within the test equipment. During the experimental phase, the behaviour of the fuel element generates acoustic waves which can be detected by two microphones placed upstream and downstream of the test device. Studies showed the interest to realize temporal and spectral analyses on these signals by showing the existence of signatures which can be correlated with physical phenomena as the rod failure or the test shutdown (i.e. scram). The work presented in this article results from the will to consolidate these studies. Since the main phenomenon to be tracked is the fuel rod failure, the aim would be to highlight specific events which would have been precursors of the rod failure in order to use in the future these signals for further interpretation. Such an antecedent works resumption leads to a better understanding of the experimental needs and constitutes a good initial state to prepare the new very fast digital data acquisition systems. Although all the raw data are accessible in the form of text files, analyses and graphics representations were not straightforward to reproduce from the ancient studies since that, on one hand, people who were in charge of the original work left the laboratory and on the other hand because it is not easy when the time passes, even with our own work, to be able to remember the steps of data manipulations and the exact setup: - During the ancient experiments the use of analog data acquisition systems required to digitize tapes to be able to realize computer treatments. That had had for consequence to lose the initial dating. This one must be correctly edited to do temporal comparisons. - Analyses require functions for calculations whose parameters has to be well-known to reach the same results. We thus wished to manage our workflow in the idea that it can be easily reproducible on all the experiments. The object of the work presented in this article was to put in practice this strong bind between the data, treatments and generation of the document in order not to hesitate to do the iteration principle in action. We do not have to be afraid by the data driven analyses. According to the philosophy of the literate programming, the text of the technical document is woven with the computer code that produces all the printed output as tables, graphs for the study eliminating hence the unrealistic cut and paste. This difficulty is not specific to the nuclear domain. For many years, researchers have been worked out solutions to this mundane issue. And, presently, new technologies and high-level programming languages offer us actual answers. We will firstly present the tools applied in our laboratory to implement this workflow, then we will describe the global perception carried out to continue the study of the Acoustic Emission signals recorded by the two microphones during the fourteen last CABRI R.I.A. test.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

Benzo[d]thiazole Based Wide Bandgap Donor Polymers Enable 19.54% Efficiency Organic Solar Cells Along with Desirable Batch-to-Batch Reproducibility and General Applicability

The limited selection pool of high-performance wide bandgap (WBG) polymer donors is a bottleneck problem of the nonfullerene acceptor (NFA) based organic solar cells (OSCs) that impedes the further improvement of their photovoltaic performances. In this work, a series of new WBG polymers, namely PH-BTz, PS-BTz, PF-BTz, and PCl-BTz, are developed by using the bicyclic difluoro-benzo[d]thiazole (BTz) as the acceptor block and benzo[1,2-b:4,5-b']dithiophene (BDT) derivatives as the donor units. By introducing S, F, and Cl atoms to the alkylthienyl sidechains on BDT, the resulting polymers exhibit lowered energy levels and enhanced aggregation properties. The fluorinated PBTz-F not only exhibits a low-lying HOMO level, but also has stronger face-on packing order and results in more uniform fibril-like interpenetrating networks in the related PF-BTz:L8-BO blend. A high-power conversion efficiency (PCE) of 18.57% is achieved. Moreover, PBTz-F also exhibits a good batch-to-batch reproducibility and general applicability. In addition, ternary blend OSCs based on the host PBTz-F:L8-BO blend and PM6 guest donor exhibits a further enhanced PCE of 19.54%, which is among the highest values of OSCs.

36 MATERIALS SCIENCE↗

A reproducing kernel enhanced approach for peridynamic solutions

The most common discretization method for peridynamic models used in engineering problems is the node-based meshfree approach. This method discretizes peridynamic domains by a set of nodes, each associated with a nodal cell with a characteristic volume, leading to a particle-based description of continuum systems. The behavior of each particle is then considered representative of its cell. This limits the convergence rate to the first order. In this paper, we introduce a reproducing kernel (RK) approximation to the field variables in the peridynamic equations to increase the order of convergence of peridynamic numerical solutions. Furthermore, the numerical results demonstrate improved convergence rates in static peridynamic problems using the proposed method.

97 MATHEMATICS AND COMPUTING↗

Asymptotically compatible reproducing kernel collocation and meshfree integration for the peridynamic Navier equation

Here, we study reproducing kernel (RK) collocation method for peridynamic Navier equation. In the first part, we apply a linear RK approximation to both displacement and dilatation, and then back-substitute dilatation and solve the peridynamic Navier equation in a pure displacement form. The RK collocation scheme converges to the nonlocal limit for a fixed nonlocal interaction length and also to the local limit as nonlocal interactions vanish. The stability is shown by comparing the collocation scheme with the standard Galerkin scheme using Fourier analysis. In the second part, we apply the RK collocation to the quasi-discrete peridynamic Navier equation and show its convergence to the correct local limit when the ratio between the nonlocal length scale and the discretization parameter is fixed. The analysis is carried out on a special family of rectilinear Cartesian grids for the RK collocation method with a designated kernel with finite support. We assume the Lamé parameters satisfy λ ≥ μ to avoid extra assumptions on the nonlocal kernel. Finally, numerical experiments are conducted to validate the theoretical results.

42 ENGINEERING↗

Reproducible emission from nonlinear random lasers

Multiple scattering of light serves as a mechanism for feedback in random lasers. Consequently, internal spatial mode patterns, lasing wavelengths, and output directionality can all be random. Strong mode interaction can occur in such devices due to spatially overlapping modes resulting in nonlinearity with respect to the pump input power. Nevertheless, temporal coherence and lasing mode amplitude can be fixed at a constant pumping rate. This is a property desirable for applications where unique randomness is exploited but expected to be reliable over time, such as physical unclonable functions. Random lasers can also be cheaply and easily fabricated, exhibit relatively low lasing thresholds and high emission intensity. However, the precise scattering properties of such structures and fluctuations in the pump field can make device emission irreproducible, thereby limiting random laser applications. Here, in this work, we directly compare the random lasing spectra from zinc oxide samples fabricated in four distinct ways: spin-coating, sputtering, solgel deposition, and atomic layer deposition. The particular method of fabrication has a strong impact. Samples made through atomic layer deposition here exhibit both reproducibility and strong nonlinearity desirable for applications. Randomness in emission spectra persists across hundreds of repeated and averaged measurements irrespective of spatial location and is demonstrably nonlinear with respect to input signal intensity.

47 OTHER INSTRUMENTATION↗

Metal Foam Morphology Affects the Run to Run Reproducibility of OER Using Nickel Catalysts

Nickel (Ni) foam-based electrodes are excellent catalysts for the oxygen evolution reaction; however, we found that the random pore-size distribution of Ni foams contributes to a significant variability in electrochemically active surface area, compromising experimental reproducibility. We provide insights into quantifying this critical material property, verified by four electrochemical laboratories.

25 ENERGY STORAGE↗

The Ghost of Performance Reproducibility Past

The importance of ensemble computing is well established. However, executing ensembles at scale introduces interesting performance fluctuations that have not been well investigated. In this paper, we trace our experience uncovering performance fluctuations of ensemble applications (primarily constituting a workflow of GROMACS tasks), and unsuccessful attempts, so far, at trying to discern the underlying cause(s) of performance fluctuations. Is the failure to discern the causative or contributing factors a failure of capability? Or imagination? Do the fluctuations have their genesis in some inscrutable aspect of the system or software? Does it warrant a fundamental reassessment and rethinking of how we assume and conceptualize performance reproducibility? Answers to these questions are not straightforward, nor are they immediate or obvious. We conclude with a discussion about the performance of ensemble applications and ruminate over the implications for how we define and measure application performance.

97 MATHEMATICS AND COMPUTING↗

Not just for programmers: How GitHub can accelerate collaborative and reproducible research in ecology and evolution

Abstract Researchers in ecology and evolutionary biology are increasingly dependent on computational code to conduct research. Hence, the use of efficient methods to share, reproduce, and collaborate on code as well as document research is fundamental. GitHub is an online, cloud‐based service that can help researchers track, organize, discuss, share, and collaborate on software and other materials related to research production, including data, code for analyses, and protocols. Despite these benefits, the use of GitHub in ecology and evolution is not widespread. To help researchers in ecology and evolution adopt useful features from GitHub to improve their research workflows, we review 12 practical ways to use the platform. We outline features ranging from low to high technical difficulty, including storing code, managing projects, coding collaboratively, conducting peer review, writing a manuscript, and using automated and continuous integration to streamline analyses. Given that members of a research team may have different technical skills and responsibilities, we describe how the optimal use of GitHub features may vary among members of a research collaboration. As more ecologists and evolutionary biologists establish their workflows using GitHub, the field can continue to push the boundaries of collaborative, transparent, and open research.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Reproducibility and Replication of Experimental Particle Physics Results

Recently, much attention has been focused on the replicability of scientific results, causing scientists, statisticians, and journal editors to examine closely their methodologies and publishing criteria. Experimental particle physicists have been aware of the precursors of non-replicable research for many decades and have many safeguards to ensure that the published results are as reliable as possible. The experiments require large investments of time and effort to design, construct, and operate. Large collaborations produce and check the results, and many papers are signed by more than three thousand authors. This paper gives an introduction to what experimental particle physics is and to some of the tools that are used to analyze the data. It describes the procedures used to ensure that results can be computationally reproduced, both by collaborators and by non-collaborators. It describes the status of publicly available data sets and analysis tools that aid in reproduction and recasting of experimental results. It also describes methods particle physicists use to maximize the reliability of the results, which increases the probability that they can be replicated by other collaborations or even the same collaborations with more data and new personnel. Examples of results that were later found to be false are given, both with failed replication attempts and one with alarmingly successful replications. While some of the characteristics of particle physics experiments are unique, many of the procedures and techniques can be and are used in other fields.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Race conditions and data partitioning: risks posed by common errors to reproducible parallel simulations

When parallel algorithms for simulation were introduced in the 1970s, their development and use interested only experts in parallel computation. This circumstance changed as multi-core processors became commonplace, putting a parallel computer into the hands of every modeler. A natural outcome is growing interest in parallel simulation among persons not intimately familiar with parallel computing. At the same time, parallel simulation tools continue to be developed with the implicit assumption that the modeler is knowledgeable about parallel programming. The unintended consequence is a rapidly growing number of users of parallel simulation tools that are unlikely to recognize when the interaction of race conditions, partitioning strategies, and simultaneous action in their simulation models make results non-reproducible, thereby calling into question the validity of conclusions drawn from the simulation data. Here, we illustrate the potential dangers of exposing parallel algorithms to users who are not experts in parallel computation with example models constructed using existing parallel simulation tools. By doing so, we hope to refocus tool developers on usability, even if this new focus incurs loss of some performance.

97 MATHEMATICS AND COMPUTING↗

Skill of ship‐following large‐eddy simulations in reproducing MAGIC observations across the northeast P acific stratocumulus to cumulus transition region

Abstract During the Marine ARM GPCI Investigation of Clouds (MAGIC) in October 2011 to September 2012, a container ship making periodic cruises between Los Angeles, CA, and Honolulu, HI, was instrumented with surface meteorological, aerosol and radiation instruments, a cloud radar and ceilometer, and radiosondes. Here large‐eddy simulation (LES) is performed in a ship‐following frame of reference for 13 four day transects from the MAGIC field campaign. The goal is to assess if LES can skillfully simulate the broad range of observed cloud characteristics and boundary layer structure across the subtropical stratocumulus to cumulus transition region sampled during different seasons and meteorological conditions. Results from Leg 15A, which sampled a particularly well‐defined stratocumulus to cumulus transition, demonstrate the approach. The LES reproduces the observed timing of decoupling and transition from stratocumulus to cumulus and matches the observed evolution of boundary layer structure, cloud fraction, liquid water path, and precipitation statistics remarkably well. Considering the simulations of all 13 cruises, the LES skillfully simulates the mean diurnal variation of key measured quantities, including liquid water path (LWP), cloud fraction, measures of decoupling, and cloud radar‐derived precipitation. The daily mean quantities are well represented, and daily mean LWP and cloud fraction show the expected correlation with estimated inversion strength. There is a −0.6 K low bias in LES near‐surface air temperature that results in a high bias of 5.6 W m −2 in sensible heat flux (SHF). Overall, these results build confidence in the ability of LES to represent the northeast Pacific stratocumulus to trade cumulus transition region.

54 ENVIRONMENTAL SCIENCES↗

Advantages of a variable‐resolution global climate model in reproducing the seasonal evolution of East Asian summer monsoon

Abstract The East Asian summer monsoon (EASM) is unique among monsoon systems that it features meridional evolution of the summer monsoon. In this study, we evaluate the performances of a Variable‐Resolution Community Earth System Model (VR‐CESM) regionally refined over eastern China (14 km) in reproducing the seasonal evolution of EASM precipitation over China. Compared with reference datasets, VR‐CESM shows better performance than the corresponding globally uniform coarse‐resolution model CESM (quasi‐uniform 1°), especially over western China where complex local topography exists. The northward monsoon migration is closely related to low‐level southerly flows and vertical moisture advection, which are more reasonably simulated in VR‐CESM. The four critical timings of the EASM (monsoon onset, withdrawal, peak, and duration) are also better captured in VR‐CESM than in CESM. The corresponding spatial Pearson correlation coefficients of the four critical timings with respect to reference datasets are about 0.1 higher in VR‐CESM than those in CESM. Both models are most accurate in simulating monsoon onset and least accurate at simulating the monsoon peak. The overestimated zonal thermal contrast in CESM is responsible for the earlier monsoon onset and excessive precipitation in September over the Yangtze River valley. Finer resolution in VR‐CESM, especially over the Tibetan Plateau (TP), appears to be a main factor in simulating better zonal thermal contrast and seasonal evolution of the EASM.

54 ENVIRONMENTAL SCIENCES↗