Search NASA⌕ Search

SEARCH · Search NASA

Results for “Reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2017

REPRODUCING THE CORRELATIONS OF TYPE C LOW-FREQUENCY QUASI-PERIODIC OSCILLATION PARAMETERS IN XTE J1550–564 WITH A SPIRAL STRUCTURE

While it has been observed that the parameters intrinsic to the type C low-frequency quasi-periodic oscillations are related in a nonlinear manner among themselves, there has been, up to now, no model to explain or reproduce how the frequency, the FWHM, and the rms amplitude of the type C low-frequency quasi-periodic oscillations behave with respect to one another. Here we are using a simple toy model representing the emission from a standard disk and a spiral such as that caused by the accretion–ejection instability to reproduce the overall observed behavior and shed some light on its origin. This allows us to prove the ability of such a spiral structure to be at the origin of flux modulation over more than an order of magnitude in frequency.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Enhanced climate reproducibility testing with false discovery rate correction

Simulating the Earth's climate is an important and complex problem, thus climate models are similarly complex, comprised of millions of lines of code. In order to appropriately utilize the latest computational and software infrastructure advancements in Earth system models running on modern hybrid computing architectures to improve their performance, precision, accuracy, or all three; it is important to ensure that model simulations are repeatable and robust. This introduces the need for establishing statistical or non-bit-for-bit reproducibility, since bit-for-bit reproducibility may not always be achievable. Here, we propose a short-simulation ensemble-based test for an atmosphere model to evaluate the null hypothesis that modified model results are statistically equivalent to that of the original model. We implement this test in version 2 of the US Department of Energy's Energy Exascale Earth System Model (E3SM). The test evaluates a standard set of output variables across the two simulation ensembles and uses a false discovery rate correction to account for multiple testing. The false positive rates of the test are examined using re-sampling techniques on large simulation ensembles and are found to be lower than the currently implemented bootstrapping-based testing approach in E3SM. We also evaluate the statistical power of the test using perturbed simulation ensemble suites, each with a progressively larger magnitude of change to a tuning parameter. The new test is generally found to exhibit more statistical power than the current approach, being able to detect smaller changes in parameter values with higher confidence.

Kelleher, Michael E. [Oak Ridge National Laborator↗
DOE OSTI2019

Vectorization of Random Number Generation and Reproducibility of Concurrent Particle Transport Simulation

Efficient random number generation with high quality statistical properties and exact reproducibility of Monte Carlo simulations are important requirements in many areas of computational science. VecRNG is a package providing pseudo-random number generation (pRNG) in the context of a new library VecMath. This library bundles up several general-purpose mathematical utilities, data structures, and algorithms having both SIMD and SIMT (GPUs) support based on VecCore.~Several state-of-the-art RNG algorithms are implemented as kernels supporting parallel generation of random numbers in scalar, vector, and Cuda workflows. In this report, we will present design considerations, implementation details, and computing performance of parallel pRNG engines on both CPU and GPU. Reproducibility of propagating multiple particles in parallel for HEP event simulation is demonstrated, using GeantV-based examples, for both sequential and fine-grain track-level concurrent simulation wor kflows. Strategies for efficient uses of vectorized pRNG and non-overlapping streams of random number sequences in concurrent computing environments is discussed as well.

Jun, S. Y.↗
DOE OSTI2015

The Acoustic Emission signal acquired by the microphones placed in the CABRI test device along the fourteen last R.I.A. experiments: an example of reproducible research in nuclear science

The CABRI facility is an experimental nuclear reactor of the French Atomic Energy Commission. It is located at the Cadarache Research Centre in southern France and it is designed to act as a support to the French nuclear infrastructure. The purpose of the new testing programme termed, 'CABRI International Programme' (CIP) is to study the behaviour of PWR-type fuel rods at high burnup, equipped with an 'advanced' cladding, under Reactivity Initiated Accident (RIA) conditions (such as the scenario of a control rod ejection). Within the framework of this programme, piloted and funded by the French Institute of Nuclear Radioprotection and Safety (IRSN), ten tests are to be conducted with a frequency of two tests per year. The LPRE laboratory of the CEA which is in charge of the Preparation, realisation and breakdown of the test results studies the possibility to set up a new test analysis based on the processing of signals coming from sensors placed within the test equipment. During the experimental phase, the behaviour of the fuel element generates acoustic waves which can be detected by two microphones placed upstream and downstream of the test device. Studies showed the interest to realize temporal and spectral analyses on these signals by showing the existence of signatures which can be correlated with physical phenomena as the rod failure or the test shutdown (i.e. scram). The work presented in this article results from the will to consolidate these studies. Since the main phenomenon to be tracked is the fuel rod failure, the aim would be to highlight specific events which would have been precursors of the rod failure in order to use in the future these signals for further interpretation. Such an antecedent works resumption leads to a better understanding of the experimental needs and constitutes a good initial state to prepare the new very fast digital data acquisition systems. Although all the raw data are accessible in the form of text files, analyses and graphics representations were not straightforward to reproduce from the ancient studies since that, on one hand, people who were in charge of the original work left the laboratory and on the other hand because it is not easy when the time passes, even with our own work, to be able to remember the steps of data manipulations and the exact setup: - During the ancient experiments the use of analog data acquisition systems required to digitize tapes to be able to realize computer treatments. That had had for consequence to lose the initial dating. This one must be correctly edited to do temporal comparisons. - Analyses require functions for calculations whose parameters has to be well-known to reach the same results. We thus wished to manage our workflow in the idea that it can be easily reproducible on all the experiments. The object of the work presented in this article was to put in practice this strong bind between the data, treatments and generation of the document in order not to hesitate to do the iteration principle in action. We do not have to be afraid by the data driven analyses. According to the philosophy of the literate programming, the text of the technical document is woven with the computer code that produces all the printed output as tables, graphs for the study eliminating hence the unrealistic cut and paste. This difficulty is not specific to the nuclear domain. For many years, researchers have been worked out solutions to this mundane issue. And, presently, new technologies and high-level programming languages offer us actual answers. We will firstly present the tools applied in our laboratory to implement this workflow, then we will describe the global perception carried out to continue the study of the Acoustic Emission signals recorded by the two microphones during the fourteen last CABRI R.I.A. test.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2023

Benzo[d]thiazole Based Wide Bandgap Donor Polymers Enable 19.54% Efficiency Organic Solar Cells Along with Desirable Batch-to-Batch Reproducibility and General Applicability

The limited selection pool of high-performance wide bandgap (WBG) polymer donors is a bottleneck problem of the nonfullerene acceptor (NFA) based organic solar cells (OSCs) that impedes the further improvement of their photovoltaic performances. In this work, a series of new WBG polymers, namely PH-BTz, PS-BTz, PF-BTz, and PCl-BTz, are developed by using the bicyclic difluoro-benzo[d]thiazole (BTz) as the acceptor block and benzo[1,2-b:4,5-b']dithiophene (BDT) derivatives as the donor units. By introducing S, F, and Cl atoms to the alkylthienyl sidechains on BDT, the resulting polymers exhibit lowered energy levels and enhanced aggregation properties. The fluorinated PBTz-F not only exhibits a low-lying HOMO level, but also has stronger face-on packing order and results in more uniform fibril-like interpenetrating networks in the related PF-BTz:L8-BO blend. A high-power conversion efficiency (PCE) of 18.57% is achieved. Moreover, PBTz-F also exhibits a good batch-to-batch reproducibility and general applicability. In addition, ternary blend OSCs based on the host PBTz-F:L8-BO blend and PM6 guest donor exhibits a further enhanced PCE of 19.54%, which is among the highest values of OSCs.

36 MATERIALS SCIENCE↗
DOE OSTI2018

A reproducing kernel enhanced approach for peridynamic solutions

The most common discretization method for peridynamic models used in engineering problems is the node-based meshfree approach. This method discretizes peridynamic domains by a set of nodes, each associated with a nodal cell with a characteristic volume, leading to a particle-based description of continuum systems. The behavior of each particle is then considered representative of its cell. This limits the convergence rate to the first order. In this paper, we introduce a reproducing kernel (RK) approximation to the field variables in the peridynamic equations to increase the order of convergence of peridynamic numerical solutions. Furthermore, the numerical results demonstrate improved convergence rates in static peridynamic problems using the proposed method.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Asymptotically compatible reproducing kernel collocation and meshfree integration for the peridynamic Navier equation

Here, we study reproducing kernel (RK) collocation method for peridynamic Navier equation. In the first part, we apply a linear RK approximation to both displacement and dilatation, and then back-substitute dilatation and solve the peridynamic Navier equation in a pure displacement form. The RK collocation scheme converges to the nonlocal limit for a fixed nonlocal interaction length and also to the local limit as nonlocal interactions vanish. The stability is shown by comparing the collocation scheme with the standard Galerkin scheme using Fourier analysis. In the second part, we apply the RK collocation to the quasi-discrete peridynamic Navier equation and show its convergence to the correct local limit when the ratio between the nonlocal length scale and the discretization parameter is fixed. The analysis is carried out on a special family of rectilinear Cartesian grids for the RK collocation method with a designated kernel with finite support. We assume the Lamé parameters satisfy λ ≥ μ to avoid extra assumptions on the nonlocal kernel. Finally, numerical experiments are conducted to validate the theoretical results.

42 ENGINEERING↗
DOE OSTI2024

Reproducible emission from nonlinear random lasers

Multiple scattering of light serves as a mechanism for feedback in random lasers. Consequently, internal spatial mode patterns, lasing wavelengths, and output directionality can all be random. Strong mode interaction can occur in such devices due to spatially overlapping modes resulting in nonlinearity with respect to the pump input power. Nevertheless, temporal coherence and lasing mode amplitude can be fixed at a constant pumping rate. This is a property desirable for applications where unique randomness is exploited but expected to be reliable over time, such as physical unclonable functions. Random lasers can also be cheaply and easily fabricated, exhibit relatively low lasing thresholds and high emission intensity. However, the precise scattering properties of such structures and fluctuations in the pump field can make device emission irreproducible, thereby limiting random laser applications. Here, in this work, we directly compare the random lasing spectra from zinc oxide samples fabricated in four distinct ways: spin-coating, sputtering, solgel deposition, and atomic layer deposition. The particular method of fabrication has a strong impact. Samples made through atomic layer deposition here exhibit both reproducibility and strong nonlinearity desirable for applications. Randomness in emission spectra persists across hundreds of repeated and averaged measurements irrespective of spatial location and is demonstrably nonlinear with respect to input signal intensity.

47 OTHER INSTRUMENTATION↗
DOE OSTI2026

Metal Foam Morphology Affects the Run to Run Reproducibility of OER Using Nickel Catalysts

Nickel (Ni) foam-based electrodes are excellent catalysts for the oxygen evolution reaction; however, we found that the random pore-size distribution of Ni foams contributes to a significant variability in electrochemically active surface area, compromising experimental reproducibility. We provide insights into quantifying this critical material property, verified by four electrochemical laboratories.

25 ENERGY STORAGE↗
DOE OSTI2022

The Ghost of Performance Reproducibility Past

The importance of ensemble computing is well established. However, executing ensembles at scale introduces interesting performance fluctuations that have not been well investigated. In this paper, we trace our experience uncovering performance fluctuations of ensemble applications (primarily constituting a workflow of GROMACS tasks), and unsuccessful attempts, so far, at trying to discern the underlying cause(s) of performance fluctuations. Is the failure to discern the causative or contributing factors a failure of capability? Or imagination? Do the fluctuations have their genesis in some inscrutable aspect of the system or software? Does it warrant a fundamental reassessment and rethinking of how we assume and conceptualize performance reproducibility? Answers to these questions are not straightforward, nor are they immediate or obvious. We conclude with a discussion about the performance of ensemble applications and ruminate over the implications for how we define and measure application performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Not just for programmers: How GitHub can accelerate collaborative and reproducible research in ecology and evolution

Abstract Researchers in ecology and evolutionary biology are increasingly dependent on computational code to conduct research. Hence, the use of efficient methods to share, reproduce, and collaborate on code as well as document research is fundamental. GitHub is an online, cloud‐based service that can help researchers track, organize, discuss, share, and collaborate on software and other materials related to research production, including data, code for analyses, and protocols. Despite these benefits, the use of GitHub in ecology and evolution is not widespread. To help researchers in ecology and evolution adopt useful features from GitHub to improve their research workflows, we review 12 practical ways to use the platform. We outline features ranging from low to high technical difficulty, including storing code, managing projects, coding collaboratively, conducting peer review, writing a manuscript, and using automated and continuous integration to streamline analyses. Given that members of a research team may have different technical skills and responsibilities, we describe how the optimal use of GitHub features may vary among members of a research collaboration. As more ecologists and evolutionary biologists establish their workflows using GitHub, the field can continue to push the boundaries of collaborative, transparent, and open research.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

Reproducibility and Replication of Experimental Particle Physics Results

Recently, much attention has been focused on the replicability of scientific results, causing scientists, statisticians, and journal editors to examine closely their methodologies and publishing criteria. Experimental particle physicists have been aware of the precursors of non-replicable research for many decades and have many safeguards to ensure that the published results are as reliable as possible. The experiments require large investments of time and effort to design, construct, and operate. Large collaborations produce and check the results, and many papers are signed by more than three thousand authors. This paper gives an introduction to what experimental particle physics is and to some of the tools that are used to analyze the data. It describes the procedures used to ensure that results can be computationally reproduced, both by collaborators and by non-collaborators. It describes the status of publicly available data sets and analysis tools that aid in reproduction and recasting of experimental results. It also describes methods particle physicists use to maximize the reliability of the results, which increases the probability that they can be replicated by other collaborations or even the same collaborations with more data and new personnel. Examples of results that were later found to be false are given, both with failed replication attempts and one with alarmingly successful replications. While some of the characteristics of particle physics experiments are unique, many of the procedures and techniques can be and are used in other fields.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Race conditions and data partitioning: risks posed by common errors to reproducible parallel simulations

When parallel algorithms for simulation were introduced in the 1970s, their development and use interested only experts in parallel computation. This circumstance changed as multi-core processors became commonplace, putting a parallel computer into the hands of every modeler. A natural outcome is growing interest in parallel simulation among persons not intimately familiar with parallel computing. At the same time, parallel simulation tools continue to be developed with the implicit assumption that the modeler is knowledgeable about parallel programming. The unintended consequence is a rapidly growing number of users of parallel simulation tools that are unlikely to recognize when the interaction of race conditions, partitioning strategies, and simultaneous action in their simulation models make results non-reproducible, thereby calling into question the validity of conclusions drawn from the simulation data. Here, we illustrate the potential dangers of exposing parallel algorithms to users who are not experts in parallel computation with example models constructed using existing parallel simulation tools. By doing so, we hope to refocus tool developers on usability, even if this new focus incurs loss of some performance.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Skill of ship‐following large‐eddy simulations in reproducing MAGIC observations across the northeast P acific stratocumulus to cumulus transition region

Abstract During the Marine ARM GPCI Investigation of Clouds (MAGIC) in October 2011 to September 2012, a container ship making periodic cruises between Los Angeles, CA, and Honolulu, HI, was instrumented with surface meteorological, aerosol and radiation instruments, a cloud radar and ceilometer, and radiosondes. Here large‐eddy simulation (LES) is performed in a ship‐following frame of reference for 13 four day transects from the MAGIC field campaign. The goal is to assess if LES can skillfully simulate the broad range of observed cloud characteristics and boundary layer structure across the subtropical stratocumulus to cumulus transition region sampled during different seasons and meteorological conditions. Results from Leg 15A, which sampled a particularly well‐defined stratocumulus to cumulus transition, demonstrate the approach. The LES reproduces the observed timing of decoupling and transition from stratocumulus to cumulus and matches the observed evolution of boundary layer structure, cloud fraction, liquid water path, and precipitation statistics remarkably well. Considering the simulations of all 13 cruises, the LES skillfully simulates the mean diurnal variation of key measured quantities, including liquid water path (LWP), cloud fraction, measures of decoupling, and cloud radar‐derived precipitation. The daily mean quantities are well represented, and daily mean LWP and cloud fraction show the expected correlation with estimated inversion strength. There is a −0.6 K low bias in LES near‐surface air temperature that results in a high bias of 5.6 W m −2 in sensible heat flux (SHF). Overall, these results build confidence in the ability of LES to represent the northeast Pacific stratocumulus to trade cumulus transition region.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2022

Advantages of a variable‐resolution global climate model in reproducing the seasonal evolution of East Asian summer monsoon

Abstract The East Asian summer monsoon (EASM) is unique among monsoon systems that it features meridional evolution of the summer monsoon. In this study, we evaluate the performances of a Variable‐Resolution Community Earth System Model (VR‐CESM) regionally refined over eastern China (14 km) in reproducing the seasonal evolution of EASM precipitation over China. Compared with reference datasets, VR‐CESM shows better performance than the corresponding globally uniform coarse‐resolution model CESM (quasi‐uniform 1°), especially over western China where complex local topography exists. The northward monsoon migration is closely related to low‐level southerly flows and vertical moisture advection, which are more reasonably simulated in VR‐CESM. The four critical timings of the EASM (monsoon onset, withdrawal, peak, and duration) are also better captured in VR‐CESM than in CESM. The corresponding spatial Pearson correlation coefficients of the four critical timings with respect to reference datasets are about 0.1 higher in VR‐CESM than those in CESM. Both models are most accurate in simulating monsoon onset and least accurate at simulating the monsoon peak. The overestimated zonal thermal contrast in CESM is responsible for the earlier monsoon onset and excessive precipitation in September over the Yangtze River valley. Finer resolution in VR‐CESM, especially over the Tibetan Plateau (TP), appears to be a main factor in simulating better zonal thermal contrast and seasonal evolution of the EASM.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Kernel Manifolds: Nonlinear‐Augmentation Dimensionality Reduction Using Reproducing Kernel Hilbert Spaces

This paper generalizes recent advances on quadratic manifold (QM) dimensionality reduction by developing kernel methods-based nonlinear-augmentation dimensionality reduction. QMs, and more generally feature map-based nonlinear corrections, augment linear dimensionality reduction with a nonlinear correction term in the reconstruction map to overcome approximation accuracy limitations of purely linear approaches. While feature map-based approaches typically learn a least squares optimal polynomial correction term, we generalize this approach by learning an optimal nonlinear correction from a user-defined reproducing kernel Hilbert space. Our approach allows one to impose arbitrary nonlinear structure on the correction term, including polynomial structure, and includes feature map and radial basis function-based corrections as special cases. Furthermore, our method has relatively low training cost and has monotonically decreasing error as the latent space dimension increases. In conclusion, we compare our approach to proper orthogonal decomposition and several recent QM approaches on data from several example problems.

kernel methods↗
DOE OSTI2022

A neural network‐enhanced reproducing kernel particle method for modeling strain localization

Abstract Modeling the localized intensive deformation in a damaged solid requires highly refined discretization for accurate prediction, which significantly increases the computational cost. Although adaptive model refinement can be employed for enhanced effectiveness, it is cumbersome for the traditional mesh‐based methods to perform while modeling the evolving localizations. In this work, neural network‐enhanced reproducing kernel particle method (NN‐RKPM) is proposed, where the location, orientation, and shape of the solution transition near a localization is automatically captured by the NN approximation via a block‐level neural network (NN) optimization. The weights and biases in the blocked parameterization network control the location and orientation of the localization. The designed basic four‐kernel NN block is capable of capturing a triple junction or a quadruple junction topological pattern, while more complicated localization topological patters are captured by the superposition of multiple four‐kernel NN blocks. The standard RK approximation is then utilized to approximate the smooth part of the solution, which permits a much coarser discretization than the high‐resolution discretization needed to capture sharp solution transitions with the conventional methods. A regularization of the NN approximation is additionally introduced for discretization‐independent material responses. The effectiveness of the proposed NN‐RKPM is verified by a series of numerical verifications.

Baek, Jonghyuk↗
DOE OSTI2021

Understanding Reproducibility of Sputter‐Deposited Metastable Ferroelectric Wurtzite Al 0.6 Sc 0.4 N Films Using In Situ Optical Emission Spectrometry

High‐Sc Al 1– x Sc x N thin films are of tremendous interest because of their attractive piezoelectric and ferroelectric properties, but overall film quality and reproducibility are widely reported to suffer as x increases. In this study, structural and electrical properties of metastable Al 0.6 Sc 0.4 N films are connected with plasma changes during film growth, identified via glow discharge optical emission spectroscopy (GD‐OES), and linked to the target mode changes. This in situ GD‐OES technique uses changes in the N 2 (I) intensity, correlated with DC bias hysteresis behavior of a Al 0.6 Sc 0.4 target in metallic and poisoned modes, to identify films that subsequently exhibit unacceptable structural and electrical performance. Two representative samples deposited under identical conditions but possessing distinct properties related to phases present in the films are focused on. Films sputtered under a poisoned target mode produce pure wurtzite ferroelectric Al 0.6 Sc 0.4 N with a reversible 80 μC cm −1 polarization and 3.1 MV cm −1 coercive field. When identical chamber settings are used but the process starts in metallic mode, a mixed wurtzite/rocksalt film is deposited which exhibits nanometer‐scale changes to the film microstructure and a nonferroelectric response. These results illustrate the utility of optical emission spectroscopy for tracking target mode fluctuations when fabricating metastable materials such as high‐Sc Al 1– x Sc x N films.

36 MATERIALS SCIENCE↗