SEARCH · Search NASA

Results for “reproducibility of results”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

All sources NASA NTRS DOE OSTI

DOE OSTI2021

Learning Universal Authorship Representations

This code contains all the utilities required to reproduce the results of our EMNLP 2021 paper "Learning Universal Authorship Representations". It contains the utilities required for download the datasets, training our model, and performing all evaluations necessary for reproducing the results in the paper. Here's the abstract of our work: Determining whether two documents were composed by the same author, also known as authorship verification, has traditionally been tackled using statistical methods. Recently, authorship representations learned using neural networks have been found to outperform alternatives, particularly in large-scale settings involving hundreds of thousands of authors. But do such representations learned in a particular domain transfer to other domains? Or are these representations inherently entangled with domain-specific features? To study these questions, we conduct the first large-scale study of cross-domain transfer for authorship verification considering zero-shot transfers involving three disparate domains: Amazon reviews, fanfiction short stories, and Reddit comments. We find that although a surprising degree of transfer is possible between certain do- mains, it is not so successful between others. We examine properties of these domains that influence generalization and propose simple but effective methods to improve transfer.

Rivera Soto, RafaelA↗

NASA NTRS2010

Effect of Post-HALT Annealing on Leakage Currents in Solid Tantalum Capacitors

Degradation of leakage currents is often observed during life testing of tantalum capacitors and is sometimes attributed to the field-induced crystallization in amorphous anodic tantalum pentoxide dielectrics. However, degradation of leakage currents and the possibility of annealing of degraded capacitors have not been investigated yet. In this work the effect of annealing after highly accelerated life testing (HALT) on leakage currents in various types of solid tantalum capacitors was analyzed. Variations of leakage currents with time during annealing at temperatures from 125 oC to 180 oC, thermally stimulated depolarization (TSD) currents, and I-V characteristics were measured to understand the conduction mechanism and the reason for current degradation. Annealing resulted in a gradual decrease of leakage currents and restored their initial values. Repeat HALT after annealing resulted in reproducible degradation of leakage currents. The observed results are explained based on ionic charge instability (drift/diffusion of oxygen vacancies) in the tantalum pentoxide dielectrics using a modified Schottky conduction mechanism.

Teverovsky, Alexander↗

DOE OSTI2021

Predicting nepheline precipitation in waste glasses using ternary submixture model and machine learning

Nepheline precipitation in nuclear waste glasses during vitrification can be detrimental due to its negative effect on chemical durability. Developing models to accurately predict nepheline precipitation from compositions is important to increase waste loading since existing models can be overly conservative. In this study, an expanded dataset containing 955 glasses was compiled from literature data, where 355 glasses are for high-level waste (HLW). Previously developed submixture models were refitted using the new dataset, where a misclassification rate of 7.8% was achieved. Nine machine learning (ML) algorithms (e.g., k-nearest neighbor, Gaussian process regression, artificial neural network, support vector machine, decision tree, etc.) were applied to evaluate their ability of predicting nepheline precipitation from compositions. Model accuracy, precision, recall/sensitivity, and F1 score were systemically compared between different ML algorithms and modeling protocols. Good model prediction with an accuracy ~0.9 (misclassification rate of ~10%) was observed with different algorithms under certain protocol. This study evaluated various ML models to predict nepheline precipitations in waste glasses, highlighting the importance of data preparation, modeling protocol, and their effect on model stability and reproducibility. The results provide insights into applying ML to predict glass properties and suggest areas for future research on modeling nepheline precipitations.

Lu, Xiaonan↗

NASA NTRS2016

Using Docker Containers to Extend Reproducibility Architecture for the NASA Earth Exchange (NEX)

NASA Earth Exchange (NEX) is a data, supercomputing and knowledge collaboratory that houses NASA satellite, climate and ancillary data where a focused community can come together to address large-scale challenges in Earth sciences. As NEX has been growing into a petabyte-size platform for analysis, experiments and data production, it has been increasingly important to enable users to easily retrace their steps, identify what datasets were produced by which process chains, and give them ability to readily reproduce their results. This can be a tedious and difficult task even for a small project, but is almost impossible on large processing pipelines. We have developed an initial reproducibility and knowledge capture solution for the NEX, however, if users want to move the code to another system, whether it is their home institution cluster, laptop or the cloud, they have to find, build and install all the required dependencies that would run their code. This can be a very tedious and tricky process and is a big impediment to moving code to data and reproducibility outside the original system. The NEX team has tried to assist users who wanted to move their code into OpenNEX on Amazon cloud by creating custom virtual machines with all the software and dependencies installed, but this, while solving some of the issues, creates a new bottleneck that requires the NEX team to be involved with any new request, updates to virtual machines and general maintenance support. In this presentation, we will describe a solution that integrates NEX and Docker to bridge the gap in code-to-data migration. The core of the solution is saemi-automatic conversion of science codes, tools and services that are already tracked and described in the NEX provenance system, to Docker - an open-source Linux container software. Docker is available on most computer platforms, easy to install and capable of seamlessly creating and/or executing any application packaged in the appropriate format. We believe this is an important step towards seamless process deployment in heterogeneous environments that will enhance community access to NASA data and tools in a scalable way, promote software reuse, and improve reproducibility of scientific results.

earth exchange↗

DOE OSTI2020

Reproducibility and Replication of Experimental Particle Physics Results

Recently, much attention has been focused on the replicability of scientific results, causing scientists, statisticians, and journal editors to examine closely their methodologies and publishing criteria. Experimental particle physicists have been aware of the precursors of non-replicable research for many decades and have many safeguards to ensure that the published results are as reliable as possible. The experiments require large investments of time and effort to design, construct, and operate. Large collaborations produce and check the results, and many papers are signed by more than three thousand authors. This paper gives an introduction to what experimental particle physics is and to some of the tools that are used to analyze the data. It describes the procedures used to ensure that results can be computationally reproduced, both by collaborators and by non-collaborators. It describes the status of publicly available data sets and analysis tools that aid in reproduction and recasting of experimental results. It also describes methods particle physicists use to maximize the reliability of the results, which increases the probability that they can be replicated by other collaborations or even the same collaborations with more data and new personnel. Examples of results that were later found to be false are given, both with failed replication attempts and one with alarmingly successful replications. While some of the characteristics of particle physics experiments are unique, many of the procedures and techniques can be and are used in other fields.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

DOE OSTI2023

Microphysical Modeling of Fault Slip and Stability Transition in Hydrothermal Conditions

Abstract Field and laboratory observations indicate that the frictional behaviors of faults depend on hydrothermal conditions. We extend the microphysical Chen‐Niemeijer‐Spiers (CNS) model to hydrothermal conditions by using the observed temperature variation of indentation hardness to infer the temperature dependence of a microphysical parameter . This parameter is assumed constant in previous versions of the CNS model. A simple spring‐slider system is used to simulate the fault system and investigate the steady‐state frictional behaviors of wet granite gouges. Our numerical results quantitatively reproduce experimental data showing the frictional‐plastic transition. The results also describe the transition from velocity‐strengthening at low temperatures (<160°C), to velocity‐weakening at intermediate temperatures (160°C–370°C), then back to velocity‐strengthening at high temperatures (>370°C). In our extended CNS model, these results suggest that the dominant shear deformation mechanism does transition from frictional granular flow to fully plastic creep with increasing temperature.

58 GEOSCIENCES↗

NASA NTRS1993

Cluster evolution as a probe of primordial density fluctuations

Although COBE's detection of large angle microwave background anisotropies fixes the amplitude of density fluctuations on length scales k exp -1 approximately = (300-6000) h(exp -1)Mpc, what is crucial for the level of large scale clustering is the amplitude of density fluctuations on scales (5-50) h(exp -1)Mpc. The level of dynamical clustering is usually parameterized by the size of the mass fluctuations in 8 h exp -1 Mpc spheres, sigma sub 8. For the cold dark matter model, COBE gives sigma sub 8 approximately = 1, while models with extra large scale power give sigma sub 8 approximately = 1/2. The most massive clusters of galaxies (greater than or approximately = 10 exp 15 solar mass) form from rare 'peak patches' found in the initial mass density distribution. Their abundance as a function of redshift is a sensitive probe of the wave number band k(exp -1) approx. (3-8) h(exp -1)Mpc, hence of sigma sub 8, and so cluster evolution can discriminate among models allowed by the COBE results. We use our Hierarchical Peaks Method, which accurately reproduces the results of P3M N-body simulations, to calculate the evolution of cluster x-ray flux counts, luminosity, and temperature functions as a function of sigma sub 8 for CDM models and those with more large scale power. We find that the EMSS and Edge et al. cluster samples support sigma sub 8 in the range from approx. 0.6-0.9, and that models with more large scale power (and hence flatter fluctuation spectra in the cluster regime) fit the x-ray bright end better.

Bond, J. Richard↗

NASA NTRSDate not supplied

NASA Life Sciences Portal (NLSP): Supporting Scientific Transparency and Reproducibility

NASA’s Life Sciences Ports (NLSP) serves the scientific community by providing curated data from space life science experiment. The Human Research Program (HRP) with the help of NLSP is currently transforming their life sciences data archive systems and processes to improve compliance with the FAIR principles [1]. Some of these improvements will at the same time support the twin pillars of Open Science [2]: transparency of methods and reproducibility of results. Scientific transparency is marked by the easily intelligible communication of what has been investigated: what were the procedures for collecting sample and the characteristics of samples collected? what kinds of measurements were made, what were the environmental conditions of the measurements? What were the analysis techniques of the collected data? Reproducibility of the results and findings from the investigation requires a high level of transparency for all but the simplest investigations; the slightest deviation in communicating and replicating complex experimental procedures or data analyses can often yield quite different data and even findings, thwarting their validation. One of the ways the NLSP is aiming to improve the communication of scientific information is through the use of ontology-driven metadata. Ontologies are powerful, graph-based knowledge representation structures, which can be leveraged to increase data interoperability, the area of the FAIR principles in which many data systems most lack compliance. Over the past decade, there has been a concerted effort in the biomedical community to develop modular and narrowly focused domain and application-specific ontologies in a common, open-source framework, the Open Biological and Biomedical Ontology (OBO) Foundry [3]. The open sharing and modular nature of this effort promises huge increases in harmonized data sharing for systems that leverage these models. Which is in line with the FAIR Data Principles of Findability, Accessibility, Interoperability, and Reuse for scientific data management and stewardship. 1. Wilkinson, M.D., et al., The FAIR Guiding Principles for scientific data management and stewardship. Sci Data, 2016. 3: p. 160018. 2. National Academies of Sciences, E. and Medicine, Open Science by Design: Realizing a Vision for 21st Century Research. 2018, Washington, DC: The National Academies Press. 232. 3. Smith, B., et al., The OBO Foundry: coordinated evolution of ontologies to support biomedical data integration. Nat Biotechnol, 2007. 25(11): p. 1251-5.

Life Sciences data↗

NASA NTRSDate not supplied

NASA Life Sciences Portal (NLSP): Supporting Scientific Transparency and Reproducibility

Life Sciences data↗

DOE OSTI2021

Evolution of microstructures in radiation fields using a coupled binary-collision Monte Carlo phase field approach

The simulation of radiation effects in materials broadly falls into two categories. At short time and length scales lies the modeling of primary radiation damage, such as point defect creation, energy deposition, and ballistic mixing. This is followed by the modeling at longer time scales of thermally activated microstructure evolution and defect reactions, such as recombination, clustering, and coarsening. The binary collision Monte Carlo method is an established, numerically efficient method for the computation of primary radiation damage. Conversely, the phase field method is a state of the art method for microstructure evolution on longer time and length scales. Here we present a concurrent coupling of these two methods, overcoming the difference between the discrete object Monte Carlo paradigm for primary radiation damage and the continuum field variable approach for microstructure evolution. The coupling is bidirectional, in which the microstructure evolution in the MOOSE finite element frame- work provides the spatial scattering data set for the charged particle trans- port and receives point defect, mass transport, and heat source terms from the simulated collision cascades that contribute to the field variable evolution. The concurrent coupling scheme is implemented in the code Magpie and demonstrated by investigating patterning for a model irradiated immiscible binary alloy. The results from the coupled binary collision Monte Carlo/phase field simulations reproduce the results of analytical models for phase separation, phase mixing, and patterning, supporting the approach and indicating its utility for modeling real materials systems.

36 MATERIALS SCIENCE↗

DOE OSTI2022

Full versus quasiparticle self-consistency in vertex-corrected GW approaches

Here, using seven semiconductors/insulators with band gaps covering the range from 1 eV to 10 eV we systematically explore the performance of two different variants of self-consistency associated with famous Hedin's system of equations: the full self-consistency and the so called quasi-particle approximation to it. The pros and cons of these two variants of self-consistency are sufficiently well documented in literature for the simplest GW approximation to the Hedin's equations. Our study, therefore, aims primarily at the level of theory beyond GW approximation, i.e. at the level of theory which includes vertex corrections. Whereas quasi-particle self-consistency has certain advantages at GW level (well known fact), the situation becomes quite different when vertex corrections are included. In the variant with full self-consistency, vertex corrections (both for polarizability and for self energy) systematically reduce the calculated band gaps making them closer to the experimental values. In the variant with quasi-particle self-consistency, however, an inclusion of the same diagrams has considerably larger effect and calculated band gaps become severely underestimated. Different effect of vertex corrections in two variants of self-consistency can be related to the Z-factor cancellation which plays positive role in quasi-particle self-consistency at GW level of theory but appears to be destructive for the quasi-particle approximation when higher order diagrams are included. Second result of our study is that we were able to reproduce the results obtained with the Questaal code using our FlapwMBPT code when the same variant of self-consistency (quasi-particle) and the same level of vertex corrections (for polarizability only, static approximation for screened interaction, and Tamm-Dancoff approximation for the Bethe-Salpeter equation) are used.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

NASA NTRS2022

The DIARieS Ecosystem – A Software Ecosystem to Simplify Discovery, Implementation, Analysis, Reproducibility, and Sharing of Scientific Results and Environments in Heliophysics

The infrastructure of the Heliophysics discipline has promising components but with several missing gaps, drastically reducing research and development efficiency. Developing an online discovery and analysis software ecosystem will close several of these gaps. The five main focuses on this ecosystem should be Discovery, Implementation, Analysis, Reproducibility, and Sharing of results (DIARieS). In this paper, we give a detailed description of how the proposed software ecosystem should operate, and point out the large range of possible applications to benefit many disparate groups, such as researchers, operational staff, decision-makers, and educators. The infrastructure components and technological capabilities necessary for its completion are either currently available or in development, making such an ecosystem possible for the first time. One main focus of current infrastructure investments must be to adapt and connect these pieces together into a cohesive whole to increase our research and development efficiency.

infrastructure↗

DOE OSTI2023

Dataset for 'Ombadi et al. (2023). A warming-induced reduction in snow fraction amplifies rainfall extremes, Nature'

This package contains the main codes, sample input data and main result files to reproduce the analysis and results presented in the article: “Ombadi et al. (2023), A warming-induced reduction in snow fraction amplifies rainfall extremes, Nature”. The folder consists of the following: (1) “Raw data”: a folder that contains sample input data which is used in some of the codes for demonstration purposes. It also contains data that was not pre-processed such as Elevation data; (2) “Results”: this folder contains files of the main results presented in the paper including: “Annual-Max-Series”, “Change-rainfall-extremes”, “Change-snow-fraction”, “Warming levels_By scenario_model_year” and “Masks”. Description of these folders is detailed in the "Readme.rtf" file; (3) Python jupyter notebooks (Extract_Annual Max Series (AMS).ipynb, Elevation-dependent amplification of rainfall extremes.ipynb, Sensitivity_to_global_warming.ipynb) demonstrate the main steps of analysis. Further description of those notebooks is provided in the "Readme.rtf" file; (4) R code for extreme value analysis (Extreme_Value_Analysis.R). The sample and pre-processed dataset in "Raw data" is obtained from publicly available repositories of CMIP6 and ERA5 datasets; see Methods for more detail. This research was supported by Office of Science, Office of Biological and Environmental Research of the US Department of Energy under contract no. DE-AC02-05CH11231 for the CASCADE Scientific Focus (funded by the Regional and Global Model Analysis Program area within the Earth and Environmental Systems Modeling Program) and the iNAIADS Early Career Research Project (funded by the Environmental Systems Science program).

54 ENVIRONMENTAL SCIENCES↗

DOE OSTI2023

Towards Redefining the Reproducibility in Quantum Computing: A Data Analysis Approach on NISQ Devices

Although the building of quantum computers has kept making rapid progress in recent years, noise is still the main challenge for any application to leverage the power of quantum computing. Existing works addressing noise in quantum devices proposed noise reduction when deploying a quantum algorithm to a specified quantum computer. The reproducibility issue of quantum algorithms has been raised since the noise levels vary on different quantum computers. Importantly, existing works largely ignore the fact that the noise of quantum devices varies as time goes by. Therefore, reproducing the results on the same hardware will even become a problem. We analyze the reproducibility of quantum machine learning (QML) algorithms based on daily model training and execution data collection. Our analysis shows a correlation between our QML models’ test accuracy and quantum computer hardware’s calibration features. We also demonstrate that noisy simulators for quantum computers are not a reliable tool for quantum machine learning applications.

Senapati, Priyabrata↗

DOE OSTI2024

Gupta-et-al_2024_EarthsFuture

Results from Gupta et al. submitted to Earth's Future. All code to reproduce the experiment and make the figures can be found here: https://github.com/rg727/Gupta-etal_2024_EarthsFuture The data provided in this repository are (1) Weather Regime Data , (2) Hydroclimate Data, and (3) CALFEWS output. In (1), there are Markov chains of daily weather regimes generated over the 600-year paleo-period. In (2), there are three sets of data: Historical daily CDEC data for 12 input locations into CALFEWS, 600-year long daily paleo data (streamflow and snow) for each input location, and (3) 600-year long daily climate-change data (4 degree temperature increase + 7% precipitation scaling applied to (2)) which serves as the "climate change scenario" in the study. Please reference the GitHub repository on how to use these data to reproduce the results. The CALFEWS output for the Paleo and Climate Change scenarios is stored in (3). More information can be found in the Gupta-et-al_2024_EarthsFuture-README file.

Gupta, Rohini↗

NASA NTRS2019

Quasi-Static and Dynamic Analysis of Composite Panels: Characterization and Validation

The predictive capability of a progressive damage model to simulate the quasi-static and dynamic crushing of composite specimens is evaluated in this article. The material model is commercially available within the ABAQUS finite element software package, i.e., “Damage for Fiber reinforced Composites”, and is often utilized to simulate damage progression and failure in static and dynamic failure scenarios such as crushing of a composite plate. The composite specimens used in this study consist of quasi-isotropic C-shaped and corrugated composite panels comprised of carbon fiber/epoxy braided fabric. The C-shaped panel specimen is used to calibrate the model and optimize material/model properties through comparison with experimental results. Then, using the same model parameters, the quasi-static crush response of the corrugated panel is predicted. The predictive capability of the model is further demonstrated by simulating the dynamic crushing response of both C-shaped and corrugated panels using the same material properties as in the quasi-static crush case with only minor adjustments for the dynamic elastic properties (i.e., rate dependent modulus). Results show that the in-built material model available in ABAQUS can successfully reproduce experimental results for both panels (C-shaped and corrugated) subjected to both quasi-static and dynamic loading scenarios. A small-scale parametric study on physical (experimentally measurable) and nonphysical (purely mathematical) model parameters was also conducted. It can be concluded that, achieving successful simulation results requires a more in-depth understanding of the influence of model parameter variation on the mechanical response of the composite together with the strategies and challenges of the utilized modeling methodology (e.g., FEA mesh density, etc.).

Naghipour Ghezeljeh, Paria↗

DOE OSTI2021

Toward Complete Leading-Order Predictions for Neutrinoless Double β Decay

The amplitude for the neutrinoless double β (0νββ) decay of the two-neutron system nn → ppe – e – constitutes a key building block for nuclear-structure calculations of heavy nuclei employed in large-scale 0νββ searches. Assuming that the 0νββ process is mediated by a light-Majorana-neutrino exchange, a systematic analysis in chiral effective field theory shows that already at leading order a contact operator is required to ensure renormalizability. Here, we develop a method to estimate the numerical value of its coefficient (in analogy to the Cottingham formula for electromagnetic contributions to hadron masses) and validate the result by reproducing the charge-independence-breaking contribution to the nucleon-nucleon scattering lengths. Our central result, while derived in dimensional regularization, is given in terms of the renormalized amplitude Aν(|p|, |p'|) , matching to which will allow one to determine the contact-term contribution in regularization schemes employed in nuclear-structure calculations. Our results thus greatly reduce a crucial uncertainty in the interpretation of searches for 0νββ decay.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

DOE OSTI2021

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) peak intensity normalization for complex mixture analyses

Fourier transform ion cyclotron resonance mass spectrometry (FT-ICR-MS) is a preferred technique for analyzing complex organic mixtures. Currently, there is no consensus normalization approach, nor an objective method for selecting one, for quantitative analyses of FT-ICR-MS data. We investigate a method to evaluate and score the amount of bias various normalization approaches introduce into the data. We evaluate the ability of the Statistical Procedure for the Analysis of Normalization Strategies (SPANS) to guide selection of appropriate normalization approaches for two different FT-ICR-MS datasets. Further, we test the robustness of SPANS results to changes in SPANS parameter values and assess the impact of using various normalization approaches on downstream statistical analyses. The normalization approach identified by SPANS differed for the two datasets. Normalization methods impacted the statistical significance of peaks differently, underscoring the importance of carefully evaluating potential methods. More consistent SPANS scores resulted when at least 120 significant peaks are used, where larger sets of peaks were obtained by increasing the p-value threshold. Interestingly, we show that Total Sum Scaling and highest peak normalization, used in previous studies, underperformed relative to SPANS-recommended normalization approaches. While there is no single, best normalization method for all datasets, SPANS provides a mechanism to identify an appropriate normalization method for analyzing FT-ICR-MS data quantitatively. As a result, the number of peaks used in the background distributions of SPANS contributes more significantly to the reproducibility of results than the p-value thresholds used to obtain those peaks.

47 OTHER INSTRUMENTATION↗