Search NASA⌕ Search

SEARCH · Search NASA

Results for “Reproducibility”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Reproducibility and Optimization Techniques for Evaluated Data in the Resolved Resonance Region [Slides]

This presentation discusses the experimental setup input parameters that are basic quantities for reproducibility. The goal is to increase quality of the evaluated data and decrease time needed for an evaluation. Full reproducibility should be achieved efficiently by including any type of constraints in the optimization procedure and to improve computational time to reach convergence. The goal of an automated reproducibility aligns to the goal of “certified” evaluated data, i.e., defined by a well-defined metric.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A reproducibility study of "Augmenting Genetic Algorithms with Deep Neural Networks for Exploring the Chemical Space"

Nigam et al. reported a genetic algorithm (GA) utilizing the SELFIES representation and also propose an adaptive, neural network-based penalty that is supposed to improve the diversity of the generated molecules. The main claims of the paper are that this GA outperforms other generative techniques (as measured by the penalized logP) and that a neural network-based adaptive penalty increases the diversity of the generated molecules. In this work, we investigated the reproducibility of their claims. Overall, we were able to reproduce comparable results using the SELFIES-based GA, but mostly by exploiting deficiencies of the (easily optimizable) fitness function (i.e., generating long, sulfur containing chains). In addition, we reproduce results showing that the discriminator can be used to bias the generation of molecules to ones that are similar to the reference set. Lastly, we attempted to quantify the evolution of the diversity, understand the influence of some hyperparameters, and propose improvements to the adaptive penalty.

Jablonka, Kevin Maik↗

Image-based modeling of coupled electro-chemo-mechanical behavior of Li-ion battery cathode using an interface-modified reproducing kernel particle method

Abstract An interface-modified reproducing kernel particle method (IM-RKPM) is introduced in this work to allow for a direct model construction from image pixels of heterogeneous polycrystalline Li-ion battery microstructures. The interface-modified reproducing kernel (IM-RK) approximation is constructed through scaling of a kernel function by a regularized distance function in conjunction with strategic placement of interface node locations. This leads to RK shape functions with either weak or strong discontinuities across material interfaces, suitable for modeling various interface mechanics. With the placement of a triple junction node and distance-based scaling of kernel functions, the resulting IM-RK shape function also possesses proper discontinuities at the triple junctions. This IM-RK approximation effectively remedies the well-known Gibb’s oscillation in the smooth approximation of discontinuities. Different from the conventional meshfree approaches for interface discontinuities, this IM-RK approach is done without additional degrees of freedom associated with the enrichment functions, and it is formulated with the standard procedures in the RK shape function construction. This work focuses on identifying the accuracy and convergence properties of IM-RKPM for modeling the coupled electro-chemo-mechanical system. A linear patch test is formulated and numerically tested for the electro-chemo-mechanical coupled problem with a Butler–Volmer boundary condition representing the physical conditions in Li-ion battery microstructures. This is followed by verification of the optimal rates of convergence of IM-RKPM for solving the coupled problem with higher order solutions. The image-based modeling of Li-ion battery microstructures in the numerical examples demonstrates the applicability of the proposed method to realistic Li-ion battery materials modeling.

25 ENERGY STORAGE↗

Reproducible, Component-based Modeling with TopoFlow, A Spatial Hydrologic Modeling Toolkit

Modern geoscientists have online access to an abundance of different data sets and models, but these resources differ from each other in myriad ways and this heterogeneity works against interoperability as well as reproducibility. The purpose of this paper is to illustrate the main issues and some best practices for addressing the challenge of reproducible science in the context of a relatively simple hydrologic modeling study for a small Arctic watershed near Fairbanks, Alaska. This study requires several different types of input data in addition to several, coupled model components. All data sets, model components and processing scripts (e.g. for preparation of data and figures, and for analysis of model output) are fully documented and made available online at persistent URLs. Similarly, all source code for the models and scripts is open-source, version controlled and made available online via GitHub. Each model component has a Basic Model Interface (BMI) to simplify coupling and its own HTML help page that includes a list of all equations and variables used. The set of all model components (TopoFlow) has also been made available as a Python package for easy installation. Three different graphical user interfaces for setting up TopoFlow runs are described, including one that allows model components to run and be coupled as web services.

58 GEOSCIENCES↗

A quasi-meshfree method for constructing boundary-aware reproducing bases on geometrically complex domains using manifold geodesics

In many applications, physical domains are geometrically complex making it challenging to perform coarse-scale approximation. A defeaturing process is often used to simplify the domain in preparation for approximation and analysis at the coarse scale. In this paper, a methodology is presented for constructing a coarse-scale reproducing basis on geometrically complex domains given an initial fine-scale mesh of the fully featured domain. The initial fine-scale mesh can be of poor quality and extremely refined. The construction of the basis functions begins with a coarse-scale covering of the domain and generation of weighting functions with local support. Manifold geodesics are used to define distances within the local support for general applicability to non-convex domains. Conventional moving least squares is used to construct the coarse-scale reproducing basis. Applications in quasi-interpolation and linear elasticity are presented.

97 MATHEMATICS AND COMPUTING↗

Towards an Open (Data) Science Analytics-Hub for Reproducible Multi-Model Climate Analysis at Scale

Open Science is key to future scientific research and promotes a deep transformation in the whole scientific research process encouraging the adoption of transparent and collaborative scientific approaches aimed at knowledge sharing. Open Science is increasingly gaining attention in the current and future research agenda worldwide. To effectively address Open Science goals, besides Open Access to results and data, it is also paramount to provide tools or environments to support the whole research process, in particular the design, execution and sharing of transparent and reproducible experiments, including data provenance (or lineage) tracking. This work introduces the Climate Analytics-Hub, a new component on top of the Earth System Grid Federation (ESGF), which joins big data approaches and parallel computing paradigms to provide an Open Science environment for reproducible multi-model climate change data analytics experiments at scale. An operational implementation has been set up at the SuperComputing Centre of the Euro- Mediterranean Center on Climate Change, with the main goal of becoming a reference Open Science hub in the climate community regarding the multi-model analysis based on the Coupled Model Intercomparison Project (CMIP).

analytics-hub↗

CUF-Links: Continuous and Ubiquitous FAIRness Linkages for Reproducible Research

Despite much creative work on methods and tools, reproducibility— the ability to repeat the computational steps used to obtain a research result—remains elusive. One reason for these difficulties is that extant tools for capturing research processes, while powerful, often fail to capture vital connections as research projects grow in extent and complexity. We explain here how these interstitial connections can be preserved via simple methods that integrate easily with current work practices to capture basic information about every data product consumed or produced in a project. By thus extending the scope of findable, accessible, interoperable, and reusable (FAIR) data in both time and space to enable the creation of a continuous chain of Continuous and Ubiquitous FAIRness linkages (CUFlinks) from inputs to outputs, such mechanisms can facilitate capture of the provenance linkages that are essential to reproducible research. Herein we give examples of mechanisms that can facilitate the use of these methods, and review how they have been applied in practice.

97 MATHEMATICS AND COMPUTING↗

Planetary normal mode computation: Parallel algorithms, performance and reproducibility

This report is an extension of work entitled “Computing planetary interior normal modes with a highly parallel polynomial filtering eigensolver.” by Shi et al., [1] originally presented at the SC18 conference. A highly parallel polynomial filtered eigensolver was developed and exploited to calculate the planetary normal modes. The proposed method is ideally suited for computing interior eigenpairs for large-scale eigenvalue problems as it greatly enhances memory and computational efficiency. In this article, the second-order finite element method is used to further improve the accuracy as only the first-order finite element method was deployed in the previous work. The parallel algorithm, its parallel performance up to 20k processors, and the great computational accuracy are illustrated. The reproducibility of the previous work was successfully performed on the Student Cluster Competition at the SC19 conference by several participant teams using a completely different Mars-model dataset on different clusters. Both weak and strong scaling performances of the reproducibility by the participant teams were impressive and encouraging. The analysis and reflection of their results are demonstrated and future direction is discussed.

97 MATHEMATICS AND COMPUTING↗

Ongoing solution reproducibility of earth system models as they progress toward exascale computing

We present a methodology for solution reproducibility for the Energy Exascale Earth System Model during its ongoing software infrastructure development to prepare for exascale computers. The nonlinear chaotic nature of climate system simulations precludes traditional model verification approaches since machine precision differences—resulting from code refactoring, changes in software environment, and so on—grow exponentially to a different weather state. Here, we leverage the nature of climate as a statistical description of the atmosphere in order to establish model reproducibility. We evaluate the degree to which two-sample equality of distribution tests can confidently detect the change in climate from minor tuning parameter changes on model output variables in order to establish the level of difference that indicates a new climate. To apply this (baselined test), we target a section of the model’s development cycle wherein no intentional science changes have been applied to its source code. We compare an ensemble of short simulations that were conducted using a verified model configuration against a new ensemble with the same configuration but with the latest software infrastructure (Common Infrastructure for Modeling the Earth, CIME5.0), compiler versions, and software libraries. We also compare these against ensemble simulations conducted using the original version of the software infrastructure (CIME4.0) of the earlier model configuration, but with the latest compilers and software libraries, to test the impact of new compilers and libraries in isolation from additional software infrastructure. The two-sample equality of distribution tests indicates that these ensembles indeed represent the same climate.

97 MATHEMATICS AND COMPUTING↗

Efforts to enhance reproducibility in a human performance research project

Background: Ensuring the validity of results from funded programs is a critical concern for agencies that sponsor biological research. In recent years, the open science movement has sought to promote reproducibility by encouraging sharing not only of finished manuscripts but also of data and code supporting their findings. While these innovations have lent support to third-party efforts to replicate calculations underlying key results in the scientific literature, fields of inquiry where privacy considerations or other sensitivities preclude the broad distribution of raw data or analysis may require a more targeted approach to promote the quality of research output. Methods: We describe efforts oriented toward this goal that were implemented in one human performance research program, Measuring Biological Aptitude, organized by the Defense Advanced Research Project Agency's Biological Technologies Office. Our team implemented a four-pronged independent verification and validation (IV&V) strategy including 1) a centralized data storage and exchange platform, 2) quality assurance and quality control (QA/QC) of data collection, 3) test and evaluation of performer models, and 4) an archival software and data repository. Results: Our IV&V plan was carried out with assistance from both the funding agency and participating teams of researchers. QA/QC of data acquisition aided in process improvement and the flagging of experimental errors. Holdout validation set tests provided an independent gauge of model performance. Conclusions: In circumstances that do not support a fully open approach to scientific criticism, standing up independent teams to cross-check and validate the results generated by primary investigators can be an important tool to promote reproducibility of results.

59 BASIC BIOLOGICAL SCIENCES↗

Reproducibility and Variability of I/O Performance on BG/Q: Lessons Learned from a Data Aggregation Algorithm

Reading and writing data efficiently from different tiers of storage is necessary for most scientific simulations to achieve good performance at scale. Many software solutions have been developed to decrease the I/O bottleneck. One wellknown strategy, in the context of collective I/O operations, is the two-phase I/O scheme. This strategy consists of selecting a subset of processes to aggregate contiguous pieces of data before performing reads/writes. In our previous work, we implemented the two-phase I/O scheme with a MPI-based topology-aware algorithm. Our algorithm showed very good performance at scale compared to the standard I/O libraries such as POSIX I/O and MPI I/O. However, the algorithm had several limitations hindering a satisfying reproducibility of our experiments. In this paper, we extend our work by 1) identifying the obstacles we face to reproduce our experiments and 2) discovering solutions that reduce the unpredictability of our results.

97 MATHEMATICS AND COMPUTING↗

A standards perspective on genomic data reusability and reproducibility

Genomic and metagenomic sequence data provides an unprecedented ability to re-examine findings, offering a transformative potential for advancing research, developing computational tools, enhancing clinical applications, and fostering scientific collaboration. However, effective and ethical reuse of genomics data is hampered by numerous technical and social challenges. The International Microbiome and Multi’Omics Standards Alliance (IMMSA, https://www.microbialstandards.org/) and the Genomic Standards Consortium (GSC, https://gensc.org) hosted a 5-part seminar series “A Year of Data Reuse” in 2024 to explore challenges and opportunities of data reuse and reproducibility across disparate domains of the genomic sciences. Addressing these challenges will require a multifaceted approach, including common metadata reporting, clear communication, standardized protocols, improved data management infrastructure, ethical guidelines, and collaborative policies that prioritize transparency and accessibility. We offer strategies to enable responsible and technically feasible data reuse, recognition of data reproducibility challenges, and emphasizing the importance of cross-disciplinary efforts in the pursuit of open science and data-driven innovation.

59 BASIC BIOLOGICAL SCIENCES↗

Neural Network-Enhanced Reproducing Kernel Particle Method for Image-Based Multiphysics Damage Modeling of Energy Storage Materials

Energy storage materials undergo significant stresses during charge/discharge cycling, which makes understanding their reliability and durability fundamental in predicting performance and service life. Strong electrochemical-mechanical coupling and highly anisotropic material properties contribute to the formation and propagation of micro-cracking, largely along material interfaces and grain boundaries. With microstructural images supplied by the National Renewable Energy Laboratory (NREL), image-based modeling techniques are used to represent the complex material microstructures that dictate the coupled physics of these systems. Traditional electrochemical-mechanical models rely on mesh-based finite element methods, which can lead to difficulties in capturing crack propagation due to mesh dependency. Additionally, commonly used damage models, such as the continuous damage model and the cohesive zone model, often have steep tradeoffs between discontinuous field accuracy and computational expense. In this work, a neural network-enhanced reproducing kernel particle method (NN-RKPM) [1] is leveraged to accurately capture damage and crack propagation throughout the material by learning the location, orientation, and sharpness of discontinuity while allowing for a coarser nodal distribution than that necessary for capturing sharp solution transitions using traditional mesh-based methods. NN-RKPM is used to inform how crack opening and closure in turn affect the coupled chemical equations and material microstructure. Reference: [1] Baek, J., Chen, J. S., Susuki, K., "Neural Network enhanced Reproducing Kernel Particle Method for Modeling Localizations," International Journal for Numerical Methods in Engineering, Vol. 123, pp 4422-4454, https://doi.org/10.1002/nme.7040, 2022.

damage modeling↗

Reproducing the observed energy-dependent structure of Earth's electron radiation belts during storm recovery with an event-specific diffusion model

Here, we present dynamic simulations of energy-dependent losses in the radiation belt “slot region” and the formation of the two-belt structure for the quiet days after the 1 March storm. The simulations combine radial diffusion with a realistic scattering model, based data-driven spatially and temporally resolved whistler-mode hiss wave observations from the Van Allen Probes satellites. The simulations reproduce Van Allen Probes observations for all energies and L shells (2–6) including (a) the strong energy dependence to the radiation belt dynamics (b) an energy-dependent outer boundary to the inner zone that extends to higher L shells at lower energies and (c) an “S-shaped” energy-dependent inner boundary to the outer zone that results from the competition between diffusive radial transport and losses. We find that the characteristic energy-dependent structure of the radiation belts and slot region is dynamic and can be formed gradually in ~15 days, although the “S shape” can also be reproduced by assuming equilibrium conditions. The highest-energy electrons (E > 300 keV) of the inner region of the outer belt (L ~ 4–5) also constantly decay, demonstrating that hiss wave scattering affects the outer belt during times of extended plasmasphere. Through these simulations, we explain the full structure in energy and L shell of the belts and the slot formation by hiss scattering during storm recovery. We show the power and complexity of looking dynamically at the effects over all energies and L shells and the need for using data-driven and event-specific conditions.

58 GEOSCIENCES↗

How Reproducible are Surface Areas Calculated from the BET Equation?

Abstract Porosity and surface area analysis play a prominent role in modern materials science. At the heart of this sits the Brunauer–Emmett–Teller (BET) theory, which has been a remarkably successful contribution to the field of materials science. The BET method was developed in the 1930s for open surfaces but is now the most widely used metric for the estimation of surface areas of micro‐ and mesoporous materials. Despite its widespread use, the calculation of BET surface areas causes a spread in reported areas, resulting in reproducibility problems in both academia and industry. To prove this, for this analysis, 18 already‐measured raw adsorption isotherms were provided to sixty‐one labs, who were asked to calculate the corresponding BET areas. This round‐robin exercise resulted in a wide range of values. Here, the reproducibility of BET area determination from identical isotherms is demonstrated to be a largely ignored issue, raising critical concerns over the reliability of reported BET areas. To solve this major issue, a new computational approach to accurately and systematically determine the BET area of nanoporous materials is developed. The software, called “BET surface identification” (BETSI), expands on the well‐known Rouquerol criteria and makes an unambiguous BET area assignment possible.

36 MATERIALS SCIENCE↗

Demonstration of the Reproducibility Challenges in the Sintering Behavior of Lithium‐Stuffed Garnets in Scaling up Synthesis

Lithium-stuffed garnets, such as Li 7 La 3 Zr 2 O 12 (LLZO), are promising candidates for next-generation solid-state batteries because of their high room-temperature ionic conductivity and chemical stability against lithium metal anodes, which are crucial for achieving higher energy density. However, realizing LLZO's potential in practical devices requires synthesis methods that can be scaled reliably to large batch sizes for manufacturing. Herein, we investigate the sintering reproducibility of LLZO synthesized at larger scales using ultrasonic spray pyrolysis, a cost-effective and scalable synthesis route. Two 100 g batches of Al-doped LLZO are prepared and their sintering behavior is examined in detail. Both Al-LLZO batches contain over 90 wt.% cubic-phase LLZO, and both batches exhibit room temperature conductivities greater than 1 × 10 −4 S cm −1 at a relative density above 0.8. However, variations in secondary phases and subtle differences in Al content lead to significant differences in densification and microstructure. These results demonstrate that LLZO's sintering behavior is highly sensitive to small changes in secondary phases and Al content, creating reproducibility challenges when moving from laboratory- to manufacturing-scale synthesis.

36 MATERIALS SCIENCE↗

Sensitivity Analysis of Material Model Parameters to Reproduce Crushing of Composite Tubes

The ability to predict the behavior of composite structures under impact conditions is an important challenge for engineering analysis since it requires modeling beyond the elasticity and into failure initiation and propagation. Crushing of laminate composites is the result of a combination of several and complex failure mechanisms, which are not always possible to reproduce together in a single finite element analysis (FEA). In this paper, models based on lamina-level failure criteria have been used to predict the onset of damage; once failure initiates, the mechanisms of failure propagation require reducing the material properties using several degradation schemes. Among the composite material models available in LS-DYNA, MAT-ENHANCED-COMPOSITE-DAMAGE (MAT-54) has been chosen for modeling the axial crushing of composite tubes with various resistant sections and wall slope, given the reduced experimental input parameters to insert into the card definition. Comparison with experimental data shows that such material model is able to successfully reproduce the crushing behavior, even if the numerical results are strictly correlated with some model parameters which cannot be measured experimentally or are purely mathematical expedients. Therefore, extensive calibration of these parameters by trial-and-error is necessary to achieve successful simulation results. Once done it for a structure made of the same composite material, it is possible to adopt the same card configuration for all the other cases, obtaining good agreement between numerical and experimental results. After, the attempt to model the same geometries using the MAT-LAMINATED-COMPOSITE-FABRIC (MAT-58) card with the same parameter values previously calibrated has been done obtaining results comparable with the experimental data.

36 MATERIALS SCIENCE↗

Large-Area Graphene Films as Target Surfaces for Highly Reproducible Matrix-Assisted Laser Desorption Ionization Suitable for Quantitative Mass Spectrometry

Due to the known sweet-spot issues that intrinsically arise from inhomogeneous formation of matrix-analyte crystals utilized as samples in matrix-assisted laser desorption ionization (MALDI) mass spectrometry, its reproducibility and thus its applications for quantification have been somewhat limited. In this paper, we report a simple strategy to improve the uniformity of matrix-analyte crystal spots, which we realized by adapting large-area graphene films, i.e., non-inert, interacting surfaces, as target surfaces. In this example, the graphitic surfaces of the graphene films interact with excess matrix molecules during the sample drying process, which induces spontaneous formation of optically uniform MALDI sample crystal layers on the film surfaces. Further, mass spectrometric imaging reveals that the visible uniformity is indeed accompanied by reproducible MALDI ionization over an entire sample spot, which greatly suppresses the appearance of sweet spots. The results of this study confirm that the proposed method achieves good linear responses of ion intensity to the analyte concentration (R{sup 2} > 0.99) with small relative standard deviations (σ < 10%), which is a range applicable for quantitative measurements using MALDI mass spectrometry. .

77 NANOSCIENCE AND NANOTECHNOLOGY↗