Search NASA⌕ Search

SEARCH · Search NASA

Results for “Reliability Prediction”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 235 records · Page 13
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

New framework for benchmarking decadal predictions leveraging the PCMDI Metric Package with interactive visualization

Reliable climate predictions across multiple timescales are increasingly critical as climate-related risks continue to rise. With the growing number and diversity of climate prediction systems, systematic intercomparison has become essential. Here, we present a comprehensive evaluation framework based on the PCMDI Metric Package to assess the performance of multiple decadal climate prediction systems. Unlike uninitialized simulations, initialized predictions exhibit bias and predictive skill that evolve with forecast lead time. To address this, we introduce (1) model-by-lead-time portrait plots, which efficiently summarize metrics of global temperature, precipitation, and Arctic/Antarctic sea-ice extent, and (2) an HTML-based interactive visualization platform that provides detailed regional and seasonal diagnostics of model bias, skill scores, and ensemble spread for each model and lead time. Comparisons with uninitialized simulations further quantify the relative impacts of initialization and external forcing on prediction skill. The proposed framework provides a scalable and transparent approach for multi-model climate prediction assessments and can be readily extended to a wide range of operational and research forecasting systems.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2024

Storm surges and extreme sea levels: Review, establishment of model intercomparison and coordination of surge climate projection efforts (SurgeMIP).

Coastal flood damage is primarily the result of extreme sea levels. Climate change is expected to drive an increase in these extremes. While proper estimation of changes in storm surges is essential to estimate changes in extreme sea levels, there remains low confidence in future trends of surge contribution to extreme sea levels. Alerting local populations of imminent extreme sea levels is also critical to protecting coastal populations. Both predicting and projecting extreme sea levels require reliable numerical prediction systems. The SurgeMIP (surge model intercomparison) community has been established to tackle such challenges. Efforts to intercompare storm surge prediction systems and coordinate the community 's prediction and projection efforts are introduced. An overview of past and recent advances in storm surge science such as physical processes to consider and the recent development of global forecasting systems are briefly introduced. Selected historical events and drivers behind fast increasing service and knowledge requirements for emergency response to adaptation considerations are also discussed. The community 's initial plans and recent progress are introduced. These include the establishment of an intercomparison project, the identification of research and development gaps, and the introduction of efforts to coordinate projections that span multiple climate scenarios.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Reliable p K a Prediction through Efficient Incorporation of Anharmonicity within the Nuclear–Electronic Orbital Framework

Accurate pK a prediction is critical for understanding chemical reactivity and molecular properties across a wide range of applications. Computational approaches usually invoke a harmonic treatment of the vibrational modes for zero-point energies, as well as thermal and entropic contributions. Herein, we present a general protocol for relative pK a prediction that incorporates the significant anharmonic effects using nuclear–electronic orbital (NEO) theory. This protocol is validated against experimental data for a range of molecules in acetonitrile, including protonated nitrogen bases, nitrophenols, anilines, and diamines, as well as cobalt electrocatalysts. For simple acids, the NEO approach offers only a slight improvement over conventional density functional theory with the standard harmonic vibrational treatment, whereas for hydrogen-bonded acids, the NEO approach offers more significantly improved performance at a comparable computational cost. This accessible methodology provides a practical route for accurate pKa prediction in challenging systems and is extendable to related thermodynamic properties such as hydricities and proton-coupled redox potentials.

Density functional theory↗
DOE OSTI2024

Reliability and Durability Testing of Glass Ceramic Seals for Praxair’s Oxygen Transport Membranes

Praxair has been developing Oxygen Transport Membrane (OTM) systems for the efficient conversion of natural gas to syngas which can subsequently be used in chemical synthesis, generation of industrial gases (H2, CO) or synthesis of liquid fuels. One of the technical barriers for commercialization of this technology is the reliability of key components of OTM systems – Seals and Membranes. Praxair has been testing these components at the process conditions expected for commercial systems. However, long-term durability data is currently limited and is not sufficiently available to make well informed predictions of OTM reliability which represents one of the major risks that could prevent the technology from being fully commercialized. Pacific Northwest National Laboratory (PNNL) has been developing high temperature seals and ceramic membranes for applications similar to OTM systems such as Solid Oxide Fuel Cells. The goal of the cooperative development between Praxair and PNNL is to generate long-term data such that there is sufficient confidence in the OTM seal and membrane reliability predictions to enable the process technology to be fully commercialized.

03 NATURAL GAS↗
DOE OSTI2022

Physics-informed graph neural networks for predicting cetane number with systematic data quality analysis

Designing alternative fuels for advanced compression ignition engines necessitates a predictive model for cetane number (CN). In this study, the physics-informed graph neural networks are introduced for a reliable CN prediction by considering molecular features pertinent to the physical properties of molecules that affect CN. The reliability of measured data is another key factor to consider for improving the predictive model. Various experimental instruments for measuring CN exist, including standard and non-standard methods. In this regard, a systematic data quality analysis was carried out for the total 630 CNs collected from literature and new measurements in this study using Advanced Fuel Ignition Delay Analyzer (AFIDA). The results from this data curation process were reflected in the model by imposing lower sample weights on the data coming from less reliable measurement techniques. This approach effectively maximized the prediction accuracy while incorporating data from all available sources. Using the sample weights decreased the mean absolute error (MAE) up to 0.8 CN units. The accuracy was also improved by introducing the CN-related physical properties (the number of hydrogen bond donors and acceptors); the test set MAE is 5.74 and 7.01 for the model with and without such properties, respectively. Investigating molecular structural effects on CN was also carried out to gain chemical insights into factors used to design new fuel candidates. The dimensionality reduction analysis of feature vectors showed a clear clustering in terms of functional groups and CN and the structural effect derived from the model was consistent with the physicochemical insights. Finally, this physics-informed model and data curation would be helpful for accurate CN prediction and inform rational fuel design.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

MS25: Materials Science-Focused Benchmark Data Set for Machine Learning Interatomic Potentials

Here, we present MS25, a benchmark data set for evaluating machine learning interatomic potentials (MLIPs) across diverse materials-relevant systems including MgO surfaces, liquid water, zeolites, a catalytic Pt surface reaction, high-entropy alloys (HEAs), and disordered Zr-oxides. Five MLIP architectures (MACE, NequIP, Allegro, MTP, and Torch-ANI) are trained and tested, focusing not only on traditional metrics (energies, forces, and stresses) but also explicitly validating derived physical observables such as lattice constants, volumes, and reaction barriers. We find that most models reach comparable accuracy on standard error metrics across the simple systems, although equivariant MLIPs offer 1.5–2× improvements over nonequivariant MLIPs in energy and force error for structurally complex or compositionally disordered environments such as HEAs and Zr–O systems. Our analysis highlights that low errors in energy and force predictions do not guarantee reliable observables, emphasizing the necessity of explicit validation. We demonstrate limitations in cross-framework transferability, as models trained on one zeolite framework (CHA) fail to reliably generalize to predictions of structurally distinct frameworks (e.g., MFI). Size-extensive tests show some dependence on system size for MgO, resulting from forced periodicity. The HEA and Zr–O data sets are identified as challenging tests for future benchmarks and MLIP model architecture developments as they show significant differentiation in error between MLIP architectures and are still relatively difficult at 1000 training images. Moving forward, we recommend that benchmarking efforts shift their focus from marginal accuracy improvements in energy and force errors toward identifying and understanding model failure modes, rigorously assessing transferability, and evaluating how their errors affect observable predictions. For researchers looking to choose an MLIP architecture, we suggest selecting equivariant MLIP architectures if the complexity of the system is a challenge. For simple materials problems, auxiliary features such as integration with molecular dynamics engines, trade-offs between computational data set generation cost vs MLIP inference speed, and framework integration may play a more important decision factor than small differences in error metrics that are unlikely to matter for production-level research.

chemical structure↗
DOE OSTI2020

A First–Principles–Based Sub–Lattice Formalism for Predicting Off–Stoichiometry in Materials for Solar Thermochemical Applications: The Example of Ceria

Theoretical models that reliably can predict off-stoichiometry in materials via accurate descriptions of underlying thermodynamics are crucial for energy applications. For example, transition-metal and rare-earth oxides that can tolerate a large number of oxygen vacancies, such as CeO 2 and doped CeO 2 , can split water and carbon dioxide via a two-step, oxide-based solar thermochemical (STC) cycle. The search for new STC materials with a performance superior to that of state-of-the-art CeO 2 can benefit from predictions accurately describing the thermodynamics of oxygen vacancies. The sub-lattice formalism, a common tool used to fit experimental data and build temperature-composition phase diagrams, can be useful in this context. Here, sub-lattice models are derived solely from zero-temperature quantum mechanics calculations to estimate fairly accurate temperature- and oxygen-partial-pressure-dependent off-stoichiometries in CeO 2 and Zr-doped CeO 2 . Physical motivations for deriving some of the “excess” sub-lattice model parameters directly from quantum mechanical calculations, instead of fitting to minimize deviations from experimental and/or theoretical data, are identified. As a result, important limitations and approximations of the approach used are specified and extensions to multi-cation oxides are also suggested to help identify novel candidates for water and carbon dioxide splitting and related applications.

08 HYDROGEN↗
DOE OSTI2021

Scenario adaptive disruption prediction study for next generation burning-plasma tokamaks

Next generation High Performance (HP) tokamaks risk damage from unmitigated disruptions at high current and power. Achieving reliable disruption prediction for a device's HP operation based on its Low Performance (LP) data is a key to its success. In this letter, through explorative data analysis and dedicated numerical experiments on multiple existing tokamaks, we demonstrate how the operational regimes of tokamaks can affect the power of a trained disruption predictor. First, our results suggest data-driven disruption predictors trained on abundant LP discharges work poorly on the HP regime of the same tokamak, which is a consequence of the distinct distributions of the tightly correlated signals related to disruptions in these two regimes. Second, we find that matching operational parameters among tokamaks strongly improves cross-machine accuracy which implies our model learns from the underlying scalings of dimensionless physics parameters like q 95 , β p and confirms the importance of these parameters in disruption physics and cross machine domain matching from the data-driven perspective. Finally, our results show in the absence of HP data from the target devices, the best predictivity of the HP regime for the target machine can be achieved by combining LP data from the target with HP data from other machines. Furthermore, these results provide a possible disruption predictor development strategy for next generation tokamaks, such as ITER and SPARC, and highlight the importance of developing baseline scenario discharges of future tokamaks on existing machines to collect more relevant disruptive data.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

An improved dataset for predicting mammal infecting viruses from genetic sequence information

There have been several attempts to develop machine learning (ML) models to identify human infecting viruses from their genomic sequences, with varying degrees of success. Direct comparison between models is problematic, because these models are typically trained and evaluated on different datasets with alternative data splitting schemes, features, and model performance metrics. In this paper we present a standardized dataset of mammal infecting and non-infecting viral pathogens, refined from the previous work of Mollentze et al. to include the latest literature evidence, roughly doubling the number of curated host-virus records available to the community, and new host target labels, primate and mammal. The new host labels were included for several reasons, including previous reports that classification performance is better at broader taxonomic ranks and the idea that there may be more data for primate infection that might serve as a suitable proxy for zoonotic potential and avoidance of false positives for human infection due to absence of evidence. On this dataset, we report the performance of eight machine learning models for predicting mammal-infecting viruses from their genomic sequences. We find that randomly assigning cases in our improved dataset to training/testing sets, when compared to the original assignments into training/testing in Mollentze et al., increases the overall average ROC AUC of prediction of human infection from 0.663 ± 0.070 to 0.784 ± 0.013, consistent with the reduction in phylogenetic distance between train and test sets (relative entropy change from 3.00 to 0.08). The broadest host category of mammal infection can be predicted most reliably at 0.850 ± 0.020. We share our improved dataset and code to enable standardized comparisons of machine learning methods to predict human host infections. Overall, we have presented preliminary evidence that classification of virus host infection is more tractable at higher taxonomic ranks, that unsurprisingly reducing the phylogenetic distance between training and test sets can improve predictive performance, that peptide kmer features appear to be harmful to out of sample model performance, and we are left with the question of whether models for virus host prediction can reasonably be expected to perform well in out of sample scenarios given the likelihood that viruses do not share a common ancestor. Consistent with this concern, when the data is resampled such that there is no overlap between viral families in training and test sets (relative entropy > 24), models perform no better than random chance at prediction of human infection regardless of whether kmers are included (ROC AUC 0.50 ± 0.08) or not (ROC AUC 0.50 ± 0.04).

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2025

Evaluating probabilistic deep learning methods for uncertainty quantification of temperature downscaling

Deep learning (DL) has emerged as a promising tool for downscaling coarse-resolution climate data to high-resolution outputs, enabling improved regional climate predictions. A critical aspect of DL-based downscaling is the incorporation of uncertainty quantification (UQ), which enhances the interpretability and reliability of predictions—key factors for climate risk assessment and decision-making. This study develops a DL model to downscale 2 m temperature across the contiguous United States using reanalysis datasets. We systematically evaluate three epistemic UQ methods—deep ensembles (DEns), Monte Carlo dropout (MCD), and Flipout—based on their probabilistic accuracy, downscaling performance, sensitivity to geographical features, and computational efficiency. Results indicate that MCD generally outperforms Flipout and DEns in terms of calibration and downscaling accuracy. However, DEns demonstrate lower calibration errors in coastal regions, indicating its higher confidence within these areas. Flipout, in contrast, is more sensitive to elevation gradients and exhibits higher calibration errors in mountainous regions. Hence, the choice of UQ method for this task depends on the specific requirements of the application. For applications that prioritize overall calibration, downscaling accuracy, and computational efficiency, MCD is a strong candidate. These findings highlight the importance of selecting UQ methods based on application-specific requirements, such as geographical context and computational constraints. By addressing the trade-offs between UQ methods, this study provides actionable insights for improving the reliability, scalability, and utility of DL-based downscaling in climate science.

Environmental sciences↗
DOE OSTI2023

Uncertainty quantification of a deep learning fuel property prediction model

Deep learning models are being widely used in the field of combustion. Given the black-box nature of typical neural network based models, uncertainty quantification (UQ) is critical to ensure the reliability of predictions as well as the training datasets, and for a principled quantification of noise and its various sources. Deep learning surrogate models for predicting properties of chemical compounds and mixtures have been recently shown to be promising for enabling data-driven fuel design and optimization, with the ultimate goal of improving efficiency and lowering emissions from combustion engines. In this study, UQ is performed for a multi-task deep learning model that simultaneously predicts the research octane number (RON), Motor Octane Number (MON), and Yield Sooting Index (YSI) of pure components and multicomponent blends. The deep learning model is comprised of three smaller networks: Extractor 1, Extractor 2, and Predictor, and a mixing operator. The molecular fingerprints of individual components are encoded via Extractor 1 and Extractor 2, the mixing operator generates fingerprints for mixtures/blends based on linear mixing operation, and the predictor maps the fingerprint to the target properties. Two different classes of UQ methods, Monte Carlo ensemble methods and Bayesian neural networks (BNNs), are employed for quantifying the epistemic uncertainty. Combinations of Bernoulli and Gaussian distributions with DropConnect and DropOut techniques are explored as ensemble methods. All the DropConnect, DropOut and Bayesian layers are applied to the predictor network. Aleatoric uncertainty is modeled by assuming that each data point has an independent uncertainty associated with it. The results of the UQ study are further analyzed to compare the performance of BNN and ensemble methods. Although this study is confined to UQ of fuel property prediction, the methodologies are applicable to other deep learning frameworks that are being widely used in the combustion community.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2020

Integrating Predictions for Improving Defect Classification Accuracy in NDT-based Assessment of Concrete - 20229

There is an increasing need to create predictive models for defect classification in concrete using the output of non-destructive testing (NDT) techniques. Recent advancement of machine-learning algorithms has offered several techniques for developing classification models for different types of data sets. However, the performance of these algorithms is very uncertain, mainly when applied to small and noisy datasets. For example, when human access is limited (e.g., nuclear facility), robot-based NDT is preferred. But compared to manual tests with humans present on site, the data sets are small, and more noise can exist. Therefore, it is imperative to develop new approaches to ensure a consistently high classification accuracy for inadequate data sets. This study explores the classification performance on NDT dataset using classifiers from different machine-learning algorithms, namely k-Nearest Neighbor (kNN), Decision Tree, Naive Bayes, Logistic Regression, and Support Vector Machine (SVM). The authors further integrated the predictions from these classifiers using proposed methods. The integration strategy combines the output of the classifiers based on two different measures, accuracy, and performance (ACC and PERF), using equations such as sum, average, and square-root-of-sums-of-squares (SRSS). Our results reveal varying classification accuracies across individual classifiers with different misclassifications across the test data set. The integration strategy provided significant improvement in the classification accuracy compared to the individual classifiers. Furthermore, the results indicate minimal variation across the integration methods as compared to the variation across the individual classifiers. To conclude, prediction integration offers a unique approach for combining the output of multiple classifiers to create redundancies with the potential of achieving high classification performance and improved reliability in predictive models for defect detection in concrete. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2024

Comparison of MeshGraphNet Techniques for Subsurface Behavior Prediction during CO2 Sequestration

Carbon sequestration is a vital part of the effort to mitigate anthropogenic climate change. Previously, we have shown that Graph Neural Networks (GNNs) provide the ability to extract meaningful insights during prediction of subsurface behavior in carbon storage projects. However, these models have struggled with long-term prediction accuracy due to error accumulation caused by autoregressive prediction. This research leverages the Illinois Basin – Decatur Project (IBDP) dataset to examine strategies for minimizing loss over time in a MeshGraphNet GNN model to improve reliability of predictions while minimizing inferencing time.

Holcomb, Paul↗
DOE OSTI2023

Addressing APC Data Sparsity in Predicting Occupancy and Delay of Transit Buses: A Multitask Learning Approach

Public transit is a vital mode of transportation in urban areas, and its efficiency is crucial for the daily commute of millions of people. To improve the reliability and predictability of transit systems, researchers have developed separate single-task learning models to predict the occupancy and delay of buses at the stop or route level. However, these models provide a narrow view of delay and occupancy at each stop and do not account for the correlation between the two. We propose a novel approach that leverages broader generalizable patterns governing delay and occupancy for improved prediction. We introduce a multitask learning toolchain that takes into account General Transit Feed Specification feeds, Automatic Passenger Counter data, and contextual temporal and spatial information. The toolchain predicts transit delay and occupancy at the stop level, improving the accuracy of the predictions of these two features of a trip given sparse and noisy data. We also show that our toolchain can adapt to fewer samples of new transit data once it has been trained on previous routes/trips as compared to state-of-the-art methods. Finally, we use actual data from Chattanooga, Tennessee, to validate our approach. We compare our approach against the state-of-the-art methods and we show that treating occupancy and delay as related problems improves the accuracy of the predictions. We show that our approach improves delay prediction significantly by as much as 4% in F1 scores while producing equivalent or better results for occupancy.

Zulqarnain, Ammar Bin↗
DOE OSTI2024

Multiscale Modeling of Nanoparticle Precipitation in Oxide Dispersion-Strengthened Steels Produced by Laser Powder Bed Fusion

Laser Powder Bed Fusion (LPBF) enables the efficient production of near-net-shape oxide dispersion-strengthened (ODS) alloys, which possess superior mechanical properties due to oxide nanoparticles (e.g., yttrium oxide, Y-O, and yttrium-titanium oxide, Y-Ti-O) embedded in the alloy matrix. To better understand the precipitation mechanisms of the oxide nanoparticles and predict their size distribution under LPBF conditions, we developed an innovative physics-based multiscale modeling strategy that incorporates multiple computational approaches. These include a finite volume method model (Flow3D) to analyze the temperature field and cooling rate of the melt pool during the LPBF process, a density functional theory model to calculate the binding energy of Y-O particles and the temperature-dependent diffusivities of Y and O in molten 316L stainless steel (SS), and a cluster dynamics model to evaluate the kinetic evolution and size distribution of Y-O nanoparticles in as-fabricated 316L SS ODS alloys. The model-predicted particle sizes exhibit good agreement with experimental measurements across various LPBF process parameters, i.e., laser power (110–220 W) and scanning speed (150–900 mm/s), demonstrating the reliability and predictive power of the modeling approach. The multiscale approach can be used to guide the future design of experimental process parameters to control oxide nanoparticle characteristics in LPBF-manufactured ODS alloys. Additionally, our approach introduces a novel strategy for understanding and modeling the thermodynamics and kinetics of precipitation in high-temperature systems, particularly molten alloys.

Wang, Zhengming (ORCID:0000000241627112)↗
DOE OSTI2025

High-throughput computational framework for high-order anharmonic thermal transport in cubic and tetragonal crystals

Accurate first-principles prediction of lattice thermal conductivity (κL) remains challenging in identifying materials with extreme thermal behavior. While the harmonic approximation with three-phonon scattering (HA + 3ph) is now routine, reliable κL prediction often requires higher-order anharmonic effects, including self-consistent phonon renormalization, three- and four-phonon scattering, and off-diagonal heat flux (SCPH + 3, 4ph + OD). We present a state-of-the-art high-throughput workflow that unifies these effects and apply it to 773 cubic and tetragonal crystals spanning diverse chemistries and structures. From 562 dynamically stable compounds, we assess the hierarchical impacts of higher-order anharmonicity. For around 60% of materials, HA + 3ph predictions closely match those from SCPH + 3, 4ph + OD. SCPH generally increases κ L , by over 8 times in extreme cases, whereas four-phonon scattering universally suppresses κ L , sometimes to 15% of the HA + 3ph value. Off-diagonal contributions are negligible in high-κ L systems but can rival diagonal terms in highly anharmonic low-κ L compounds. We highlight four case studies, Rb 2 TlAlH 6 , Cu 3 VSe 4 , CuBr, and KTlCl 4 , that exhibit distinct extreme behaviors. This work delivers not only a robust workflow for high-fidelity κL dataset but also a quantitative framework to determine when higher-order effects are essential. The hierarchy of κ L results, from the HA + 3ph to SCPH + 3, 4ph + OD level, offers a scalable, interpretable route to discovering next-generation extreme thermal materials.

36 MATERIALS SCIENCE↗