Search NASA⌕ Search

SEARCH · Search NASA

Results for “Explainability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Uncertainty-Aware and Explainable Human Error Detection in the Operation of Nuclear Power Plants

The timely and accurate identification of incidents, such as human factor error, is important to restore nuclear power plants (NPPs) to a stable state. However, the identification of abnormal operating conditions is difficult because of the existence of multiple scenarios. In addition, to implement mitigation actions rapidly after an incident occurs, operators must accurately identify an incident by monitoring the trends of many variables. The mental burden posed by this can increase human error and cause failure in identifying incidents. Failure to identify incidents directly results in erroneous mitigation measures, which are detrimental to NPPs. In this study, we leverage uncertainty-aware models to identify such errors and thereby increase the chances of mitigating them. We use the data collected from a physical test bed. The goal is to identify both certain and accurate models. For this, the two main aspects of focus in this study are explainable artificial intelligence (XAI) and uncertainty quantification (UQ). While XAI elucidates the decision pathway, UQ evaluates decision reliability. Their integration paints a comprehensive picture, signifying that understanding decisions and their confidence should be interlinked. Thus, in this study we leverage UQ measures (e.g. entropy and mutual information) along with Shapley additive explanations to gain insights into the features contributing to both accuracy and uncertainty in error identification. Furthermore, our results show that uncertainty-aware models combined with XAI tools can explain the artificial intelligence–prescribed decisions, with the potential of better explaining errors for the operators.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2024

Demonstration and Evaluation of Explainable and Trustworthy Predictive Technology for Condition-based Maintenance

The domestic nuclear power plant (NPP) fleet has historically relied on labor-intensive and time-consuming predictive maintenance (PdM) programs, thus driving up operation and maintenance (O&M) costs to achieve high-capacity factors. Artificial intelligence (AI) and machine-learning (ML) can help simplify complex problems such as diagnosing equipment degradation to enable more effective decision-making efforts. The benefits of AI will be felt through more efficient plant O&M, improved work processes, and better integration of people and technology. Together, these benefits hold the promise to make nuclear power more sustainable by reducing O&M costs while improving employee engagement. While AI and ML technologies hold significant promise for the nuclear industry, there are challenges or barriers to their adoption. Explainability and trustworthiness of AI are two salient challenges that need to be addressed for wider deployment of these technologies in NPPs. This research focuses specifically on addressing the explainability and trustworthiness of AI technologies to advance the human, technical, and organization (HTO) readiness levels in adopting a risk-informed PdM strategy at commercial NPPs. In addition, this approach can be adapted to enhance the acceptability of AI in other nuclear applications with a few application-specific modifications. The technical approach ensuring wider adoption of AI technologies was developed by Idaho National Laboratory (INL)—in collaboration with Public Service Enterprise Group (PSEG), Nuclear, LLC—by utilizing the circulating water system (CWS) at two PSEG-owned plant sites for demonstration. Focused user studies were performed in collaboration with subject matter experts (SMEs) from PSEG and other nuclear domains to enhance human and organization readiness by building trust in AI-informed technologies. VIsualization for PrEdictive maintenance Recommendation (VIPER)—a Battelle Energy Alliance, LLC, copyrighted software—was developed and expanded to provide a user-centric visualization by incorporating inputs from the collaborating utility, human factors engineering guidelines, and data analysts. The VIPER software enables users, who may be unfamiliar with ML in general, to be interactively engaged by asking technical questions about PdM, work orders, diagnosis results and their confidence levels, the kind of data being used, and the types of ML algorithms employed. This interactive engagement enhances explainability and builds trust. One of the enabling accomplishments was the integration of large language models (LLMs), both text-based and vision-based, in the VIPER software.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2020

Explainable Artificial Intelligence Recommendation System by Leveraging the Semantics of Adverse Childhood Experiences: Proof-of-Concept Prototype Development

The study of adverse childhood experiences and their consequences has emerged over the past 20 years. Although the conclusions from these studies are available, the same is not true of the data. Accordingly, it is a complex problem to build a training set and develop machine-learning models from these studies. Classic machine learning and artificial intelligence techniques cannot provide a full scientific understanding of the inner workings of the underlying models. This raises credibility issues due to the lack of transparency and generalizability. Explainable artificial intelligence is an emerging approach for promoting credibility, accountability, and trust in mission-critical areas such as medicine by combining machine-learning approaches with explanatory techniques that explicitly show what the decision criteria are and why (or how) they have been established. Hence, thinking about how machine learning could benefit from knowledge graphs that combine “common sense” knowledge as well as semantic reasoning and causality models is a potential solution to this problem. In this study, we aimed to leverage explainable artificial intelligence, and propose a proof-of-concept prototype for a knowledge-driven evidence-based recommendation system to improve mental health surveillance. We used concepts from an ontology that we have developed to build and train a question-answering agent using the Google DialogFlow engine. In addition to the question-answering agent, the initial prototype includes knowledge graph generation and recommendation components that leverage third-party graph technology. To showcase the framework functionalities, we here present a prototype design and demonstrate the main features through four use case scenarios motivated by an initiative currently implemented at a children’s hospital in Memphis, Tennessee. Ongoing development of the prototype requires implementing an optimization algorithm of the recommendations, incorporating a privacy layer through a personal health library, and conducting a clinical trial to assess both usability and usefulness of the implementation. This semantic-driven explainable artificial intelligence prototype can enhance health care practitioners’ ability to provide explanations for the decisions they make.

adverse childhood experiences↗
DOE OSTI2025

Ca X ML: Chemistry‐informed machine learning explains mutual changes between protein conformations and calcium ions in calcium‐binding proteins using structural and topological features

Proteins' flexibility is a feature in communicating changes in cell signaling instigated by binding with secondary messengers, such as calcium ions, associated with the coordination of muscle contraction, neurotransmitter release, and gene expression. When binding with the disordered parts of a protein, calcium ions must balance their charge states with the shape of calcium-binding proteins and their versatile pool of partners depending on the circumstances they transmit. Accurately determining the ionic charges of those ions is essential for understanding their role in such processes. However, it is unclear whether the limited experimental data available can be effectively used to train models to accurately predict the charges of calcium-binding protein variants. Here, we developed a chemistry-informed, machine-learning algorithm that implements a game theoretic approach to explain the output of a machine-learning model without the prerequisite of an excessively large database for high-performance prediction of atomic charges. We used the ab initio electronic structure data representing calcium ions and the structures of the disordered segments of calcium-binding peptides with surrounding water molecules to train several explainable models. Network theory was used to extract the topological features of atomic interactions in the structurally complex data dictated by the coordination chemistry of a calcium ion, a potent indicator of its charge state in protein. Our design created a computational tool of Ca X ML, which provided a framework of explainable machine learning model to annotate ionic charges of calcium ions in calcium-binding proteins in response to the chemical changes in an environment. Our framework will provide new insights into protein design for engineering functionality based on the limited size of scientific data in a genome space.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Explaining word embeddings with perfect fidelity: a case study in predicting research impact

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

Coarse-grained models↗
DOE OSTI2021

Assessing Decadal Predictability in an Earth-System Model Using Explainable Neural Networks

We show that explainable neural networks can identify regions of oceanic variability that contribute predictability on decadal timescales in a fully coupled Earth-system model. The neural networks learn to use sea-surface temperature anomalies to predict future continental surface temperature anomalies. We then use a neural-network explainability method called layerwise relevance propagation to infer which oceanic patterns lead to accurate predictions made by the neural networks. In particular, regions within the North Atlantic Ocean and North Pacific Ocean lend the most predictability for surface temperature across continental North America. We apply the proposed methodology to decadal variability, although the concept is generalizable to other timescales of predictability. Furthermore, while our approach focuses on predictable patterns of internal variability within climate models, it should be generalizable to observational data as well. Our study contributes to the growing evidence that explainable neural networks are important tools for advancing geoscientific knowledge.

58 GEOSCIENCES↗
DOE OSTI2026

Violent mergers can explain the inflated state of some of the fastest stars in the Galaxy

A significant number of hypervelocity stars with velocities between 1500 − 2500 km s −1 have recently been observed. The only plausible explanation so far is that they were produced through thermonuclear supernovae in white dwarf binaries. Since these stars are thought to be surviving donors of Type Ia supernovae, a surprising finding was that these stars are inflated, with radii an order of magnitude higher than expected for Roche-lobe-filling donors. Recent attempts at explaining them have combined 3D hydrodynamical supernova explosion simulations with 1D stellar modelling to explain the impact of supernova shocks on runaway white dwarfs. However, only the hottest and most compact of those runaway stars can so far marginally be reproduced by detailed models of runaways from supernova explosions. In this and a companion paper, we introduce a new AREPO simulation of two massive CO white dwarfs that explode via a violent merger. During the merger, the primary white dwarf ignites when the secondary is on its last orbit and plunging towards the primary. In the corresponding aftermath, the core of the secondary white dwarf of 0.16 M ⊙ remains bound, moving at a velocity of ∼2800 km s −1 . We mapped this object into MESA and show that this runaway star can explain the observations of two hypervelocity stars that were dubbed D6-1 and D6-3 based on their original discovery motivated by the D6 scenario, though the violent merger scenario presented here is somewhat distinct from the D6 scenario.

Astronomy and AstroPhysics↗
DOE OSTI2026

Global Explainability of A Deep Abstaining Classifier for Cancer Pathology Reports

We present a global explainability method to characterize sources of errors in a real-world multitask deep abstaining classifier (DAC), in the context of cancer histology prediction. Our multitask classifier, currently deployed for automated annotation of cancer pathology reports from NCI-SEER registries, was trained and evaluated on 1.04 million hand-annotated samples and makes simultaneous predictions of cancer site, subsite, histology, laterality, and behavior for each report. The DAC framework enables the model to abstain on ambiguous reports and confusing classes to achieve the target accuracy on the retained (non-abstained) samples, but at the cost of decreased coverage. Requiring 97% accuracy on the histology task caused our model to retain only 22% of all samples, mostly the less ambiguous and common classes. Local explainability with the GradInp technique provided a computationally efficient way of obtaining contextual reasoning for hundreds of thousands of individual predictions. Our method, involving dimensionality reduction of approximately 13000 aggregated local explanations (ALE), offers a tractable path to true global explainability. It enabled identification of sources of errors in histology classification, globally, as hierarchical complexity among classes, label noise, insufficient information, and conflicting evidence. This suggests several strategies for iterative improvement of our DAC, including well-designed exclusion criteria, focused annotation, and reduced penalties for errors involving hierarchically related classes.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

SAGE Intrusion Detection System: Sensitivity Analysis Guided Explainability for Machine Learning.

This report details the results of a three-fold investigation of sensitivity analysis (SA) for machine learning (ML) explainability (MLE): (1) the mathematical assessment of the fidelity of an explanation with respect to a learned ML model, (2) quantifying the trustworthiness of a prediction, and (3) the impact of MLE on the efficiency of end-users through multiple users studies. We focused on the cybersecurity domain as the data is inherently non-intuitive. As ML is being using in an increasing number of domains, including domains where being wrong can elicit high consequences, MLE has been proposed as a means of generating trust in a learned ML models by end users. However, little analysis has been performed to determine if the explanations accurately represent the target model and they themselves should be trusted beyond subjective inspection. Current state-of-the-art MLE techniques only provide a list of important features based on heuristic measures and/or make certain assumptions about the data and the model which are not representative of the real-world data and models. Further, most are designed without considering the usefulness by an end-user in a broader context. To address these issues, we present a notion of explanation fidelity based on Shapley values from cooperative game theory. We find that all of the investigated MLE explainability methods produce explanations that are incongruent with the ML model that is being explained. This is because they make critical assumptions about feature independence and linear feature interactions for computational reasons. We also find that in deployed, explanations are rarely used due to a variety of reason including that there are several other tools which are trusted more than the explanations and there is little incentive to use the explanations. In the cases when the explanations are used, we found that there is the danger that explanations persuade the end users to wrongly accept false positives and false negatives. However, ML model developers and maintainers find the explanations more useful to help ensure that the ML model does not have obvious biases. In light of these findings, we suggest a number of future directions including developing MLE methods that directly model non-linear model interactions and including design principles that take into account the usefulness of explanations to the end user. We also augment explanations with a set of trustworthiness measures that measure geometric aspects of the data to determine if the model output should be trusted.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Exploring Whether Super-puffs can be Explained as Ringed Exoplanets

An intriguing, growing class of planets are the “super-puffs,” objects with exceptionally large radii for their masses and thus correspondingly low densities (≲0.3 g cm{sup −3}). Here we consider whether they could have large inferred radii because they are in fact ringed. This would naturally explain why super-puffs have thus far only shown featureless transit spectra. We find that this hypothesis can work in some cases but not all. The close proximity of the super-puffs to their parent stars necessitates rings with a rocky rather than icy composition. This limits the radius of the rings, and makes it challenging to explain the large size of Kepler 51b, 51c, 51d, and 79d unless the rings are composed of porous material. Furthermore, the short tidal locking timescales for Kepler 18d, 223d, and 223e mean that these planets may be spinning too slowly, resulting in a small oblateness and rings that are warped by their parent star. Kepler 87c and 177c have the best chance of being explained by rings. Using transit simulations, we show that testing this hypothesis requires photometry with a precision of somewhere between ∼10 ppm and ∼50 ppm, which roughly scales with the ratio of the planet and star’s radii. We conclude with a note about the recently discovered super-puff HIP 41378f.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2026

Using Explainable Artificial Intelligence to Predict Perovskite Solar Cell Electrical Metastability from Operando Photoluminescence Images in Accelerated Stress Testing

Metal halide perovskite (MHP) solar cells exhibit a metastable response to bias governed by coupled ionic–electronic processes, complicating the conventional reciprocity relation between luminescence intensity and device open-circuit voltage (V oc ). This limits the use of luminescence as a diagnostic for device screening or accelerated stress testing, motivating new approaches that can interpret photoluminescence (PL) signals under nonequilibrium conditions. From the artificial intelligence perspective, we develop an explainable deep learning framework that integrates convolutional neural networks (CNN), long short-term memory (LSTM) layers, and an attention mechanism to learn spatiotemporal features from operando photoluminescence PL image sequences. The model achieves a mean absolute error of ±0.027 V in predicting open-circuit voltage transients and reduces extreme-tail errors by up to 78% compared to physics-based reciprocity calculations. Gradient-weighted Class Activation Mapping (Grad-CAM) provides interpretability by highlighting physically meaningful regions such as electrode edges and emergent defect features. From the engineering application perspective, this framework enables accurate, contactless prediction of device V oc and identification of degradation-relevant features during accelerated aging of perovskite solar cells. This approach demonstrates how explainable AI can enhance operando diagnostics and reliability analysis in photovoltaic devices under nonequilibrium conditions.

14 SOLAR ENERGY↗
DOE OSTI2023

Explainable machine learning model for multi-step forecasting of reservoir inflow with uncertainty quantification

We propose an explainable machine learning (ML) model with uncertainty quantification (UQ) to improve multi-step reservoir inflow forecasting. Traditional ML methods have challenges in forecasting inflows multiple days ahead, and lack explainability and UQ. To address these limitations, we introduce an encoder–decoder long short-term memory (ED-LSTM) network for multi-step forecasting, employ the SHapley Additive exPlanation (SHAP) technique for understanding the influence of hydrometeorological factors on inflow prediction, and develop a novel UQ method for prediction trustworthiness. We apply these methods to forecast 7-day inflow in snow-dominant and rain-driven reservoirs. The results demonstrate the effectiveness of the ED-LSTM model, with high forecasting accuracy for short lead times. Our UQ method provides reliable uncertainty estimates, covering 90% of data with a 90% confidence level. The SHAP analysis reveals the importance of historical inflow and precipitation as influential factors. These findings and methods may support reservoir operators in optimizing water resources management decisions.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Aerosol Influences on Cloud Water: Insights From ARM EPCAPE Observations With Explainable Machine Learning

This study employs an explainable machine learning (ML) framework (XGBoost-SHapley Additive exPlanations analysis) to investigate controlling factors on cloud liquid water path (LWP) using EPCAPE observations near the California coast. Aerosols are found to be the dominant factor explaining LWP variability, surpassing meteorological factors (MFs). By isolating aerosol effects from meteorological influences, the ML reveals a negative linear relationship between LWP and cloud droplet number concentration (Nd) in log space, likely driven by entrainment drying via evaporation-entrainment feedback. This aligns with the negative regime of the inverted-V relationship reported in previous studies, while no positive LWP responses are found due to a limited number of precipitating cases in EPCAPE. Furthermore, the sensitivity of LWP to Nd shows a non-linear dependence on MFs like moisture contrast between surface and free troposphere and lower-tropospheric stability. This occurs due to the interplay between the MFs' direct effects on entrainment drying and indirect effects through LWP adjustments.

EPCAPE observations↗
DOE OSTI2021

EXPLAINABLE AND TRUSTWORTHY DIAGNOSTICS ACHIEVABLE THROUGH PROCESS-BASED AUTOMATED REASONING

An approach has been developed that incorporates domain knowledge to obtain a more explainable and trustworthy equipment health monitoring diagnosis than might otherwise be obtained from a purely data-driven method. Physics-based models serve to constrain the realizable solution space and render a more trusted diagnosis. An automated reasoning algorithm performs backward chaining to infer a diagnosis that is consistent with logic statements that have been evaluated as true. This diagnosis is made explainable to an operator by providing the forward chaining path that elucidates for inspection and validity testing those truths implied by the diagnosis.

automated reasoning↗
DOE OSTI2024

TCR-H: explainable machine learning prediction of T-cell receptor epitope binding on unseen datasets

Artificial-intelligence and machine-learning (AI/ML) approaches to predicting T-cell receptor (TCR)-epitope specificity achieve high performance metrics on test datasets which include sequences that are also part of the training set but fail to generalize to test sets consisting of epitopes and TCRs that are absent from the training set, i.e., are ‘unseen’ during training of the ML model. We present TCR-H, a supervised classification Support Vector Machines model using physicochemical features trained on the largest dataset available to date using only experimentally validated non-binders as negative datapoints. TCR-H exhibits an area under the curve of the receiver-operator characteristic (AUC of ROC) of 0.87 for epitope ‘hard splitting’ (i.e., on test sets with all epitopes unseen during ML training), 0.92 for TCR hard splitting and 0.89 for ‘strict splitting’ in which neither the epitopes nor the TCRs in the test set are seen in the training data. Furthermore, we employ the SHAP (Shapley additive explanations) eXplainable AI (XAI) method for post hoc interrogation to interpret the models trained with different hard splits, shedding light on the key physiochemical features driving model predictions. TCR-H thus represents a significant step towards general applicability and explainability of epitope:TCR specificity prediction.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Role of Uncertainty Quantification in the Explainability of Large Language Models for the Nuclear Industry

The meteoric rise of generative artificial intelligence (AI) large language models (LLMs) has created an opportunity to utilize them to increase efficiencies in a multitude of industries. While LLMs carry great potential to revolutionize the manner in which work is performed, numerous known deficiencies limit their utility, including the black box nature of the models, the stochastic nature of the response (i.e., presenting the same prompt multiple times results in different responses), and the potential for hallucination. Widespread adoption of LLMs in safety-critical industries such as nuclear will require some form of explainability to assure end users that the LLM’s response to a given query is valid. Model uncertainty is inherently linked to the concepts of trust and explainability, and can be used to identify situations in which the model is insufficiently certain about its answer. Although uncertainty is not enough in and of itself to determine the suitability of an answer—a model can be very certain of an inaccurate answer—it still provides valuable supporting information. Practical methodologies for gauging or quantifying the uncertainty in LLM outputs are presented herein, along with examples based on nuclear-specific prompts.

22 - GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2022

The channel-source hypothesis: Empirical evidence for in-channel sourcing of dissolved organic carbon to explain hysteresis in a headwater mountain stream

Catchment hydrologists have long puzzled over the question: How can catchments rapidly generate storm flows and pulses of solutes in response to storm events? Conceptual models viewing catchments as composed of discrete source areas generating flow at unique time scales and with unique chemical characteristics have been used to explain the observed changes in flow and water chemistry. Surprisingly, those conceptual models usually do not treat the stream channel as one of the potential source areas. We propose the channel-source hypothesis in which the stream itself should be considered as a potential source with the same rigour as other contributing areas. We pose this in the spirit of the scientific use of the word: a hypothesis is not a proven idea but rather a provisional supposition serving as the basis for further study. We suggest that the channel should be considered as a potential source for dissolved organic carbon (DOC). Channels store substantial amounts of organic matter, and stream ecologists have long studied stream carbon cycling. From those studies, we know that leaching and decomposition can generate DOC from particulate organic carbon (POC). Further, POC is stored in channel ‘dead-zones’—regions of low flow velocity - that can be activated as flow velocity increases, thus releasing accumulated DOC during storms. All catchments are different; there is no reason to assume that channel sources are always important, in every catchment, in every storm. Thus, the channel-source hypothesis does not replace existing conceptual models. Instead, it adds another potential mechanism that may explain DOC dynamics observed in streams. The channel-source hypothesis has substantial implications for catchment studies examining sources of DOC in stream water or using DOC as a tracer to determine the locations of, and proportional contributions of, different source areas for streamflow generation.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Combination of deep neural network with attention mechanism enhances the explainability of protein contact prediction

Abstract Deep learning has emerged as a revolutionary technology for protein residue‐residue contact prediction since the 2012 CASP10 competition. Considerable advancements in the predictive power of the deep learning‐based contact predictions have been achieved since then. However, little effort has been put into interpreting the black‐box deep learning methods. Algorithms that can interpret the relationship between predicted contact maps and the internal mechanism of the deep learning architectures are needed to explore the essential components of contact inference and improve their explainability. In this study, we present an attention‐based convolutional neural network for protein contact prediction, which consists of two attention mechanism‐based modules: sequence attention and regional attention. Our benchmark results on the CASP13 free‐modeling targets demonstrate that the two attention modules added on top of existing typical deep learning models exhibit a complementary effect that contributes to prediction improvements. More importantly, the inclusion of the attention mechanism provides interpretable patterns that contain useful insights into the key fold‐determining residues in proteins. We expect the attention‐based model can provide a reliable and practically interpretable technique that helps break the current bottlenecks in explaining deep neural networks for contact prediction. The source code of our method is available at https://github.com/jianlin-cheng/InterpretContactMap .

59 BASIC BIOLOGICAL SCIENCES↗