Search NASA⌕ Search

SEARCH · Search NASA

Results for “Explanability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 415 records · Page 23

SAGE Intrusion Detection System: Sensitivity Analysis Guided Explainability for Machine Learning.

This report details the results of a three-fold investigation of sensitivity analysis (SA) for machine learning (ML) explainability (MLE): (1) the mathematical assessment of the fidelity of an explanation with respect to a learned ML model, (2) quantifying the trustworthiness of a prediction, and (3) the impact of MLE on the efficiency of end-users through multiple users studies. We focused on the cybersecurity domain as the data is inherently non-intuitive. As ML is being using in an increasing number of domains, including domains where being wrong can elicit high consequences, MLE has been proposed as a means of generating trust in a learned ML models by end users. However, little analysis has been performed to determine if the explanations accurately represent the target model and they themselves should be trusted beyond subjective inspection. Current state-of-the-art MLE techniques only provide a list of important features based on heuristic measures and/or make certain assumptions about the data and the model which are not representative of the real-world data and models. Further, most are designed without considering the usefulness by an end-user in a broader context. To address these issues, we present a notion of explanation fidelity based on Shapley values from cooperative game theory. We find that all of the investigated MLE explainability methods produce explanations that are incongruent with the ML model that is being explained. This is because they make critical assumptions about feature independence and linear feature interactions for computational reasons. We also find that in deployed, explanations are rarely used due to a variety of reason including that there are several other tools which are trusted more than the explanations and there is little incentive to use the explanations. In the cases when the explanations are used, we found that there is the danger that explanations persuade the end users to wrongly accept false positives and false negatives. However, ML model developers and maintainers find the explanations more useful to help ensure that the ML model does not have obvious biases. In light of these findings, we suggest a number of future directions including developing MLE methods that directly model non-linear model interactions and including design principles that take into account the usefulness of explanations to the end user. We also augment explanations with a set of trustworthiness measures that measure geometric aspects of the data to determine if the model output should be trusted.

97 MATHEMATICS AND COMPUTING↗

Nucleon and pio distribution functions in the valence region.

An experimental and theoretical perspective is provided on the behavior of unpolarized distribution functions for the nucleon and pion on the valence-quark domain, namely, Bjorken x {ge} 0.4. This domain is a key to much of hadron physics; e.g., a hadron is defined by its flavor content and that is a valence-quark property. Furthermore, its accurate parametrization is crucial to the provision of reliable input for large collider experiments. The focus is on experimental extractions of distribution functions via electron and muon inelastic scattering, and from Drell-Yan interactions; and on theoretical treatments that emphasize an explanation of the distribution functions, providing an overview of major contemporary approaches and issues. Valence-quark physics is a compelling subject, which probes at the heart of our understanding of the standard model. There are numerous outstanding and unresolved challenges, which experiment and theory must confront. In connection with experiment, an explanation that an upgraded Jefferson Laboratory facility is well suited to provide new data on the nucleon is provided, while a future electron-ion collider could provide essential new data for the mesons. There is also great potential in using Drell-Yan interactions, at FNAL, CERN, J-PARC, and GSI, to push into the large-x domain for both mesons and nucleons. Furthermore, it is argued that explanation, in contrast to modeling and parametrization, requires a widespread acceptance of the need to adapt theory: to the lessons learnt already from the methods of nonperturbative quantum-field theory and a fuller exploitation of those methods.

43 PARTICLE ACCELERATORS↗

Description of Transmutation Library for Fuel Cycle System Analyses

This report documents the Transmutation Library that is used in Fuel Cycle System Analyses. This version replaces the 2008 version.[Piet2008] The Transmutation Library has the following objectives: • Assemble past and future transmutation cases for system analyses. • For each case, assemble descriptive information such as where the case was documented, the purpose of the calculation, the codes used, source of feed material, transmutation parameters, and the name of files that contain raw or source data. • Group chemical elements so that masses in separation and waste processes as calculated in dynamic simulations or spreadsheets reflect current thinking of those processes. For example, the CsSr waste form option actually includes all Group 1A and 2A elements. • Provide mass fractions at input (charge) and output (discharge) for each case. • Eliminate the need for either “fission product other” or “actinide other” while conserving mass. Assessments of waste and separation cannot use “fission product other” or “actinide other” as their chemical behavior is undefined. • Catalog other isotope-specific information in one place, e.g., heat and dose conversion factors for individual isotopes. • Describe the correlations for how input and output compositions change as a function of UOX burnup (for LWR UOX fuel) or fast reactor (FR) transuranic (TRU) conversion ratio (CR) for either FR-metal or FR-oxide. This document therefore includes the following sections: • Explanation of the data set information, i.e., the data that describes each case. In no case are all of the data presented in the Library included in previous documents. In assembling the Library, we return to raw data files to extract the case and isotopic data, into the specified format. • Explanation of which isotopes and elements are tracked. For example, the transition metals are tracked via the following: two Zr isotopes, Zr-other, Tc99, Tc-other, two Mo-Ru-Rh-Pd isotopes, Mo-Ru-Rh-Pd-other, four other specific TM isotopes, and TM-other. Mo-Ru-Rh-Pd are separated because their content constrains the loading of waste in glass, so we have to know the mass of those elements independent of others. • Rules for collapsing long lists of isotopes (~1000) to the 81 items in the library. For each tracked isotope, we define which short-lived isotopes’ mass (at t=0) is included with the mass of the tracked isotope at t=0, which short-lived radioactive progeny must be accounted for when the tracked isotope decays, and to which of the other 80 items the mass of the tracked isotope goes when it decays. • Explanation of where raw data files can be found on the fuel cycle data portal. • Explanation of generic cross section sets • Explanation of isotope-specific parameters such as heat and dose conversion factors • Explanation of the LWR UOX burnup and FR TRU CR correlations.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Leveraging large language models to automate the identification of healthcare access barriers for veterans

Objective: To develop and evaluate an automated system for identifying healthcare barriers focusing on transportation issues in veterans’ clinical notes using large language models (LLMs) and to assess the impact of different prompting strategies on classification performance and explanation consistency. Methods: We developed a hybrid system combining pattern matching for templated notes with LLM analysis for free-text notes. Using 2000 manually annotated clinical notes, we compared four prompting strategies (dual-role short, dual-role long, analysis-first, analysis-only) across Mistral-7B and Llama-3.1 models. We evaluated classification performance using standard metrics and assessed explanation consistency through embedding similarity analysis. Results: The analysis-first strategy achieved superior performance, with Mistral-7B reaching an F1 score of 0.914, outperforming traditional machine learning approaches (GBM: 0.786, BERT: 0.811). LLMs demonstrated higher explanation consistency within models (mean cosine similarity 0.887–0.908) compared to cross-model similarities (0.767–0.872). Pattern matching successfully handled 6.7% of templated notes deterministically. Mistral-7B showed greater internal consistency but higher abstention rates compared to Llama-3.1. Conclusion: Requiring LLMs to analyze evidence before classification improves both accuracy and explanation consistency for identifying transportation barriers in clinical notes. This approach enables automated barrier detection at scale while providing clinically relevant explanations, supporting both population-level healthcare planning and individual patient care decisions.

Healthcare access barriers↗

ACES-GNN: can graph neural network learn to explain activity cliffs?

Graph Neural Networks (GNNs) have revolutionized molecular property prediction by leveraging graph-based representations, yet their opaque decision-making processes hinder broader adoption in drug discovery. This study introduces the Activity-Cliff-Explanation-Supervised GNN (ACES-GNN) framework, designed to simultaneously improve predictive accuracy and interpretability by integrating explanation supervision for activity cliffs (ACs) into GNN training. ACs, defined by structurally similar molecules with significant potency differences, pose challenges for traditional models due to their reliance on shared structural features. By aligning model attributions with chemist-friendly interpretations, the ACES-GNN framework bridges the gap between prediction and explanation. Validated across 30 pharmacological targets, ACES-GNN consistently enhances both predictive accuracy and attribution quality for ACs compared to unsupervised GNNs. Our results demonstrate a positive correlation between improved predictions and accurate explanations, offering a robust and adaptable framework to better understand and interpret ACs. This work underscores the potential of explanation-guided learning to advance interpretable artificial intelligence in molecular modeling and drug discovery.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Explaining machine-learning models for gamma-ray detection and identification

As more complex predictive models are used for gamma-ray spectral analysis, methods are needed to probe and understand their predictions and behavior. Recent work has begun to bring the latest techniques from the field of Explainable Artificial Intelligence (XAI) into the applications of gamma-ray spectroscopy, including the introduction of gradient-based methods like saliency mapping and Gradient-weighted Class Activation Mapping (Grad-CAM), and black box methods like Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). In addition, new sources of synthetic radiological data are becoming available, and these new data sets present opportunities to train models using more data than ever before. In this work, we use a neural network model trained on synthetic NaI(Tl) urban search data to compare some of these explanation methods and identify modifications that need to be applied to adapt the methods to gamma-ray spectral data. We find that the black box methods LIME and SHAP are especially accurate in their results, and recommend SHAP since it requires little hyperparameter tuning. We also propose and demonstrate a technique for generating counterfactual explanations using orthogonal projections of LIME and SHAP explanations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

US-UK Collaboration on Fossil Energy Advanced Materials: Task 1—Steam Oxidation

This presentation goes over the following from the US-UK collaboration on Fossil Energy Advanced Materials: Task 1, Steam Oxidation: US-led or co-led deliverables, Phase II products (US), 2011-present, Phase III products, Phase III Plan, an explanation of sCO 2 compared with sH 2 O, an explanation of Ni-base Alloys, an explanation of 300 Series (18Cr-8Ni)/E-Brite, an explanation of the typical Microchannel HX Fabrication process, and an explanation of diffusion bonded Ni-base superalloys.

02 PETROLEUM↗

Measure Utility, Gain Trust: Practical Advice for XAI Researchers

Research into explanation of machine learning models, i.e. explainable AI (XAI), has seen a sympathetic exponential growth alongside deep artificial neural networks throughout the past decade. For historical reasons explanation and trust have been intertwined. However this focus on trust is too narrow, and has led the research community astray from tried and true empirical methods that lead to more defensible scientific knowledge about people and explanations. To address this, we contribute a practical path forward for researchers in the XAI field. We recommend researchers focus on the utility and impact of their explanations instead of trust. We outline five broad use cases where explanations are useful and, for each, we describe pseudo-experiments that rely on objective empirical measurements and falsifiable hypotheses. We believe that this experimental rigor is necessary to contribute to scientific knowledge in the field of XAI.

Davis, Brittany F.↗

Explaining word embeddings with perfect fidelity: a case study in predicting research impact

The best-performing approaches for scholarly document quality prediction are based on embedding models. In addition to their performance when used in classifiers, embedding models can also provide predictions even for words that were not contained in the labelled training data for the classification model, which is important in the context of the ever-evolving research terminology. Although model-agnostic explanation methods, such as Local interpretable model-agnostic explanations, can be applied to explain machine learning classifiers trained on embedding models, these produce results with questionable correspondence to the model. We introduce a new feature importance method, Self-Model Entities Rated (SMER), for logistic regression-based classification models trained on word embeddings. We show that SMER has theoretically perfect fidelity with the explained model, as the average of logits of SMER scores for individual words (SMER explanation) exactly corresponds to the logit of the prediction of the explained model. Quantitative and qualitative evaluation is performed through five diverse experiments conducted on 50,000 research articles (papers) from the CORD-19 corpus. In conclusion, through an AOPC curve analysis, we experimentally demonstrate that SMER produces better explanations than LIME, SHAP and global tree surrogates.

Coarse-grained models↗

Dark Matter Candidates: A Status Report

A brief review is given of various dark matter candidates. In particular, some of the most studied candidates like axions, inert Higgs doublet, sterile neutrinos, supersymmetric particles and Kaluza-Klein particles are discussed. In particular, indirect detection methods are reviewed, with gamma-ray detection being a particularly important method. The situation also for indirect detection through antimatter cosmic rays has recently become quite interesting with new results having dark matter as one of the possible explanations. Problems of this explanation and possible solutions are discussed, and the importance of new measurements is emphasized. If the explanation is indeed dark matter, some unusual and unexpected properties would be needed. One should always keep the possible 'conventional' astrophysical explanations, like electron and positron radiation from pulsars in mind. To get a conclusive case for dark matter, detection in other channels will likely be needed, such as direct detection or detection of angular and spectral signatures in gamma-rays.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Origin of anomeric effect: A density functional steric analysis

The anomeric effect (the tendency of heteroatomic substituents adjacent to a heteroatom within the cyclohexane ring to prefer the axial orientation instead of the sterically less hindered equatorial position) is traditionally explained through either the dipole moment repulsion or the hyperconjugation effect. In this work, by employing our recent work in density functional steric analysis, we provide a novel two-component explanation, which is consistent with the common belief in chemistry that the effect has a stereoelectronic origin. With α-D-glucopyranose as the prototype, we systematically explore its conformational space and generate 32 isomers, leading to a total of 80 axial–equatorial conformation pairs. The energy difference analysis of these pairs shows that while statistically speaking the tendency is valid, the anomeric effect is not always true and can be violated. Three energy components, exchange–correlation, classical electrostatic, and density functional steric, are found to be directly proportional to the total energy difference between axial and equatorial isomers. We also found that the total dipole moment change, not the hyperconjugation effect, is a reasonable indicator of the total energy difference. However, all these correlations alone are not strong enough to provide a compellingly convincing explanation for the general validity of the effect. With the help of strong correlations between energy components, an explanation with two energy components, steric and electrostatic, was proposed in this work. We show that the axial–equatorial energy difference in general, with the anomeric effect as a special case, is dictated by two factors of the stereoelectronic origin, steric hindrance and classical electrostatic interactions, synchronously working together. Another explanation in terms of exchange–correlation and electrostatic interactions has also been obtained in this work.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

TEMPERATURE STRUCTURE AND METALLICITY IN H II REGIONS

The metallicities implied by collisionally excited lines (CELs) of heavy elements in H II regions are systematically lower than those implied by recombination lines (RLs) by factors of approx2, introducing uncertainties of the same order in the metallicities inferred for the interstellar medium of any star-forming galaxy. Most explanations of this discrepancy are based on the different sensitivities of CELs and RLs to electron temperature, and invoke either some extra heating mechanism producing temperature fluctuations in the ionized region or the addition of cold gas in metal-rich inclusions or ionized by cosmic rays or X-rays. These explanations will change the temperature structure of the ionized gas from the one predicted by simple photoionization models, and depending on which one is correct, will imply different metallicities for the emitting gas. We select nine H II regions with observed spectra of high quality and show that simple models with metallicities close to the ones implied by oxygen CELs reproduce easily their temperature structure, measured with T{sub e}([N II])/T{sub e}([O III]), and their oxygen CELs emission. We discuss the strong constraints that this agreement places on the possible explanations of the discrepancy and suggest that the simplest explanation, namely errors in the line recombination coefficients by factors approx2, might be the correct one. In such case, CELs will provide the best estimates of metallicity.

79 ASTRONOMY AND ASTROPHYSICS↗

A description of the Galactic Center excess in the Minimal Supersymmetric Standard Model

Observations with the Fermi Large Area Telescope (LAT) indicate an excess in gamma rays originating from the center of our Galaxy. A possible explanation for this excess is the annihilation of Dark Matter particles. We have investigated the annihilation of neutralinos as Dark Matter candidates within the phenomenological Minimal Supersymmetric Standard Model (pMSSM). An iterative particle filter approach was used to search for solutions within the pMSSM. We found solutions that are consistent with astroparticle physics and collider experiments, and provide a fit to the energy spectrum of the excess. The neutralino is a Bino/Higgsino or Bino/Wino/Higgsino mixture with a mass in the range 84–92 GeV or 87–97 GeV annihilating into W bosons. A third solutions is found for a neutralino of mass 174–187 GeV annihilating into top quarks. The best solutions yield a Dark Matter relic density 0.06<Ωh{sup 2}<0.13. These pMSSM solutions make clear forecasts for LHC, direct and indirect DM detection experiments. If the pMSSM explanation of the excess seen by Fermi-LAT is correct, a DM signal might be discovered soon.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

THE YOUNG PLANET-MASS OBJECT 2M1207b: A COOL, CLOUDY, AND METHANE-POOR ATMOSPHERE

The properties of 2M1207b, a young ({approx}8 Myr) planet-mass companion, have lacked a satisfactory explanation for some time. The combination of low luminosity, red near-IR colors, and L-type near-IR spectrum (previously consistent with T{sub eff} {approx} 1600 K) implies an abnormally small radius. Early explanations for the apparent underluminosity of 2M1207b invoked an edge-on disk or the remnant of a recent protoplanetary collision. The discovery of a second planet-mass object (HR8799b) with similar luminosity and colors as 2M1207b indicates that a third explanation, one of a purely atmospheric nature, is more likely. By including clouds, non-equilibrium chemistry, and low gravity, an atmosphere with effective temperature consistent with evolution cooling-track predictions is revealed. Consequently, 2M1207b, and others like it, requires no new physics to explain nor do they belong to a new class of objects. Instead they most likely represent the natural extension of cloudy substellar atmospheres down to low T{sub eff} and log (g). If this atmosphere only explanation for 2M1207b is correct, then very young planet-mass objects with near-IR spectra similar to field T dwarfs may be rare.

79 ASTRONOMY AND ASTROPHYSICS↗