SEARCH · Search NASA
Results for “Explanability”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Simple explanations and reasoning: From philosophy of science to expert systems
A preliminary prototype of a simple explanation system was constructed. Although the system, based on the idea of storytelling, did not incorporate all of the principles of simple explanation, it did demonstrate the potential of the approach. The system incorporated a hypertext system, an inference engine, and facilities for constructing contrast type explanations. The continued development of such a system should prove to be valuable. By extending the resources of the expert system paradigm, the knowledge engineer is not forced to learn a new set of skills, and the domain knowledge already acquired by him is not lost. Further, both the beginning user and the more advanced user can be accommodated. For the beginning user, corrective explanations and ES explanations provide facilities for more clearly understanding the way in which the system is functioning. For the more advanced user, the instance and state explanations allow him to focus on the issues at hand. The simple model of explanation attempts to exploit and show how the why and how facilities of the expert system paradigm can be extended by attending to the pragmatics of explanation and adding texture to the ordinary pattern of reasoning in a rule based system.
Explanation production by expert planners
Although the explanation capability of expert systems is usually listed as one of the distinguishing characteristics of these systems, the explanation facilities of most existing systems are quite primitive. Computer generated explanations are typically produced from canned text or by direct translation of the knowledge structures. Explanations produced in this manner bear little resemblance to those produced by humans for similar tasks. The focus of our research in explanation is the production of justifications for decisions by expert planning systems. An analysis of justifications written by people for planning tasks has been taken as the starting point. The purpose of this analysis is two-fold. First, analysis of the information content of the justifications will provide a basis for deciding what knowledge must be represented if human-like justifications are to be produced. Second, an analysis of the textual organization of the justifications will be used in the development of a mechanism for selecting and organizing the knowledge to be included in a computer-produced explanation. This paper describes a preliminary analysis done of justifications written by people for a planning task. It is clear that these justifications differ significantly from those that would be produced by an expert system by tracing the firing of production rules. The results from the text analysis have been used to develop an augmented phrase structured grammar (APSG) describing the organization of the justifications. The grammar was designed to provide a computationally feasible method for determining textual organization that will allow the necessary information to be communicated in a cohesive manner.
Improving the explanation capabilities of advisory systems
A major limitation of current advisory systems (e.g., intelligent tutoring systems and expert systems) is their restricted ability to give explanations. The goal of our research is to develop and evaluate a flexible explanation facility, one that can dynamically generate responses to questions not anticipated by the system's designers and that can tailor these responses to individual users. To achieve this flexibility, we are developing a large knowledge base, a viewpoint construction facility, and a modeling facility. In the long term we plan to build and evaluate advisory systems with flexible explanation facilities for scientists in numerous domains. In the short term, we are focusing on a single complex domain in biological science, and we are working toward two important milestones: (1) building and evaluating an advisory system with a flexible explanation facility for freshman-level students studying biology, and (2) developing general methods and tools for building similar explanation facilities in other domains.
Improving the explanation capabilities of advisory systems
A major limitation of current advisory systems (e.g., intelligent tutoring systems and expert systems) is their restricted ability to give explanations. The goal of our research is to develop and evaluate a flexible explanation facility, one that can dynamically generate responses to questions not anticipated by the system's designers and that can tailor these responses to individual users. To achieve this flexibility, we are developing a large knowledge base, a viewpoint construction facility, and a modeling facility. In the long term we plan to build and evaluate advisory systems with flexible explanation facilities for scientists in numerous domains. In the short term, we are focusing on a single complex domain in biological science, and we are working toward two important milestones: (1) building and evaluating an advisory system with a flexible explanation facility for freshman-level students studying biology; and (2) developing general methods and tools for building similar explanation facilities in other domains.
A human-in-the-loop explanation framework for morphologically transparent AI predictions from whole-slide images
Deep learning models enable the prediction of clinical endpoints from whole-slide images (WSIs), but many such models function as “black boxes”, lacking transparency about whether and which histomorphological patterns drive their predictions, hindering interpretability and clinical adoption. Here we propose a human-in-the-loop explanation framework, MorphoXAI, which provides both local and global interpretability for deep learning models by incorporating human-expert interpretations. At the global level, it reveals the histomorphological patterns on which the model consistently relies to distinguish between classes of WSIs, as well as the patterns associated with confusion between classes. At the local level, it indicates which of these patterns are used in the prediction of an individual WSI and which regions within the slide correspond to such patterns. We validated our method across multiple deep learning–based WSI analysis tasks spanning different tissue types. The results show that our framework generates explanations that accurately reflect the histomorphology underlying the model’s predictions at both global and local levels. For interpretability and clinical utility in diagnostic contexts, human evaluation results showed that our explanations were easy to interpret, rich in diagnostic features, and directly helpful for diagnostic decision-making, thereby enhancing pathologist-AI collaboration. Our work highlights that unifying global and local explanations and grounding them in expert-interpreted morphology enhances the interpretability and verifiability of deep learning models, thereby facilitating the transparent deployment of such models in clinical practice.
An architecture for rule based system explanation
A system architecture is presented which incorporate both graphics and text into explanations provided by rule based expert systems. This architecture facilitates explanation of the knowledge base content, the control strategies employed by the system, and the conclusions made by the system. The suggested approach combines hypermedia and inference engine capabilities. Advantages include: closer integration of user interface, explanation system, and knowledge base; the ability to embed links to deeper knowledge underlying the compiled knowledge used in the knowledge base; and allowing for more direct control of explanation depth and duration by the user. User models are suggested to control the type, amount, and order of information presented.
Robust Explanations using Diverse Adversarially Trained Ensembles, Multi-Modal Contrastive Learning, and Attribution-based Confidence Metrics
The primary objective of this project is to strengthen the trustworthiness of AI systems by designing algorithms that make their internal decision-making processes more understandable to human users. This involves creating clear, interpretable explanations for AI decisions and developing metrics to assess these explanations' validity and reliability. Significant progress has been achieved through (i) developing symbolic explanations, (ii) generating meaningful interpretive insights, (iii) establishing accuracy and confidence metrics, and (iv) devising methods to evaluate the knowledge boundaries of AI models. To date, the research findings have been shared in peer-reviewed publications, with accompanying scientific and technical information (STI) detailed below.
Graphical explanation in an expert system for Space Station Freedom rack integration
The rationale and methodology used to incorporate graphics into explanations provided by an expert system for Space Station Freedom rack integration is examined. The rack integration task is typical of a class of constraint satisfaction problems for large programs where expertise from several areas is required. Graphically oriented approaches are used to explain the conclusions made by the system, the knowledge base content, and even at more abstract levels the control strategies employed by the system. The implemented architecture combines hypermedia and inference engine capabilities. The advantages of this architecture include: closer integration of user interface, explanation system, and knowledge base; the ability to embed links to deeper knowledge underlying the compiled knowledge used in the knowledge base; and allowing for more direct control of explanation depth and duration by the user. The graphical techniques employed range from simple statis presentation of schematics to dynamic creation of a series of pictures presented motion picture style. User models control the type, amount, and order of information presented.
Explanations in knowledge systems - Design for Explainable Expert Systems
The Explainable Expert Systems framework (EES), in which the focus is on capturing those design aspects that are important for producing good explanations, including justifications of the system's actions, explications of general problem-solving strategies, and descriptions of the system's terminology, is discussed. EES was developed as part of the Strategic Computing Initiative of DARPA. Both the general principles from which the system was derived and how the system was derived from those principles can be represented in EES. The Program Enhancement Advisor, which is the main prototype on which the explanation work has been developed and tested, is presented. PEA is an advice system that helps users improve their Common Lisp programs by recommending transformations that enhance the user's code. How EES produces better explanations is shown.
Explanation-based generalization of partially ordered plans
Most previous work in analytic generalization of plans dealt with totally ordered plans. These methods cannot be directly applied to generalizing partially ordered plans, since they do not capture all interactions among plan operators for all total orders of such plans. We introduce a new method for generalizing partially ordered plans. This method is based on providing explanation-based generalization (EBG) with explanations which systematically capture the interactions among plan operators for all the total orders of a partially-ordered plan. The explanations are based on the Modal Truth Criterion which states the necessary and sufficient conditions for ensuring the truth of a proposition at any point in a plan, for a class of partially ordered plans. The generalizations obtained by this method guarantee successful and interaction-free execution of any total order of the generalized plan. In addition, the systematic derivation of the generalization algorithms from the Modal Truth Criterion obviates the need for carrying out a separate formal proof of correctness of the EBG algorithms.
Portfolios in Stochastic Local Search: Efficiently Computing Most Probable Explanations in Bayesian Networks
Portfolio methods support the combination of different algorithms and heuristics, including stochastic local search (SLS) heuristics, and have been identified as a promising approach to solve computationally hard problems. While successful in experiments, theoretical foundations and analytical results for portfolio-based SLS heuristics are less developed. This article aims to improve the understanding of the role of portfolios of heuristics in SLS. We emphasize the problem of computing most probable explanations (MPEs) in Bayesian networks (BNs). Algorithmically, we discuss a portfolio-based SLS algorithm for MPE computation, Stochastic Greedy Search (SGS). SGS supports the integration of different initialization operators (or initialization heuristics) and different search operators (greedy and noisy heuristics), thereby enabling new analytical and experimental results. Analytically, we introduce a novel Markov chain model tailored to portfolio-based SLS algorithms including SGS, thereby enabling us to analytically form expected hitting time results that explain empirical run time results. For a specific BN, we show the benefit of using a homogenous initialization portfolio. To further illustrate the portfolio approach, we consider novel additive search heuristics for handling determinism in the form of zero entries in conditional probability tables in BNs. Our additive approach adds rather than multiplies probabilities when computing the utility of an explanation. We motivate the additive measure by studying the dramatic impact of zero entries in conditional probability tables on the number of zero-probability explanations, which again complicates the search process. We consider the relationship between MAXSAT and MPE, and show that additive utility (or gain) is a generalization, to the probabilistic setting, of MAXSAT utility (or gain) used in the celebrated GSAT and WalkSAT algorithms and their descendants. Utilizing our Markov chain framework, we show that expected hitting time is a rational function - i.e. a ratio of two polynomials - of the probability of applying an additive search operator. Experimentally, we report on synthetically generated BNs as well as BNs from applications, and compare SGSs performance to that of Hugin, which performs BN inference by compilation to and propagation in clique trees. On synthetic networks, SGS speeds up computation by approximately two orders of magnitude compared to Hugin. In application networks, our approach is highly competitive in Bayesian networks with a high degree of determinism. In addition to showing that stochastic local search can be competitive with clique tree clustering, our empirical results provide an improved understanding of the circumstances under which portfolio-based SLS outperforms clique tree clustering and vice versa.
Explanation of the seasonal variation of cosmic multiple muon events observed with the NOvA Near Detector
The flux of cosmic ray muons at the Earth’s surface exhibits seasonal variations due to changes in the temperature of the atmosphere affecting the production and decay of mesons in the upper atmosphere. Using 1473 live days of data collected by the NuMI Off-axis 𝜈 𝑒 Appearance (NOvA) Near Detector during 2018–2022, we studied the seasonal pattern in the multiple-muon event rate. The data confirm an anticorrelation between the multiple-muon event rate and effective atmospheric temperature, consistent across all the years of data. Previous analyses from MINOS and NOvA saw a similar anticorrelation but did not include an explanation. We find that this anticorrelation is driven by altitude–geometry effects as the average muon production height changes with the season. This has been studied with a CORSIKA cosmic ray simulation package by varying atmospheric parameters, and provides an explanation to a longstanding discrepancy between the seasonal phases of single and multiple-muon events.
Integrating adaptive learning with post hoc model explanation and symbolic regression to build interpretable surrogate models
Abstract We develop a materials informatics workflow to build an interpretable surrogate model for micromagnetic simulations. Our goal is to predict the energy barrier of a moving isolated skyrmion in rare-earth-free $$\hbox {Mn}_4$$ Mn 4 N. Our approach integrates adaptive learning with post hoc model explanation and symbolic regression methods. We discuss an unexplored acquisition function (information condensing active learning) within the adaptive learning loop and compare it with the known standard deviation function for efficient navigation of the search space. Model-agnostic post hoc explanation techniques then uncover trends learned by the trained model, which we then leverage to constrain the expressions used for symbolic regression. Graphical abstract
Io's sodium cloud - Explanation of the east-west asymmetries
An explanation of the east-west asymmetries of Io's sodium clouds is suggested based upon the force experienced by sodium atoms as they resonantly scatter sunlight and upon the ability of this force to alter the cloud atom orbits, as determined by the gravitational fields of Io and Jupiter. Orbit calculations illustrating the time-dependent changes in the sodium atom trajectories produced by the solar radiation force as a function of Io's orbital location are presented in support of this explanation. This theory agrees with the idea that sodium atoms are emitted from the satellite nonisotropically and at a constant rate. These conclusions may be important in studying the global effects of active volcanoes and the intense satellite plasma torus discovered for Io from Voyager I spacecraft measurements.
CLIPS enhanced with objects, backward chaining and explanation facilities
In this project we extend C Language Production System (CLIPS), an existing Expert System shell, by creating three new options. Specifically, first we create a compatible with CLIPS environment that allows for defining objects and object hierarchies, second we provide means to implement backward chaining in a pure forward chaining environment, and finally we give some simple explanation facilities for the derivations the system has made. Objects and object hierarchies are extended so that facts can be automatically inferred, and placed in the fact base. Backward chaining is implemented by creating run time data structures which hold the derivation process allowing for a depth first search. The backward chaining mechanism works not only with ground facts, but also creates bindings for every query that involves variables, and returns the truth value of such a query as well as the relevant variable bindings. Finally, the WHY and HOW explanation facilities allow for a complete examination of the derivation process, the rules triggered, and the bindings created. The entire system is integrated with the original CLIPS code, and all of its routines can be invoked as CLIPS commands.
Initialization and Restart in Stochastic Local Search: Computing a Most Probable Explanation in Bayesian Networks
For hard computational problems, stochastic local search has proven to be a competitive approach to finding optimal or approximately optimal problem solutions. Two key research questions for stochastic local search algorithms are: Which algorithms are effective for initialization? When should the search process be restarted? In the present work we investigate these research questions in the context of approximate computation of most probable explanations (MPEs) in Bayesian networks (BNs). We introduce a novel approach, based on the Viterbi algorithm, to explanation initialization in BNs. While the Viterbi algorithm works on sequences and trees, our approach works on BNs with arbitrary topologies. We also give a novel formalization of stochastic local search, with focus on initialization and restart, using probability theory and mixture models. Experimentally, we apply our methods to the problem of MPE computation, using a stochastic local search algorithm known as Stochastic Greedy Search. By carefully optimizing both initialization and restart, we reduce the MPE search time for application BNs by several orders of magnitude compared to using uniform at random initialization without restart. On several BNs from applications, the performance of Stochastic Greedy Search is competitive with clique tree clustering, a state-of-the-art exact algorithm used for MPE computation in BNs.
Testing Explanations of Short Baseline Neutrino Anomalies
The experimental observation of neutrino oscillations profoundly impacted the physics of neutrinos, from being well understood theoretically to requiring new physics beyond the standard model of particle physics. Indeed, the mystery of neutrino masses implies the presence of new particles never observed before, often called sterile neutrinos, as they would not undergo standard weak interactions. And while neutrino oscillation measurements entered the precision era, reaching percent-level precision, many experimental results show significant discrepancies with the standard model, at baselines much shorter than typical oscillation baselines, like LSND, MiniBooNE, gallium experiments, and reactor antineutrino measurements. These short baseline anomalies could be explained by the addition of a light sterile neutrino, with mass in the $1-10~\text{eV}$ range, however, in strong tension with many null experimental observations. Other explanations that rely on sterile neutrinos with masses in the $1-500~\text{MeV}$ could resolve the tension. Here we test both classes of models. On the one hand, we look for datasets collected at a short baseline which can constrain heavy sterile neutrino models. We find that the minimal model is fully constrained, but several extensions of this model could weaken the current constraint and be tested with current and future datasets. On the other hand, we test the presence of neutrino oscillations at short baselines, induced by a light sterile state, with the data collected by the MicroBooNE experiment, a liquid argon time projection chamber specifically designed to resolve the details of each neutrino interaction. We report null results from both analyses, further constraining the space of possible explanations for the short baseline anomalies. If new physics lies behind the short baseline anomaly puzzle, it is definitely not described by a simple model.