Search NASA⌕ Search

SEARCH · Search NASA

Results for “model interpretability”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 127 records · Page 7

Impact of Statistical Learning Methods on the Predictive Power of Multivariate Normal Tissue Complication Probability Models

Purpose: To study the impact of different statistical learning methods on the prediction performance of multivariate normal tissue complication probability (NTCP) models. Methods and Materials: In this study, three learning methods, stepwise selection, least absolute shrinkage and selection operator (LASSO), and Bayesian model averaging (BMA), were used to build NTCP models of xerostomia following radiotherapy treatment for head and neck cancer. Performance of each learning method was evaluated by a repeated cross-validation scheme in order to obtain a fair comparison among methods. Results: It was found that the LASSO and BMA methods produced models with significantly better predictive power than that of the stepwise selection method. Furthermore, the LASSO method yields an easily interpretable model as the stepwise method does, in contrast to the less intuitive BMA method. Conclusions: The commonly used stepwise selection method, which is simple to execute, may be insufficient for NTCP modeling. The LASSO method is recommended.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Ripening of Rh Nanoparticle Catalysts in Reverse Water–Gas Shift via a Data-Driven Model Combining Physics, Theory, and Experiment

Degradation via sintering is an ongoing challenge that impedes the broad commercial success of supported metallic nanoparticle catalysts. To mitigate degradation via informed catalyst design and process operations, here we aim to disambiguate the underlying mechanisms of sintering by combining theory and experiment in a quantitative framework. While mechanistic sintering models exist, they only model a single sintering pathway, even though multiple sintering mechanisms can occur simultaneously or dominate at different stages of the process. Data-driven machine learning models have emerged as a means to represent complex processes through data regression. However, machine learning models have very large data needs and lack mechanistic insights due to their black-box encoding. To develop an interpretive model of catalyst degradation via sintering, we constructed a hybrid model combining mechanistic “physics-based” models and data-driven methods to obtain both reliable predictions and mechanistic insights regarding experimentally observed sintering phenomena. Focusing on nanoparticle sintering in the Rh–TiO 2 catalyst for the reverse water–gas shift (RWGS) reaction, the hybrid model couples a mechanistic term for Ostwald ripening with energy values calculated via density functional theory (DFT) with a parametric, data-driven discrepancy function term for unmodeled mechanisms. The hybrid model is trained using Bayesian inference with data collected from small-angle X-ray scattering (SAXS) in situ experiments wherein average nanoparticle diameter versus time was measured at three relevant operating temperatures. The calibrated hybrid model results show that an Ostwald ripening-only model parameterized with fixed DFT energies does not fully capture the time and temperature dependence of the SAXS-observed sintering kinetics, and that an additional functional contribution, or DFT energy calibration, is required to reconcile simulation and experiment. Analysis of the hybrid-model error confirms that the hybrid model outperforms both the purely mechanistic and purely data-driven alternatives in terms of expected predictive accuracy for time-evolving average particle sizes. Furthermore, the results support the hypothesis that the Ostwald ripening mechanism is less important for explaining the sintering phenomena as operating temperature increases under an assumed fixed DFT parameterization. This could be explained in one of two ways: either latent, unmodeled sintering mechanisms dominate at higher temperatures, or the DFT uncertainty increases with temperature. The proposed modeling approach directly links theory to experiments and simulations via a statistical hybrid modeling framework and can be extended to other catalytic systems to improve predictive models and mechanistic understanding.

Bayesian hybrid modeling↗

Jet classification using high-level features from anatomy of top jets

Recent advancements in deep learning models have significantly enhanced jet classification performance by analyzing low-level features (LLFs). However, this approach often leads to less interpretable models, emphasizing the need to understand the decision-making process and to identify the high-level features (HLFs) crucial for explaining jet classification. To address this, we consider the top jet tagging problems and introduce an analysis model (AM) that analyzes selected HLFs designed to capture important features of top jets. Our AM mainly consists of the following three modules: a relation network analyzing two-point energy correlations, mathematical morphology and Minkowski functionals for generalizing jet constituent multiplicities, and a recursive neural network analyzing subjet constituent multiplicity to enhance sensitivity to subjet color charges. We demonstrate that our AM achieves performance comparable to the Particle Transformer (ParT) while requiring fewer computational resources in a comparison of top jet tagging using jets simulated at the hadronic calorimeter angular resolution scale. Furthermore, as a more constrained architecture than ParT, the AM exhibits smaller training uncertainties because of the bias-variance tradeoff. We also compare the information content of AM and ParT by decorrelating the features already learned by AM. Lastly, we briefly comment on the results of AM with finer angular resolution inputs.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Evaluation and projection of long period return values of extreme daily precipitation in the CMIP5 and CMIP6 models

Using a non-stationary Generalized Extreme Value statistical method, we calculate selected extreme daily precipitation indices and their 20 year return values from the CMIP5 and CMIP6 climate models over the historical and future periods. We evaluate model performance of these indices and their return values in replicating similar quantities calculated from multiple gridded observational products. Difficulties in interpreting model quality in the context of observational uncertainties are discussed. Projections are framed in terms of specified global warming target temperatures rather than at specific times and under specific emissions scenarios. The change in framing shifts projection uncertainty due to differences in model climate sensitivity from the values of the projections to the timing of the global warming target. At their standard resolutions, we find there are no meaningful differences between the two generations of models in their quality or projections of simulated extreme daily precipitation.

54 ENVIRONMENTAL SCIENCES↗

An interpretable machine learning framework to understand bikeshare demand before and during the COVID-19 pandemic in New York City

In recent years, bikesharing systems have become increasingly popular as affordable and sustainable micromobility solutions. Advanced mathematical models such as machine learning are required to generate good forecasts for bikeshare demand. Here, this study proposes a machine learning modeling framework to estimate hourly demand in a large-scale bikesharing system. Two Extreme Gradient Boosting models were developed: one using data from before the COVID-19 pandemic (March 2019 to February 2020) and the other using data from during the pandemic (March 2020 to February 2021). Furthermore, a model interpretation framework based on SHapley Additive exPlanations was implemented. Based on the relative importance of the explanatory variables considered in this study, share of female users and hour of day were the two most important explanatory variables in both models. However, the month variable had higher importance in the pandemic model than in the pre-pandemic model.

99 GENERAL AND MISCELLANEOUS↗

Preliminary Findings of the South Africa Power System Capacity Expansion and Operational Modelling Study: Preprint

Wind and solar power contract prices have recently become cheaper than many conventional new-build alternatives in South Africa and trends suggest a continued increase in the share of variable renewable energy (vRE) on South Africa's power system with coal technology seeing the greatest reduction in capacity, see 'Figure 6: Percentage share by Installed Capacity (MW)' in [1]. Hence it is essential to perform a state-of-the-art grid integration study examining the effects of these high penetrations of vRE on South Africa's power system. Under the 21st Century Power Partnership (21CPP), funded by the U.S. Department of Energy, the National Renewable Energy Laboratory (NREL) has significantly augmented existing models of the South African power system to investigate future vRE scenarios. NREL, in collaboration with Eskom's Planning Department, further developed, tested and ran a combined capacity expansion and operational model of the South African power system including spatially disaggregated detail and geographical representation of system resources. New software to visualize and interpret modelling outputs has been developed, and scenario analysis of stepwise vRE build targets reveals new insight into associated planning and operational impacts and costs. The model, built using PLEXOS, is split into two components, firstly a capacity expansion model and secondly a unit commitment and economic dispatch model. The capacity expansion model optimizes new generation decisions to achieve the lowest cost, with a full understanding of capital cost and an approximated understanding of operational costs. The operational model has a greater set of detailed operational constraints and is run at daily resolutions. Both are run from 2017 through 2050. This investigation suggests that running both models in tandem may be the most effective means to plan the least cost South African power system as build plans seen to be more expensive than optimal by the capacity expansion model can produce greater operational cost savings seen only in the operational model.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Scale-Dependent Fracture-Matrix Interactions and Their Impact on Radionuclide Transport: Development of efficient particle-tracking methods

Matrix Diffusion and Adsorption within a rock matrix are important mechanisms for retarding transport of radionuclides in fractured rock. Due to computational limitations and difficulties in characterizing complex subsurface systems, diffusive exchange between a fracture network and surrounding rock matrix is often modeled using simplified conceptual representations. There is significant uncertainty in “effective” parameters used in these models, such as the “effective matrix diffusivity”. Often, these parameters are estimated by fitting sparse breakthrough data, and estimated values fall outside meaningful ranges, because simplified interpretive models do not consider complex three-dimensional flow. There is limited understanding of the relationship between the effective parameters and rock mass characteristics including network structure and matrix properties. There is also evidence for an apparent scale-dependence in “effective matrix diffusion” coefficients. These observations raise questions on whether fracture-matrix interaction parameters estimated from small-scale tracer tests can be used for predicting radionuclide fate and transport at the scale of DOE field sites. High-resolution three-dimensional Discrete-Fracture-Network-Matrix (DFNM) models based on well-defined local scale transport equations can help to address some of these questions. Due to tremendous advances in computational technology over the last 10 years, DFNM modeling in relatively large domains is now feasible. The overarching objective of our research is to use DFNM modeling to improve fundamental understanding of how effective parameters in conceptual models are related to fracture network structure and matrix properties. An advanced three-dimensional DFNM model is being developed, which combines upscaled particle-tracking algorithms for fracture-matrix interaction and a parallel fracture-network flow simulator. The particle-tracking algorithms allow complexity in flow fields at different scales, and track transport across fracture-matrix interfaces based on rigorous local approximations to the transport equations. This modeling approach can incorporate aperture variability, multi-scale preferential flow and matrix heterogeneity. We developed efficient particle-tracking methods for handling matrix diffusion and adsorption on fracture walls and demonstrated their efficiency for use within the context of large-scale complex fracture network models with variability in apertures across a network of fractures and within individual fractures.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Fully generalized, turbulent trace impurity transport with Gkeyll and Flan in the DIII-D far-SOL

The Monte Carlo trace impurity turbulent transport code Flan is introduced for the first time. Flan follows impurities in a turbulent background plasma from Gkeyll using the Lorentz force to resolve the full particle gyro orbit. Collisions are handled using the Nanbu collision algorithm (Nanbu 1997 Phys. Rev. E 55 4642–52), and ionization/recombination is handled via ADAS coupling. The far-SOL of a generic DIII-D L-mode is simulated with and without the collision model to show how collisions affect radial tungsten transport. Anomalous diffusion coefficient (D r ) and pinch velocity (v p ) profiles are extracted from fits to the results. With collisions, D r and v p are between 0–1.0 m 2 s −1 and −100–100 m s −1 , respectively. Without collisions, D r and v p are between 0–0.3 m 2 s −1 and −50–50 m s −1 , respectively. Exponential fits to the radial W density profiles and experimental data from W deposition along a collector probe are in good agreement, demonstrating Flan as a useful interpretive modeling tool. Additional simulations show that impurity transport away from the wall increases with atomic number, though it is not clear why. Flan has the potential to better interpret existing data and improve reactor scale predictions of core contamination because the underlying physics model is very general and does not rely on arbitrary user-defined transport coefficients.

DIII-D↗

Validation and parameterization of a novel physics-constrained neural dynamics model applied to turbulent fluid flow

We report, in fluid physics, data-driven models to enhance or accelerate time to solution are becoming increasingly popular for many application domains, such as alternatives to turbulence closures, system surrogates, or for new physics discovery. In the context of reduced order models of high-dimensional time-dependent fluid systems, machine learning methods grant the benefit of automated learning from data, but the burden of a model lies on its reduced-order representation of both the fluid state and physical dynamics. In this work, we build a physics-constrained, data-driven reduced order model for Navier–Stokes equations to approximate spatiotemporal fluid dynamics in the canonical case of isotropic turbulence in a triply periodic box. The model design choices mimic numerical and physical constraints by, for example, implicitly enforcing the incompressibility constraint and utilizing continuous neural ordinary differential equations for tracking the evolution of the governing differential equation. We demonstrate this technique on a three-dimensional, moderate Reynolds number turbulent fluid flow. In assessing the statistical quality and characteristics of the machine-learned model through rigorous diagnostic tests, we find that our model is capable of reconstructing the dynamics of the flow over large integral timescales, favoring accuracy at the larger length scales. More significantly, comprehensive diagnostics suggest that physically interpretable model parameters, corresponding to the representations of the fluid state and dynamics, have attributable and quantifiable impact on the quality of the model predictions and computational complexity.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

An interpretable machine learning model for advancing terrestrial ecosystem predictions

We apply an interpretable Long Short-Term Memory (iLSTM) network for land-atmosphere carbon flux predictions based on time series observations of seven environmental variables. iLSTM enables interpretability of variable importance and variable-wise temporal importance to the prediction of targets by exploring internal network structures. The application results indicate that iLSTM not only improves prediction performance by capturing different dynamics of individual variables, but also reasonably interprets the different contribution of each variable to the target and its different temporal relevance to the target. This variable and temporal importance interpretation of iLSTM advances terrestrial ecosystem model development as well as our predictive understanding of the system.

Lu, Dan↗

Observed and modeled patterns of covariability between low-level cloudiness and the structure of the trade-wind layer

We present patterns of covariability between low-level cloudiness and the trade-wind boundary layer structure using long-term measurements at a site representative of dynamical regimes with moderate subsidence or weak ascent. We compare these with ECMWF’s Integrated Forecast System and 10 CMIP5 models. By using single-time step output at a single location, we find that models can produce a fairly realistic trade-wind layer structure in long-term means, but with unrealistic variability at shorter-time scales. The unrealistic variability in modeled cloudiness near the lifting condensation level (LCL) is due to stronger than observed relationships with mixed-layer relative humidity (RH) and temperature stratification at the mixed-layer top. Those relationships are weak in observations, or even of opposite sign, which can be explained by a negative feedback of convection on cloudiness. Cloudiness near cumulus tops at the tradewind inversion instead varies more pronouncedly in observations on monthly time scales, whereby larger cloudiness relates to larger surface winds and stronger trade-wind inversions. However, these parameters appear to be a prerequisite, rather than strong controlling factors on cloudiness, because they do not explain submonthly variations in cloudiness. Models underestimate the strength of these relationships and diverge in particular in their responses to large-scale vertical motion. No model stands out by reproducing the observed behavior in all respects. As a result, these findings suggest that climate models do not realistically represent the physical processes that underlie the coupling between trade-wind clouds and their environments in present-day climate, which is relevant for how we interpret modeled cloud feedbacks.

58 GEOSCIENCES↗

Interpretable machine learning-guided design of Fe-based soft magnetic alloys

Here, we present a machine learning (ML) guided approach to predict saturation magnetization (𝑀 S ) and coercivity (𝐻 C ) in Fe-rich soft magnetic alloys, particularly Fe-Si-B systems. ML models trained on experimental data reveal that increasing Si and B content reduces 𝑀 S from 1.81 T (DFT ≈ 2.04 T) to ≈1.54 T (DFT ≈ 1.56T) in Fe-Si-B, which is attributed to decreased magnetic density and structural modifications. Experimental validation of ML predicted magnetic saturation on Fe-1Si-1B (2.09 T), Fe-5Si-5B (2.01 T), and Fe-10Si-10B (1.54 T) alloy compositions further supports our findings. These trends are consistent with density functional theory predictions, which link increased electronic disorder and band broadening to lower 𝑀 S values. Experimental validation on selected alloys confirms the predictive accuracy of the ML model, with good agreement across compositions. Beyond predictive accuracy, detailed uncertainty quantification and model interpretability including through feature importance and partial dependence analysis reveal that 𝑀 S is governed by a nonlinear interplay between Fe content and early transition metal ratios, while 𝐻 C is more sensitive to processing conditions such as ribbon thickness and thermal treatment windows. The ML framework was further applied to Fe-Si-B/Cr/Cu/Zr/Nb alloys in a pseudoquaternary compositional space, which shows comparable magnetic properties to NANOMET (Fe 84.8 ⁢Si 0.5 ⁢B 9.4 ⁢Cu 0.8⁢ P 3.5 ⁢C 1 ), FINEMET (Fe 73.5 ⁢Si 13.5 ⁢B 9 Cu 1 ⁢Nb 3 ), NANOPERM (Fe 88 ⁢Zr 7⁢ B 4 ⁢Cu 1 ), and HITPERM (Fe 44 ⁢Co 44 ⁢Zr 7⁢ B 4 ⁢Cu 1 . Our findings demonstrate the potential of the ML framework for accelerated search of high-performance soft magnetic materials.

density functional theory↗

Challenges in the Use of AI-Driven Non-Destructive Spectroscopic Tools for Rapid Food Analysis

Routine, remote, and process analysis for foodstuffs is gaining attention and can provide more confidence for the food supply chain. A new generation of rapid methods is emerging both in the literature and in industry based on spectroscopy coupled with AI-driven modelling methods. Current published studies using these advanced methods are plagued by weaknesses, including sample size, abuse of advanced modelling techniques, and the process of validation for both the acquisition method and modelling. This paper aims to give a comprehensive overview of the analytical challenges faced in research and industrial settings where screening analysis is performed while providing practical solutions in the form of guidelines for a range of scenarios. After extended literature analysis, we conclude that there is no easy way to enhance the accuracy of the methods by using state-of-the-art modelling methods and the key remains that capturing good quality raw data from authentic samples in sufficient volume is very important along with robust validation. A comprehensive methodology involving suitable analytical techniques and interpretive modelling methods needs to be considered under a tailored experimental design whenever conducting rapid food analysis.

59 BASIC BIOLOGICAL SCIENCES↗

Prediction of Creep-Induced Strain Using a Symbolic Regression-Based Model

Material creep under high-temperature conditions limits the lifetime and safety of structural systems such as advanced nuclear reactors. Conventional creep testing is slow and often produces inconsistent results across nominally identical experiments, making lifetime prediction uncertain. Here, to address these challenges, this work develops a data-driven symbolic regression (SR) model that consolidates results from duplicate creep tests and predicts the remaining strain-time curve of an ongoing experiment. The method uses piece-wise multi-objective SR with physical constraints to generate analytic, interpretable functions describing transient creep strain. Applied to Inconel Alloy 617 data, the approach achieved relative mean absolute errors of 1.0–9.5%, providing closed-form predictions of strain evolution. These results demonstrate a first step toward reducing the duration and cost of long-term creep testing while retaining physically interpretable model forms.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

DECOVALEX-2023: Task A Final Report

Task A, also known as HGFrac, examines the fracturing processes that may occur in the Callovo-Oxfordian claystone (COx) in the context of the high-level (HLW) and intermediate-level long-lived (ILW-LL) radioactive waste repository in France. Understanding these processes and improving numerical models to reproduce them will aid in the design, optimization, and safety of the repository. Heat and gas fracturing are studied in two independent subtasks following a stepwise approach: laboratory tests/benchmark exercises, in-situ experiment, and finally, an application case. The in-situ heater experiment aimed to thermally induce a hydraulic fracture; temperature and pore pressure were monitored to detect any evidence of fracturing. The in-situ gas injection experiment aimed to study the effect of the stress orientation and gas injection kinetics on the gas fracturing process. The occurrence of fracturing was monitored by gas pressure measured in the injection interval. In both experiments, the excavation-induced fracture network around the heater/injection boreholes played an important role in the reduction of the compressive stress state, leading to both a tensile and shear failure response of the COx. During the first two years of the project, the research teams working on each task developed and/or proposed numerical approaches for reproducing the occurrence of fracturing in the in-situ experiments. The failure criteria were defined by reproducing the measurements from laboratory extension tests for the heat fracturing subtask. In the gas fracturing subtask, their approaches were used for simulating several benchmark exercises, and an inter-comparison between models was carried out. In both tasks, the developed approaches were compared with a simplified approach considering poro-elasticity for the mechanical behaviour of the COx. Most of the developed approaches are based on a continuous medium that takes into account variations in hydraulic properties due to mechanical degradation, such as plastic deformation or damage. Other approaches implicitly modelled weak planes or embedded discontinuities to reproduce fracture propagation. The potential for fracture initiation was also studied through of a discrete approach. In the second half of the project, the research teams mainly focused on interpretative modelling of two in-situ experiments and a blind prediction exercise to test their respective approaches. The models developed by the research teams were also applied at the repository scale to evaluate fracture initiation in a case study under vi unfavourable conditions, particularly in terms of spacing between High-Level Waste cells. The results showed that the poro-elasticity approach could be an efficient tool for understanding the main processes occurring in the COx. One example is the explicit representation of the excavation-induced fracture network around the boreholes, which yielded acceptable results compared to the measurement data. However, advanced approaches were needed to evaluate the potential increase of the excavation-induced fracture network extend and better understand fracture initiation. The stress analyses carried out by the teams revealed that hydraulic boundary conditions had a strong impact on fracture initiation in the heater experiment. Furthermore, in most cases, the results required higher pore pressure increments to reach fracturing than those measured in the experiment. This implies that the measurements may have been biased by the packer’s capacity to fully isolate the piezometric chambers, leading to lower pressures. On the contrary, there was no agreement on the fracturing mode, as some reported either shear or tensile fracturing, while others reported a combination of the two modes. In the case of gas fracturing, the research teams were limited to the comparison of a single point, which complicated their task. Nonetheless, the numerical results were able to reproduce the measurements and capture processes such as longitudinal gas flow through the excavation-induced fracture network, as suggested by some evidence in the observation piezometric chambers. The numerical models also agreed with the measurements in the sense of higher probability of developing along the injection borehole than radially towards the sound rock. The approaches developed by the research teams showed that they are capable of analysing and reproducing fracture initiation in the COx. However, areas of future work should focus on the fracture propagation and fracture aperture, which were out of the scope of this task. To this end, additional data must be gathered for the parameter characterisation and validation of the numerical models. Nonetheless, various approaches showed promising results as they were able to reproduce fracture development under certain conditions.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

Balancing Trade-offs: Adaptive Differential Privacy in Interpretable Machine Learning Models

In the advancing field of machine learning, balancing accuracy, interpretability, and privacy represents a significant challenge. The problem is exacerbated by the widespread deployment of pre-trained models locally in diverse applications, which could lead to various amounts of privacy leakage. Conventional Differential Privacy strategies, in which uniform noises are applied to model gradients, guarantee data privacy at the expense of accuracy and interpretability. This paper introduces a Feature-Sensitive Adaptive Differential Privacy (FADP) framework with a unique noise-adding strategy. Noises are adaptively added based on feature importance clustering, where important features are considered for interpretability. By employing a unique masking technique, FADP selectively preserves crucial features with minimal noise interference, maintaining accuracy while enhancing interpretability. The FADP framework addresses the limitations of traditional DP methods by preserving critical channels and improving interpretability — a vital requirement in machine learning applications that demand transparency in model decisions. Through comprehensive testing, FADP is shown to balance the trade-offs among accuracy, privacy, and interpretability, marking a substantial advancement in the field of privacy-preserving machine learning.

Farhad Riya, Farhin [University of Tennessee, Knox↗

Particle Markov Chain Monte Carlo Approach to Inference in Transient Surface Kinetics

Here, in this work, we develop a novel Bayesian approach to study the adsorption and desorption of CO onto a Pd(111) surface, a process of great importance in natural sciences. The motivation for this work comes from the recent availability of time-resolved infrared spectroscopy data and the need for model interpretability and uncertainty quantification in chemical processes. The objective is to learn the relevant parameters that characterize the process: coverage with time, rate constants, activation energies, and pre-exponential factors. Our approach consists of three main schemes: (i) a problem design and probabilistic model for the whole system, (ii) a particle Markov chain Monte Carlo sampler to learn the hidden coverages and rate constant parameters, and (iii) two Bayesian formulations to infer the activation energies and pre-exponential factors. The flexibility of the Bayesian framework allows for uncertainty quantification where possible and integration of mathematical constraints in the model to reflect the system physically. We found that our results for the activation energies and pre-exponential factor are in agreement with those reported in the experimental literature, independently, and we provide discussions on the advantages and disadvantages as well as applicability to other systems.

36 MATERIALS SCIENCE↗