Search NASA⌕ Search

SEARCH · Search NASA

Results for “Explainable machine learning”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 343 records · Page 19

Big Data Synchrophasor Monitoring and Analytics for Resiliency Tracking (BDSMART)

This report contains key findings from a project titled Big Data Synchrophasor Monitoring and Analytics for Resiliency Tracking (BDSMART), which was carried out through a collaborative effort of a team of researchers from Texas A&M Engineering Experiment Station, Temple University, and Quanta Technology, LLC. The in-kind support came from OSIsoft (acquired by AVEVA), which provided their PI Historian software to demonstrate the use case of streaming PMU data. The first section of the report describes the project goals and objectives related to the development of Machine Learning (ML) models capable of detecting and classifying events by processing phasor measurements captured in the field by Phasor Measurement Units (PMUs). The data for this study was contributed by the utilities/ISOs from the Western and Eastern interconnects and ERCOT, further referred to as Interconnect B (IC B), Interconnect A (IC A), and Interconnect C (IC C), respectively. The approach that the BDSMART Research Team proposed and the key research tasks defined by the team are outlined in this section. The next section describes the technical approach. We first discuss the data constraints related to the PMU measurements and data interpretation constraints imposed by the data contributors. They provided neither the topological information of the grid nor PMU placement locations and captured recorded data at very few locations in the system with the reporting rate of either 30 or 60 fps. The recordings are mostly positive sequence voltage, frequency, and ROCOF, and in some limited cases, three-phase voltages and currents. We then reflect on the bad data issues that stem from poor recording practices and vague definitions of the PMU status bits to supposedly be used for bad data identification. Finally, the data discovery points to imprecise time stamps with incomplete event start/end time, as well as inconsistent and incomplete event labeling, which combined make the implementation of the data models using supervising learning quite challenging. Following the data discovery study, we hypothesize that because the IC B data has the most complete labels, we should focus our model development on that data and then test it on data from other interconnects. We also define the common metrics used to evaluate the results from the ML algorithm tests. We concluded this section by summarizing the common ML models we used and explaining how we implemented and tested them. The issues from this section are expanded in the Training Dataset Report from this project. The final section of this report deals with the accomplishments and conclusions. As the accomplishments, we formulate the problem we are solving and what is achieved by solving the problem. We then reflect on each of the analytics tools we developed and point out the performance of each tool when applied to solving the mentioned problems. We reference this work for further details to the papers we published on each tool. In the conclusions, we give recommendations on how to improve future PMU recording practices to facilitate the ML algorithm implementation and guidance for the future standardization work aimed at clarifying the ambiguities associated with the PMU status bits. We finally list future tasks that can bring about further improvements in the proposed algorithms. The issues from this section are expanded in the Training, and Test Dataset Report filed at the project completion date.

97 MATHEMATICS AND COMPUTING↗

On the Need to Align Intent and Implementation in Uncertainty Quantification for Machine Learning

Quantifying uncertainties for machine learning (ML) models is a foundational challenge in modern data analysis. This challenge is compounded by at least two key aspects of the field: (a) inconsistent terminology surrounding uncertainty and estimation across disciplines, and (b) the varying technical requirements for establishing trustworthy uncertainties in diverse problem contexts. In this position paper, we aim to clarify the depth of these challenges by identifying these inconsistencies and articulating how different contexts impose distinct epistemic demands. We examine the current landscape of estimation targets (e.g., prediction, inference, simulation-based inference), uncertainty constructs (e.g., frequentist, Bayesian, fiducial), and the approaches used to map between them. Drawing on the literature, we highlight and explain examples of problematic mappings. To help address these issues, we advocate for standards that promote alignment between the \textit{intent} and \textit{implementation} of uncertainty quantification (UQ) approaches. We discuss several axes of trustworthiness that are necessary (if not sufficient) for reliable UQ in ML models, and show how these axes can inform the design and evaluation of uncertainty-aware ML systems. Our practical recommendations focus on scientific ML, offering illustrative cases and use scenarios, particularly in the context of simulation-based inference (SBI).

Trivedi, Shubhendu [MIT] (ORCID:0000000312374301)↗

The evolution of precipitate crystal structures in an Al-Mg-Si(-Cu) alloy studied by a combined HAADF-STEM and SPED approach

Highlights: • Scanning electron diffraction and machine learning complement lattice imaging • Developed approach gives increased objectivity in precipitate phase analysis • Precipitate phase fractions and solute content obtained with superior statistics • 0.01 at.% Cu changes the Al-Mg-Si system precipitation during overageing • Q’-phase eventually dominates precipitate structures and shows varying Cu-occupancy - Abstract: This work presents a detailed investigation into the effect of a low Cu addition (0.01 at.%) on precipitation in an Al-0.80Mg-0.85Si alloy during ageing. The precipitate crystal structures were assessed by scanning transmission electron microscopy combined with a novel scanning precession electron diffraction approach, which includes machine learning. The combination of techniques enabled evaluation of the atomic arrangement within individual precipitates, as well as an improved estimate of precipitate phase fractions at each ageing condition, through analysis of a statistically significant number of precipitates. Based on the obtained results, the total amount of solute atoms locked inside precipitates could be approximated. It was shown that even with a Cu content close to impurity levels, the Al-Mg-Si system precipitation was significantly affected with overageing. The principal change was due to a gradually increasing phase fraction of the Cu-containing Q′-phase, which eventually was seen to dominate the precipitate structures. The structural overtake could be explained based on a continuous formation of the thermally stable Q′-phase, with Cu atomic columns incorporating less Cu than what could potentially be accommodated.

36 MATERIALS SCIENCE↗

Integrating Deep Learning and Hydrodynamic Modeling to Improve the Great Lakes Forecast

The Laurentian Great Lakes, one of the world’s largest surface freshwater systems, pose a modeling challenge in seasonal forecast and climate projection. While physics-based hydrodynamic modeling is a fundamental approach, improving the forecast accuracy remains critical. In recent years, machine learning (ML) has quickly emerged in geoscience applications, but its application to the Great Lakes hydrodynamic prediction is still in its early stages. This work is the first one to explore a deep learning approach to predicting spatiotemporal distributions of the lake surface temperature (LST) in the Great Lakes. Our study shows that the Long Short-Term Memory (LSTM) neural network, trained with the limited data from hypothetical monitoring networks, can provide consistent and robust performance. The LSTM prediction captured the LST spatiotemporal variabilities across the five Great Lakes well, suggesting an effective and efficient way for monitoring network design in assisting the ML-based forecast. Furthermore, we employed an explainable artificial intelligence (XAI) technique named SHapley Additive exPlanations (SHAP) to uncover how the features impact the LSTM prediction. Our XAI analysis shows air temperature is the most influential feature for predicting LST in the trained LSTM. The relatively large bias in the LSTM prediction during the spring and fall was associated with substantial heterogeneity of air temperature during the two seasons. In contrast, the physics-based hydrodynamic model performed better in spring and fall yet exhibited relatively large biases during the summer stratification period. Finally, we developed a statistical integration of the hydrodynamic modeling and deep learning results based on the Best Linear Unbiased Estimator (BLUE). The integration further enhanced prediction accuracy, suggesting its potential for next-generation Great Lakes forecast systems.

Xue, Pengfei (ORCID:000000025702421X)↗

Deep learning-driven insights into super protein complexes for outer membrane protein biogenesis in bacteria

To reach their final destinations, outer membrane proteins (OMPs) of gram-negative bacteria undertake an eventful journey beginning in the cytosol. Multiple molecular machines, chaperones, proteases, and other enzymes facilitate the translocation and assembly of OMPs. These helpers usually associate, often transiently, forming large protein assemblies. They are not well understood due to experimental challenges in capturing and characterizing protein-protein interactions (PPIs), especially transient ones. Using AF2Complex, we introduce a high-throughput, deep learning pipeline to identify PPIs within the Escherichia coli cell envelope and apply it to several proteins from an OMP biogenesis pathway. Among the top confident hits obtained from screening ~1500 envelope proteins, we find not only expected interactions but also unexpected ones with profound implications. Subsequently, we predict atomic structures for these protein complexes. These structures, typically of high confidence, explain experimental observations and lead to mechanistic hypotheses for how a chaperone assists a nascent, precursor OMP emerging from a translocon, how another chaperone prevents it from aggregating and docks to a β-barrel assembly port, and how a protease performs quality control. This work presents a general strategy for investigating biological pathways by using structural insights gained from deep learning-based predictions.

60 APPLIED LIFE SCIENCES↗

Characterization of Extreme Hydroclimate Events in Earth System Models using ML/AI

Focal Area(s): (1) We put forward the concepts of data assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization and unsupervised learning applied to downscale information within Earth System Models (ESMs). (2) We discuss predictive modeling through the use of AI techniques and other tools to design a prediction system comprising of a hierarchy of models (e.g., AI-driven model/component/parameterization selection) to improve the characterization of extreme hydroclimate events in ESMs. The AI-based models will run five-six order of magnitude times faster, yet will provide similar accuracy, allowing us to provide range bounds on uncertainty faster and thus enabling faster extreme event identification. (3) Further, we consider the insight gleaned from complex data (observed/simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI to improve the characterization of extreme hydroclimate events in ESMs.

54 ENVIRONMENTAL SCIENCES↗

Intrusive analysis for NEK5000: development of intrusive uncertainty quantification for high-dimensional, high-fidelity codes.

We study the use of lower-fidelity training data for uncertainty quantification of complex simulation models. In our approach, computationally expensive full-model outputs are approximated applying proper orthogonal-decomposition-based dimensionality reduction to the full model. A Gaussian-processes-based machine learning approach is then used to model the difference (or, rather, the correspondence) between the higher-fidelity and the lower-fidelity data. This stochastic model can be constructed by using few additional full code evaluations; it is used to calibrate arbitrarily many lower-quality outputs in order to create additional training data for sampling-based analysis. In effect, an adequate approximation of the simulation model’s response to uncertainty can be constructed at a modest computational cost. In fact, we aim at the number of full-model evaluations comparable to that required for a linear approximation (as opposed to numbers traditionally associated with sampling in high-dimensional spaces). In this report, we explain the basic algorithm, suggest some performance tuning options, and give a first characterization for the class of simulation models of nuclear engineering for which the suggested report is effective.

Roderick, O.↗

Adoption of image-driven machine learning for microstructure characterization and materials design: A Perspective

Microstructure characterization enables the development of structure-processing-property relationships critical to several research areas within the broad field of materials science, from alloy design to the assessment of corrosion resistance, and failure analysis. Conventional approaches to material characterization have relied on either qualitative inference by the human ex-pert or software applications that can extract high-level features from images, such as boundary segmentation, average grain diameter, etc. Such approaches rely heavily on subject matter expert user intervention and knowledge of what phases or more generally, what microstructural features, are of interest. The recent surge in the adoption of machine learning techniques to address problems in materials engineering has brought with it an increased interest and application of Image Driven Machine Learning (IDML) approaches. In this work, we review the applications of IDML to the field of materials characterization. A canonical hierarchy of stages is defined, which when put sequentially together completes an IDML study: problem definition, dataset building, model selection and training, model evaluation, and integration with existing instrumentation or simulation workflow. The studies reviewed in this work are analyzed from the perspective of each of these stages. Such a review permits agranular assessment of the field, for example the impact of IDML on materials characterization at the nanoscale, the size of a typical dataset required to train a semantic segmentation model on electron microscopy images, ubiquitousness of transfer learning in the domain, etc. Finally, we discuss the importance of interpretability and explainability in the field of IDML for materials characterization, and provide an overview of two emerging techniques in the field: semantic segmentation and generative adversarial networks.

Baskaran, Arun↗

Towards verifiable cancer digital twins: tissue level modeling protocol for precision medicine

Cancer exhibits substantial heterogeneity, manifesting as distinct morphological and molecular variations across tumors, which frequently undermines the efficacy of conventional oncological treatments. Developments in multiomics and sequencing technologies have paved the way for unraveling this heterogeneity. Nevertheless, the complexity of the data gathered from these methods cannot be fully interpreted through multimodal data analysis alone. Mathematical modeling plays a crucial role in delineating the underlying mechanisms to explain sources of heterogeneity using patient-specific data. Intra-tumoral diversity necessitates the development of precision oncology therapies utilizing multiphysics, multiscale mathematical models for cancer. This review discusses recent advancements in computational methodologies for precision oncology, highlighting the potential of cancer digital twins to enhance patient-specific decision-making in clinical settings. We review computational efforts in building patient-informed cellular and tissue-level models for cancer and propose a computational framework that utilizes agent-based modeling as an effective conduit to integrate cancer systems models that encode signaling at the cellular scale with digital twin models that predict tissue-level response in a tumor microenvironment customized to patient information. Furthermore, we discuss machine learning approaches to building surrogates for these complex mathematical models. These surrogates can potentially be used to conduct sensitivity analysis, verification, validation, and uncertainty quantification, which is especially important for tumor studies due to their dynamic nature.

60 APPLIED LIFE SCIENCES↗

Conductivity Spectroscopy for Investigation and Discovery of Photovoltaic Materials

Conductivity spectroscopy is an extremely powerful set of methods for probing the properties of optoelectronic materials, especially photovoltaics, where photoconductivity is one of the best spectroscopic proxies for performance. Despite this power, they are substantially less commonly used than time-resolved photoluminescence (for instance) because they tend to be more expensive to implement (THz) and/or require specialized knowledge (GHz) to construct instruments, which are not widely available. The goal of this review is to illustrate the utility of these experiments in the discovery and study of photovoltaic absorber materials and simultaneously make them more accessible to the community by providing a central tutorial resource. We provide a comprehensive review of how conductivity spectroscopy has developed over the past decade and been applied in the discovery and development of photovoltaic materials, with a primary focus on emerging solution-processable technologies. Along the way we aim to demystify conductivity spectroscopy with focused tutorial sections that explain the physical models used to fit the data and illustrate how to think about “high-frequency conductivity”.

14 SOLAR ENERGY↗

An Efficient Bayesian Approach to Learning Droplet Collision Kernels: Proof of Concept Using “Cloudy,” a New n -Moment Bulk Microphysics Scheme

The small-scale microphysical processes governing the formation of precipitation particles cannot be resolved explicitly by cloud resolving and climate models. Instead, they are represented by microphysics schemes that are based on a combination of theoretical knowledge, statistical assumptions, and fitting to data (“tuning”). Historically, tuning was done in an ad hoc fashion, leading to parameter choices that are not explainable or repeatable. Recent work has treated it as an inverse problem that can be solved by Bayesian inference. The posterior distribution of the parameters given the data—the solution of Bayesian inference—is found through computationally expensive sampling methods, which require over $\mathcal{O}$(10 5 ) evaluations of the forward model; this is prohibitive for many models. We present a proof of concept of Bayesian learning applied to a new bulk microphysics scheme named “Cloudy,” using the recently developed Calibrate-Emulate-Sample (CES) algorithm. Cloudy models collision-coalescence and collisional breakup of cloud droplets with an adjustable number of prognostic moments and with easily modifiable assumptions for the cloud droplet mass distribution and the collision kernel. The CES algorithm uses machine learning tools to accelerate Bayesian inference by reducing the number of forward evaluations needed to $\mathcal{O}$(10 2 ). It also exhibits a smoothing effect when forward evaluations are polluted by noise. In a suite of perfect-model experiments, we show that CES enables computationally efficient Bayesian inference of parameters in Cloudy from noisy observations of moments of the droplet mass distribution. In an additional imperfect-model experiment, a collision kernel parameter is successfully learned from output generated by a Lagrangian particle-based microphysics model.

54 ENVIRONMENTAL SCIENCES↗

Uncertainty Quantification for Data-Driven Machine Learning Models in Nuclear Engineering Applications: Where We Are and What Do We Need?

Machine learning (ML) has been leveraged to tackle a diverse range of tasks in almost all branches of nuclear engineering. Many of the successes in ML applications can be attributed to the recent performance breakthroughs in deep learning, the growing availability of computational power, data, and easy-to-use ML libraries. However, these empirical successes have often outpaced our formal understanding of the ML algorithms. An important but under-rated area is uncertainty quantification (UQ) of ML. ML-based models are subject to approximation uncertainty when they are used to make predictions, due to sources including but not limited to, data noise, data coverage, extrapolation, imperfect model architecture and the stochastic training process. The goal of this paper is to clearly explain and illustrate the importance of UQ of ML. We will elucidate the differences in the basic concepts of UQ of physics-based models and data-driven ML models. Various sources of uncertainties in physical modeling and data-driven modeling will be discussed, demonstrated, and compared. We will also present and demonstrate a few techniques to quantify the ML prediction uncertainties, including Monte Carlo dropout, deep ensemble, Bayesian neural networks, Gaussian Processes and conformal prediction. Lastly, we will discuss the need for building a verification, validation and UQ framework to establish ML credibility.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Generative Models for Crystalline Materials

Understanding structure-property relationships in materials is fundamental in condensed matter physics and materials science. Over the past few years, machine learning (ML) has emerged as a powerful tool for advancing this understanding and accelerating materials discovery. Early ML approaches primarily focused on constructing and screening large material spaces to identify promising candidates for various applications. More recently, research efforts have increasingly shifted toward generating crystal structures using end-to-end generative models. This review analyzes the current state of generative modeling for crystal structure prediction and de novo generation. It examines crystal representations, outlines the generative models used to design crystal structures, and evaluates their respective strengths and limitations. Furthermore, the review highlights experimental considerations for evaluating generated structures and provides recommendations for suitable existing software tools. Emerging topics, such as modeling disorder and defects, integration in advanced characterization, incorporating synthetic feasibility constraints, and model explainability are explored. Ultimately, this work aims to inform both experimental scientists looking to adapt suitable ML models to their specific circumstances and ML specialists seeking to understand the unique challenges related to inverse materials design and discovery.

Metni, Houssam [Karlsruhe Inst. of Technology (KIT↗

Internal variability and forcing influence model–satellite differences in the rate of tropical tropospheric warming

Climate-model simulations exhibit approximately two times more tropical tropospheric warming than satellite observations since 1979. The causes of this difference are not fully understood and are poorly quantified. Here, we apply machine learning to relate the patterns of surface-temperature change to the forced and unforced components of tropical tropospheric warming. This approach allows us to disentangle the forced and unforced change in the model-simulated temperature of the midtroposphere (TMT). In applying the climate-model-trained machine-learning framework to observations, we estimate that external forcing has produced a tropical TMT trend of 0.25 ± 0.08 K⋅decade −1 between 1979 and 2014, but internal variability has offset this warming by 0.07 ± 0.07 K⋅decade −1 . Using the Community Earth System Model version 2 (CESM2) large ensemble, we also find that a discontinuity in the variability of prescribed biomass-burning aerosol emissions artificially enhances simulated tropical TMT change by 0.04 K⋅decade −1 . The magnitude of this aerosol-forcing bias will vary across climate models, but since the latest generation of climate models all use the same emissions dataset, the bias may systematically enhance climate-model trends over the satellite era. Our results indicate that internal variability and forcing uncertainties largely explain differences in satellite-versus-model warming and are important considerations when evaluating climate models.

54 ENVIRONMENTAL SCIENCES↗

Quantifying Uncertainties of Microscopic Nuclear Theories

The goal of this project was to produce modernized nuclear data libraries of light ion fusion reactions and actinide fission product yields. Such libraries are essential input in both stockpile science but also fundamental science (nucleosynthesis, superheavy nuclei). When such libraries are used to model complex processes involving exotic, very short-lived nuclei that have not been measured experimentally, the underlying nuclear data come from theoretical models, the predictive power of which needs to be quantified, together with its uncertainties. By teaming up nuclear theorists and statisticians, we built comprehensive theoretical frameworks based on the most fundamental theories of fusion and fission that we coupled for the first time with a Bayesian framework with machine learning techniques for uncertainty quantification. Our results include: (1) statistical emulators for neutron-alpha scattering, and the fission product mass yields of neutron-induced fission of 239 Pu, which will enable developers to generate random realizations of the corresponding data for uncertainty quantification (UQ) in applications; (2) the first complete and most accurate ab initio prediction of the S-factor for proton capture on 7 Be, which – combined with UQ – may become the recommended value at solar energies in the next review of solar fusion cross sections; (3) statistical emulators for fission mass yields in the neutron-induced fission of 239 Pu; (4) the first quantification of how uncertainties in nuclear forces impact fission properties that are relevant in nucleosynthesis simulations to explain how heavy elements are formed in the Cosmos.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Using automated machine learning for the upscaling of gross primary productivity

Estimating gross primary productivity (GPP) over space and time is fundamental for understanding the response of the terrestrial biosphere to climate change. Eddy covariance flux towers provide in situ estimates of GPP at the ecosystem scale, but their sparse geographical distribution limits larger-scale inference. Machine learning (ML) techniques have been used to address this problem by extrapolating local GPP measurements over space using satellite remote sensing data. However, the accuracy of the regression model can be affected by uncertainties introduced by model selection, parameterization, and choice of explanatory features, among others. Recent advances in automated ML (AutoML) provide a novel automated way to select and synthesize different ML models. In this work, we explore the potential of AutoML by training three major AutoML frameworks on eddy covariance measurements of GPP at 243 globally distributed sites. We compared their ability to predict GPP and its spatial and temporal variability based on different sets of remote sensing explanatory variables. Explanatory variables from only Moderate Resolution Imaging Spectroradiometer (MODIS) surface reflectance data and photosynthetically active radiation explained over 70 % of the monthly variability in GPP, while satellite-derived proxies for canopy structure, photosynthetic activity, environmental stressors, and meteorological variables from reanalysis (ERA5-Land) further improved the frameworks' predictive ability. We found that the AutoML framework Auto-sklearn consistently outperformed other AutoML frameworks as well as a classical random forest regressor in predicting GPP but with small performance differences, reaching an r 2 of up to 0.75. We deployed the best-performing framework to generate global wall-to-wall maps highlighting GPP patterns in good agreement with satellite-derived reference data. This research benchmarks the application of AutoML in GPP estimation and assesses its potential and limitations in quantifying global photosynthetic activity.

54 ENVIRONMENTAL SCIENCES↗

Observational benchmarks inform representation of soil organic carbon dynamics in land surface models

Abstract. Representing soil organic carbon (SOC) dynamics in Earth system models (ESMs) is a key source of uncertainty in predicting carbon–climate feedbacks. Machine learning models can help identify dominant environmental controllers and establish their functional relationships with SOC stocks. The resulting knowledge can be integrated into ESMs to reduce uncertainty and improve predictions of SOC dynamics over space and time. In this study, we used a large number of SOC field observations (n=54 000), geospatial datasets of environmental factors (n=46), and two machine learning approaches (namely random forest, RF, and generalized additive modeling, GAM) to (1) identify dominant environmental controllers of global and biome-specific SOC stocks, (2) derive functional relationships between environmental controllers and SOC stocks, and (3) compare the identified environmental controllers and predictive relationships with those in models used in Phase 6 of the Coupled Model Intercomparison Project (CMIP6). Our results showed that the diurnal temperature, drought index, cation exchange capacity, and precipitation were important observed environmental predictors of global SOC stocks. While the RF model identified 14 environmental factors that describe climatic, vegetation, and edaphic conditions as important predictors of global SOC stocks (R2=0.61, RMSE = 0.46 kg m−2), current ESMs oversimplify the relationships between environmental factors and SOC, with precipitation, temperature, and net primary productivity explaining > 96 % of the variability in ESM-modeled SOC stocks. Further, our study revealed notable disparities among the functional relationships between environmental factors and SOC stocks simulated by ESMs compared with observed relationships. To improve SOC representations in ESMs, it is imperative to incorporate additional environmental controls, such as the cation exchange capacity, and refine the functional relationships to align more closely with observations.

54 ENVIRONMENTAL SCIENCES↗