Search NASA⌕ Search

SEARCH · Search NASA

Results for “grad-cam”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

12 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2023

Identifying common stored product insects using automated deep learning methods

Monitoring stored product insect pests is a common practice for post-harvest management of stored grain and grain-based commodities, which helps ensure product quality from harvest to final consumer. Current methods of sampling and monitoring can be time-consuming, labor-intensive, expensive and require expertise in insect identification. Therefore, this study aims to develop an image-based automated identification system for common stored product insect species using deep-learning methods. Top-down images of the common stored product adult insect species of Rhyzopertha dominica, Cryptolestes ferrugineus, Tribolium castaneum, Sitophilus oryzae, and Oryzaephilus surinamensis were acquired and analyzed. Deep learning-based, state-of-the-art Convolutional Neural Networks (CNN) models (ResNet-50, MobileNet-v2, DarkNet-53, and EfficientNet-b0) were fine-tuned with a transfer learning approach to classify the insect species. All models were able to correctly identify the insect species with at least 96% accuracy and with few misclassifications. One issue with trained CNNs is that they do not explain the reasoning for the classification and are often called a “black box”. Therefore, visualization methods called Gradient-weighted Class Activation Mapping (Grad-CAM) were implemented to explore the black box network. The Grad-CAM uses heat maps to highlight the major image features that the network focused on to make insect species predictions. The Grad-CAM verifies the network's prediction and also helps improve network performance. This study contributes to the overall goal of developing a camera-based system for monitoring stored grain insects. As a result, the developed system would empower warehouse, flour mills, and other food facilities with a tool to quickly and accurately identify insect species in stored product environments and could be implemented as part of a close to real-time monitoring system.

60 APPLIED LIFE SCIENCES↗
NASA NTRS2021

Predicting the Functional State of Protein Kinases Using Interpretable Graph Neural Networks

Kinases are a family of proteins that function as molecular switches, regulating several essential cellular activities such as cell proliferation. Dysfunctional kinases are implicated in several types of cancers and hence they are actively pursued as drug targets. Given the vast number of complex kinase structures that are available in the protein data bank (PDB), there is a necessity to develop methodologies that can identify structurally important moieties of the kinases in an automated fashion, for such techniques can be instrumental in identifying novel drug targets. In this work, we develop a graph neural network (GNN) based deep learning framework for classifying the functionally active and inactive states of a large set of eukaryotic protein kinases, making use of their 3D structure from the PDB. We show that GNN based machine learning models can classify protein states with an accuracy greater than 97%. We further use the GNN models to automatically identify regions of the kinases that are important for its function. For this purpose, Gradient-weighted Class Activation Mapping (Grad-CAM) was implemented on the protein graphs. Remarkably, Grad-CAM consistently identifies the highly conserved DFG motif as the most important part of the protein across the entire kinome, without any prior input. Other regions of the hydrophobic core such as the HRD motif were also identified by the interpretable GNN framework, consistent with the literature. We discuss the significance of each of these regions in detail.

Ashwin Ravichandran↗
DOE OSTI2026

Using Explainable Artificial Intelligence to Predict Perovskite Solar Cell Electrical Metastability from Operando Photoluminescence Images in Accelerated Stress Testing

Metal halide perovskite (MHP) solar cells exhibit a metastable response to bias governed by coupled ionic–electronic processes, complicating the conventional reciprocity relation between luminescence intensity and device open-circuit voltage (V oc ). This limits the use of luminescence as a diagnostic for device screening or accelerated stress testing, motivating new approaches that can interpret photoluminescence (PL) signals under nonequilibrium conditions. From the artificial intelligence perspective, we develop an explainable deep learning framework that integrates convolutional neural networks (CNN), long short-term memory (LSTM) layers, and an attention mechanism to learn spatiotemporal features from operando photoluminescence PL image sequences. The model achieves a mean absolute error of ±0.027 V in predicting open-circuit voltage transients and reduces extreme-tail errors by up to 78% compared to physics-based reciprocity calculations. Gradient-weighted Class Activation Mapping (Grad-CAM) provides interpretability by highlighting physically meaningful regions such as electrode edges and emergent defect features. From the engineering application perspective, this framework enables accurate, contactless prediction of device V oc and identification of degradation-relevant features during accelerated aging of perovskite solar cells. This approach demonstrates how explainable AI can enhance operando diagnostics and reliability analysis in photovoltaic devices under nonequilibrium conditions.

14 SOLAR ENERGY↗
DOE OSTI2020

DeepMerge: Classifying high-redshift merging galaxies with deep neural networks

In this work, we investigate and demonstrate the use of convolutional neural networks (CNNs) for the task of distinguishing between merging and non-merging galaxies in simulated images, and for the first time at high redshifts (i.e. $z=2$). We extract images of merging and non-merging galaxies from the Illustris-1 cosmological simulation and apply observational and experimental noise that mimics that from the Hubble Space Telescope; the data without noise form a "pristine" data set and that with noise form a "noisy" data set. The test set classification accuracy of the CNN is $79\%$ for pristine and $76\%$ for noisy. The CNN outperforms a Random Forest classifier, which was shown to be superior to conventional one- or two-dimensional statistical methods (Concentration, Asymmetry, the Gini, $M_{20}$ statistics etc.), which are commonly used when classifying merging galaxies. We also investigate the selection effects of the classifier with respect to merger state and star formation rate, finding no bias. Finally, we extract Grad-CAMs (Gradient-weighted Class Activation Mapping) from the results to further assess and interrogate the fidelity of the classification model.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2022

Combining Deep Learning With Physics Based Features in Explosion-Earthquake Discrimination

This paper combines the power of deep-learning with the generalizability of physics-based features, to present an advanced method for seismic discrimination between earthquakes and explosions. The proposed method contains two branches: a deep learning branch operating directly on seismic waveforms or spectrograms, and a second branch operating on physics-based parametric features. These features are high-frequency P/S amplitude ratios and the difference between local magnitude (ML) and coda duration magnitude (MC). The combination achieves better generalization performance when applied to new regions than models that are developed solely with deep learning. Further, we also examined which parts of the waveform data dominate deep learning decisions (i.e., via Grad-CAM). Such visualization provides a window into the black-box nature of the machine-learning models and offers new insight into how the deep learning derived models use data to make decisions.

58 GEOSCIENCES↗
DOE OSTI2022

Detecting damaged buildings using real-time crowdsourced images and transfer learning

After significant earthquakes, we can see images posted on social media platforms by individuals and media agencies owing to the mass usage of smartphones these days. These images can be utilized to provide information about the shaking damage in the earthquake region both to the public and research community, and potentially to guide rescue work. This paper presents an automated way to extract the damaged buildings images after earthquakes from social media platforms such as Twitter and thus identify the particular user posts containing such images. Using transfer learning and ~ 6500 manually labelled images, we trained a deep learning model to recognize images with damaged buildings in the scene. The trained model achieved good performance when tested on newly acquired images of earthquakes at different locations and when ran in near real-time on Twitter feed after the 2020 M7.0 earthquake in Turkey. Furthermore, to better understand how the model makes decisions, we also implemented the Grad-CAM method to visualize the important regions on the images that facilitate the decision.

58 GEOSCIENCES↗
DOE OSTI2023

Gamma-ray Spectrum Explanations (GRSE) v0.1

This code repository provides tools for producing various types of machine learning model explanations trained on gamma-ray spectra. It includes implementations of saliency mapping, Grad-CAM, LIME, and Kernel SHAP, including improvements and clarifications to some of the approaches that are specific to their use with gamma-ray spectral data. These tools are in support of the manuscript M.S. Bandstra et al., "Explaining machine-learning models for gamma-ray detection and identification," under review at PLOS ONE.

Bandstra, Mark↗
DOE OSTI2023

Explaining machine-learning models for gamma-ray detection and identification

As more complex predictive models are used for gamma-ray spectral analysis, methods are needed to probe and understand their predictions and behavior. Recent work has begun to bring the latest techniques from the field of Explainable Artificial Intelligence (XAI) into the applications of gamma-ray spectroscopy, including the introduction of gradient-based methods like saliency mapping and Gradient-weighted Class Activation Mapping (Grad-CAM), and black box methods like Local Interpretable Model-agnostic Explanations (LIME) and SHapley Additive exPlanations (SHAP). In addition, new sources of synthetic radiological data are becoming available, and these new data sets present opportunities to train models using more data than ever before. In this work, we use a neural network model trained on synthetic NaI(Tl) urban search data to compare some of these explanation methods and identify modifications that need to be applied to adapt the methods to gamma-ray spectral data. We find that the black box methods LIME and SHAP are especially accurate in their results, and recommend SHAP since it requires little hyperparameter tuning. We also propose and demonstrate a technique for generating counterfactual explanations using orthogonal projections of LIME and SHAP explanations.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2026

Phase Picking Beyond Local Distances: Where Waveform Filtering Still Matters for Deep Learning Models

Waveform filtering is a standard step in traditional seismic phase picking but often receives little attention in deep learning workflows, where models are typically trained on raw or minimally processed waveforms. Although this strategy performs well for local events, we show that performance can degrade substantially at regional distances. To address this limitation, we introduce two ways to incorporate multiband-filtered waveforms into deep learning phase pickers. The stacking approach concatenates filtered inputs along the channel dimension, while the branching approach processes each frequency band through a dedicated network branch before feature fusion. Both approaches can substantially improve performance across epicentral distances of 0° to 20°, but their effectiveness depends strongly on the selected frequency bands. Tests with multiple filter banks show that filter-bank design should be treated as part of model optimization rather than as a fixed preprocessing choice. Grad-CAM analysis of the branching model indicates that band importance varies among waveform samples and across training realizations, with only a weak overall preference for the 0.25 to 0.5 Hz band. These results show that no single filter band is consistently optimal and demonstrate that explicit feature engineering remains valuable for robust deep learning-based seismic phase picking.

58 GEOSCIENCES↗
DOE OSTI2025

Toward Complete Merger Identification at Cosmic Noon with Deep Learning

As we enter the era of large imaging surveys such as Roman, Rubin, and Euclid, a deeper understanding of potential biases and selection effects in optical astronomical catalogs created with the use of ML-based methods is paramount. This work focuses on a deeper understanding of the performance and limitations of deep learning-based classifiers as tools for galaxy merger identification. We train a ConvNeXT-Pico model on mock HST CANDELS images from the IllustrisTNG50 simulation. Our focus is on a more challenging classification of galaxy mergers and non-mergers at higher redshifts 1 < z < 1.5, including minor mergers and lower mass galaxies down to the stellar mass of 108M⊙. We demonstrate, for the first time, that a deep learning model, such as the one developed in this work, can successfully identify even minor and low mass mergers even at these redshifts. Our model achieves overall accuracy, purity, and completeness of over 73%. We show that some galaxy mergers can only be identified from certain observation angles, leading to a potential upper limit in overall accuracy. Using Grad-CAMs and UMAPs, we more deeply examine the performance and observe a visible gradient in the latent space with stellar mass and specific star formation rate, but no visible gradient with merger mass ratio or merger stage.

Schechter, Aimee L. [U. Colorado, Boulder]↗
DOE OSTI2023

An extended focused assessment with sonography in trauma ultrasound tissue-mimicking phantom for developing automated diagnostic technologies

Medical imaging-based triage is critical for ensuring medical treatment is timely and prioritized. However, without proper image collection and interpretation, triage decisions can be hard to make. While automation approaches can enhance these triage applications, tissue phantoms must be developed to train and mature these novel technologies. Here, we have developed a tissue phantom modeling the ultrasound views imaged during the enhanced focused assessment with sonography in trauma exam (eFAST). The tissue phantom utilized synthetic clear ballistic gel with carveouts in the abdomen and rib cage corresponding to the various eFAST scan points. Various approaches were taken to simulate proper physiology without injuries present or to mimic pneumothorax, hemothorax, or abdominal hemorrhage at multiple locations in the torso. Multiple ultrasound imaging systems were used to acquire ultrasound scans with or without injury present and were used to train deep learning image classification predictive models. Performance of the artificial intelligent (AI) models trained in this study achieved over 97% accuracy for each eFAST scan site. We used a previously trained AI model for pneumothorax which achieved 74% accuracy in blind predictions for images collected with the novel eFAST tissue phantom. Grad-CAM heat map overlays for the predictions identified that the AI models were tracking the area of interest for each scan point in the tissue phantom. Overall, the eFAST tissue phantom ultrasound scans resembled human images and were successful in training AI models. Tissue phantoms are critical first steps in troubleshooting and developing medical imaging automation technologies for this application that can accelerate the widespread use of ultrasound imaging for emergency triage.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2025

Automated Fire Detection for Industrial Settings with Pretrained Convolutional Networks

Early fire detection in industrial environments is critical to preventing equipment damage, personal injury, and operational disruptions. Traditional smoke detectors, while effective, often experience delays due to the time required for smoke to reach sensors, allowing fires to spread. Manual fire watch operations and human surveillance of camera feeds are resource-intensive and prone to human error. To address these challenges, this paper explores the application of convolutional neural networks for automated fire detection, specifically in industrial settings. By leveraging 11 different pre-trained machine vision models from TensorFlow and enhancing them with transfer learning on a custom-built industrial fire dataset, we optimized fire detection performance. Here, we analyzed each machine vision model architecture in terms of its depth, width, and input image resolution, considering both resource requirements and detection accuracy. We further explored the option of combining multiple models into an ensemble classifier to evaluate whether the performance improvements could justify the much greater computational complexity and other practical impacts. A cost-benefit analysis is presented to evaluate the trade-offs between performance and computational expense. Our findings identify that EfficientNetV2L, specifically tailored for industrial applications, provides the optimal balance between costs involved in training and using the model versus the overall fire detection performance. Additionally, we present a qualitative analysis of model performance using the technique of gradient-based class activation mapping to provide explainability by visualizing model decisions.

artificial intelligence↗