Saliency-driven system models for cell analysis with deep learning
Not Available
60 APPLIED LIFE SCIENCES↗
SEARCH · Search NASA
Results for “saliency prediction”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Structure–Property Linkage in Alloys Using Graph Neural Network and Explainable Artificial Intelligence
Deep learning tools have recently shown significant potential for accelerating the prediction of microstructure–property linkage in materials. While deep neural networks like convolution neural networks (CNNs) can extract physics information from 3D microstructure images, they often require a large network architecture and substantial training time. In this research, we trained a graph neural network (GNN) using phase field generated microstructures of Ni-Al alloys to predict the evolution of mechanical properties. We found that a single GNN is capable of accurately predicting the strengthening of Ni-Al alloys with microstructures of varying sizes and dimensions, which cannot otherwise be done with a CNN. Additionally, GNN requires significantly less GPU utilization than CNN and offers more interpretable explanation of predictions using saliency analysis as features are manually defined in the graph. We also utilize explainable artificial intelligence tool Bayesian Inference to determine the coefficients in the power law equation that governs coarsening of precipitates. Overall, our work demonstrates the ability of the GNN to accurately and efficiently extract relevant information from material microstructures without having restrictions on microstructure size or dimension and offers an interpretable explanation.
Chemistry↗
An information maximization model of eye movements
We propose a sequential information maximization model as a general strategy for programming eye movements. The model reconstructs high-resolution visual information from a sequence of fixations, taking into account the fall-off in resolution from the fovea to the periphery. From this framework we get a simple rule for predicting fixation sequences: after each fixation, fixate next at the location that minimizes uncertainty (maximizes information) about the stimulus. By comparing our model performance to human eye movement data and to predictions from a saliency and random model, we demonstrate that our model is best at predicting fixation locations. Modeling additional biological constraints will improve the prediction of fixation sequences. Our results suggest that information maximization is a useful principle for programming eye movements.
NASA Discipline Neuroscience↗
Explainable deep learning for insights in El Niño and river flows
The El Niño Southern Oscillation (ENSO) is a semi-periodic fluctuation in sea surface temperature (SST) over the tropical central and eastern Pacific Ocean that influences interannual variability in regional hydrology across the world through long-range dependence or teleconnections. Recent research has demonstrated the value of Deep Learning (DL) methods for improving ENSO prediction as well as Complex Networks (CN) for understanding teleconnections. However, gaps in predictive understanding of ENSO-driven river flows include the black box nature of DL, the use of simple ENSO indices to describe a complex phenomenon and translating DL-based ENSO predictions to river flow predictions. Here we show that eXplainable DL (XDL) methods, based on saliency maps, can extract interpretable predictive information contained in global SST and discover SST information regions and dependence structures relevant for river flows which, in tandem with climate network constructions, enable improved predictive understanding. Our results reveal additional information content in global SST beyond ENSO indices, develop understanding of how SSTs influence river flows, and generate improved river flow prediction, including uncertainty estimation. Observations, reanalysis data, and earth system model simulations are used to demonstrate the value of the XDL-CN based methods for future interannual and decadal scale climate projections.
54 ENVIRONMENTAL SCIENCES↗
Explainable deep learning for insights in El Niño and river flows
The El Niño Southern Oscillation (ENSO) is a semi-periodic fluctuation in sea surface temperature (SST) over the tropical central and eastern Pacific Ocean that influences interannual variability in regional hydrology across the world through long-range dependence or teleconnections. Recent research has demonstrated the value of Deep Learning (DL) methods for improving ENSO prediction as well as Complex Networks (CN) for understanding teleconnections. However, gaps in predictive understanding of ENSO-driven river flows include the black box nature of DL, the use of simple ENSO indices to describe a complex phenomenon and translating DL-based ENSO predictions to river flow predictions. Here we show that eXplainable DL (XDL) methods, based on saliency maps, can extract interpretable predictive information contained in global SST and discover SST information regions and dependence structures relevant for river flows which, in tandem with climate network constructions, enable improved predictive understanding. Our results reveal additional information content in global SST beyond ENSO indices, develop understanding of how SSTs influence river flows, and generate improved river flow prediction, including uncertainty estimation. Observations, reanalysis data, and earth system model simulations are used to demonstrate the value of the XDL-CN based methods for future interannual and decadal scale climate projections.
SST↗
Predicting Pilot Performance in Off-Nominal Conditions: A Meta-Analysis and Model Validation
Pilot response to off-nominal (very rare) events represents a critical component to understanding the safety of next generation airspace technology and procedures. We describe a meta-analysis designed to integrate the existing data regarding pilot accuracy of detecting rare, unexpected events such as runway incursions in realistic flight simulations. Thirty-five studies were identified and pilot responses were categorized by expectancy, event location, and whether the pilot was flying with a highway-in-the-sky display. All three dichotomies produced large, significant effects on event miss rate. A model of human attention and noticing, N-SEEV, was then used to predict event noticing performance as a function of event salience and expectancy, and retinal eccentricity. Eccentricity is predicted from steady state scanning by the SEEV model of attention allocation. The model was used to predict miss rates for the expectancy, location and highway-in-the-sky (HITS) effects identified in the meta-analysis. The correlation between model-predicted results and data from the meta-analysis was 0.72.
Wickens, C.D.↗
Salience Assignment for Multiple-Instance Data and Its Application to Crop Yield Prediction
An algorithm was developed to generate crop yield predictions from orbital remote sensing observations, by analyzing thousands of pixels per county and the associated historical crop yield data for those counties. The algorithm determines which pixels contain which crop. Since each known yield value is associated with thousands of individual pixels, this is a multiple instance learning problem. Because individual crop growth is related to the resulting yield, this relationship has been leveraged to identify pixels that are individually related to corn, wheat, cotton, and soybean yield. Those that have the strongest relationship to a given crop s yield values are most likely to contain fields with that crop. Remote sensing time series data (a new observation every 8 days) was examined for each pixel, which contains information for that pixel s growth curve, peak greenness, and other relevant features. An alternating-projection (AP) technique was used to first estimate the "salience" of each pixel, with respect to the given target (crop yield), and then those estimates were used to build a regression model that relates input data (remote sensing observations) to the target. This is achieved by constructing an exemplar for each crop in each county that is a weighted average of all the pixels within the county; the pixels are weighted according to the salience values. The new regression model estimate then informs the next estimate of the salience values. By iterating between these two steps, the algorithm converges to a stable estimate of both the salience of each pixel and the regression model. The salience values indicate which pixels are most relevant to each crop under consideration.
Wagstaff, Kiri L.↗
Visual Attention Allocation Between Robotic Arm and Environmental Process Control: Validating the STOM Task Switching Model
Fifty six participants time shared a spacecraft environmental control system task with a realistic space robotic arm control task in either a manual or highly automated version. The former could suffer minor failures, whose diagnosis and repair were supported by a decision aid. At the end of the experiment this decision aid unexpectedly failed. We measured visual attention allocation and switching between the two tasks, in each of the eight conditions formed by manual-automated arm X expected-unexpected failure X monitoring- failure management. We also used our multi-attribute task switching model, based on task attributes of priority interest, difficulty and salience that were self-rated by participants, to predict allocation. An un-weighted model based on attributes of difficulty, interest and salience accounted for 96 percent of the task allocation variance across the 8 different conditions. Task difficulty served as an attractor, with more difficult tasks increasing the tendency to stay on task.
STOM attention allocation model validation↗
Explainable tokamak-agnostic forecasting of fusion plasma instability via megahertz turbulent fluctuations
Scientific applications of artificial intelligence (AI) often remain limited by device-specific training and unexplained “black-box” approaches, creating fundamental barriers to cross-system generalization. This challenge is critical for nuclear fusion, where future reactors will have limited operational data for AI training. Here, we demonstrate that our neural network, trained solely on megahertz-scale turbulence measurements from one machine (DIII-D), forecasts Type-I edge localized mode (ELM) onsets in a different tokamak (KSTAR) through zero-shot weight transfer following physics-consistent preprocessing without device-specific retraining. Through an explainable AI framework combining gradient-weighted class activation mapping with physics validation, we reveal that our network can internalize physics relationships governing the ELM instabilities rather than memorizing device-specific patterns. The network perceives spatiotemporal features that correlate consistently with independently calculated instability growth rates, magnetohydrodynamic stability limits, and pedestal structure dynamics. Statistical analyses of dimensionally-reduced saliency features reveal the identical triangular features between the saliency representations, instability growth rates, and prediction probability across tokamaks, providing evidence that our forecasting system can show tokamak-agnostic generalization. This work contributes to a foundation for explainable scientific AI systems, where cross-system developments are essential for transcending traditional domain-specific constraints.
AI↗
Sensor selection and tool wear prediction with data‐driven models for precision machining
Abstract Estimation of tool wear in precision machining is vital in the traditional subtractive machining industry to reduce processing cost, improve manufacturing efficiency and product quality. In this vein, fusion of time and frequency‐domain features of commonly sensed signals can provide an early indication of tool wear and improve its prediction accuracy for prognostics and health management. This paper presents a data‐driven methodology and a complete tool chain for the inference of precision machining tool wear from fused machine measurements, such as cutting force, power, audio and vibration signals, and quantify the usefulness of each measurement. Indicators of tool wear are extracted from time‐domain signal statistics, frequency‐domain analysis, and time‐frequency domain analysis. Correlation coefficients between the extracted features (indicators) and the tool wear are used to select the most informative features. Principal Component Analysis and Partial Least‐Squares are used to reduce the dimensionality of the feature space. Regression models, including linear regression, support vector regression, Decision tree regression, neural network regression and Gaussian process regression, are used to predict the tool wear using data from a Haas milling machine performing spiral boss face milling. The performance of the regression models based on subsets of sensors validates the preliminary estimates about the saliency of the sensors. The experimental results show that the proposed methods can predict the machine tool wear precisely, with readily available sensor measurements. Neural network and Gaussian process regression were able to achieve good estimates of tool wear at different machine operating conditions. The most informative signal in predicting tool wear was shown to be the vibration signal. Time‐frequency domain features were the most informative features among the combination of features of three domains. In addition, using partial least squares components extracted from the original features of signals led to higher prediction accuracy.
Han, Seulki↗
AIF for Vis (Active Inference for simulating human interpretation of data visualization) [SWR-26-084]
AIF for Vis contains the Active Inference models and analysis scripts used to study a simple visualization-interpretation task: estimating the average value of two bars in a bar chart. The work is a proof of concept for translating hypothesized cognitive strategies into executable, inspectable process models. We implement two idealized strategies inspired by dual-process accounts of visualization-aided decision making: *Fast model: a compressed, heuristic strategy that estimates the visual midpoint of the two bars and maintains a single belief over their average. *Slow model: a sequential, analytic strategy that estimates the two bar heights separately and maintains them in working memory before computing an average. Both models use a common Active-Inference-inspired framework for sequential perception, belief updating, action selection, and reporting. Their different internal representations produce distinct predicted vulnerabilities: *the Fast model is more susceptible to tick-salience bias; *the Slow model is more susceptible to working-memory decay. The repository includes the model implementations, scripts used for the experiments reported in the paper, precomputed trial-level results, and plotting scripts.
Goldwyn, Harrison [National Laboratory of the Rock↗
Employing Eye Trackers to Reduce Nuisance Alarms
When process operators anticipate an alarm prior to its annunciation, that alarm loses information value and becomes a nuisance. This study investigated using eye trackers to measure and adjust the salience of alarms with three methods of gaze-based acknowledgement (GBA) of alarms that estimate operator anticipation. When these methods detected possible alarm anticipation, the alarm’s audio and visual salience was reduced. A total of 24 engineering students (male = 14, female = 10) aged between 18 and 45 were recruited to predict alarms and control a process parameter in three scenario types (parameter near threshold, trending, or fluctuating). The study evaluated whether behaviors of the monitored parameter affected how frequently the three GBA methods were utilized and whether reducing alarm salience improved control task performance. The results did not show significant task improvement with any GBA methods (F(3,69) = 1.357, p = 0.263, partial η 2 = 0.056). However, the scenario type affected which GBA method was more utilized (X 2 (2, N = 432) = 30.147, p < 0.001). Alarm prediction hits with gaze-based acknowledgements coincided more frequently than alarm prediction hits without gaze-based acknowledgements (X 2 (1, N = 432) = 23.802, p < 0.001, OR = 3.877, 95% CI 2.25–6.68, p < 0.05). Participant ratings indicated an overall preference for the three GBA methods over a standard alarm design (F(3,63) = 3.745, p = 0.015, partial η 2 = 0.151). This study provides empirical evidence for the potential of eye tracking in alarm management but highlights the need for additional research to increase validity for inferring alarm anticipation.
99 - GENERAL AND MISCELLANEOUS↗
NASA Tech Briefs, November 2010
Topics covered include: Portable Handheld Optical Window Inspection Device; Salience Assignment for Multiple-Instance Data and Its Application to Crop Yield Prediction; Speech Acquisition and Automatic Speech Recognition for Integrated Spacesuit Audio Systems ; Predicting Long-Range Traversability from Short-Range Stereo-Derived Geometry; Browser-Based Application for Telemetry Monitoring of Robotic Assets; Miniature Low-Noise G-Band I-Q Receiver; Methods of Using a Magnetic Field Response Sensor Within Closed, Electrically Conductive Containers; Differential Resonant Ring YIG Tuned Oscillator; Microfabricated Segmented-Involute-Foil Regenerator for Stirling Engines; Reducing Seal Adhesion in Low Impact Docking Systems; Optimal Flow Control Design; Corrosion-Resistant Container for Molten-Material Processing; Reusable Hot-Wire Cable Cutter; Deployment of a Curved Truss; High-Volume Airborne Fluids Handling Technologies to Fight Wildfires; Modeling of Alkane Oxidation Using Constituents and Species; Fabrication of Lanthanum Telluride 14-1-11 Zintl High-Temperature Thermoelectric Couple; A Computer Model for Analyzing Volatile Removal Assembly; Analysis of Nozzle Jet Plume Effects on Sonic Boom Signature; Optical Sidebands Multiplier; Single Spatial-Mode Room-Temperature-Operated 3.0 to 3.4 micrometer Diode Lasers; Self-Nulling Beam Combiner Using No External Phase Inverter; Portable Dew Point Mass Spectrometry System for Real-Time Gas and Moisture Analysis; Maximum Likelihood Time-of-Arrival Estimation of Optical Pulses via Photon-Counting Photodetectors; Handheld White Light Interferometer for Measuring Defect Depth in Windows; Decomposition Algorithm for Global Reachability on a Time-Varying Graph; Autonomous GN and C for Spacecraft Exploration of Comets and Asteroids; Efficient Web Services Policy Combination; Using CTX Image Features to Predict HiRISE-Equivalent Rock Density; Isolation of the Paenibacillus phoenicis, a Spore-Forming Bacterium; Monolithically Integrated, Mechanically Resilient Carbon-Based Probes for Scanning Probe Microscopy; Cell Radiation Experiment System; Process to Produce Iron Nanoparticle Lunar Dust Simulant Composite; Inversion Method for Early Detection of ARES-1 Case Breach Failure; Use of ILTV Control Laws for LaNCETS Flight Research;and Evaluating Descent and Ascent Trajectories Near Non-Spherical Bodies.
Source record↗
ZeoNet: 3D convolutional neural networks for predicting adsorption in nanoporous zeolites
Zeolites are one of the most widely used materials in the chemical industry due to their nanometer-sized pores that can adsorb and react upon molecules selectively. With hundreds of known framework topologies and hundreds of thousands of computationally predicted structures, the ability to rapidly predict zeolite performance allows researchers to prioritize their efforts on the most promising structures for a given application. Although the accuracy of forcefield-based atomistic simulations has advanced significantly in the past two decades, these simulations can be computationally expensive, especially for long-chain, complex molecules. Here, we present ZeoNet, a representation learning framework using convolutional neural networks (ConvNets) and 3D volumetric representations for predicting adsorption in zeolites. ZeoNet was trained on the task of predicting Henry's constants for adsorption, k H , of n-octadecane in more than 330 000 known and predicted zeolite materials. Employing a 3D grid based on the distances to solvent-accessible surfaces, a volumetric representation that can be generated efficiently, the best-performing ZeoNet achieved a correlation coefficient r 2 = 0.977 and a mean-squared error MSE = 3.8 in ln k H , which corresponds to an error of 9.3 kJ mol -1 in adsorption free energy. In comparison, a model based on hand-designed geometric features has values of r 2 = 0.783 and MSE = 35.7. ZeoNet is also relatively efficient and can process ≈8 structures per second on an Nvidia RTX 2080TI GPU, orders of magnitude faster than forcefield-based simulations. A systematic analysis was conducted to investigate how the choice of ConvNet architectures, the linear dimension (L) and spatial resolution (Δd) of the distance grids, batch size, optimizer, and learning rate impact the model performance. We found that ConvNets based on the ResNet architecture offer the best tradeoff between expressiveness and efficiency. The performance for all models reaches a plateau at L = 30–45 Å and depends less sensitively on grid resolution, with a small benefit around Δd = 0.30–0.45 Å. Finally, saliency maps were visualized to identify which regions of the materials contributed the most to model predictions. It was found, interestingly, that the predictions are driven primarily by the accessible pore volume rather than the region occupied by the framework atoms.
36 MATERIALS SCIENCE↗
Learning to Predict Crystal Plasticity at the Nanoscale: Deep Residual Networks and Size Effects in Uniaxial Compression Discrete Dislocation Simulations
The density and configurational changes of crystal dislocations during plastic deformation influence the mechanical properties of materials. These influences have become clearest in nanoscale experiments, in terms of strength, hardness and work hardening size effects in small volumes. The mechanical characterization of a model crystal may be cast as an inverse problem of deducing the defect population characteristics (density, correlations) in small volumes from the mechanical behavior. In this work, we demonstrate how a deep residual network can be used to deduce the dislocation characteristics of a sample of interest using only its surface strain profiles at small deformations, and then statistically predict the mechanical response of size-affected samples at larger deformations. As a testbed of our approach, we utilize high-throughput discrete dislocation simulations for systems of widths that range from nano- to micro- meters. We show that the proposed deep learning model significantly outperforms a traditional machine learning model, as well as accurately produces statistical predictions of the size effects in samples of various widths. By visualizing the filters in convolutional layers and saliency maps, we find that the proposed model is able to learn the significant features of sample strain profiles.
36 MATERIALS SCIENCE↗
Uncovering Structure–Conductivity Relationships in Anion Exchange Membranes (AEMs) Using Interpretable Machine Learning
Anion exchange membranes (AEMs) play a vital role in the performance of water electrolyzers and fuel cells, yet their discovery and optimization remain challenging due to the complexity of structure–property relationships. In this study, we introduce a machine learning framework that leverages conditional graph neural networks (cGNNs) and descriptor-based models and a hybrid graph neural network (HGARE) to predict and interpret ionic conductivity. The descriptor-based pipeline employs principal component analysis (PCA), ablation, and SHAP analysis to identify factors governing anion conductivity, revealing electronic, topological, and compositional descriptors as key contributors. Beyond prediction, dimensionality reduction and clustering are performed by employing t-SNE and KMeans as well as SOM, which reveal distinct membranes clusters, some of which were enriched with high anion conductivity. Among graph-based approaches, the graph convolutional (GCN) achieved strong predictive performance, while the Hybrid Graph Autoencoder-Regressor Ensemble (HGARE) achieved the highest accuracy. Additionally, atom-level saliency maps from GCN provide spatial explanations for conductive behavior, revealing the importance of polarizable and flexible regions. This work contributes to the accelerated and data-driven design of high-performance AEMs.
Naghshnejad, Pegah [Department of Chemical Enginee↗
Natural Language Processing to Inform Agent-Based Modeling: With Application to Modeling Adoption of Medium-Duty Electric Vehicles
Agent-based socio-technical modeling of medium- and heavy-duty (MDHD) electric vehicle (EV) adoption has the potential to provide analysis, prediction, and gui. This paper describes new applications of text analysis developed through machine learning (ML) to build and understand relevant topics and their saliency in the published discourse on adoption of MDHD EVs. This work contributes to the state of the art in topic mining models by defining a new metric of topic ranking (START) that quantifies the importance of predefined topics within the corpus using weighted results for predefined topics from two topic modeling approaches: Latent Dirichlet Allocation (LDA) and BERTopic. The START metric is then demonstrated in practice to model how academia and industry view the EV adoption process based on the respective texts published by these groups. Results show that academic literature places more emphasis on categories of interests such as norms/attitudes and adopter knowledge, while trade journals tend to emphasize long-term cost more than academia. The two bodies of literature agree on the importance of policy and incentives in MDHD EV adoption. Together these results illustrate the potential to use ML-based text analysis to populate the characteristics of agent-based socio-technical models.
Electric vehicle adoption, fleet electrification, ↗
Salience Assignment for Multiple-Instance Regression
We present a Multiple-Instance Learning (MIL) algorithm for determining the salience of each item in each bag with respect to the bag's real-valued label. We use an alternating-projections constrained optimization approach to simultaneously learn a regression model and estimate all salience values. We evaluate this algorithm on a significant real-world problem, crop yield modeling, and demonstrate that it provides more extensive, intuitive, and stable salience models than Primary-Instance Regression, which selects a single relevant item from each bag.
regression↗