Search NASA⌕ Search

SEARCH · Search NASA

Results for “ATTENTION”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

CAMFeND: Credibility-Aware Multimodal Fake News Detection with Rotational Attention

In the evolving digital landscape, fake news is a significant challenge, influencing public perception and decision-making. Traditional detection approaches focus on single-modal data or simple multimodal fusion, often overlooking deeper interactions and news credibility. We propose a novel model addressing these limitations by introducing rotational attention and news domain information as a feature. Unlike static attention mechanisms, our rotational attention dynamically shifts query, key, and value roles across text and image inputs, enabling richer cross-modal interaction. Incorporating news domain information further enhances the model’s reliability by associating news posts with top domains extracted from Google search results, reducing false detections. This approach assesses both the content and the broader web context in which the news is discussed. Our model outperforms existing state-of-the-art methods by providing deeper, layered multimodal integration and domain information analysis, resulting in a more robust and adaptive fake news detection system.

Gupta, Nidhi↗
DOE OSTI2026

Attention-based 3D – convolutional neural network model for mechanical property predictions using visible light images in metal additive manufacturing

Additive manufacturing (AM), while commonly used for rapid prototyping and creating components with complex geometries, has not been widely adopted for critical applications across the aerospace, automotive, defense, energy, and medical industries. This is, in part, due to the challenges of controlling flaws and uncertainty in the mechanical behavior of additively manufactured components. In recent years, there has been an increase in research aimed at predicting the final mechanical properties of additively manufactured components during the printing process. To address these issues, a 3D-CNN model was trained using low-cost in situ visible-light camera data, anomaly classifications, and the chosen process parameters to predict the ultimate tensile strength (UTS), yield strength (YS), total elongation (TE), and uniform elongation (UE). The 3D-CNN layers of the model employed attention mechanisms to prioritize features in the data, thereby improving prediction accuracy. Furthermore, the effect of each process parameter and anomaly class is investigated using attention-based dynamic sigmoid weighted gates to interpret the influence each class has on the final prediction. Different combinations of the in situ data were fed into the 3D-CNN, with varying amounts of image layers, to determine the ideal combination for predicting mechanical properties in situ. Here, the 3D-CNN model achieved mean absolute percentage errors (MAPE) below 5% for both UTS and YS while using only a single camera input and under half of the available image layers.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Instantiation of HCML Demonstrating Bayesian Predictive Modeling for Attentional Control

The research team developed models of Attentional Control (AC) that are unique to existing modeling approaches in the literature. The goal was to enable the research team to (1) make predictions about AC and human performance in real-world scenarios and (2) to make predictions about individual characteristics based on human data. First, the team developed a proof-of-concept approach for representing an experimental design and human subjects data in a Bayesian model, then demonstrated an ability to draw inferences about conditions of interest relevant to real-world scenarios. Ultimately, this effort was successful, and we were able to make reasonable (meaning supported by behavioral data) inferences about conditions of interest to develop a risk model for AC (where risk is defined as a mismatch between AC and attentional demand). The team additionally defined a path forward for a human-constrained machine learning (HCML) approach to make predictions about an individual's state based on performance data. The effort represents a successful first step in both modeling efforts and serves as a basis for future work activities. Numerous opportunities for future work have been defined.

97 MATHEMATICS AND COMPUTING↗
NASA NTRS2001

A dissociation between attention and selection

It is widely assumed that the allocatian of spatial attention results in the "selection" of attended objects or regions of space. That is, once a stimulus is attended, all its feature dimensions are processed irrespective of their relevance to behavioral goals. This assumption is based in part on experiments showing significant interference for attended stimuli when the response to an irrelevant dimension conflicts with the response to the relevant dimension (e.g., the Stroop effect). Here we show that such interference is not due to attending per se. In two spatial cuing experiments, we found that it was possible to restrict processing of attended stimuli to task-relevant dimensions. This new evidence supports two novel conclusions: (a) Selection involves more than the focusing of attention per se: and (b) task expectations play a key role in detertnining the depth of processing of the elementary feature dimensions of attended stimuli.

Attention↗
NASA NTRS2003

A dynamic model of stress and sustained attention

This paper examines the effects of stress on sustained attention. With recognition of the task itself as the major source of cognitive stress, a dynamic model is presented that addresses the effects of stress on vigilance and, potentially, a wide variety of attention performance tasks.

Review↗
NASA NTRS1997

Superimposition, symbology, visual attention, and the head-up display

In two experiments we examined a number of related factors postulated to influence head-up display (HUD) performance. We addressed the benefit of reduced scanning and the cost of increasing the number of elements in the visual field by comparing a superimposed HUD with an identical display in a head-down position in varying visibility conditions. We explored the extent to which the characteristics of HUD symbology support a division of attention by contrasting conformal symbology (which links elements of the display image to elements of the far domain) with traditional instrument landing system (ILS) symbology. Together the two experiments provide strong evidence that minimizing scanning between flight instruments and the far domain contributes substantially to the observed HUD performance advantage. Experiment 1 provides little evidence for a performance cost attributable to visual clutter. In Experiment 2 the pattern of differences in lateral tracking error between conformal and traditional ILS symbology supports the hypothesis that, to the extent that the symbology forms an object with the far domain, attention may be divided between the superimposed image and its counterpart in the far domain.

Attention↗
NASA NTRS1994

A Distance Measure for Attention Focusing and Anomaly Detection in Systems Monitoring

Any attempt to introduce automation into the monitoring of complex physical systems must start from a robust anomaly detection capability. This task is far from straightforward, for a single definition of what constitutes an anomaly is difficult to come by. In addition, to make the monitoring process efficient, and to avoid the potential for information overload on human operators, attention focusing must also be addressed. When an anomaly occurs, more often than not several sensors are affected, and the partially redundant information they provide can be confusing, particularly in a crisis situation where a response is needed quickly. Previous results on extending traditional anomaly detection techniques are summarized. The focus of this paper is a new technique for attention focusing.

attention focusing systems monitoring anomaly dete↗
NASA NTRS2023

Ch. 12. A Theoretical Approach to Management of Limited Attentional Resources to Support the m:N Operation in Advanced Air Mobility Ecosystem

Advanced air mobility (AAM) technologies incorporate increasingly autonomous systems that allow fully remote, independent, and intelligent operation of air vehicles to support the transportation of goods and passengers within and across urban and rural areas. With a myriad of automated technologies enabling the AAM ecosystem, the human operator’s role will likely be a passive supervisory monitor of the air vehicles, involving increasingly fewer humans (m) that manage many more autonomous systems (N), or m:N operations. Unfortunately, the human performance literature suggests that human operators will exhibit poor supervision of numerous autonomous agents due to the limits of attentional resources in the operators. In the general human information-processing model, a human operator exercises a limited pool of attentional resources to engage various information-processing stages including detecting, perceiving, comprehending, and predicting objects around them. Yamani and Horrey (2018) expanded the human information-processing model to characterize a tradeoff between information-processing demand and resource relief that automation brings in the context of automated driving. In their model, a driver interacting with an automated driving system is assumed to reallocate resources “freed” by automation to support other information-processing stages required for successful task performance. A future AAM ecosystem enabled by an orchestration of advanced automated systems, however, requires a single operator to interact with more than one air vehicle with varying levels and degrees of automated systems, making the traditional framework of human-automation interaction insufficient. To address this gap, we provide a review of the literature on situation assessment and trust, two constructs identified as critical for a fuller understanding of intimate and intricate interactions between a human operator and multiple air vehicles equipped with increasingly autonomous systems. Then, we propose an expansion of Yamani and Horrey’s (2018) model to motivate systematic research on the human operator’s role, identify factors that influence resource allocation and guide human-centered design of an interface supporting the m:N operation in the AAM environment.

Advanced Air Mobility↗
DOE OSTI2026

LAF-Net: A Deep Residual and Cross-Attention Framework for Day-Ahead Load Forecasting: Preprint

Accurate day-ahead load forecasting is essential for reliable power system operations and market efficiency. System operators such as the Midcontinent Independent System Operator (MISO) rely on forecasts from multiple vendors, yet combining them effectively remains a persistent challenge due to vendor-specific biases. This paper presents a novel LSTM-Attention Fusion Network with Error Representation (LAF-Net) that enhances day-ahead hourly load forecasting through deep residual learning and multi-modal cross-attention. The proposed model builds a historical error memory from past vendor performance and dynamically queries it with future hour context to generate adaptive, hour-specific trust weights for each vendor. A bounded residual correction further refines forecasts by mitigating systematic and temporally localized errors. Tested on real MISO LBA data with multi-vendor forecasts, LAF-Net consistently outperforms the best vendor baseline across all 38 LBAs, achieving more than a 40% reduction in system-level mean absolute error (MAE) during peak load hours relative to the best vendor baseline.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

Multihead Attention U‐Net for Magnetic Particle Imaging–Computed Tomography Image Segmentation

Magnetic particle imaging (MPI) is an emerging noninvasive molecular imaging modality with high sensitivity and specificity, exceptional linear quantitative ability, and potential for successful applications in clinical settings. Computed tomography (CT) is typically combined with the MPI image to obtain more anatomical information. Herein, a deep learning‐based approach for MPI‐CT image segmentation is presented. The dataset utilized in training the proposed deep learning model is obtained from a transgenic mouse model of breast cancer following administration of indocyanine green (ICG)‐conjugated superparamagnetic iron oxide nanoworms (NWs‐ICG) as the tracer. The NWs‐ICG particles progressively accumulate in tumors due to the enhanced permeability and retention (EPR) effect. The proposed deep learning model exploits the advantages of the multihead attention mechanism and the U‐Net model to perform segmentation on the MPI‐CT images, showing superb results. In addition, the model is characterized with a different number of attention heads to explore the optimal number for our custom MPI‐CT dataset.

Juhong, Aniwat↗
DOE OSTI2021

Microstructural Injury to Corpus Callosum and Intrahemispheric White Matter Tracts Correlate With Attention and Processing Speed Decline After Brain Radiation

The corpus callosum (CC) and intrahemispheric white matter tracts (IHWM) subserve critical aspects of attention and processing speed. We analyzed imaging biomarkers of microstructural injury within these regions and association with attention and processing speed performance before and after radiation therapy in primary brain tumor patients.

62 RADIOLOGY AND NUCLEAR MEDICINE↗
DOE OSTI2026

Peak2Patch: High-Fidelity Functional Group Identification through Attention-Based Fusion of Infrared and Mass Spectra

Identifying molecular structure based on spectroscopic readings is a key task in a variety of chemical and biological applications. Common spectroscopy techniques, such as Infrared (IR) Spectroscopy and Mass Spectrometry (MS), provide detailed information on the structure of molecular compounds but nonetheless require expert-level knowledge to decode. Machine learning has emerged as a potential solution for automating structure prediction from chemical spectra; however, current approaches generally focus on single sensor modalities, neglecting to leverage the complementary information contained within differing spectra. In this paper, we introduce Peak2Patch, a novel approach to fusion-enhanced prediction of functional groups from IR and mass spectra. First, we perform a detailed comparison of backbone networks for encoding both sparse mass spectra and dense IR spectra and demonstrate the superior performance of transformer neural networks over current state-of-the-art convolutional neural networks. Second, we evaluate three broad categories of fusion: early (raw feature), middle (deep feature), and late (decision) fusion, demonstrating the potential of a deep feature fusion-based approach. Lastly, we present Peak2Patch, our attention-based fusion scheme, which leverages cross-attention to mix features between encoded tokens of the two modalities. We validate our approach on a publicly available multimodal spectroscopic data set of 790k simulated molecules, demonstrating a large improvement in functional group prediction over both the previous state-of-the-art and our own strong single-modal baselines.

Jacobson, Philip [Sandia National Laboratories (SN↗
DOE OSTI2021

Attention Network Forecasts Time-to-Failure in Laboratory Shear Experiments

Rocks under stress deform by creep mechanisms that include formation and slip on small-scale internal cracks. Intragranular cracks and slip along grain contacts release energy as elastic waves termed acoustic emissions (AE). AEs are thought to contain predictive information that can be used for fault failure forecasting. Here, we present a method using unsupervised classification and an attention network to forecast labquakes using AE waveform features. Our data were generated in a laboratory setting using a biaxial shearing device with granular fault gouge intended to mimic the conditions of tectonic faults. Here, we analyzed the temporal evolution of AEs generated throughout several hundred laboratory earthquake cycles. We used a Conscience Self-Organizing Map (CSOM) to perform topologically ordered vector quantization based on waveform properties. The resulting map was used to interactively cluster AEs. We examined the clusters over time to identify those with predictive ability. Finally, we used a variety of LSTM and attention-based networks to test the predictive power of the AE clusters. By tracking cumulative waveform features over the seismic cycle, the network is able to forecast the time-to-failure (TTF) of lab earthquakes. Our results show that analyzing the data to isolate predictive signals and using a more sophisticated network architecture are key to robustly forecasting labquakes. In the future, this method could be applied on tectonic faults to monitor earthquakes and augment early warning systems.

58 GEOSCIENCES↗
DOE OSTI2023

CEGANN: Crystal Edge Graph Attention Neural Network for multiscale classification of materials environment

Abstract We introduce Crystal Edge Graph Attention Neural Network (CEGANN) workflow that uses graph attention-based architecture to learn unique feature representations and perform classification of materials across multiple scales (from atomic to mesoscale) and diverse classes ranging from metals, oxides, non-metals to hierarchical materials such as zeolites and semi-ordered mesophases. CEGANN can classify based on a global, structure-level representation such as space group and dimensionality (e.g., bulk, 2D, clusters, etc.). Using representative materials such as polycrystals and zeolites, we demonstrate its transferability in performing local atom-level classification tasks, such as grain boundary identification and other heterointerfaces. CEGANN classifies in (thermal) noisy dynamical environments as demonstrated for representative zeolite nucleation and growth from an amorphous mixture. Finally, we use CEGANN to classify multicomponent systems with thermal noise and compositional diversity. Overall, our approach is material agnostic and allows for multiscale feature classification ranging from atomic-scale crystals to heterointerfaces to microscale grain boundaries.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Attention-based functional-group coarse-graining: a deep learning framework for molecular prediction and design

Machine learning (ML) offers considerable promise for the design of new molecules and materials. In real-world applications, the design problem is often domain-specific, and suffers from insufficient data, particularly labeled data, for ML training. In this study, we report a data-efficient, deep-learning framework for molecular discovery that integrates a coarse-grained functional-group representation with a self-attention mechanism to capture intricate chemical interactions. Our approach exploits group-contribution concepts to create a graph-based intermediate representation of molecules, serving as a low-dimensional embedding that substantially reduces the data demands typically required for training. Using a self-attention mechanism to learn the subtle but highly relevant chemical context of functional groups, the method proposed here consistently outperforms existing approaches for predictions of multiple thermophysical properties. In a case study focused on adhesive polymer monomers, we train on a limited dataset comprising only 6,000 unlabeled and 600 labeled monomers. The resulting chemistry prediction model achieves over 92% accuracy in forecasting properties directly from SMILES strings, exceeding the performance of current state-of-the-art techniques. Furthermore, the latent molecular embedding is invertible, enabling the design pipeline to automatically generate new monomers from the learned chemical subspace. We illustrate this functionality by targeting several properties, including high and low glass transition temperatures (Tg), and demonstrate that our model can identify new candidates with values that surpass those in the training set. The ease with which the proposed framework navigates both chemical diversity and data scarcity offers a promising route to accelerate and broaden the search for functional materials.

Han, Ming [Univ. of Chicago, IL (United States)]↗
DOE OSTI2025

SA-GAT-SR: self-adaptable graph attention networks with symbolic regression for high-fidelity material property prediction

Recent advances in machine learning have demonstrated an enormous utility of deep learning approaches, particularly Graph Neural Networks (GNNs) for materials science. These methods have emerged as powerful tools for high-throughput prediction of material properties, offering a compelling enhancement and alternative to traditional first-principles calculations. While the community has predominantly focused on developing increasingly complex and universal models to enhance predictive accuracy, such approaches often lack physical interpretability and insights into materials behavior. Here, we introduce a novel computational paradigm—Self-Adaptable Graph Attention Networks integrated with Symbolic Regression (SA-GAT-SR)—that synergistically combines the predictive capability of GNNs with the interpretative power of symbolic regression. Our framework employs a self-adaptable encoding algorithm that automatically identifies and adjust attention weights so as to screen critical features from an expansive 180-dimensional feature space while maintaining O(n) computational scaling. The integrated SR module subsequently distills these features into compact analytical expressions that explicitly reveal quantum-mechanically meaningful relationships, achieving 23 × acceleration compared to conventional SR implementations that heavily rely on first-principle calculations-derived features as input. This work suggests a new framework in computational materials science, bridging the gap between predictive accuracy and physical interpretability, offering valuable physical insights into material behavior.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Reconstruction of unstable heavy particles using deep symmetry-preserving attention networks

Abstract Reconstructing unstable heavy particles requires sophisticated techniques to sift through the large number of possible permutations for assignment of detector objects to the underlying partons. An approach based on a generalized attention mechanism, symmetry preserving attention networks (SPA-NET), has been previously applied to top quark pair decays at the Large Hadron Collider which produce only hadronic jets. Here we extend the SPA-NET architecture to consider multiple input object types, such as leptons, as well as global event features, such as the missing transverse momentum. In addition, we provide regression and classification outputs to supplement the parton assignment. We explore the performance of the extended capability of SPA-NET in the context of semi-leptonic decays of top quark pairs as well as top quark pairs produced in association with a Higgs boson. We find significant improvements in the power of three representative studies: a search for$$t\bar{t}H$$ t t ¯ H , a measurement of the top quark mass, and a search for a heavy$${Z}^{{\prime} }$$ Z ′ decaying to top quark pairs. We present ablation studies to provide insight on what the network has learned in each case.

Physics↗
DOE OSTI2021

Attention-based generative models for de novo molecular design

An implementation of attention within the variational autoencoder framework for continuous representation of molecules. The addition of attention significantly increases model performance for complex tasks such as exploration of novel chemistries.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗