Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 325 records · Page 18

4D Multimodal Co-attention Fusion Network with Latent Contrastive Alignment for Alzheimer’s Diagnosis

Multimodal neuroimaging provides complementary structural and functional insights into both human brain organization and disease-related dynamics. Recent studies demonstrate enhanced diagnostic sensitivity for Alzheimer’s disease (AD) through synergistic integration of neuroimaging data (e.g., sMRI, fMRI) with tabular data (e.g., behavioral and cognitive tests). However, the intrinsic heterogeneity across modalities (e.g., 4D spatiotemporal fMRI dynamics vs. 3D anatomical sMRI structure) presents critical challenges for discriminative feature fusion, often leading to information loss or biased fusion. To bridge this gap, we propose M2M-AlignNet: a multimodal co-attention network with latent alignment for early AD diagnosis using sMRI and fMRI. At the core of our approach is a multi-patch-to-multi-patch (M2M) contrastive loss function that quantifies and reduces representational discrepancies via weighted patch correspondence, explicitly aligning fMRI components across brain regions with their sMRI structural substrates without one-to-one constraints. Additionally, we propose a latent-as-query co-attention module to autonomously discover fusion patterns, circumventing modality prioritization biases while minimizing feature redundancy. We conduct extensive experiments to confirm the effectiveness of our method and highlight the correspondence between fMRI and sMRI as AD biomarkers.

Wei, Yuxiang [Georgia Institute of Technology]↗

Hybrid Quantum–Classical Graph Transformers for Efficient Sentiment Analysis

Quantum Machine Learning (QML) offers a promising paradigm that leverages quantum computing principles to develop efficient and expressive models for learning from complex and structured data. Recent advances in natural language processing (NLP) and artificial intelligence (AI) have demonstrated capabilities in understanding, generating, and reasoning over linguistic and multimodal information. In this work, we present the Quantum Graph Transformer (QGT), a hybrid quantum–classical architecture that extends graph transformer capabilities through quantum self-attention. The QGT models variable-length sentences as token graphs, where both the embedding encoding and the self-attention mechanisms are implemented using parameterized quantum circuits (PQCs), enabling efficient contextual learning with significantly fewer trainable parameters. We train QGT using both fully connected and 𝑘 -nearest-neighbor graph structures and evaluate it on five benchmark sentiment-classification datasets. Experimental results show that QGT consistently achieves higher or comparable accuracy to existing quantum NLP models and outperforms a Classical Graph Transformer (CGT) baseline with identical architecture, achieving 29.4 × fewer parameters while requiring 3–5 × fewer samples to reach comparable performance. These findings highlight the potential of graph-based quantum models as scalable and data-efficient architectures for natural language understanding.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Industrial Inspection with Open Eyes: Advance with Machine Vision Technology

Machine vision systems have evolved significantly with the technology advances to tackle the challenges from modern manufacturing industry. A wide range of industrial inspection applications for quality control are benefiting from visual information captured by different types of cameras variously configured in a machine vision system. This chapter screens the state of the art in machine vision technologies in the light of hardware, software tools, and major algorithm advances for industrial inspection. The inspection beyond visual spectrum offers a significant complementary to the visual inspection. The combination with multiple technologies makes it possible for the inspection to achieve a better performance and efficiency in varied applications. The diversity of the applications demonstrates the great potential of machine vision systems for industry.

acoustic↗

Economic Analyses in Squamous Cell Carcinoma of the Head and Neck: A Review of the Literature From a Clinical Perspective

The purpose of this review was to describe cost-effectiveness and cost analysis studies across treatment modalities for squamous cell carcinoma of the head and neck (SCCHN), while placing their results in context of the current clinical practice. We performed a literature search in PubMed for English-language studies addressing economic analyses of treatment modalities for SCCHN published from January 2000 to March 2013. We also performed an additional search for related studies published by the National Institute for Health and Clinical Excellence in the United Kingdom. Identified articles were classified into 3 clinical approaches (organ preservation, radiation therapy modalities, and chemotherapy regimens) and into 2 types of economic studies (cost analysis and cost-effectiveness/cost-utility studies). All cost estimates were normalized to US dollars, year 2013 values. Our search yielded 23 articles: 13 related to organ preservation approaches, 5 to radiation therapy modalities, and 5 to chemotherapy regimens. In general, studies analyzed different questions and modalities, making it difficult to reach a conclusion. Even when restricted to comparisons of modalities within the same clinical approach, studies often yielded conflicting findings. The heterogeneity across economic studies of SCCHN should be carefully understood in light of the modeling assumptions and limitations of each study and placed in context with relevant settings of clinical practices and study perspectives. Furthermore, the scarcity of comparative effectiveness and quality-of-life data poses unique challenges for conducting economic analyses for a resource-intensive disease, such as SCCHN, that requires a multimodal care. Future research is needed to better understand how to compare the costs and cost-effectiveness of different modalities for SCCHN.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Energy-efficient multimodal mobility networks in transportation digital twins: Strategies and optimization

The study proposes a comprehensive Transportation Mobility (TransitMo) framework covering conceptual design, model formulation, optimization, simulation, and impact analysis of the transportation mobility system. TransitMo is composed of a transportation digital twin developed in Simulation of Urban MObility (SUMO) and an Intelligent Traffic Management and Control Center (ITMCC) that identifies the best ways to improve the movement of people within urban areas using various modes of transportation. This study encompasses advanced modeling techniques, algorithms, and strategic testing to optimize energy efficiency and mobility in a multimodal shared mobility network. TransitMo’s practical applications are exemplified through a city-scaled simulation network in Chattanooga, TN, employing demographic data to analyze historical traffic patterns and forecast future demands. Central to this methodology are three models: the User Preference Model (UP), the Energy Consumption Model (EC), and the System Optimization Model (SO). These models work in concert to iteratively devise the optimal travel incentives and minimize the total system cost in a real-time manner. In conclusion, test results verified that the proposed adaptive incentive program and optimized bus scheduling can improve network performance by increasing public transit ridership.

42 ENGINEERING↗

Localized keyhole pore prediction during laser powder bed fusion via multimodal process monitoring and X-ray radiography

Systematic fault detection and control during laser powder bed fusion (L-PBF) has been a long-standing objective for system manufacturers and researchers in the additive manufacturing (AM) industry. This manuscript investigates a data fusion approach for detection of keyhole porosity formation during laser irradiation of Ti-6Al-4V substrates by concurrent recording of thermally induced optical emission measured using both off-axis and coaxial photodiode sensors, and acoustic emission. Subsurface defect formation was monitored via high-speed synchrotron X-ray imaging at 20,000 frames per second, enabling temporal registration of keyhole pore formation events to the monitoring signals at a resolution of 50 µs. We developed data fusion machine learning (ML) models for localized prediction of keyhole pore formation at various time scales ranging from 0.5 ms to 2 ms. The signal segments were featurized using two independent approaches: (1) power spectral density (PSD) and (2) highly comparative time series analysis (HCTSA) framework. The extracted features from different sensor modalities were fused together to construct a multimodal feature space and sequential feature selection was used to determine the most informative features for training the ML models. The predictive performance was evaluated for three classifying algorithms: Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Gaussian Naive Bayes (GNB). As a result, pore formation events were predicted with up to 0.95 F1-score, 1.0 recall and 0.94 accuracy. The most heavily weighted features indicate that model performance is chiefly governed by the acoustic monitoring signal, with a secondary contribution from the optical emission sensors.

36 MATERIALS SCIENCE↗

Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy

Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN)-based deep learning models is a powerful technique to detect features from material micrographs in an automated manner. In contrast to microstructural classification, supervised CNN models for segmentation tasks require pixel-wise annotation labels. However, manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. Microstructural characterization especially needs to be expedited for faster material discovery by changing alloy compositions. Here, in this study, we attempt to overcome such limitations by utilizing multimodal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy images of 347H stainless steel as training data and electron backscatter diffraction micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. The viability of our method is evaluated by considering a set of deep CNN architectures. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.

36 MATERIALS SCIENCE↗

Adult Rhabdomyosarcoma Survival Improved With Treatment on Multimodality Protocols

Purpose: Rhabdomyosarcoma (RMS) is a pediatric sarcoma rarely occurring in adults. For unknown reasons, adults with RMS have worse outcomes than do children. Methods and Materials: We analyzed data from all patients who presented to Memorial Sloan-Kettering Cancer Center between 1990 and 2011 with RMS diagnosed at age 16 or older. One hundred forty-eight patients met the study criteria. Ten were excluded for lack of adequate data. Results: The median age was 28 years. The histologic diagnoses were as follows: embryonal 54%, alveolar 33%, pleomorphic 12%, and not otherwise specified 2%. The tumor site was unfavorable in 67% of patients. Thirty-three patients (24%) were at low risk, 61 (44%) at intermediate risk, and 44 (32%) at high risk. Forty-six percent were treated on or according to a prospective RMS protocol. The 5-year rate of overall survival (OS) was 45% for patients with nonmetastatic disease. The failure rates at 5 years for patients with nonmetastatic disease were 34% for local failure and 42% for distant failure. Among patients with nonmetastatic disease (n=94), significant factors associated with OS were histologic diagnosis, site, risk group, age, and protocol treatment. On multivariate analysis, risk group and protocol treatment were significant after adjustment for age. The 5-year OS was 54% for protocol patients versus 36% for nonprotocol patients. Conclusions: Survival in adult patients with nonmetastatic disease was significantly improved for those treated on RMS protocols, most of which are now open to adults.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Learning the temporal evolution of multivariate densities via normalizing flows

In this work, we propose a method to learn multivariate probability distributions using sample path data from stochastic differential equations. Specifically, we consider temporally evolving probability distributions (e.g., those produced by integrating local or nonlocal Fokker–Planck equations). Here, we analyze this evolution through machine learning assisted construction of a time-dependent mapping that takes a reference distribution (say, a Gaussian) to each and every instance of our evolving distribution. If the reference distribution is the initial condition of a Fokker–Planck equation, what we learn is the time-T map of the corresponding solution. Specifically, the learned map is a multivariate normalizing flow that deforms the support of the reference density to the support of each and every density snapshot in time. We demonstrate that this approach can approximate probability density function evolutions in time from observed sampled data for systems driven by both Brownian and Lévy noise. We present examples with two- and three-dimensional, uni- and multimodal distributions to validate the method.

97 MATHEMATICS AND COMPUTING↗

Provenance infrastructure for multimodal x-ray experiments and reproducible analysis

Exploring relationships between material structures and desired properties for material discovery and synthesis requires mining machine-readable databases and linking experimental data with sufficiently rich annotations. There are vast gaps both in these data sources and in the connections between them. We present a provenance-based data management and analysis framework that enables capturing, persisting, and reanalyzing experimental data from the BNL National Synchrotron Light Source II (NSLS-II). Our system leverages NSLS-II Bluesky for experimental data acquisition, captures analysis parameters, and enriches the search space with results from the scientific literature. We describe in detail the challenges and benefits of data-driven approaches when comparing computational and experimental data, such as the multiple possible scenarios of interest in often ill-posed inverse problems and missing data. Our infrastructure utilizes the NSLS-II event model and includes discovery mechanisms across multiple beamlines. A text and data mining module using natural language processing classification techniques extracts relevant information from the literature that is added to data sources. We describe a provenance acquisition and analysis rerun workflow based on a directed acyclic graph that keeps track of process and data provenance and allows to replay analysis.

97 MATHEMATICS AND COMPUTING↗

CQM-Analysis v1.0

This repository contains the data and code necessary to reproduce the primary analysis and figures for the manuscript "Pathways to productivity: mapping the relationship between multimodal transportation infrastructure, commute quality, and economic vitality for the United States workforce". The analysis demonstrates a newly defined commute quality metric (CQM) characterizing the quality, as a monetized consumer surplus, of travel for the purpose of work for every census tract in the continental United States. The analysis additionally demonstrates the correlation of that CQM with key economic vitality indicators. Specifically median household income and unemployment rate.

Spurlock, C Anna [Lawrence Berkeley National Labor↗

Investigation of the bandwidth of multimode optical fibers used with 1550-nm LED and laser sources

Multimode optical fibers are not intended to be used with 1550-nm sources; however, it is desirable to utilize 1300/1550-nm wavelength division multiplexing (WDM) on some multimode fibers at Kennedy Space Center (KSC). No information from fiber vendors nor from the literature is available to support this use. Preliminary studies at KSC have suggested that these fibers might be usable at 1550-nm if the fibers possessed enough bandwidth when sourced by LEDs. Detailed bandwidth studies were made on 12 multimode fibers using 1300- and 1550-nm lasers and LEDs. The results showed that the modal bandwidth at 1550-nm was about 50 percent of the 1300-nm value and that the chromatic dispersion could be predicted by extrapolating the vendor's specifications for wavelengths outside the 1550-nm region. Utilizing these data, predictions of the fiber's optical bandwidth were accurately made. Problems with launch conditions and possible differential attenuation at connectors was noted at 1300-nm but was less significant at 1550-nm. It appears that the multimode fibers studied will offer adequate performance in the 1550-nm region for a number of current KSC needs. Studies of additional fibers are encouraged to gain more confidence and better understanding of the 1550-nm bandwidth of KSC's multimode optical fibers before committing to 1300/1550-nm WDM.

White, Preston A., III↗

Multimodal Communication System

Current communications modules operate as a point-to-point radio. This system operates in a point-to-multipoint utilizing multiple radio frequencies for both high bandwidth data to satellite communication. This communication module further allows for multiple users to interact with the communication system in a true cloud type application so that the modules can be handled in a fleet situation.

Vacaliuc, Bogdan↗

Striving to translate shale physics across ten orders of magnitude: What have we learned?

Shales will play an important role in the successful transition of energy from fossil-based resources to renewables in the coming decades. Aside from being a significant source of low-carbon intensity fuels, like natural gas, they also serve as geologic seals of subsurface formations that may be used to isolate nuclear waste, sequester CO 2 , or store intermittent energy (e.g., solar hydrogen). Despite their importance, shales pose significant engineering and environmental challenges due to their nanoporous structure and extreme heterogeneity that spans at least ~10 orders of magnitude in spatial scale. Two challenges inhibit a system-level understanding: (1) the physics of fluid flow and phase behavior in shales are poorly understood due to the dominant molecular interactions between minerals and fluids under confinement, and (2) the apparent lack of scale separation that prevents a reliable (closed) description of the physics at any single scale of observation. In this review, we focus on the latter issue and discuss scale translation, which in its broadest sense is transforming data or simulations from one spatiotemporal scale to another. While effective scale translation is not exclusive to shales, but all geologic porous media, the need for it is especially acute in shales given their high degree of heterogeneity. Classical theories like homogenization, while indispensable, fail when scales are not separated. Other methods, like numerical upscaling, scale-translate in only one direction: small to large, but not the reverse, called downscaling. However, the confluence of advances in three areas are bringing challenging problems such as shales within reach: increased computational power and scalable algorithms; high-resolution imaging and multi-modal data acquisition; and machine learning to process massive amounts of data. While these advances equip geoscientists with a wide array of experimental and computational tools, no individual tool can probe the entire gamut of heterogeneity in shales. Their effective use, therefore, requires an ability to bridge between various data types obtained at different scales. The aim of this review is to present a coherent account of computational and experimental methods that may be used to achieve just that, i.e., to perform scale translation. We provide a broader definition of scale translation, one that transcends classical homogenization and upscaling methods, but is consistent with them and accommodates notions like downscaling and data translation. After a brief introduction to homogenization, we review hybrid methods, numerical upscaling and its recent extensions, multiscale computing, high-resolution imaging, and machine learning. We place particular emphasis on multiscale computing and propose an algorithmic framework to bridge between the pore (micro) and Darcy (macro) scales. Throughout the paper, we draw comparisons between the various methods and highlight their (often hidden) similarities, differences, benefits, and pitfalls. We finally conclude with two case studies on shales that exemplify some of the methods presented.

58 GEOSCIENCES↗

Accurate and transferable multitask prediction of chemical properties with an atoms-in-molecules neural network

Atomic and molecular properties could be evaluated from the fundamental Schrodinger’s equation and therefore represent different modalities of the same quantum phenomena. Here, we present AIMNet, a modular and chemically inspired deep neural network potential. We used AIMNet with multitarget training to learn multiple modalities of the state of the atom in a molecular system. The resulting model shows on several benchmark datasets state-of-the-art accuracy, comparable to the results of orders of magnitude more expensive DFT methods. It can simultaneously predict several atomic and molecular properties without an increase in the computational cost. With AIMNet, we show a new dimension of transferability: the ability to learn new targets using multimodal information from previous training. The model can learn implicit solvation energy (SMD method) using only a fraction of the original training data and an archive median absolute deviation error of 1.1 kcal/mol compared to experimental solvation free energies in the MNSol database.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Study of Lasing in Liquid-Crystal Systems with Microgratings

Waveguide lasing in a layer of nematic liquid crystal (NLC) doped with a dye has been experimentally observed. A chromium micrograting with a period of 1.76 μm has been deposited on the surface one of the cell substrates to provide a distributed feedback (DFB) and partially extract laser radiation to a quartz substrate. The DFB provides lasing in the ninth diffraction order for waveguide TM modes. Laser radiation is observed at the output of the substrate end face at an angle of 67.0° ± 1.5° with respect to the normal to the waveguide plane. An increase in voltage across the micrograting electrodes leads to red-shifted multimode lasing. A numerical simulation of a structure imitating the experimental cell has shown good agreement between the calculation results and experimental data.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Toward more-robust, AI-enabled subsurface seismic imaging for geotechnical applications

Non-invasive seismic imaging has the potential to cost-effectively evaluate large volumes of subsurface material to inform geotechnical site investigation. However, seismic imaging using full waveform inversion (FWI) requires significant computational time and is dependent on an initial starting model. As a result, FWI has not yet been widely adopted into geotechnical practice. Previous efforts, on relatively simple two-layered models, indicate that data-driven artificial intelligence (AI) models may be as effective as FWI at predicting 2D images of shear wave velocity (V s ). Furthermore, the AI model predictions can be made almost instantaneously after data acquisition and do not require an initial starting model. We examine the generality of these findings by developing a new AI model for subsurface seismic imaging, whereby we make several notable contributions. First, we architect a multimodal AI model that combines time- and frequency-domain representations of the seismic wavefield to predict a 50 m by 20 m subsurface image of V s . Second, we developed a new diverse dataset of 100,000 images with their corresponding seismic wavefields to train the AI model. Third, we propose four physics-informed data augmentations for data-driven seismic imaging. Fourth, we develop two prediction consistency tests to evaluate the model’s performance when the true subsurface is unknown. Our final model, which has been made publicly available, is capable of predicting a subsurface V s image from a single seismic wavefield with an average, mean absolute percent error (MAPE) of 24 %. The predictive model is applied to a field dataset and shown to be consistent with local geology and shear-wave refraction measurements from the same location.

Artificial intelligence↗

Systems for the Intermodal Routing of Spent Nuclear Fuel

The safe and secure movement of spent nuclear fuel from shutdown and active reactor facilities to intermediate or long term storage sites may, in some instances, require the use of several modes of transportation to accomplish the move. To that end, a fully operable multi-modal routing system is being developed within Oak Ridge National Laboratory s (ORNL) WebTRAGIS (Transportation Routing Analysis Geographic Information System). This study aims to provide an overview of multi-modal routing, the existing state of the TRAGIS networks, the source data needs, and the requirements for developing structural relationships between various modes to create a suitable system for modeling the transport of spent nuclear fuel via a multimodal network. Modern transportation systems are comprised of interconnected, yet separate, modal networks. Efficient transportation networks rely upon the smooth transfer of cargoes at junction points that serve as connectors between modes. A key logistical impediment to the shipment of spent nuclear fuel is the absence of identified or designated transfer locations between transport modes. Understanding the potential network impacts on intermodal transportation of spent nuclear fuel is vital for planning transportation routes from origin to destination. By identifying key locations where modes intersect, routing decisions can be made to prioritize cost savings, optimize transport times and minimize potential risks to the population and environment. In order to facilitate such a process, ORNL began the development of a base intermodal network and associated routing code. The network was developed using previous intermodal networks and information from publicly available data sources to construct a database of potential intermodal transfer locations with likely capability to handle spent nuclear fuel casks. The coding development focused on modifying the existing WebTRAGIS routing code to accommodate intermodal transfers and the selection of prioritization constraints and modifiers to determine route selection. The limitations of the current model and future directions for development are discussed, including the current state of information on possible intermodal transfer locations for spent fuel.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗