Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 109 records · Page 6
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Multimodal azimuthal oscillations in electron beam generated E×B plasma

Electron beam (e-beam) generated plasmas with applied crossed electric and magnetic (E×B) fields are promising for low-damage material processing. However, these plasmas can be subject to the formation of azimuthally propagating structures that enhance the radial transport of energetic charged species, which can harm the gentle processing capability of the plasma. In this work, we investigate the azimuthal structure formation in an e-beam generated E×B plasma using experimental diagnostics and 2D3V particle-in-cell simulations. Our findings demonstrate the formation of multiple simultaneously occurring azimuthally propagating modes that exhibit nontrivial radial dependence. It is suggested that the multimodal azimuthal spectrum is caused in part by the radial outflow of ions in the plasma.

electron beams↗
DOE OSTI2026

Hybrid Quantum–Classical Graph Transformers for Efficient Sentiment Analysis

Quantum Machine Learning (QML) offers a promising paradigm that leverages quantum computing principles to develop efficient and expressive models for learning from complex and structured data. Recent advances in natural language processing (NLP) and artificial intelligence (AI) have demonstrated capabilities in understanding, generating, and reasoning over linguistic and multimodal information. In this work, we present the Quantum Graph Transformer (QGT), a hybrid quantum–classical architecture that extends graph transformer capabilities through quantum self-attention. The QGT models variable-length sentences as token graphs, where both the embedding encoding and the self-attention mechanisms are implemented using parameterized quantum circuits (PQCs), enabling efficient contextual learning with significantly fewer trainable parameters. We train QGT using both fully connected and 𝑘 -nearest-neighbor graph structures and evaluate it on five benchmark sentiment-classification datasets. Experimental results show that QGT consistently achieves higher or comparable accuracy to existing quantum NLP models and outperforms a Classical Graph Transformer (CGT) baseline with identical architecture, achieving 29.4 × fewer parameters while requiring 3–5 × fewer samples to reach comparable performance. These findings highlight the potential of graph-based quantum models as scalable and data-efficient architectures for natural language understanding.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Transformations of Ti-5Al-5V-5Cr-3Mo powder due to reuse in laser powder bed fusion: A surface analytical approach

Reactive Ti-5Al-5 V-5Cr-3Mo (Ti5553) alloy feedstock powder is used in laser powder-bed fusion (LPBF). Due to the large quantity of powder necessary for LPBF, powder is reused numerous times under oxidizing environments. Transformations to the powder’s native surface oxide may impact the LPBF process and lead to deviations from expected behavior of printed parts. In this work, we present a multimodal characterization of new and reused powder by combining X-ray photoelectron spectroscopy (XPS), time-of-flight secondary ion mass spectrometry (ToF-SIMS), and transmission electron microscopy (TEM) of focused ion beam (FIB)-prepared cross sections of individual powder particles to understand how the surface oxide composition changes, grows, and becomes hydroxylated and hydrated due to reuse. We show that the native oxide film of Ti5553 powder is a hydroxylated mixed oxide composed of TiO 2 and Al 2 O 3 that grows from 5.6 +/- 0.7 nm to 8.3 +/- 1.1 nm due to reuse. Al oxide regions of the surface oxide become more hydroxylated during reuse in comparison to Ti oxide regions. This indicates that an outer oxide of pure TiO 2 may enhance recyclability of Ti5553 powder while the presence of Al oxides may decrease recyclability. Further characterization of surface transformations due to recyclability may eliminate unnecessary variation in LPBF.

36 MATERIALS SCIENCE↗
DOE OSTI2025

In situ Visible Light and Thermal Imaging Data from a Laser Powder Bed Fusion Additive Manufacturing Process Co-Registered to X-ray Computed Tomography and Fatigue Data

This dataset is comprised of in situ sensing data collected during a laser-based powder bed fusion additive manufacturing process, as well as rasterized scan path information, post-build X-ray computed tomography (XCT), and fatigue test results. A total of 64 cylinders, approximately 15 mm in diameter and 102 mm tall, were printed out of stainless steel 316H on a Colibrium Additive Concept Laser M2 Series 5 machine. Parameters known to produce dense material were used to construct 56 of these cylinders, while the remaining 8 cylinders were printed with relatively high energy density parameters prone to producing keyhole pores. In addition, two spatter generation blocks were constructed upstream of the 64 cylinders such that ejecta produced during the melting of the spatter generators were stochastically seeded onto the 64 cylinders. Based on previous experiments, these spatter particles were theorized to produce stochastic lack-of-fusion pores. During the construction of the build, high-resolution images of reflected light in the visible spectrum were captured both before and after recoating for each print layer. Additionally, temporally integrated thermal imaging in the near infrared spectrum produced integrated sum and max images on a layerwise basis. The multimodal in situ data has been co-registered to the build plate coordinate system, allowing for identification of process anomalies (e.g., spatter particles) apparent in the two sensors. Following construction of the build, the cylinders were subjected to XCT to identify internal flaws, and the resulting data have also been registered to the build plate coordinate system. Finally, 60 of the 64 cylinders were machined into fatigue coupons conforming to ASTM E466 and subsequently subjected to either high- or -low-cycle fatigue testing. The results of the fatigue tests have also been included in the dataset, and the XCT data corresponded to the approximate location of the gauge sections of the machine fatigue specimen geometry.

42 ENGINEERING↗
DOE OSTI2024

Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy

Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN)-based deep learning models is a powerful technique to detect features from material micrographs in an automated manner. In contrast to microstructural classification, supervised CNN models for segmentation tasks require pixel-wise annotation labels. However, manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. Microstructural characterization especially needs to be expedited for faster material discovery by changing alloy compositions. Here, in this study, we attempt to overcome such limitations by utilizing multimodal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy images of 347H stainless steel as training data and electron backscatter diffraction micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. The viability of our method is evaluated by considering a set of deep CNN architectures. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Generalist multimodal AI: A review of architectures, challenges and opportunities

Multimodal models are expected to be a critical component to future advances in artificial intelligence. Here, this field is starting to grow rapidly with a surge of new design elements motivated by the success of foundation models in natural language processing (NLP) and vision. It is widely hoped that further extending the foundation models to multiple modalities (e.g., text, image, video, sensor, time series, graph, etc.) will ultimately lead to generalist multimodal models, i.e. one model across different data modalities and tasks. However, there is little research that systematically analyzes recent multimodal models (particularly the ones that work beyond text and vision) with respect to the underling architecture proposed. Therefore, this work provides a fresh perspective on generalist multimodal models (GMMs) via a novel architecture and training configuration specific taxonomy. This includes factors such as Unifiability, Modularity, and Adaptability that are pertinent and essential to the wide adoption and application of GMMs. The review further highlights key challenges and prospects for the field and guide the researchers into the new advancements.

Artificial intelligence (AI)↗
DOE OSTI2020

Capture and Decomposition of the Nerve Agent Simulant, DMCP, Using the Zeolitic Imidazolate Framework (ZIF-8)

Understanding mechanisms of decontamination of chemical warfare agents (CWA) is an area of intense research aimed at developing new filtration materials to protect soldiers and civilians in case of state-sponsored or terrorist attack. In this study, we employed complementary structural, chemical, and dynamic probes and in situ data collection, to elucidate the complex chemistry, capture, and decomposition of the CWA simulant, dimethyl chlorophosphonate (DMCP). Our work reveals key details of the reactive adsorption of DMCP and demonstrates the versatility of zeolitic imidazolate framework (ZIF-8) as a plausible material for CWA capture and decomposition. Here, the in situ synchrotron-based powder X-ray diffraction (PXRD) and pair distribution function (PDF) studies, combined with Fourier transform infrared (FT-IR) spectroscopy, X-ray photoelectron spectroscopy (XPS), zinc K-edge X-ray absorption near edge structure (XANES), and Raman spectroscopies, showed that the unique structure, chemical state, and topology of ZIF-8 enable accessibility, adsorption, and hydrolysis of DMCP into the pores and revealed the importance of linker chemistry and Zn 2+ sites for nerve agent decomposition. DMCP decontamination and decomposition product(s) formation were observed by thermogravimetric analysis, FT-IR spectroscopy, and phosphorus (P) K-edge XANES studies. Differential PDF analysis indicated that the average structure of ZIF-8 (at the 30 Å scale) remains unchanged after DMCP dosing and provided information on the dynamics of interactions of DMCP with the ZIF-8 framework. Using in situ PXRD and diffuse reflectance infrared Fourier transform spectroscopy (DRIFTS), we showed that nearly 90% regeneration of the ZIF-8 structure and complete liberation of DMCP and decomposition products occur upon heating.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2025

Exploring Capability of Multimodal Foundation Model for Image-based Fault Detection of Photovoltaic Modules

Multimodal Foundation Model (MFM), like ChatGPT and Gemini, have emerged as powerful tools for their exceptional natural language processing capabilities and their emerging potential in image analysis. This paper investigates the application of MFMs for photovoltaic (PV) fault detection through image analysis, focusing on ChatGPT 4.0 and Gemini 1.5 Pro. Three types of PV images and the corresponding common PV faults are detected: bird droppings using visible images, cell cracks via electroluminescence (EL) images, and hotspots using infrared (IR) images. Among the two models, Gemini 1.5 Pro demonstrated superior performance, achieving near-perfect results with an average F1 score of 0.97, consistently outperforming ChatGPT 4.0 in accuracy and reliability. Unlike traditional machine learning (ML) models, MFMs can operate in a zero shot manner that does not require additional training by the user, and the input images are not limited by size, angle, scope, or PV technology. The strong adaptability and user-friendliness make MFM a promising tool for analyzing PV images and advancing health monitoring for PV modules.

Li, Baojie↗
DOE OSTI2022

Plasmonics-Enhanced Optical Imaging Systems for Bioenergy Research

This final report summarizes the key accomplishments of the project entitled “Plasmonics-Enhanced Optical Imaging Systems for Bioenergy Research”. The goal of this project is aimed at developing innovative and improved imaging instrumentation for visualization and quantitative characterization of biomarkers related to molecular processes and cellular function in living plants. This project focused on three specific aims: 1) Development of Advanced Multimodal Optical Sensing And Imaging Combinatory (MOSAIC) System for SERS biosensing and bioimaging. 2) Evaluation of MOSAIC technologies for sensing and imaging gene expression in plant systems using plasmonic nanoprobes. 3) Application of MOSAIC techniques to bioenergy research.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Multimodal Few-Shot Segmentation of Electron Micrographs

Scanning transmission electron microscopy (STEM) is one of the most used methods of analyzing the chemistry and composition of materials. By analyzing microstructures, these microscopes can help scientists better understand the molecular underpinnings of microelectronics, batteries, and more. However, STEM data can be difficult to interpret, so recent developments have been made in applications of machine learning to analyze these images. The PNNL-developed pyCHIP Classifier has achieved results in segmenting STEM these images via few-shot learning, a method which requires little data and human input, perfect for quickly analysis. In my internship I (Eli Meyers) investigated a multimodal improvement of this classifier by incorporating energy dispersive x-ray spectroscopy (EDS) data into the classification process for a more accurate segmentation. Furthermore, I encoded the spectral data by training a mass spectrometry encoder on the EDS data to extract a more meaningful representation of the data.

36 MATERIALS SCIENCE↗
DOE OSTI2024

AC Magnetometry Using Nano-ferrofluid Cladded Multimode Interferometric Fiber Optic Sensors for Power Grid Monitoring Applications

The AC magnetic field response of the superparamagnetic nano-ferrofluid is an interplay between the Neel and Brownian relaxation processes and is generally quantified via the susceptibility measurements at high frequencies. The high frequency limit is dictated by these relaxation times which need to be shorter than the time scale of the time varying magnetic field for the nano-ferrofluid to be considered in an equilibrium state at each time instant. Even though the high frequency response of ferrofluid has been extensively investigated for frequencies up to GHz range by non-optical methods, harnessing dynamic response by optical means for AC magnetic field sensing in fiber-optic-based sensors-field remains unexplored. Instead, the incorporation of nano-ferrofluid as sensing materials has been only limited to DC magnetic field sensing, often citing their long response time as a limiting factor to AC field sensing. This work reports the finding of high frequency (up to 15 kHz) AC magnetic field sensing capability of nanomagnetic fluid as the cladding material of a fiber-optic multimode interferometry (MMI) structure optimized for the fourth self-imaging spectral response. The key parameter enabling high frequency response is the short response time (<1 ms) achieved by optimizing both the sensing structure and nano-ferrofluid solution. Focus has been imparted on 60 Hz line-frequency profiles of various current/magnetic fields to test the efficacy of these sensors in metering and monitoring current and current-induced magnetic fields in the electrical power grid systems. The magnetic field sensitivity of 240 mV/Gauss per dBm of transmitted power was achieved for 60 Hz field applied via Helmholtz coil, whereas the 60 Hz AC current sensitivity of 2.83 mV/A was measured due to magnetic field induced by current in a straight conducting wire.

42 ENGINEERING↗
DOE OSTI2024

Bridging length scales in hard materials with ultra-small angle X-ray scattering – a critical review

Owing to their exceptional properties, hard materials such as advanced ceramics, metals and composites have enormous economic and societal value, with applications across numerous industries. Understanding their microstructural characteristics is crucial for enhancing their performance, materials development and unleashing their potential for future innovative applications. However, their microstructures are unambiguously hierarchical and typically span several length scales, from sub-ångstrom to micrometres, posing demanding challenges for their characterization, especially for in situ characterization which is critical to understanding the kinetic processes controlling microstructure formation. This review provides a comprehensive description of the rapidly developing technique of ultra-small angle X-ray scattering (USAXS), a nondestructive method for probing the nano-to-micrometre scale features of hard materials. USAXS and its complementary techniques, when developed for and applied to hard materials, offer valuable insights into their porosity, grain size, phase composition and inhomogeneities. We discuss the fundamental principles, instrumentation, advantages, challenges and global status of USAXS for hard materials. Using selected examples, we demonstrate the potential of this technique for unveiling the microstructural characteristics of hard materials and its relevance to advanced materials development and manufacturing process optimization. We also provide our perspective on the opportunities and challenges for the continued development of USAXS, including multimodal characterization, coherent scattering, time-resolved studies, machine learning and autonomous experiments. Our goal is to stimulate further implementation and exploration of USAXS techniques and inspire their broader adoption across various domains of hard materials science, thereby driving the field toward discoveries and further developments.

36 MATERIALS SCIENCE↗
DOE OSTI2024

Synthesis and Characterization of Enhanced Conductivity Copper Composites [Abstract]

In this project, PNNL will extrude and characterize copper alloy wires embedded with graphene additives using the billets provided by NAECO via the shear assisted processing and extrusion (ShAPE TM ) technology. PNNL will measure the electrical properties of the extruded wires and also perform multimodal imaging to determine the microstructural features as required. These activities will be performed as part of a voucher service provided by PNNL for the CABLE Manufacturing Prize stewarded by DOE Advanced Manufacturing Office. This 6-month effort will be executed at PNNL with a budget of $100,000.00 (USD).

36 MATERIALS SCIENCE↗
DOE OSTI2022

A novel sub-pilot-scale direct-contact ultrasonic dehydration technology for sustainable production of distillers dried grains (DDG)

DDG is a major source of protein, calcium, phosphorus, and sulfur is arguably the most important byproduct of the bioethanol industry with increasing demand over the past few years. Reducing energy consumption in the DDG production process and energy recovery from DDG is vital for sustainable bioethanol productions. In this paper, a novel direct-contact multi-frequency, multimode, and modulated (MMM) ultrasonic dryer (US) was developed for the first time and has been applied in dehydration of wet distillers' grain (WDG). Ultrasonic drying (US) was combined with a convective airflow (HA) at different temperatures of 25 (room temperature), 50 and 70 °C to evaluate the impact of US, HA, and US + HA on drying kinetics, activation energy, chemical compositions, microstructure, and color of DDG. Semi-empirical kinetic models were developed and evaluating drying performances showed that the application of ultrasound significantly enhanced the drying rate and decreased the drying time (by 46%), especially at low drying temperatures. The activation energy for moisture removal in the presence of ultrasound was about 50% of that without ultrasound. The final dried distillers' grains product processed by ultrasonic drying had a brighter color, a higher available protein, a higher digestible protein (the lowest acid detergent insoluble crude protein), and a better surface profile with no compromise on minerals and fiber contents.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

Ice-nucleating agents in sea spray aerosol identified and quantified with a holistic multimodal freezing model

Sea spray aerosol (SSA) is a widely recognized important source of ice-nucleating particles (INPs) in the atmosphere. However, composition-specific identification, nucleation processes, and ice nucleation rates of SSA-INPs have not been well constrained. Microspectroscopic characterization of ambient and laboratory-generated SSA confirms that water-borne exudates from planktonic microorganisms composed of a mixture of proteinaceous and polysaccharidic compounds act as ice-nucleating agents (INAs). These data and data from previously published mesocosm and wave channel studies are subsequently used to further develop the stochastic freezing model (SFM) producing ice nucleation rate coefficients for SSA-INPs. The SFM simultaneously predicts immersion freezing and deposition and homogeneous ice nucleation by SSA particles under tropospheric conditions. Predicted INP concentrations agree with ambient and laboratory measurements. In addition, this holistic freezing model is independent of the source and exact composition of the SSA particles, making it well suited for implementation in cloud and climate models.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Ultrafast CMOS image sensors and data-enabled super-resolution for multimodal radiographic imaging and tomography

We summarize recent progress in ultrafast Complementary Metal Oxide Semiconductor (CMOS) image sensor development and the application of neural networks for post-processing of CMOS and charge-coupled device (CCD) image data to achieve sub-pixel resolution (thus ‘super-resolution’). The combination of novel CMOS pixel designs and data-enabled image post-processing provides a promising path towards ultrafast high-resolution multi-modal radiographic imaging and tomography applications.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Ultrafast CMOS image sensors and data-enabled super-resolution for multimodal radiographic imaging and tomography

We summarize recent progress in ultrafast Complementary Metal Oxide Semiconductor (CMOS) image sensor development and the application of neural networks for post-processing of CMOS and charge-coupled device (CCD) image data to achieve sub-pixel resolution (thus $super$-$resolution$). The combination of novel CMOS pixel designs and data-enabled image post-processing provides a promising path towards ultrafast high-resolution multi-modal radiographic imaging and tomography applications.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2023

High Cooperativity Using a Confocal-Cavity–QED Microscope

Cavity quantum electrodynamics (QED) with cooperativity far greater than unity enables high-fidelity quantum sensing and information processing. The high-cooperativity regime is often reached through the use of short single-mode resonators. More complicated multimode resonators, such as the near-confocal optical Fabry-Pérot cavity, can provide intracavity atomic imaging in addition to high cooperativity. This capability has recently proved important for exploring quantum many-body physics in the driven-dissipative setting. In this work, we show that a confocal-cavity–QED microscope can realize cooperativity in excess of 110. This cooperativity is on par with the very best single-mode cavities (which are far shorter) and 21 times greater than single-mode resonators of similar length and mirror radii. The 1.7-µm imaging resolution is naturally identical to the photon-mediated interaction range. We measure these quantities by determining the threshold of cavity superradiance when small optically tweezed BoseEinstein condensates are pumped at various intracavity locations. Transmission measurements of an ex situ cavity corroborate these results. We provide a theoretical description that shows how cooperativity enhancement arises from the dispersive coupling to the atoms of many near-degenerate modes.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗