Search NASA⌕ Search

SEARCH · Search NASA

Results for “Multimodal analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9
All sourcesNASA NTRSDOE OSTI
DOE OSTI2026

A unified large language model–based framework for heterogeneous PV image diagnosis

With advances in imaging technologies, modern photovoltaic (PV) systems generate large volumes of heterogeneous image data, including visible, electroluminescence (EL), and infrared (IR) images. Existing PV image analysis models, particularly deep learning approaches, are typically task-specific and lack cross-modality generalization. To address this limitation, this paper proposes an open-source large language model (LLM)–based unified framework for heterogeneous PV image diagnostics. Through task-aware diagnostic prompting, the framework enables analysis of visible, EL, and IR images within a single pipeline, supporting both zero-shot and few-shot inference and binary and multiclass classification. It is compatible with state-of-the-art multimodal LLMs, including ChatGPT, Gemini, Claude, Qwen, and CLIP. The framework is evaluated on PV module condition classification (clean, soiling, snow, hail, and bird droppings) using visible images, cell crack detection using EL images, and hotspot detection using IR images. GPT-5.1 in few-shot mode achieves the best performance, with classification accuracy exceeding 97.3%. Open-source models such as Qwen and CLIP also deliver competitive results on visible images (around 90% accuracy), though their performance is more limited on EL and IR modalities. On the full ELPV dataset, the framework achieves 83.5% zero-shot accuracy, within 2.8% of the supervised CNN baseline, confirming scalability to larger benchmarks. Practical aspects such as reproducibility, response latency, and confidence estimation are systematically analyzed. The framework operates across PV image modalities without modality- or task-specific training, making it well suited as a rapid pre-screening tool to support downstream detailed diagnostics. A benchmark dataset of diverse labeled PV images is also released.

Li, Baojie↗
DOE OSTI2022

A multimodal approach to revisiting oxidation defects in Cr 2 O 3

The oxidation of chromium in air at 700 °C was investigated with a focus on point defect behavior and transport during oxide layer growth. A comprehensive set of characterization techniques targeted characteristics of chromium oxide microstructure and chemical composition analysis. TEM showed that the oxide was thicker with longer oxidation times and that, for the thicker oxides, voids formed at the metal/oxide interface. PAS revealed that the longer the oxidation time, there was an overall reduction in vacancy-type defects, though chromium monovacancies were not found in either case. EIS found that the longer oxidized material was more electrochemically stable and that, while all oxides displayed p-type character, the thicker oxide had an overall lower charge carrier density. Together, the results suggest anion oxygen interstitials and chromium vacancy cluster complexes drive transport in an oxidizing environment at this temperature, providing invaluable insight into the mechanisms that regulate corrosion.

36 MATERIALS SCIENCE↗
DOE OSTI2021

Cluster defects in gibbsite nanoplates grown at acidic to neutral pH

Gibbsite [α-Al(OH) 3 ] is the solubility limiting phase for aluminum across a wide pH range, and it is a common mineral phase with many industrial applications. The growth mechanism of this layered-structure material, however, remains incompletely understood. Synthesis of gibbsite at low to circumneutral pH yields nanoplates with substantial interlayer disorder. Here we examine defects in this material in detail, and the effects of recrystallization in highly alkaline sodium hydroxide solution at 80 °C. We employed a multimodal approach, including scanning electron microscopy, magic-angle spinning nuclear magnetic resonance (MAS-NMR), Raman and infrared spectroscopies, X-ray diffraction (XRD), and X-ray total scattering pair distribution function (XPDF) analysis to characterize the ageing of the nanoplates over several days. Additionally, XRD and XPDF indicate that gibbsite nanoplates precipitated at circumneutral pH contain dense, truncated sheets imparting a local difference in interlayer distance. These interlayer defects appear well described by flat Al 13 aluminum hydroxide nanoclusters nearly isostructural with gibbsite sheets present under synthesis conditions and trapped as interlayer inclusions during growth. Ageing at elevated temperature in alkaline solutions gradually improves crystallinity, showing a gradual increase in H-bonding between interlayer OH groups. Between 7 to 8 vol% of the initial gibbsite nanoparticles exhibit this defect, with the majority of differences disappearing after 2–4 hours of recrystallization in alkaline solution. The results not only identify the source of disorder in gibbsite formed under acidic/neutral conditions but also point to a possible cluster-mediated growth mechanism evident through inclusion of relict oligomers with gibbsite-like topology trapped in the interlayer spaces.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2020

Multimodal Nanoscale Tomographic Imaging for Battery Electrodes

Accurate representations of the 3D structure within a lithium-ion battery are key to understanding performance limitations. However, obtaining exact reconstructions of electrodes, where the active particles, the carbon black and polymeric binder domain, and the pore space are visualized is challenging. In this work, it is shown that multimodal imaging can be used to overcome this challenge. High-resolution ptychographic X-ray computed tomography are combined with lower resolution but higher contrast transmission X-ray tomographic microscopy to obtain 3D reconstructions of pristine and cycled graphite-silicon composite electrodes. This cross-correlation enables quantitative analysis of the surface of active particles, including the heterogeneity of carbon-black and binder domain and solid-electrolyte interphase coverage. Capturing the active particles as well as the carbon black-binder domain allows using these segmented structures for electrochemical simulations to highlight the influence of the particle embedding on local state of charge heterogeneities.

25 ENERGY STORAGE↗
DOE OSTI2022

LES of HCCI combustion of iso-octane/air in a flat-piston rapid compression machine

Homogeneous Charge Compression Ignition (HCCI) engines promise better efficiency and cleaner emissions than conventional piston engines, but can be challenging to control. Rapid compression machines (RCM) provide a simplified configuration for investigating HCCI combustion behavior, which is necessary for effective control of engine ignition timing and peak pressures. Here, in this study, we assess the utility of large eddy simulations (LES) for predicting HCCI combustion in a 3-D configuration. To this end, LES with finite-rate chemistry employing a 99-species iso-octane/air mechanism of two RCM operating conditions are performed. The RCM configuration under consideration was designed by Strozzi et al. with a flat piston to introduce large amounts of thermal stratification representative of realistic HCCI engine conditions, through the generation of corner vortices. It is shown that the simulation provides reasonable agreement with temperature fluctuations (7% difference), as well as ignition delay in the short ignition case (1 ms difference), while the long ignition case (35 ms difference) highlights more substantial deficiencies that are still within the expected uncertainty from the employed chemical mechanism. Flame propagation modes predicted by LES agree with experimental observations: spontaneous ignition is seen in the short ignition case, while deflagration is more prominent in the long ignition case. Analysis of global and local quantities classify the short ignition case in a mixed ignition regime, and the long ignition case in the mild ignition regime. These results demonstrate the utility of FRC-LES for investigations of multimode combustion regimes of HCCI combustion in a 3-D configuration.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2025

Decoding diffraction and spectroscopy data with machine learning: A tutorial

This Tutorial provides a step-by-step guide on how to apply supervised machine-learning techniques to analyze diffraction and spectroscopy data. This Tutorial details four models—a reconstruction-focused model, a regression-focused model, a hybrid reconstruction/regression model, and a multimodal model—that use x-ray diffraction profiles and vibrational density of states spectra to predict various microstructural descriptors. In this Tutorial, we cover data pre-processing steps, constructions of the models via dimensionality reduction and regression, training, and analysis of these models. Comparisons of the model’s performance are provided, highlighting the strength and weakness of the various approaches utilized.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Joint Analysis of Program Data Representations using Machine Learning for Improved Software Assurance and Development Capabilities

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2026

Cascading economic losses from port disruptions under capacity constrained multimodal freight networks

This study quantifies how throughput disruptions at major seaports cascade through capacity-constrained multimodal freight networks and interregional production systems. We couple an agent-based model (ABM) multimodal freight simulation that resolves rerouting, terminal queueing, and inventory drawdown under binding modal and facility capacities with a multiregional output loss input-output (MRIIM) model that propagates realized delivery shortfalls across regions and sectors. The framework is demonstrated for the Port of Los Angeles using Freight Analysis Framework flows and Bureau of Economic Analysis input-output accounts and is evaluated over a 52-week horizon under deterministic sector targeted shocks and stochastic disruption realizations with uncertain severity and duration. Results indicate nonlinear amplification: realized national losses concentrate in manufacturing and transportation/warehousing even when exogenous port shocks are dispersed, suggesting that congestion spillback and limited short-run substitution can dominate the initial shock allocation. We further evaluate a tabular reinforcement-learning (Q-learning) intervention layer that selects among a small set of implementable system level levers (truck-to-rail and truck-to-barge shift settings) without overriding shipper routing, finding that such interventions reduce total losses for moderate disruptions but yield diminishing returns once substitute modes approach capacity. By linking operational freight behavior to system wide impacts under uncertainty, the proposed ABM-MRIIM pipeline provides a reusable workflow for port disruption stress testing, identification of structurally critical sectors/corridors, and evaluation of resilience interventions under realistic capacity limits.

42 ENGINEERING↗
DOE OSTI2024

Comparing Sensor Fusion and Multimodal Chemometric Models for Monitoring U(VI) in Complex Environments Representative of Irradiated Nuclear Fuel

Optical sensors and chemometric models were leveraged for the quantification of uranium(VI) (0–100 μg mL –1 ), europium (0–150 μg mL –1 ), samarium (0–250 μg mL –1 ), praseodymium (0–350 μg mL –1 ), neodymium (0–1000 μg mL –1 ), and HNO 3 (2–4 M) with varying corrosion product (iron, nickel, and chromium) levels using laser fluorescence, Raman scattering, and ultraviolet–visible–near-infrared absorption spectra. In this paper, an efficient approach to developing and evaluating tens of thousands of partial least-squares regression (PLSR) models, built from fused optical spectra or multimodal acquisitions, is discussed. Each PLSR model was optimized with unique preprocessing combinations, and features were selected using genetic algorithm filters. The 7-factor D-optimal design training set contained just 55 samples to minimize the number of samples. The performance of PLSR models was evaluated by using an automated latent variable selection script. PLS1 regression models tailored to each species outperformed a global PLS2 model. PLS1 models built using fused spectra data and a multimodal (i.e., analyzed separately) approach yielded similar information, resulting in percent root-mean-square error of prediction values of 0.9–5.7% for the seven factors. Further, the optical techniques and data processing strategies established in this study allow for the direct analysis of numerous species without measuring luminescence lifetimes or relying on a standard addition approach, making it optimal for near-real-time, in situ measurements. Nuclear reactor modeling helped bound training set conditions and identified elemental ratios of lanthanide fission products to characterize the burnup of irradiated nuclear fuel. Leveraging fluorescence, spectrophotometry, experimental design, and chemometrics can enable the remote quantification and characterization of complex systems with numerous species, monitor system performance, help identify the source of materials, and enable rapid high-throughput experiments in a variety of industrial processes and fundamental studies.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Multimodal Atomic Force Microscopy for the Characterization of Metallic Particulates

This study investigates the utility of multimodal, or functional, atomic force microscopy (AFM) for the characterization of individual particles and particle ensembles. In single-particle analyses, AFM imaging modes provided orthogonal insights into mechanical and magnetic properties that are not accessible through conventional electron microscopy. Although the experiments were inherently delicate and time-intensive, with an observed sample loss rate of approximately 30%, these techniques enabled the qualitative differentiation of grain structures and magnetic domains, underscoring the challenges of experimental robustness. For particle ensembles, AFM enabled the extraction of reliable 2D and 3D particle size distributions. However, efforts to chemically differentiate particles based on mechanical property contrasts were limited by scaling effects and the qualitative nature of the data. Overall, multimodal AFM offers valuable complementary information, but its application requires careful consideration of methodological limitations, particularly for sample integrity, calibration, and scalability of mechanical and magnetic measurements. This report is organized to first address the application of AFM to single-particle analysis using various imaging modalities, followed by its potential role in ensemble-level particle screening and differentiation.

36 MATERIALS SCIENCE↗
DOE OSTI2025

Predicting Trends in VOC Through Rapid, Multimodal Characterization of State-of-the-Art p-i-n Perovskite Devices

Perovskite photovoltaic technologies are approaching commercial deployment, yet single junction and tandem architectures both still have significant room to improve power conversion efficiency and stability. The ability to perform rapid screening of material quality after altering processing conditions is critical to accelerating the optimization and commercialization of perovskite-based technologies. Currently, researchers utilize a wide range of stand-alone metrology tools to isolate sources of power loss throughout a device stack, which can be slow and labor intensive. Here, we demonstrate the use of a multimodal metrology approach to rapidly determine the maximum achievable and predicted open circuit voltages of >100 perovskite devices during fabrication. Acquisition of these different data is facilitated by combining them into a single integrated measurement platform. We show that these data and automated analysis can be used to rapidly understand and ultimately predict quantitative trends in open circuit voltages of state-of-the-art device architectures. The data and automated analysis workflow presented provides a reliable approach to quickly identify absorber and charge transport layer combinations that can lead to improved open circuit voltages.

14 SOLAR ENERGY↗
DOE OSTI2023

Plasmonic nanorod probes’ journey inside plant cells for in vivo SERS sensing and multimodal imaging

Nanoparticle-based platforms are gaining strong interest in plant biology and bioenergy research to monitor and control biological processes in whole plants. However, in vivo monitoring of biomolecules using nanoparticles inside plant cells remains challenging due to the impenetrability of the plant cell wall to nanoparticles beyond the exclusion limits (5-20 nm). To overcome this physical barrier, we have designed unique bimetallic silver-coated gold nanorods (AuNR@Ag) capable of entering plant cells, while conserving key plasmonic properties in the near-infrared (NIR). To demonstrate cellular internalization and tracking of the nanorods inside plant tissue, we used a comprehensive multimodal imaging approach that included transmission electron microscopy (TEM), confocal fluorescence microscopy, two-photon luminescence (TPL), X-ray fluorescence microscopy (XRF), and photoacoustics imaging (PAI). We successfully acquired SERS signals of nanorods in vivo inside plant cells of tobacco leaves. On the same leaf samples, we applied orthogonal imaging methods, TPL and PAI techniques for in vivo imaging of the nanorods. This study first demonstrates the intracellular internalization of AuNR@Ag inside whole plant systems for in vivo SERS analysis in tobacco cells. This work demonstrates the potential of this nanoplatform as a new nanotool for intracellular in vivo biosensing for plant biology.

Cupil-Garcia, Vanessa↗
DOE OSTI2022

Multimodal sensor fusion framework for residential building occupancy detection

For several years now, smart building energy systems have been a research area of intensive activity. In light of the increasing need for sustainable buildings and energy systems, this trend motivates an increasing need for a solution to reduce carbon dioxide emissions and improve energy efficiency. This work proposes a high-performing and transferable occupancy detection framework that combines sensor data from different data modalities, including time series environmental data (temperature, humidity, and illuminance), image data, and acoustic energy data using ensemble method. To draw out the best prediction performance in each modality, the proposed framework was developed, including various models that were designed to learn the occupancy patterns reflected in the physical data streams. To tackle the time series environmental data, we designed two variants of an occupancy detection spatiotemporal pattern network (Occ-STPN) that performs both feature level and decision level fusion, respectively. We also propose a new metric; the fading memory mean square error (FMMSE), that provides a fair evaluation and penalization of delayed occupancy predictions. Multiple open-sourced datasets, including the Electricity Consumption and Occupancy and the University of California, Irvine's (UCI) building occupancy detection dataset, along with our own real data collected from six different houses, were used to validate the algorithms' performance. The experimental results presented herein break down the performance for each sensing modality, and a detailed analysis of the performance is also discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2022

Analysis of the Spontaneous Emission Limited Linewidth of an Integrated III–V/SiN Laser

Abstract This article describes a calculation of the spontaneous emission limited linewidth of a semiconductor laser consisting of hybrid or heterogeneously integrated, silicon and III–V intracavity components. Central to the approach are a) description of the multi‐element laser cavity in terms of composite laser/free‐space eigenmodes, b) use of multimode laser theory to treat mode competition and multiwave mixing, and c) incorporation of quantum‐optical contributions to account for spontaneous emission effects. Application of the model is illustrated for the case of linewidth narrowing in an InAs quantum‐dot laser coupled to a high‐ SiN cavity.

42 ENGINEERING↗
DOE OSTI2025

Using multimodal X-ray computed tomography to advance 3D petrography: A non-destructive investigation of olivine inside a carbonaceous chondrite

Rocks form in three dimensions through time and studying them provides information from inside dynamic systems we cannot otherwise observe. Yet how we typically access the interior of the rocks themselves to gain that information may limit our understanding and influence how we reconstruct the processes that formed them. Here, we demonstrate combined non-destructive 3D X-ray imaging techniques that produce quantitative densitometric and crystallographic maps of entire individual grains inside a rock. Olivine grains throughout a sample of the carbonaceous chondrite Northwest Africa (NWA) 11346 were each characterized by size, shape, composition, zoning intensity, and crystallographic orientation. The addition of 3D crystallographic mapping to calibrated 3D densitometric analysis—used to calculate chemical composition—demonstrates a fully non-destructive petrographic method and provides unique insight. For instance, in our case, using crystallographic data to delineate individual grains and then measuring the 3D size, shape, and composition of each distinguishes variably reset relict grains from those later crystallized after a melting event. Intersection in a 2D slice could not have led to this interpretation because the integration of three-dimensional size, rounding, composition, location, and crystallographic orientation measured from each grain forms the key patterns. Multimodal laboratory X-ray imaging has strong potential to advance 3D petrography.

36 MATERIALS SCIENCE↗
DOE OSTI2025

In-device Battery Failure Analysis

Lithium-ion batteries are indispensable power sources for a wide range of modern electronic devices. However, battery lifespan remains a critical limitation, directly affecting the sustainability and user experience. Conventional battery failure analysis in controlled lab settings may not capture the complex interactions and environmental factors encountered in real-world, in-device operating conditions. Here, this study analyzes the failure of commercial wireless earbud batteries as a model system within their intended usage context. Through multiscale and multimodal characterizations, the degradations from the material level to the device level are correlated, elucidating a failure pattern that is closely tied to the specific device configuration and operating conditions. The findings indicate that the ultimate failure mode is determined by the interplay of battery materials, cell structural design, and the in-device microenvironment, such as temperature gradients and their fluctuations. This holistic, in-device perspective on environmental influences provides critical insights for battery integration design, enhancing the reliability of modern electronics.

36 MATERIALS SCIENCE↗
DOE OSTI2020

Nanoscale Mass Spectrometry Multimodal Imaging via Tip-Enhanced Photothermal Desorption

Materials ranging from adhesives, pharmaceuticals, lubricants, and personal care products are traditionally studied using macroscopic characterization techniques. However, their functionality is in reality defined by details of chemical organization on often noncrystalline matter with characteristic length scales on the order of microns to nanometers. Additionally, these materials are traditionally difficult to analyze using standard vacuum-based approaches that provide nanoscale chemical characterization due to their volatile and beam-sensitive nature. Therefore, approaches that operate under ambient conditions need to be developed that allow probing of nanoscale chemical phenomena and correlated functionality. Here, we demonstrate a tool for probing and visualizing local chemical environments and correlating them to material structure and functionality using advanced multimodal chemical imaging on a combined atomic force microscopy (AFM) and mass spectrometry (MS) system using tip-enhanced photothermal desorption with atmospheric pressure chemical ionization (APCI). We demonstrate enhanced performance metrics of the technique for correlated imaging and point sampling and illustrate the applicability for the analysis of trace chemicals on a human hair, additives in adhesives on paper, and pharmaceuticals samples notoriously difficult to analyze in a vacuum environment. Overall, this approach of correlating local chemical environments to structure and functionality is key to advancing research in many fields ranging from biology, to medicine, to material science.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2024

Correlated 4D-STEM and EDS for the classification of fine Beta-precipitates in aluminum alloy AA 6063-T6

Tuning the properties of aluminum alloys AA 6063-T6 involves artificial aging to induce precipitate formation, particularly β’’ and β’ phases. Previous characterization challenges due to their similar appearance are addressed here by correlating 4D scanning transmission electron microscopy (4DSTEM) and energy-dispersive spectroscopy (EDS) mapping. This approach allows us to analyze the structure and composition of precipitates individually, overcoming limitations of conventional imaging and structural analysis techniques when the precipitates appear simultaneously, as is often the case. We present detailed characterizations of needle-shaped Beta precipitates, revealing distinct diffraction patterns (DPs) and compositional differences. The method's applicability extends beyond aluminum alloys, offering a promising strategy for complex composite material characterization with multimodal scanning transmission electron microscopy (STEM) techniques.

36 MATERIALS SCIENCE↗