Search NASA⌕ Search

SEARCH · Search NASA

Results for “Multimodal analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Towards verifiable cancer digital twins: tissue level modeling protocol for precision medicine

Cancer exhibits substantial heterogeneity, manifesting as distinct morphological and molecular variations across tumors, which frequently undermines the efficacy of conventional oncological treatments. Developments in multiomics and sequencing technologies have paved the way for unraveling this heterogeneity. Nevertheless, the complexity of the data gathered from these methods cannot be fully interpreted through multimodal data analysis alone. Mathematical modeling plays a crucial role in delineating the underlying mechanisms to explain sources of heterogeneity using patient-specific data. Intra-tumoral diversity necessitates the development of precision oncology therapies utilizing multiphysics, multiscale mathematical models for cancer. This review discusses recent advancements in computational methodologies for precision oncology, highlighting the potential of cancer digital twins to enhance patient-specific decision-making in clinical settings. We review computational efforts in building patient-informed cellular and tissue-level models for cancer and propose a computational framework that utilizes agent-based modeling as an effective conduit to integrate cancer systems models that encode signaling at the cellular scale with digital twin models that predict tissue-level response in a tumor microenvironment customized to patient information. Furthermore, we discuss machine learning approaches to building surrogates for these complex mathematical models. These surrogates can potentially be used to conduct sensitivity analysis, verification, validation, and uncertainty quantification, which is especially important for tumor studies due to their dynamic nature.

60 APPLIED LIFE SCIENCES↗
DOE OSTI2023

Multimode theory of electron hole transverse instability

We present Vlasov–Poisson three-dimensional linear stability analysis of an initially planar electron hole structure, solving for the distribution function by integration along unperturbed orbits. The non-sinusoidal potential perturbation shape (parallel to $B$ ) is expanded in eigenfunctions of the adiabatic Poisson operator, generalizing the prior assumption of a rigid shift of the equilibrium. We show that the shiftmode is then modified by a second discrete mode plus an integral over a continuum of wave-like modes. A rigorous treatment shows that the continuum can be approximated effectively by a single mode that satisfies the external wave dispersion relation, thus making the perturbation a weighted sum of three modes. We find numerically the solution for the complex instability frequency, and the corresponding three mode amplitudes determining the perturbation eigenmode. This multimode analysis refines the accuracy of the prior single-mode results, giving slightly higher growth rates at most parameters, as expected from the extra mode shape freedom. Oscillating modes near stability boundaries have larger mode distortions which help explain particle-in-cell simulations that observe instability up to ${\sim }20$ % beyond the prior shiftmode thresholds, and narrowing of the perturbation. At high magnetic field, the multimode analysis predicts a reduction of the already small growth rate.

Physics↗
DOE OSTI2024

Automated electrosynthesis reaction mining with multimodal large language models (MLLMs)

Leveraging the chemical data available in legacy formats such as publications and patents is a significant challenge for the community. Automated reaction mining offers a promising solution to unleash this knowledge into a learnable digital form and therefore help expedite materials and reaction discovery. However, existing reaction mining toolkits are limited to single input modalities (text or images) and cannot effectively integrate heterogeneous data that is scattered across text, tables, and figures. In this work, we go beyond single input modalities and explore multimodal large language models (MLLMs) for the analysis of diverse data inputs for automated electrosynthesis reaction mining. We compiled a test dataset of 65 articles (MERMES-T24 set) and employed it to benchmark five prominent MLLMs against two critical tasks: (i) reaction diagram parsing and (ii) resolving cross-modality data interdependencies. The frontrunner MLLM achieved ≥96% accuracy in both tasks, with the strategic integration of single-shot visual prompts and image pre-processing techniques. We integrate this capability into a toolkit named MERMES (multimodal reaction mining pipeline for electrosynthesis). Our toolkit functions as an end-to-end MLLM-powered pipeline that integrates article retrieval, information extraction and multimodal analysis for streamlining and automating knowledge extraction. This work lays the groundwork for the increased utilization of MLLMs to accelerate the digitization of chemistry knowledge for data-driven research.

Leong, Shi Xuan↗
DOE OSTI2021

Recurrence analysis of friction based dry-couplant ultrasonic Lamb waves in plate-like structures

Here, in this study, the effect of friction on the generation of dry-coupled Lamb waves is experimentally investigated. Recurrence analysis is performed to analyze the complex behavior of friction based dry-coupled Lamb waves. In particular, the effect of the normal force, which is necessary for a stronger dry-coupled Lamb wave generation and the friction, on the transmission of mechanical energy and determinism characteristics of Lamb waves are investigated. The results verify that larger friction coefficient and friction force are crucial for generation and propagation of strong Lamb waves supporting the fact that the main mechanism to transfer mechanical energy using dry-couplant is friction. The sensitivity of Lamb waves to the friction coefficient, highlights the importance of designing specific pads with respect to condition of the surface. Besides, the results show that the normal force and friction coefficient can change the determinism characteristics behavior of multimode Lamb waves. Furthermore, it is shown that the determinism value is sensitive to the friction coefficient and normal force. A similar trend is observed in the determinism values and friction coefficient. In general, a smaller friction coefficient indicates smaller determinism value. Additionally, it is shown that a normal load can change the behavior of a system, as observed from recurrence plots, owing to changes in the Lamb waves trajectories in the phase-space domain. In addition, it is shown that recurrence plots enable the detection of mode transitions in multimode Lamb waves. Recurrence analysis is a complementary tool to frequency domain methods for accurate analysis of multimode Lamb waves behavior.

42 ENGINEERING↗
DOE OSTI2022

Multimodal Freight Energy Model for Emerging Freight Technology Analysis

While freight movement represents a small portion of the volume in the transportation sector, they are a critical contributor for energy consumed in that sector. To reduce logistics costs, energy consumption and negative environmental impact, emerging technologies, such as digitalization of logistics and alternative powertrain, have been developed and extended applications in freight. These trends are expected to grow and provide opportunities for greater efficiencies in freight movement and corresponding energy use. Still, the complexity of freight systems present challenges in evaluating these benefits, especially in the multimodal inter-city freight. Addressing the research need, this paper develops a multimodal freight energy modeling framework for the analysis of emerging freight technology scenarios. The framework is a bi-level optimization problem: network cost minimization problem (lower-level) and energy minimization problem (upper-level). The lower-level problem is a mode-path assignment problem in multimodal inter-city freight networks, where commodity-specific congestion effects on trans-shipment links are considered. For this model, an inverse modeling approach is applied to infer parameters of the lower-level model. The upper-level problem is designed to search for an optimal scenario that has the lowest energy consumption among different levels of technology applications. The proposed model is empirically tested to analyze truck load-pooling and multimodal load-pooling scenarios through stand-alone and mixed applications using freight shipments originating from or destined to the Chicago region. This framework can be used to explore the impact of emerging freight technologies on mode-path freight flow and energy consumption in the national multimodal freight network.

ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

From Data to Insights: A Covariate Analysis of the IARPA BRIAR Dataset for Multimodal Biometric Recognition Algorithms at Altitude and Range

This paper examines covariate effects on fused whole body biometrics performance in the IARPA BRIAR dataset, specifically focusing on UAV platforms, elevated positions, and distances up to 1000 meters. The dataset includes outdoor videos compared with indoor images and controlled gait recordings. Normalized raw fusion scores relate directly to predicted false accept rates (FAR), offering an intuitive means for interpreting model results. A linear model is developed to predict biometric algorithm scores, analyzing their performance to identify the most influential covariates on accuracy at altitude and range. Weather factors like temperature, wind speed, solar loading, and turbulence are also investigated in this analysis. The study found that resolution and camera distance best predicted accuracy and findings can guide future research and development efforts in long-range/elevated/UAV biometrics and support the creation of more reliable and robust systems for national security and other critical domains.

Bolme, David↗
DOE OSTI2020

Provenance infrastructure for multimodal x-ray experiments and reproducible analysis

Exploring relationships between material structures and desired properties for material discovery and synthesis requires mining machine-readable databases and linking experimental data with sufficiently rich annotations. There are vast gaps both in these data sources and in the connections between them. We present a provenance-based data management and analysis framework that enables capturing, persisting, and reanalyzing experimental data from the BNL National Synchrotron Light Source II (NSLS-II). Our system leverages NSLS-II Bluesky for experimental data acquisition, captures analysis parameters, and enriches the search space with results from the scientific literature. We describe in detail the challenges and benefits of data-driven approaches when comparing computational and experimental data, such as the multiple possible scenarios of interest in often ill-posed inverse problems and missing data. Our infrastructure utilizes the NSLS-II event model and includes discovery mechanisms across multiple beamlines. A text and data mining module using natural language processing classification techniques extracts relevant information from the literature that is added to data sources. We describe a provenance acquisition and analysis rerun workflow based on a directed acyclic graph that keeps track of process and data provenance and allows to replay analysis.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Topological Signatures of Adversaries in Multimodal Alignments

Topological Data Analysis for Adversarial Detection (LANL O4937) - Detects adversarial examples in vision-language models using persistent homology and two-sample testing. Combines TDA features from CLIP embeddings with statistical methods (ME, SCF, SAMMD, C2ST) for robust detection across ImageNet, CIFAR-10/100.

Bhattarai, Manish↗
DOE OSTI2025

An Integrated ML/AI Framework for Digitizing, Structuring and Searching DOE U-TRU-Fuels Data with Gap Analysis of Non-DOE Records

The U.S. Department of Energy (DOE) Advanced Fuels Campaign (AFC) is advancing transmutation fuel technologies to reduce long-lived radioactive waste by converting minor actinides into shorter-lived or stable elements through irradiation in sodium-cooled fast reactors. Key experiments such as AFC-1, AFC-2, FUels for the transmutation of Trans-URanium elements In phéniX (FUTURIX)-Fortes Teneurs en Actinides (FTA), and Experimental Breeder Reactor-II (EBR-II) X501 have provided fuel fabrication, irradiation, and performance data on various transuranic-bearing fuel forms. This report documents the creation of an artificial-intelligence assisted database, which has consolidated all DOE-owned data related to Transuranic (TRU)-bearing fuel experiments and stored across it across both the Idaho National Laboratory (INL) Nuclear Data Management and Analysis System and the INL high performance computing (HPC) infrastructure. A dedicated webpage, hosted on the INL HPC system, has been developed to support role-based access and data interaction. The database architecture allows researchers to navigate large, heterogeneous archives with far greater speed and accuracy than manual search and lays the foundation for future expansion into multimodal nuclear materials analysis environments. The database represents a major step towards a nationally integrated fuels database utilizing artificial intelligence tools.

11 - NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗
DOE OSTI2025

Pilot-Scale Validation of Distributed Optical Fiber Sensors for Underground Pipeline Monitoring

Monitoring parameters such as hoop strain, pressure, and acoustic vibrations is key to detecting potential leaks, intrusions, or structural issues. Distributed optical fiber sensor (DOFS) systems provide a compelling solution for continuous, real-time monitoring over long distances. This paper details the development and pilot-scale implementation of DOFS systems for underground pipeline monitoring, evolving from a proof-of-concept stage. Multiple custom-designed DOFS interrogator units—such as optical frequency-domain reflectometry (OFDR), Brillouin optical time-domain analysis (BOTDA), and multimodal interferometer-based fiber acoustic sensor systems were tested to measure the key parameters, such as hoop strain, pipe pressure, surrounding soil temperature, and acoustic vibrations. The underground product pipeline’s outer diameter is 30 inches, the wall thickness is 1.28 inches, and 3 feet deep from the surface. The fiber deployment strategies and sensing data acquisition methods for these systems are discussed. The results demonstrate the effectiveness of DOFS in detecting hoop strain, temperature changes, and acoustic vibrations, showcasing their potential for real-time monitoring and enhancing pipeline safety. These findings from pilot-scale testing offer valuable insights into advancing pipeline monitoring technologies and improving the reliability of underground pipeline systems.

fiber optic sensors↗
DOE OSTI2025

Pilot-Scale Validation of Distributed Optical Fiber Sensors for Underground Pipeline Monitoring

Distributed fiber optic sensing is a cutting-edge technology that has found extensive applications in the monitoring of Ensuring the safety, integrity, and operational efficiency of underground product pipelines is vital for maintaining the nation’s critical infrastructure. Monitoring parameters such as hoop strain, pressure, and acoustic vibrations is key to detecting potential leaks, intrusions, or structural issues. Distributed optical fiber sensor (DOFS) systems provide a compelling solution for continuous, real-time monitoring over long distances. This paper details the development and pilot-scale implementation of DOFS systems for underground pipeline monitoring, evolving from a proof-of-concept stage. Multiple custom-designed DOFS interrogator units—such as optical frequency-domain reflectometry (OFDR), Brillouin optical time-domain analysis (BOTDA), and multimodal interferometer-based fiber acoustic sensors—were employed to measure key parameters like hoop strain, pressure, and acoustic vibrations. The underground product pipeline's outer diameter is 30 inches, the wall thickness is 1.28 inches, and the 3-foot depth. The fiber deployment strategies, and sensing data acquisition methods for these systems are discussed. The results demonstrate the effectiveness of DOFS in detecting hoop strain, temperature changes, and acoustic vibrations, showcasing their potential for real-time monitoring and enhancing pipeline safety.

distributed fiber sensing↗
DOE OSTI2024

Automated Grain Boundary (GB) Segmentation and Microstructural Analysis in 347H Stainless Steel Using Deep Learning and Multimodal Microscopy

Austenitic 347H stainless steel offers superior mechanical properties and corrosion resistance required for extreme operating conditions such as high temperature. The change in microstructure due to composition and process variations is expected to impact material properties. Identifying microstructural features such as grain boundaries thus becomes an important task in the process-microstructure-properties loop. Applying convolutional neural network (CNN)-based deep learning models is a powerful technique to detect features from material micrographs in an automated manner. In contrast to microstructural classification, supervised CNN models for segmentation tasks require pixel-wise annotation labels. However, manual labeling of the images for the segmentation task poses a major bottleneck for generating training data and labels in a reliable and reproducible way within a reasonable timeframe. Microstructural characterization especially needs to be expedited for faster material discovery by changing alloy compositions. Here, in this study, we attempt to overcome such limitations by utilizing multimodal microscopy to generate labels directly instead of manual labeling. We combine scanning electron microscopy images of 347H stainless steel as training data and electron backscatter diffraction micrographs as pixel-wise labels for grain boundary detection as a semantic segmentation task. The viability of our method is evaluated by considering a set of deep CNN architectures. We demonstrate that despite producing instrumentation drift during data collection between two modes of microscopy, this method performs comparably to similar segmentation tasks that used manual labeling. Additionally, we find that naïve pixel-wise segmentation results in small gaps and missing boundaries in the predicted grain boundary map. By incorporating topological information during model training, the connectivity of the grain boundary network and segmentation performance is improved. Finally, our approach is validated by accurate computation on downstream tasks of predicting the underlying grain morphology distributions which are the ultimate quantities of interest for microstructural characterization.

36 MATERIALS SCIENCE↗
DOE OSTI2024

CrossMP: Enabling Cross-Modality Translation between Single-Cell RNA-Seq and Single-Cell ATAC-Seq through Web-Based Portal

In recent years, there has been a growing interest in profiling multiomic modalities within individual cells simultaneously. One such example is integrating combined single-cell RNA sequencing (scRNA-seq) data and single-cell transposase-accessible chromatin sequencing (scATAC-seq) data. Integrated analysis of diverse modalities has helped researchers make more accurate predictions and gain a more comprehensive understanding than with single-modality analysis. However, generating such multimodal data is technically challenging and expensive, leading to limited availability of single-cell co-assay data. Here, we propose a model for cross-modal prediction between the transcriptome and chromatin profiles in single cells. Our model is based on a deep neural network architecture that learns the latent representations from the source modality and then predicts the target modality. It demonstrates reliable performance in accurately translating between these modalities across multiple paired human scATAC-seq and scRNA-seq datasets. Additionally, we developed CrossMP, a web-based portal allowing researchers to upload their single-cell modality data through an interactive web interface and predict the other type of modality data, using high-performance computing resources plugged at the backend.

59 BASIC BIOLOGICAL SCIENCES↗
DOE OSTI2021

Transverse mode analysis for free-space laser beams using Bayesian analysis

Reliable and rapid assessment of the transverse mode quality of a free-space laser beam has a wide range of applications in laser development, research, and utilization. It has become even more important with recent advances in developing orbital angular momentum photon beams across a broad spectral region. In this work, a general modal analysis method for a free-space multimode laser beam has been developed based on Bayesian analysis. After transforming mode decomposition into a linear system problem, a Gaussian probabilistic model is used to find a closed-form solution. The method is found to be robust with the presence of Gaussian noise. Prior knowledge about the mode content can be incorporated into the method to improve the solution for situations when coherent disturbances or contamination are present in the laser beam. This method can be used to analyze the mode content for laser beams in different bases, such as Hermite–Gaussian (HG) modes and Laguerre–Gaussian (LG) modes. Three applications of this method are presented: a detailed modal analysis of the beam image from the incoherent intensity addition of HG modes and two examples of mode decomposition using the complex wavefront from the coherent superposition of HG and LG modes. The feasibility of this method is demonstrated using various simulation results. Based on digital images of a laser beam recorded without complex wavelength-limiting optics, in principle, this method can be used in a wide wavelength range from infrared to ultraviolet, and possibly x ray.

Liu, Peifan (ORCID:0000000273816153)↗
DOE OSTI2025

Mind the gap: Bridging the divide between AI aspirations and the reality of autonomous microscopy

What does materials science look like in the “Age of Artificial Intelligence?” Each material’s domain—synthesis, characterization, and modeling—has a different answer to this question, motivated by unique challenges and constraints. This work focuses on the tremendous potential of autonomous characterization within electron microscopy. We present our recent advancements in developing domain-aware, multimodal models for microscopy analysis capable of describing complex atomic systems. We then address the critical gap between the theoretical promise of autonomous microscopy and its current practical limitations, showcasing recent successes while highlighting the necessary developments to achieve robust, real-world autonomy.

2D materials↗