Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Large-scale cascading of first-order FBG array in a highly multimode coreless fiber using femtosecond laser for distributed thermal sensing

This research focuses on the performance analysis and characterization of a fiber Bragg gratings (FBGs) array, consisting of 10 first-order FBGs inscribed by a femtosecond (FS) laser in a highly multimode coreless fiber. The study evaluates the FBG array's ability to function as a distributed thermal sensing (DTS) platform, with the coreless fiber chosen as the sensing element due to its immunity to dopant migration at high temperatures. The design of a large cascaded first-order FBG array effectively eliminates unwanted harmonic peaks across a wide spectrum range. In contrast, higher-order FBGs introduce limitations due to the overlapping of Bragg peaks with harmonics. The FBG array's performance is evaluated by measuring the reflection spectrum of each grating at different temperatures, showing a high temperature sensitivity of 15.05 pm/°C at a Bragg wavelength of 1606.3 nm, with a linear response in the temperature range of 24 - 1100 °C. The FBG array was designed for a spatial resolution of 5 mm. A mode scrambler in the sensing network is employed, which suppresses multimodal interference, characterizes FBG peak visibility, and stabilizes the interference spectrum. The stability of the FBG array is also assessed over 24 hrs at 1100 °C, and it is observed to be stable during thermal treatment. Heat treatment at 1100°C improves the signal to noise ratio of the FBG array, demonstrating the robustness and suitability of the proposed FBG array on highly multimode coreless fiber as a potential sensing platform for DTS applications in harsh environmental conditions, overcoming the issues of dopant migration presented by dopes silica optical fibers at high temperatures.

47 OTHER INSTRUMENTATION↗

Data augmentation and multimodal learning for predicting drug response in patient-derived xenografts from gene expressions and histology images

Patient-derived xenografts (PDXs) are an appealing platform for preclinical drug studies. A primary challenge in modeling drug response prediction (DRP) with PDXs and neural networks (NNs) is the limited number of drug response samples. We investigate multimodal neural network (MM-Net) and data augmentation for DRP in PDXs. The MM-Net learns to predict response using drug descriptors, gene expressions (GE), and histology whole-slide images (WSIs). We explore whether combining WSIs with GE improves predictions as compared with models that use GE alone. We propose two data augmentation methods which allow us training multimodal and unimodal NNs without changing architectures with a single larger dataset: 1) combine single-drug and drug-pair treatments by homogenizing drug representations, and 2) augment drug-pairs which doubles the sample size of all drug-pair samples. Unimodal NNs which use GE are compared to assess the contribution of data augmentation. The NN that uses the original and the augmented drug-pair treatments as well as single-drug treatments outperforms NNs that ignore either the augmented drug-pairs or the single-drug treatments. In assessing the multimodal learning based on the MCC metric, MM-Net outperforms all the baselines. Our results show that data augmentation and integration of histology images with GE can improve prediction performance of drug response in PDXs.

60 APPLIED LIFE SCIENCES↗

Unsupervised physics-informed disentanglement of multimodal data

Here, we introduce physics-informed multimodal autoencoders (PIMA) - a variational inference framework for discovering shared information in multimodal datasets. Individual modalities are embedded into a shared latent space and fused through a product-of-experts formulation, enabling a Gaussian mixture prior to identify shared features. Sampling from clusters allows cross-modal generative modeling, with a mixture-of-experts decoder that imposes inductive biases from prior scientific knowledge and thereby imparts structured disentanglement of the latent space. This approach enables cross-modal inference and the discovery of features in high-dimensional heterogeneous datasets. Consequently, this approach provides a means to discover fingerprints in multimodal scientific datasets and to avoid traditional bottlenecks related to high-fidelity measurement and characterization of scientific datasets.

97 MATHEMATICS AND COMPUTING↗

Multimodality Image Fusion-Guided Procedures: Technique, Accuracy, and Applications

Personalized therapies play an increasingly critical role in cancer care: Image guidance with multimodality image fusion facilitates the targeting of specific tissue for tissue characterization and plays a role in drug discovery and optimization of tailored therapies. Positron-emission tomography (PET), magnetic resonance imaging (MRI), and contrast-enhanced computed tomography (CT) may offer additional information not otherwise available to the operator during minimally invasive image-guided procedures, such as biopsy and ablation. With use of multimodality image fusion for image-guided interventions, navigation with advanced modalities does not require the physical presence of the PET, MRI, or CT imaging system. Several commercially available methods of image-fusion and device navigation are reviewed along with an explanation of common tracking hardware and software. An overview of current clinical applications for multimodality navigation is provided.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Microstructure Evolution of a Multimodal Gamma-Prime Ni-Based Superalloy Characterized by In Situ Diffraction

The relationship between the evolution of microstructure, deformation micromechanisms and mechanical properties is difficult to establish in multimodal size distribution $_γ$$'$ superalloys, as the microstructure evolves with both temperature and time, and multiple strengthening mechanisms across each size distribution contribute to mechanical performance. In situ X-ray scattering can offer unparalleled insight regarding microstructure evolution at the temperatures and stresses of importance to gas-turbine applications; however, in situ X-ray diffraction has not been applied to the study of multimodal $_γ$$'$ distribution superalloys. Herein, lattice parameter evolution of secondary and tertiary $_γ$$'$ precipitates in a representative superalloy, Nimonic 115, is determined between 750 °C and 950 °C and correlated to room-temperature SEM and microhardness values. A large positive lattice parameter misfit of secondary $_γ$$'$ induces precipitate splitting, and the tertiary $_γ$$'$ goes into dissolution at ~ 800 °C, but with little apparent change in hardness values. The volume fraction of $_γ$$'$ decreases above 900 °C and precipitate-matrix coherency is lost, and there is a corresponding decrease in microhardness values. Importantly, the diffraction analysis demonstrates the capability to determine critical microstructural parameters of both precipitate size distributions in situ, representing an additional tool for determining microstructure-mechanical property relationships of multimodal superalloys.

36 MATERIALS SCIENCE↗

Evaluation of UAV-derived multimodal remote sensing data for biomass prediction and drought tolerance assessment in bioenergy sorghum

Screening for drought tolerance is critical to ensure high biomass production of bioenergy sorghum in arid or semi-arid environments. The bottleneck in drought tolerance selection is the challenge of accurately predicting biomass for a large number of genotypes. Although biomass prediction by low-altitude remote sensing has been widely investigated on various crops, the performance of the predictions are not consistent, especially when applied in a breeding context with hundreds of genotypes. In some cases, biomass prediction of a large group of genotypes benefited from multimodal remote sensing data; while in other cases, the benefits were not obvious. In this study, we evaluated the performance of single and multimodal data (thermal, RGB, and multispectral) derived from an unmanned aerial vehicle (UAV) for biomass prediction for drought tolerance assessments within a context of bioenergy sorghum breeding. The biomass of 360 sorghum genotypes grown under well-watered and water-stressed regimes was predicted with a series of UAV-derived canopy features, including canopy structure, spectral reflectance, and thermal radiation features. Biomass predictions using canopy features derived from the multimodal data showed comparable performance with the best results obtained with the single modal data with coefficients of determination (R 2 ) ranging from 0.40 to 0.53 under water-stressed environment and 0.11 to 0.35 under well-watered environment. The significance in biomass prediction was highest with multispectral followed by RGB and lowest with the thermal sensor. Finally, two well-recognized yield-based drought tolerance indices were calculated from ground truth biomass data and UAV predicted biomass, respectively. Results showed that the geometric mean productivity index outperformed the yield stability index in terms of the potential for reliable predictions by the remotely sensed data. Collectively, this study demonstrated a promising strategy for the use of different UAV-based imaging sensors to quantify yield-based drought tolerance.

09 BIOMASS FUELS↗

The Role of Ligand–Ligand Interactions in Multimodal Ligand Conformational Equilibria and Surface Pattern Formation

Multimodal chromatography uses multiple modes of interaction such as charge, hydrophobic, or hydrogen bonding to separate proteins. Recently, we used molecular dynamics (MD) simulations to show that ligands immobilized on surfaces can interact and associate with neighboring ligands to form hydrophobic and charge patches, which may have important implications for the nature of protein–surface interactions. Here, we study interfacial systems of increasing complexity—from a single immobilized multimodal ligand to high density surfaces—to better understand how ligand behavior is affected by the presence of a surface and the presence of other ligands in the vicinity, and how this behavior scales to larger systems. Furthermore, we find that tethering a ligand to a surface restricts its conformations to a subset of those observed in free solution, yet the ligand maintains flexibility in the plane of the surface and can form contacts with neighboring ligands. We find that although the formation of a contact between two neighboring ligands is slightly unfavorable, three neighboring ligands exhibit a preference for the formation of a fully connected cluster. To explore how these trends in ligand association extend to a larger surface with high density of ligands, we performed coarse-grained Monte Carlo (MC) simulations of a 132-ligand surface using ligand interactions parametrized based on free energies obtained from the three-ligand MD simulations. Despite their simplicity, the coarse-grained simulations qualitatively capture the cluster size distribution of ligands observed in detailed MD simulations. Quantitative differences between the two suggest opportunities for improvements in the coarse-grained energy function for efficient predictions of cluster and pattern formations. Our approach presents a promising route to the engineering of multimodal patterns for future chromatographic resin design.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Probing corrosion using a simple and versatile in situ multimodal corrosion measurement system

In this work, we have developed a unique in situ multimodal corrosion system that is capable of acquiring electrochemical data, sample imaging/visualization and hydrogen collection, simultaneously. Each of these modalities yield valuable information pertaining to the ongoing corrosion process. Combining them can yield holistic information on the role of microstructure, processing history, presence of coatings, etc., on the sequence of steps occurring during the corrosion process, and how they correlate with the acquired electrochemical data. Four materials systems, namely AA6061-T6 aluminum alloy, AZ91 magnesium alloy, galvanized DP590 steel, and pure Zn, were investigated under open circuit potential and under potentiodynamic polarization. The multimodal corrosion system was utilized to observe processes such as surface passivation and dissolution, pit and filiform corrosion initiation and propagation, and was correlated with location and magnitude of hydrogen evolution. This approach is shown to yield a truly multimodal understanding of the ongoing corrosion processes.

36 MATERIALS SCIENCE↗

A Representation Fusion Framework for Decoupling Diagnostic Information in Multimodal Learning

Modern medicine increasingly relies on multimodal data, ranging from clinical notes to imaging and genomics, to guide diagnosis and treatment. However, integrating these heterogeneous data sources in a principled and interpretable manner remains a major challenge. We present MODES (Multi-mOdal Disentangled Embedding Space), a representation fusion framework that explicitly separates shared and modality-specific factors of variation, offering a structured latent space for multimodal information that improves both prediction and interpretability. By leveraging pre-trained unimodal foundation models, MODES mitigates the dependency on extensive paired datasets, crucial in data-scarce clinical settings. We introduce a masking strategy that optimizes representation dimensionality by eliminating low-information dimensions, to achieve compact, information-rich representations. Our framework demonstrates superior performance in predicting diagnoses and phenotypes compared to unimodal and conventional fusion models. MODES also enables robust diagnostic inference in missing data scenarios, offering an opportunity toward interpretable and efficient multimodal diagnostics in personalized healthcare.

60 APPLIED LIFE SCIENCES↗

Automated electrosynthesis reaction mining with multimodal large language models (MLLMs)

Leveraging the chemical data available in legacy formats such as publications and patents is a significant challenge for the community. Automated reaction mining offers a promising solution to unleash this knowledge into a learnable digital form and therefore help expedite materials and reaction discovery. However, existing reaction mining toolkits are limited to single input modalities (text or images) and cannot effectively integrate heterogeneous data that is scattered across text, tables, and figures. In this work, we go beyond single input modalities and explore multimodal large language models (MLLMs) for the analysis of diverse data inputs for automated electrosynthesis reaction mining. We compiled a test dataset of 65 articles (MERMES-T24 set) and employed it to benchmark five prominent MLLMs against two critical tasks: (i) reaction diagram parsing and (ii) resolving cross-modality data interdependencies. The frontrunner MLLM achieved ≥96% accuracy in both tasks, with the strategic integration of single-shot visual prompts and image pre-processing techniques. We integrate this capability into a toolkit named MERMES (multimodal reaction mining pipeline for electrosynthesis). Our toolkit functions as an end-to-end MLLM-powered pipeline that integrates article retrieval, information extraction and multimodal analysis for streamlining and automating knowledge extraction. This work lays the groundwork for the increased utilization of MLLMs to accelerate the digitization of chemistry knowledge for data-driven research.

Leong, Shi Xuan↗

Arbitrary-ratio power splitter based on nonlinear multimode interference coupler

We propose an ultra-compact multimode interference (MMI) power splitter based on nonlinear effects from simulations using nonlinear modal propagation analysis (NMPA) cooperation with finite difference Method (FDM) to access free choice of splitting ratio. Conventional multimode interference power splitter could only obtain a few discrete ratios. The power splitting ratio may be adjusted continuously while the input set power is varying by a tunable laser. In fact, using an ultra- compact MMI with a simple structure that is launched by a tunable nonlinear input fulfills the problem of arbitrary-ratio in integrated photonics circuits. Silicon on insulator (SOI) is used as the offered material due to the high contrast refractive index and Centro symmetric properties. The high-resolution images at the end of the multimode waveguide in the simulated power splitter have a high power balance, whereas access to a free choice of splitting ratio is not possible under the linear regime in the proposed length range except changes in the dimension for any ratio. The compact dimensions and ideal performance of the device are established according to optimized parameters. The proposed regime can be extended to the design of M×N arbitrary power splitters ratio for programmable logic devices in all optical digital signal processing. The results of this study indicate that nonlinear modal propagation analysis solves the miniaturization problem for all-optical devices based on MMI couplers to achieve multiple functions in a compact planar integrated circuit and also overcomes the limitations of previously proposed methods for nonlinear MMI.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Nonlinear bubble competition of the multimode ablative Rayleigh–Taylor instability and applications to inertial confinement fusion

The self-similar nonlinear evolution of the multimode ablative Rayleigh–Taylor instability (RTI) and the ablation-generated vorticity effect are studied for a range of initial conditions. We show that, unlike classical RTI, the nonlinear multimode bubble-front evolution remains in the bubble competition regime due to ablation-generated vorticity, which accelerates the bubbles, thereby preventing a transition into the bubble-merger regime. We develop an analytical bubble competition model to describe the linear and nonlinear stages of ablative RTI. Here, we show that vorticity inside the multimode bubbles is most significant at small scales with large initial perturbation. Since these small scales persist in the bubble competition regime, the self-similar growth coefficient ab can be enhanced by up to 30% relative to ablative bubble competition without vorticity effects. We use the ablative bubble competition model to explain the hydrodynamic stability boundary observed in OMEGA low-adiabat implosion experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

ADAPTIVE ANNEALED IMPORTANCE SAMPLING FOR MULTIMODAL POSTERIOR EXPLORATION AND MODEL SELECTION WITH APPLICATION TO EXTRASOLAR PLANET DETECTION

We describe an algorithm that can adaptively provide mixture summaries of multimodal posterior distributions. The parameter space of the involved posteriors ranges in size from a few dimensions to dozens of dimensions. This work was motivated by an astrophysical problem called extrasolar planet (exoplanet) detection, wherein the computation of stochastic integrals that are required for Bayesian model comparison is challenging. The difficulty comes from the highly nonlinear models that lead to multimodal posterior distributions. We resort to importance sampling (IS) to estimate the integrals, and thus translate the problem to be how to find a parametric approximation of the posterior. To capture the multimodal structure in the posterior, we initialize a mixture proposal distribution and then tailor its parameters elaborately to make it resemble the posterior to the greatest extent possible. We use the effective sample size (ESS) calculated based on the IS draws to measure the degree of approximation. The bigger the ESS is, the better the proposal resembles the posterior. A difficulty within this tailoring operation lies in the adjustment of the number of mixing components in the mixture proposal. Brute force methods just preset it as a large constant, which leads to an increase in the required computational resources. We provide an iterative delete/merge/add process, which works in tandem with an expectation-maximization step to tailor such a number online. The efficiency of our proposed method is tested via both simulation studies and real exoplanet data analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

Identification of Blue Horizontal Branch Stars with Multimodal Fusion

Blue Horizontal Branch stars (BHBs) are ideal tracers to probe the global structure of the milky Way (MW), and the increased size of the BHB star sample could be helpful to accurately calculate the MW's enclosed mass and kinematics. Large survey telescopes have produced an increasing number of astronomical images and spectra. However, traditional methods of identifying BHBs are limited in dealing with the large scale of astronomical data. A fast and efficient way of identifying BHBs can provide a more significant sample for further analysis and research. Therefore, in order to fully use the various data observed and further improve the identification accuracy of BHBs, we have innovatively proposed and implemented a Bi-level attention mechanism-based Transformer multimodal fusion model, called Bi-level Attention in the Transformer with Multimodality (BATMM). The model consists of a spectrum encoder, an image encoder, and a Transformer multimodal fusion module. The Transformer enables the effective fusion of data from two modalities, namely image and spectrum, by using the proposed Bi-level attention mechanism, including cross-attention and self-attention. As a result, the information from the different modalities complements each other, thus improving the accuracy of the identification of BHBs. The experimental results show that the F1 score of the proposed BATMM is 94.78%, which is 21.77% and 2.76% higher than the image and spectral unimodality, respectively. It is therefore demonstrated that higher identification accuracy of BHBs can be achieved by means of using data from multiple modalities and employing an efficient data fusion strategy.

79 ASTRONOMY AND ASTROPHYSICS↗

Intensity correlations, entanglement properties, and ghost imaging in multimode thermal-seeded parametric down-conversion: Theory

We address parametric down-conversion seeded by multimode pseudothermal fields. We show that this process may be used to generate multimode pairwise correlated states with entanglement properties that can be tuned by controlling the seed intensities. Parametric down-conversion seeded by multimode pseudothermal fields represents a source of correlated states, which allows one to explore the classical-quantum transition in pairwise correlations and to realize ghost imaging and ghost diffraction in regimes not yet explored by experiments.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Multimode model for projective photon-counting measurements

We present a general model to account for the multimode nature of the quantum electromagnetic field in projective photon-counting measurements. We focus on photon-subtraction experiments, where non-Gaussian states are produced conditionally. These are useful states for continuous-variable quantum-information processing. We present a general method called mode reduction that reduces the multimode model to an effective two-mode problem. We apply this method to a multimode model describing broadband parametric down-conversion, thereby improving the analysis of existing experimental results. The main improvement is that spatial and frequency filters before the photon detector are taken into account explicitly. We find excellent agreement with previously published experimental results, using fewer free parameters than before, and discuss the implications of our analysis for the optimized production of states with negative Wigner functions.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Testing nonclassicality in multimode fields: A unified derivation of classical inequalities

We consider a way to generate operational inequalities to test nonclassicality (or quantumness) of multimode bosonic fields (or multiparty bosonic systems) that unifies the derivation of many known inequalities and allows to propose new ones. The nonclassicality criteria are based on Vogel's criterion corresponding to analyzing the positivity of multimode P functions or, equivalently, the positivity of matrices of expectation values of, e.g., creation and annihilation operators. We analyze not only monomials but also polynomial functions of such moments, which can sometimes enable simpler derivations of physically relevant inequalities. As an example, we derive various classical inequalities which can be violated only by nonclassical fields. In particular, we show how the criteria introduced here easily reduce to the well-known inequalities describing (a) multimode quadrature squeezing and its generalizations, including sum, difference, and principal squeezing; (b) two-mode one-time photon-number correlations, including sub-Poisson photon-number correlations and effects corresponding to violations of the Cauchy-Schwarz and Muirhead inequalities; (c) two-time single-mode photon-number correlations, including photon antibunching and hyperbunching; and (d) two- and three-mode quantum entanglement. Other simple inequalities for testing nonclassicality are also proposed. We have found some general relations between the nonclassicality and entanglement criteria, in particular those resulting from the Cauchy-Schwarz inequality. It is shown that some known entanglement inequalities can be derived as nonclassicality inequalities within our formalism, while some other known entanglement inequalities can be seen as sums of more than one inequality derived from the nonclassicality criterion. This approach enables a deeper analysis of the entanglement for a given nonclassicality.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Seamless High-Q Microwave Cavities for Multimode Circuit Quantum Electrodynamics

Multimode cavity quantum electrodynamics ---where a two-level system interacts simultaneously with many cavity modes---provides a versatile framework for quantum information processing and quantum optics. Due to the combination of long coherence times and large interaction strengths, one of the leading experimental platforms for cavity QED involves coupling a superconducting circuit to a 3D microwave cavity. In this work, we realize a 3D multimode circuit QED system with single photon lifetimes of $2$ ms and cooperativities of $0.5-1.5\times10^9$ across 9 modes of a novel seamless cavity. We demonstrate a variety of protocols for universal single-mode quantum control applicable across all cavity modes, using only a single drive line. We achieve this by developing a straightforward flute method for creating monolithic superconducting microwave cavities that reduces loss while simultaneously allowing control of the mode spectrum and mode-qubit interaction. We highlight the flexibility and ease of implementation of this technique by using it to fabricate a variety of 3D cavity geometries, providing a template for engineering multimode quantum systems with exceptionally low dissipation. This work is an important step towards realizing hardware efficient random access quantum memories and processors, and for exploring quantum many-body physics with photons.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗