Search NASA⌕ Search

SEARCH · Search NASA

Results for “Multimode”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4
All sourcesNASA NTRSDOE OSTI
NASA NTRS2016

Multimode Directional Coupler

A multimode directional coupler is provided. In some embodiments, the multimode directional coupler is configured to receive a primary signal and a secondary signal at a first port of a primary waveguide. The primary signal is configured to propagate through the primary waveguide and be outputted at a second port of the primary waveguide. The multimode directional coupler also includes a secondary waveguide configured to couple the secondary signal from the primary waveguide with no coupling of the primary signal into the secondary waveguide. The secondary signal is configured to propagate through the secondary waveguide and be outputted from a port of the secondary waveguide.

Simons, Rainee N.↗
DOE OSTI2023

An Agenda for Multimodal Foundation Models for Earth Observation

Archives of remote sensing (RS) data are increasing swiftly as new sensing modalities with enhanced spatiotemporal resolution become operational. While promising new breakthroughs, the sheer volume of RS archives stretches the limits of human analysts and existing AI tools, as most models are: i) limited to single data modalities; ii) task-specific; iii) heavily reliant on labeled data. The emerging Foundation Models (FMs) have the potential to address these limitations. Trained on vast unlabeled datasets through self-supervised learning, FMs enable generic feature extraction that facilitate specialization to a wide variety of downstream tasks. This paper describes a vision towards an FM for multimodal Earth Observation data (FM4EO), discussing key building blocks and open challenges. We put particular emphasis on multimodal reasoning, a topic underexplored in EO. Our ultimate goal is a practical path toward FM4EO with capacity to unlock breakthroughs in few-shot learning scenarios, multimodal geographic knowledge integration, synthesis, and hypothesis generation.

Ambrozio Dias, Philipe↗
DOE OSTI2024

SCITUNE: Aligning Large Language Models with Human-Curated Scientific Multimodal Instructions

Instruction finetuning is a popular paradigm to align large language models (LLM) with human intent. Despite its popularity, this idea is less explored in improving the LLMs to align existing foundation models with scientific disciplines, concepts and goals. In this work, we present SciTune as a tuning framework to improve the ability of LLMs to follow scientific multimodal instructions. To test our methodology, we use a human-generated scientific instruction tuning dataset and train a large multimodal model LLaMA-SciTune that connects a vision encoder and LLM for science-focused visual and language understanding. LLaMA-SciTune significantly outperforms the state-of-the-art models in the generated figure types and captions in multiple scientific multimodal benchmarks. In comparison to the models that are fine-tuned with machine generated data only, LLaMA-SciTune surpasses human performance on average and in many sub-categories on the ScienceQA benchmark.

• Artificial intelligence (AI) / machine learning ↗
DOE OSTI2025

Fast Sideband Control of a Weakly Coupled Multimode Bosonic Memory

Circuit quantum electrodynamics (cQED) with superconducting cavities coupled to nonlinear circuits like transmons offers a promising platform for hardware-efficient quantum information processing. We address critical challenges in realizing this architecture by weakening the dispersive coupling while also demonstrating fast, high-fidelity multimode control by dynamically amplifying gate speeds through transmon-mediated sideband interactions. This approach enables transmon-cavity SWAP gates, for which we achieve speeds up to 30 times larger than the bare dispersive coupling. Combined with transmon rotations, this allows for efficient, universal state preparation in a single cavity mode, though achieving unitary gates and extending control to multiple modes remains a challenge. In this work, we overcome this by introducing two sideband control strategies: (1) a shelving technique that prevents unwanted transitions by temporarily storing populations in sideband-transparent transmon states and (2) a method that exploits the dispersive shift to synchronize sideband transition rates across chosen photon-number pairs to implement transmon-cavity SWAP gates that are selective on photon number. We leverage these protocols to prepare Fock and binomial code states across any of ten modes of a multimode cavity with millisecond cavity coherence times. We demonstrate the encoding of a qubit from a transmon into arbitrary vacuum and Fock state superpositions, as well as entangled NOON states of cavity mode pairs— a scheme extendable to arbitrary multimode Fock encodings. Furthermore, we implement a new binomial encoding gate that converts arbitrary transmon superpositions into binomial code states in $\qty{4}{\micro\second}$ (less than $1/\chi$), achieving an average post-selected final state fidelity of $\qty{96.3}{\percent}$ across different fiducial input states.

Huang, Jordan [Rutgers U., Piscataway]↗
DOE OSTI2026

FATHOMS-RAG: A Framework for the Assessment of Thinking and Observation in Multimodal Systems that use Retrieval Augmented Generation

Retrieval-augmented generation (RAG) has emerged as a promising paradigm for improving factual accuracy in large language models (LLMs). We introduce a benchmark designed to evaluate RAG pipelines as a whole, evaluating a pipelines ability to ingest several modalities of information. We present (1) a curated dataset of 93 questions designed to evaluate a pipeline's ability to ingest textual data, tables, images, multimodal data, and cross-document multimodal data; (2) a phrase-level recall metric for correctness; (3) a nearest-neighbor embedding classifier in an attempt to classify pipeline hallucinations; (4) a comparative evaluation of 2 pipelines built with open-source retrieval mechanisms and 4 closed-source foundational models; and (5) a third-party human evaluation of the alignment of our correctness and hallucination metrics. We find that closed-source pipelines significantly outperform open-source pipelines in both the correctness and halucination metrics, with a wider performance gap in questions relying on multimodal and cross-document information. We also find after a human evaluation of our correctness and hallucination metric compared with our questions and pipeline responses, average agreement was 4.62 for correctness 4.53 for hallucination detection on a 1-5 Likert scale with 5 being strongly agree with our determination.

Hildebrand, Samuel [ORNL] (ORCID:0009000465963104)↗
DOE OSTI2020

Multimodal Imaging of Autofluorescent Sites Reveals Varied Chemical Speciation in SSZ-13 Crystals

In this report a multimodal imaging study of chabazite is used to analyse emissive deposits. A notable difference is seen in the morphology of agglomerated surface deposits and larger subsurface deposits. Comparison of the distribution of organic residue throughout the crystal volume and XRF mapping is demonstrating that a fluorescence-based technique can also be used to indirectly comment on the compositional chemistry of the inorganic framework. A multimodal imaging study of chabazite is used to show the distribution of and discriminate between different emissive deposits arising as a result of the detemplation process. Confocal imaging, 3D fluorescence lifetime imaging, 3D multispectral fluorescence imaging, and Raman mapping are used to show three different types of emissive behaviours each characterised by different spatial distributions, trends in lifetime, spectral signals, and Raman signatures. A notable difference is seen in the morphology of agglomerated surface deposits and larger subsurface deposits, which experience lifetime augmentation due to spatial confinement. The distribution of organic residue throughout the crystal volume is comparable to XRF mapping that shows Si enrichment on the outer edges and higher Al content through the centre, demonstrating that a fluorescence-based technique can also be used to indirectly comment on the compositional chemistry of the inorganic framework.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2023

Sputter-Deposited Mo Thin Films: Multimodal Characterization of Structure, Surface Morphology, Density, Residual Stress, Electrical Resistivity, and Mechanical Response

Multimodal datasets of materials are rich sources of information which can be leveraged for expedited discovery of process–structure–property relationships and for designing materials with targeted structures and/or properties. For this data descriptor article, we provide a multimodal dataset of magnetron sputter-deposited molybdenum (Mo) thin films, which are used in a variety of industries including high temperature coatings, photovoltaics, and microelectronics. In this dataset we explored a process space consisting of 27 unique combinations of sputter power and Ar deposition pressure. Here, the phase, structure, surface morphology, and composition of the Mo thin films were characterized by x-ray diffraction, scanning electron microscopy, atomic force microscopy, and Rutherford backscattering spectrometry. Physical properties—namely, thickness, film stress and sheet resistance—were also measured to provide additional film characteristics and behaviors. Additionally, nanoindentation was utilized to obtain mechanical load-displacement data. The entire dataset consists of 2072 measurements including scalar values (e.g., film stress values), 2D linescans (e.g., x-ray diffractograms), and 3D imagery (e.g., atomic force microscopy images). An additional 1889 quantities, including film hardness, modulus, electrical resistivity, density, and surface roughness, were derived from the experimental datasets using traditional methods. Minimal analysis and discussion of the results are provided in this data descriptor article to limit the authors’ preconceived interpretations of the data. Overall, the data modalities are consistent with previous reports of refractory metal thin films, ensuring that a high-quality dataset was generated. The entirety of this data is committed to a public repository in the Materials Data Facility.

36 MATERIALS SCIENCE↗
DOE OSTI2022

Multimodal analysis and characterization of the boehmite layer formed on AA6061 before and after alkaline etching

Low-enriched uranium (LEU) alloyed with 10% Mo (U-10Mo) is being considered as a promising alternative to oxide-based dispersion fuel with high-enriched uranium for use in research reactors. The configuration of this proposed LEU monolithic LEU fuel plate consists of a U-10Mo plate-type fuel foil with a 25 µm zirconium interlayer barrier clad with an aluminum alloy (AA6061). In certain research reactors, the clad AA6061 is coated with a boehmite layer to prevent corrosion. The boehmite layer has a high-pH passivation range, which makes it resistant to oxidation. Boehmite is usually formed on the AA6061 surface by autoclave processing. Before the boehmite layer is added, the surface of the AA6061 is cleaned using techniques such as polishing and wet etching. In this study, we use multimodal analysis to examine how pretreatment of AA6061 using polishing followed by alkaline etching affects the chemical composition of the boehmite layer. X-ray photoelectron microscopy (XPS), transmission electron microscopy (TEM), and x-ray diffraction (XRD) were used to study the chemical changes in the boehmite layer caused by alkaline etching pretreatment. XPS provides quantitative analysis for the Al:O ratio as well as oxidation states present on the surface, which suggests slight oxidation of the boehmite surface after alkaline etching of the AA6061 surface. We further explored this suggested oxidation of the boehmite surface using high-resolution transmission electron microscopy with selected area electron diffraction (SAED) and grazing incidence x-ray diffraction (GI-XRD), which suggested only a small amount of aluminum oxide at the surface. The multimodal analysis and imaging yielded new insights for optimizing boehmite growth on AA6061 for research reactors.

36 MATERIALS SCIENCE↗
DOE OSTI2026

Cascading economic losses from port disruptions under capacity constrained multimodal freight networks

This study quantifies how throughput disruptions at major seaports cascade through capacity-constrained multimodal freight networks and interregional production systems. We couple an agent-based model (ABM) multimodal freight simulation that resolves rerouting, terminal queueing, and inventory drawdown under binding modal and facility capacities with a multiregional output loss input-output (MRIIM) model that propagates realized delivery shortfalls across regions and sectors. The framework is demonstrated for the Port of Los Angeles using Freight Analysis Framework flows and Bureau of Economic Analysis input-output accounts and is evaluated over a 52-week horizon under deterministic sector targeted shocks and stochastic disruption realizations with uncertain severity and duration. Results indicate nonlinear amplification: realized national losses concentrate in manufacturing and transportation/warehousing even when exogenous port shocks are dispersed, suggesting that congestion spillback and limited short-run substitution can dominate the initial shock allocation. We further evaluate a tabular reinforcement-learning (Q-learning) intervention layer that selects among a small set of implementable system level levers (truck-to-rail and truck-to-barge shift settings) without overriding shipper routing, finding that such interventions reduce total losses for moderate disruptions but yield diminishing returns once substitute modes approach capacity. By linking operational freight behavior to system wide impacts under uncertainty, the proposed ABM-MRIIM pipeline provides a reusable workflow for port disruption stress testing, identification of structurally critical sectors/corridors, and evaluation of resilience interventions under realistic capacity limits.

42 ENGINEERING↗
DOE OSTI2026

Revealing Local Structures through Machine-Learning-Fused Multimodal Spectroscopy

Atomistic structures of materials offer valuable insights into their functionality. Determining these structures remains a fundamental challenge in materials science, especially for systems with defects. While both experimental and computational methods exist, each has limitations in resolving nanoscale structures. Core-level spectroscopies, such as X-ray absorption (XAS) or electron energy-loss spectroscopies (EELS), have been used to determine the local bonding environment and structure of materials. Recently, machine learning (ML) methods have been applied to extract structural and bonding information from XAS/EELS data. However, frameworks relying solely on a single data stream, defined as characterization data derived from a single element using one technique, are often insufficient because multiple local environments can yield similar spectral features, making it challenging to differentiate between competing structural hypotheses. Here, in this work, we address this challenge by integrating multimodal ab initio simulations, experimental data acquisition, and ML techniques for structure characterization. Our goal is to determine local structures and properties using EELS and XAS data from multiple elements and edges. To showcase our approach, we use various lithium nickel manganese cobalt (NMC) oxide compounds which are used for lithium ion batteries, including those with oxygen vacancies and antisite defects, as the sample material system. We successfully inferred local element content, ranging from lithium to transition metals, with quantitative agreement with experimental data. Beyond local element inference, we find that ML model based on multimodal spectroscopic data is able to determine whether local defects such as oxygen vacancy and antisites are present, a task which is impossible for single mode spectra or other experimental techniques. Furthermore, our framework is able to provide physical interpretability, bridging spectroscopy with the local atomic and electronic structures.

battery↗
DOE OSTI2020

Multimodal Deep Learning for Flaw Detection in Software Programs

We explore the use of multiple deep learning models for detecting flaws in software programs. Current, standard approaches for flaw detection rely on a single representation of a software program (e.g., source code or a program binary). We illustrate that, by using techniques from multimodal deep learning, we can simultaneously leverage multiple representations of software programs to improve flaw detection over single representation analyses. Specifically, we adapt three deep learning models from the multimodal learning literature for use in flaw detection and demonstrate how these models outperform traditional deep learning models. We present results on detecting software flaws using the Juliet Test Suite and Linux Kernel.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Using Neural Architecture Search for Improving Software Flaw Detection in Multimodal Deep Learning Models

Software flaw detection using multimodal deep learning models has been demonstrated as a very competitive approach on benchmark problems. In this work, we demonstrate that even better performance can be achieved using neural architecture search (NAS) combined with multimodal learning models. We adapt a NAS framework aimed at investigating image classification to the problem of software flaw detection and demonstrate improved results on the Juliet Test Suite, a popular benchmarking data set for measuring performance of machine learning models in this problem domain.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

An Efficient Storage-Driven Machine Learning Model for Performance in the Era of Multimodal Scientific Data

Scientific workflows are increasingly relying on machine learning (ML), simulation, and hybrid techniques to predict, understand, and optimize the behavior of complex experiments. High-performance computing has greatly improved researchers’ ability to acquire diverse data modalities in these workflows. Recent studies suggest that the performance of machine learning models can be improved by integrating data from various sources. Unfortunately, these workloads pose unprecedent pressure on the network storage to meet the demands associated with accessing these multimodal data. To mitigate the impact of intensive IO, we propose a solution that utilizes a multi-tier High-Performance Computing (HPC) distributed storage and data processing framework, placing computation where the data resides for better performance. By adopting this project, the scientific community will gain new opportunities to explore multimodal storage-driven possibilities, integrating multiple scientific data sources with advanced streaming frameworks. Additionally, our framework effectively utilizes computing resources and bridges the gaps identified by HPC experts. Our proposed approach tackles scalability and persistence challenges by leveraging native persistency, which has posed difficulties in traditional approaches. Furthermore, we seek to enhance fault-tolerance and load-balance of computations by leveraging real-time streaming in diverse scientific computing environments, thereby propelling advanced scientific computing research into the next generation.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Multimodal parameter spaces of a complex multi-channel neuron model

One of the most common types of models that helps us to understand neuron behavior is based on the Hodgkin–Huxley ion channel formulation (HH model). A major challenge with inferring parameters in HH models is non-uniqueness: many different sets of ion channel parameter values produce similar outputs for the same input stimulus. Such phenomena result in an objective function that exhibits multiple modes (i.e., multiple local minima). This non-uniqueness of local optimality poses challenges for parameter estimation with many algorithmic optimization techniques. HH models additionally have severe non-linearities resulting in further challenges for inferring parameters in an algorithmic fashion. To address these challenges with a tractable method in high-dimensional parameter spaces, we propose using a particular Markov chain Monte Carlo (MCMC) algorithm, which has the advantage of inferring parameters in a Bayesian framework. The Bayesian approach is designed to be suitable for multimodal solutions to inverse problems. We introduce and demonstrate the method using a three-channel HH model. We then focus on the inference of nine parameters in an eight-channel HH model, which we analyze in detail. We explore how the MCMC algorithm can uncover complex relationships between inferred parameters using five injected current levels. The MCMC method provides as a result a nine-dimensional posterior distribution, which we analyze visually with solution maps or landscapes of the possible parameter sets. The visualized solution maps show new complex structures of the multimodal posteriors, and they allow for selection of locally and globally optimal value sets, and they visually expose parameter sensitivities and regions of higher model robustness. We envision these solution maps as enabling experimentalists to improve the design of future experiments, increase scientific productivity and improve on model structure and ideation when the MCMC algorithm is applied to experimental data.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Impact of Biofuel Blending on Hydrocarbon Speciation and Particulate Matter from a Medium-Duty Multimode Combustion Strategy

The U.S. Department of Energy’s Co-Optima initiative simultaneous focused on diversifying fuel sources, improving efficiency, and reducing emissions through using novel combustion strategies and sustainable fuel blends. For medium-duty/heavy-duty diesel engines, research in this area has led to the development of a multimode strategy that uses premixed charge compression ignition (PCCI) at low loads and conventional diesel combustion (CDC) at mid–high loads. The aim of this study was to understand how emissions were impacted when using PCCI instead of CDC at low loads and switching to an oxygenated biofuel blend. It provides a detailed speciation of the hydrocarbon (HC) and particulate matter (PM) emissions from a multimode medium-duty engine operating at low loads in PCCI and CDC modes and high loads in CDC. The effect of the oxygenated biofuel blend on emissions was studied at all three mode–load conditions using #2 ULSD and a bio-derived fuel (25% hexyl hexanoate (HHN)) blended in #2 ULSD. The PCCI mode effectively decreased NOx, total HC, and PM/PN emissions, with a substantial decrease in larger particles (≥50 nm). A PM/PN reduction was observed at high loads with the 25% HHN fuel. While the total HC emissions were not impacted by fuel type, the detailed HC analysis exposed changes in the HC’s composition.

09 BIOMASS FUELS↗
NASA NTRS1997

Power Loss For Multimode Waveguides and its Application to Beam-Waveguide Systems

The conventional way of expressing poer loss in dB/meter for a multimode waveguiding system with finite wall condcutivity (such as a beam-waveguide system with protective shroud) can be incorrect and misleading. The power loss (indB) for a multimode waveguiding system is, in general, not linearly proportional to the length of the waveguide.

Multimode↗
NASA NTRS2021

The Role of Tactile Cueing in Multimodal Displays: Application in Complex Task Environments for Space Exploration

This paper reviews the potential role of tactile cueing in multimodal displays that will need to be developed to enable Extravehicular Activity (EVA) and telerobotic exploration for both Moon and Mars missions. Specific missions to be accomplished during EVA surface and space operations will include construction and assembly, surface and geologic exploration, and excavation for protective shelter from radiation and significant solar particle events. EVA crews have limited time, resources, mobility, visibility and dexterity, and activities take place in a perceptually impoverished environment. Limitations include a loss of spatial reference frame leading to spatial disorientation, a loss of information normally provided by the auditory and somato-sensory systems, and a restricted field of view. In addition, in non-surface operations crew must be able to navigate and operate in a fully six degree of freedom (DoF) environment. Fatigue resulting from suited EVA tasks will also critically affect performance and safety. In order to mitigate these factors, it is important that crews be well trained prior to executing actual missions. There is a critical need to design multimodal interfaces to optimize performance under these conditions for both training and actual operations. Because of the high level of complexity of the EVA environment on the one hand, and the “sensory restrictions” on the other, it makes sense to envisage enhancements to the astronauts’ visual environment. Such an enriched environment may include virtual auditory as well as tactile inputs.

multimodal displays↗
DOE OSTI2021

Multimodal imaging of oil-in-water bilgewater emulsion and biofilms

Bilgewater emulsions, consisting of oil and surfactant mixtures in water, affect many aquatic species. Thus development of methods and techniques to treat or mitigate the formation and undesired consequences of shipboard emulsions are urgently needed. This work presents new results of the evolution of the bilgewaer chemical and physical properties using in situ imaging. A Navy O/W emulsion consisting of three oils and a detergent mixture was used as the synthetic bilgewater model [1, 2]. Synthetic oil-in-water (O/W) emulsion droplets were seled in a vacuum compatible microfluidic sample holder and imaged using a suite of tools including scanning electron microscopy (SEM), time-of-flight secondary ion mass spectrometry (ToF-SIMS), confocal laser scanning microscopy, and optical microscopy, highlighting the advantage of multiscale analysis and its applications in waste reduction and treatment in the environment. The transferrable and vacuum compatible microfluidic interface, System for Analysis at the Liquid Vacuum Interface (SALVI), was used in this study. SALVI enabled surface analysis of liquids and liquid-solid interactions using ToF-SIMS and SEM [3, 4]. Its detection window is an aperture of 2 ?m in diameter open to vacuum, permitting direct detection of the liquid surface. Liquid is withheld by surface tension within the aperture. The interface is composed of a silicon nitride membrane and polydimethylsiloxane microchannel [3, 4]. A variety of samples including complex liquid mixtures, ionic liquids, single mammalian cells, live biofilms, and solid-electrolyte interface have been analyzed using in situ imaging [5, 6]. The feasibility of using SALVI for in situ SEM imaging of nanoparticles and colloids in liquid was recently demonstrated [7]. This paper shows new findings of multimodal imaging and analysis of synthesized emulsion particles in water. Figure 1 depicts a schematic of multimodal in situ imaging of bilgewater emulsion using SEM and ToF-SIMS. Furthermore, we study the interactions of microbes and bilgewater emulsion. Microbes play an important role in the ocean. We hypothesize that microbes release organics that can act as surfactants that affect bilgewater formation or weakening. We present the first systematic study of emulsions and biofilms and investigate the effects of biofilms on bilgewater emulsions. Pseudomonas was selected as a model strain. Biofilms were cultured in a microchannel to allow culture [8]. Once a thick layer of biofilms was formed, the medium solution was changed to a mixture consisting of 50 % bilgewater emulsion. Dispersed biofilms were collected at 24 hrs. and 48 hrs. after emulsions were introduced into the channel. Bilgewater emulsions, biofilms, and mixtures of bilgewater emulsions and biofilms were analyzed using multiple in situ and ex situ techniques as described earlier. Our findings indicate that biofilms change the chemical makeup of the emulsion surface compositions and emulsion droplet size distribution, confirming the hypothesis that extracellular polymeric substance (EPS) related components released from biofilms can function as surfactants and change the oil-in-water interfaces.

bilgewater, multimodal imaging, biofilms↗