Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 289 records · Page 16
All sourcesNASA NTRSDOE OSTI
DOE OSTI2024

Co-orchestration of multiple instruments to uncover structure–property relationships in combinatorial libraries

The rapid growth of automated and autonomous instrumentation brings forth opportunities for the co-orchestration of multimodal tools that are equipped with multiple sequential detection methods or several characterization techniques to explore identical samples. This is exemplified by combinatorial libraries that can be explored in multiple locations via multiple tools simultaneously or downstream characterization in automated synthesis systems. In co-orchestration approaches, information gained in one modality should accelerate the discovery of other modalities. Correspondingly, an orchestrating agent should select the measurement modality based on the anticipated knowledge gain and measurement cost. Herein, we propose and implement a co-orchestration approach for conducting measurements with complex observables, such as spectra or images. The method relies on combining dimensionality reduction by variational autoencoders with representation learning for control over the latent space structure and integration into an iterative workflow via multi-task Gaussian Processes (GPs). This approach further allows for the native incorporation of the system's physics via a probabilistic model as a mean function of the GPs. We illustrate this method for different modes of piezoresponse force microscopy and micro-Raman spectroscopy on a combinatorial Sm-BiFeO3 library. However, the proposed framework is general and can be extended to multiple measurement modalities and arbitrary dimensionality of the measured signals.

47 OTHER INSTRUMENTATION↗
DOE OSTI2021

Unraveling the Formation Mechanism of a Hybrid Zr-Based Chemical Conversion Coating with Organic and Copper Compounds for Corrosion Inhibition

Environmental-friendly chromate-free, zirconium (Zr)-based conversion coating is a promising green technology for corrosion protection. Additives in the surface treatment provides critical functionalities and performance improvements; however, mechanistic understanding as of how the additives influence the coatings remains unclear. In this study, a new organic-inorganic hybrid Zrbased conversion coating which combines copper (Cu) compounds and polyamidoamine (PAMAM), taking advantages of a complementary nature of organic and inorganic additives. Here, a multimodal approach combining electron and X-ray characterization is applied to study the interaction of Cu 2+ and PAMAM and the resulting impacts on the coating formation. Adding PAMAM changed the surface morphology, thickness, distribution of Cu in the cluster and void formation of the coatings. High PAMAM (100-200 pm) leads to little conversion coating formation, and low PAMAM (0-25 ppm) shows voids formation under the coatings. Moreover, PAMAM incorporates in the coating in a form of PAMAM-Cu complex with a higher concentration towards the surface, provides an organic layer at the surface of the coating. X-ray absorption near-edge structure (XANES) spectroscopy shows a difference between the conventional and hybrid coating treatments in an alkaline solution to simulate the E-coat process, suggesting the contribution of PAMAM in the enhanced chemical stability in an alkaline environment. Therefore, an intermediate range of addition of PAMAM (50 ppm) is optimal to 1) avoid excessive voids formation, 2) promote some Cu cluster formation and thus enhancing the Zr-based coating formation and 3) incorporate organic components into the coating to improve the adhesion of the subsequent coatings. Overall, this work furthers our knowledge on the formation mechanism of an effective and environmentally friendly hybrid conversion coating for corrosion inhibition, demonstrating a critical processing-structureproperty relationship. This study will benefit future development of green and effective surface treatment technology.

77 NANOSCIENCE AND NANOTECHNOLOGY↗
NASA NTRS2023

2023 Astrophotonics Roadmap: Pathways to Realizing Multi-Functional Integrated Astrophotonic Instruments

Photonic technologies offer numerous functionalities that can be used to realize astrophotonic instruments. The most spectacular example to date is the ESO Gravity instrument at the Very Large Telescope in Chile that combines the light-gathering power of four 8 m telescopes through a complex photonic interferometer. Fully integrated astrophotonic devices stand to offer critical advantages for instrument development, including extreme miniaturization when operating at the diffraction-limit, as well as integration, superior thermal and mechanical stabilization owing to the small footprint, and high replicability offering significant cost savings. Numerous astrophotonic technologies have been developed to address shortcomings of conventional instruments to date, including for example the development of photonic lanterns to convert from multimode inputs to single mode outputs, complex aperiodic fiber Bragg gratings to filter OH emission from the atmosphere, complex beam combiners to enable long baseline interferometry with for example, ESO Gravity, and laser frequency combs for high precision spectral calibration of spectrometers. Despite these successes, the facility implementation of photonic solutions in astronomical instrumentation is currently limited because of (1) low throughputs from coupling to fibers, coupling fibers to chips, propagation and bend losses, device losses, etc, (2) difficulties with scaling to large channel count devices needed for large bandwidths and high resolutions, and (3) efficient integration of photonics with detectors, to name a few. In this roadmap, we identify 24 key areas that need further development. We outline the challenges and advances needed across those areas covering design tools, simulation capabilities, fabrication processes, the need for entirely new components, integration and hybridization and the characterization of devices. To realize these advances the astrophotonics community will have to work cooperatively with industrial partners who have more advanced manufacturing capabilities. With the advances described herein, multi-functional integrated instruments will be realized leading to novel observing capabilities for both ground and space based platforms, enabling new scientific studies and discoveries.

astrophotonics↗
DOE OSTI2025

Ultra-high Information-content Chemical Imaging with Broadband Coherent Anti-Stokes Raman and Two-photon Fluorescence Lifetime Microscopy

Raman fingerprint spectroscopy and fluorescence lifetime imaging are emerging tools for studying metabolic profiles of biological specimens. While Raman fingerprint spectroscopy detects intrinsic molecular vibrations that reflect the molecular composition and chemical environment of a sample, fluorescence lifetime imaging measures changes in the excited-state lifetime of fluorophores that are sensitive to their microenvironments. Here, we present a multimodal imaging platform combining broadband coherent anti-Stokes Raman scattering (BCARS) and two-photon fluorescence lifetime imaging (2p-FLIM) microscopy that can acquire biologically relevant Raman fingerprint spectra and fluorescence lifetime signals in vivo and simultaneously. The tremendous chemical information obtained from spatially co-registered BCARS and 2p-FLIM images allows us to characterize the subtle differences between sub-cellular compartments and verify the potential false-positive results generated by fluorescence imaging alone. This is demonstrated by directly comparing the BCARS, 2p-FLIM, and two-photon excitation fluorescence(TPEF) signals simultaneously obtained from the same dye-stained organelle in the live, intact C. elegans expressing a green fluorescent protein (GFP) marker. In this work, we introduce the BCARS/2p-FLIM/TPEF setup scheme, the image acquisition steps, data processing, and representative results showing that the cross-modality imaging method enables rigorous characterization and in vivo detection at sub-cellular resolution. Furthermore, this protocol provides a framework for simultaneous chemical and fluorescence lifetime imaging to improve the accuracy of biological interpretation in complex living systems.

Xu, Haoyu [Georgia Institute of Technology, Atlant↗
NASA NTRS2001

Multimodal Neuroelectric Interface Development

This project aims to improve performance of NASA missions by developing multimodal neuroelectric technologies for augmented human-system interaction. Neuroelectric technologies will add completely new modes of interaction that operate in parallel with keyboards, speech, or other manual controls, thereby increasing the bandwidth of human-system interaction. We recently demonstrated the feasibility of real-time electromyographic (EMG) pattern recognition for a direct neuroelectric human-computer interface. We recorded EMG signals from an elastic sleeve with dry electrodes, while a human subject performed a range of discrete gestures. A machine-teaming algorithm was trained to recognize the EMG patterns associated with the gestures and map them to control signals. Successful applications now include piloting two Class 4 aircraft simulations (F-15 and 757) and entering data with a "virtual" numeric keyboard. Current research focuses on on-line adaptation of EMG sensing and processing and recognition of continuous gestures. We are also extending this on-line pattern recognition methodology to electroencephalographic (EEG) signals. This will allow us to bypass muscle activity and draw control signals directly from the human brain. Our system can reliably detect P-rhythm (a periodic EEG signal from motor cortex in the 10 Hz range) with a lightweight headset containing saline-soaked sponge electrodes. The data show that EEG p-rhythm can be modulated by real and imaginary motions. Current research focuses on using biofeedback to train of human subjects to modulate EEG rhythms on demand, and to examine interactions of EEG-based control with EMG-based and manual control. Viewgraphs on these neuroelectric technologies are also included.

Trejo, Leonard J.↗
DOE OSTI2025

Building a quantum computing architecture using 3D superconducting cavities

Quantum computers promise advantages over classical machines for solving certain complex problems, but building processors that truly deliver this advantage remains a central challenge, particularly due to limited coherence times. Three-dimensional superconducting radio-frequency (SRF) cavities offer an attractive platform due to their exceptionally long lifetimes. However, since these harmonic systems require nonlinear elements, such as transmons, for control, additional losses are often introduced. In this talk, I will present a multimode quantum system based on an elliptical SRF cavity hosting two cavity modes weakly coupled to an ancillary transmon circuit. This architecture is carefully engineered to preserve coherence while enabling efficient control. By optimizing the design to mitigate transmon-induced decoherence, we realize single-photon lifetimes of 20.6 ms and 15.6 ms in the two modes, with pure dephasing times exceeding 40 ms. Using sideband interactions and error-resilient protocols, such as measurement-based correction and post-selection, we demonstrate high-fidelity state control, including preparation of Fock states up to N=20 with fidelities above 95% (to our knowledge, the highest reported to date), as well as high-fidelity two-mode entanglement. These results highlight 3D SRF cavities as a robust foundation for qudit-based quantum information processing, harnessing the large Hilbert space of cavity modes. I will conclude by outlining strategies to further enhance coherence in both cavities and ancilla qubits, and discuss pathways toward scaling this architecture into a larger quantum computing platform.

Roy, Tanay [Fermilab]↗
DOE OSTI2026

Integrated fluorescence light microscopy-guided cryo-focused ion beam-milling for in situ montage cryo-ET

Cryogenic-electron tomography (cryo-ET) permits the in situ visualization of biological macromolecules at the molecular level. Owing to the variable thickness of cells, tissues and organisms, frozen specimens may need to be thinned by cryo-focused ion beam (FIB) milling to produce thin (<500 nm) cryo-lamellae suitable for cryo-ET. Locating regions of interest remains a challenge because untargeted milling can lead to inadvertent ablation and removal of regions of interest. Correlative light and electron microscopy, combined with cryo-FIB milling, can guide the identification of labeled targets in the cellular milieu. Multiple transfers between cryo-imaging instruments, cumbersome correlation algorithms, limited accuracy and low throughput have hindered the routine adoption of cryo-FIB milling within a multimodal correlative workflow for in situ structural biology. Here, in this study, we present a workflow for 3D correlative cryo-fluorescence light microscopy-FIB-ET that streamlines fluorescence light microscopy-guided FIB milling, improving throughput while preserving both structural and contextual information. The complete integration of hardware and software described here minimizes sample contamination from cross-platform exchanges and greatly enhances the efficiency of 3D targeting in cryo-milling. We then describe procedures for implementing montage parallel array cryo-ET (MPACT), which can be easily adapted to any modern life-science transmission electron microscope. MPACT supports high-throughput cryo-ET acquisitions (10 tilt series in 1.5 h) for structure determination and comprehensive contextual understanding of macromolecules within their native surroundings. A complete session from sample preparation to MPACT data processing takes 5−7 d for an individual experienced in both cryo-EM and cryo-FIB milling.

Yang, Jie E. [Univ. of Wisconsin, Madison, WI (Uni↗
DOE OSTI2023

Manifold projection image segmentation for nano-XANES imaging

As spectral imaging techniques are becoming more prominent in science, advanced image segmentation algorithms are required to identify appropriate domains in these images. We present a version of image segmentation called manifold projection image segmentation (MPIS) that is generally applicable to a broad range of systems without the need for training because MPIS uses unsupervised machine learning with a few physically motivated hyperparameters. We apply MPIS to nanoscale x-ray absorption near edge structure (XANES) imaging, where XANES spectra are collected with nanometer spatial resolution. We show the superiority of manifold projection over linear transformations, such as the commonly used principal component analysis (PCA). Moreover, MPIS maintains accuracy while reducing computation time and sensitivity to noise compared to the standard nano-XANES imaging analysis procedure. Finally, we demonstrate how multimodal information, such as x-ray fluorescence data and spatial location of pixels, can be incorporated into the MPIS framework. We propose that MPIS is adaptable for any spectral imaging technique, including scanning transmission x-ray microscopy, where the length scale of domains is larger than the resolution of the experiment.

36 MATERIALS SCIENCE↗
NASA NTRS1998

Computational Modeling of Semiconductor Dynamics at Femtosecond Time Scales

The main objective of the Joint-Research Interchange NCC2-5149 was to develop computer codes for accurate simulation of femtosecond pulse propagation in semiconductor lasers and semiconductor amplifiers [I]. The code should take into account all relevant processes such as the interband and intraband carrier relaxation mechanisms and the many-body effects arising from the Coulomb interaction among charge carriers [2]. This objective was fully accomplished. We made use of a previously developed algorithm developed at NASA Ames [3]-[5]. The new algorithm was tested on several problems of practical importance. One such problem was related to the amplification of femtosecond optical pulses in semiconductors. These results were presented in several international conferences over a period of three years. With the help of a postdoctoral fellow, we also investigated the origin of instabilities that can lead to the formation of femtosecond pulses in different kinds of lasers. We analyzed the occurrence of absolute instabilities in lasers that contain a dispersive host material with third-order nonlinearities. Starting from the Maxwell-Bloch equations, we derived general multimode equations to distinguish between convective and absolute instabilities. We find that both self-phase modulation and intensity-dependent absorption can dramatically affect the absolute stability of such lasers. In particular, the self-pulsing threshold (the so-called second laser threshold) can occur at few times the first laser threshold even in good-cavity lasers for which no self-pulsing occurs in the absence of intensity-dependent absorption. These results were presented in an international conference and published in the form of two papers.

Agrawal, Govind P.↗
DOE OSTI2025

MTUQ: a framework for estimating moment tensors, point forces, and their uncertainties

SUMMARY We introduce MTUQ, an open-source Python package for seismic source estimation and uncertainty quantification, emphasizing flexibility and operational scalability. MTUQ provides MPI-parallelized grid search and global optimization capabilities, compatibility with 1-D and 3-D Green’s function database formats, customizable data processing, C-accelerated waveform and first-motion polarity misfit functions, and utilities for plotting seismic waveforms and visualizing misfit and likelihood surfaces. Applicability to a range of full- and constrained-moment tensor, point force, and centroid inversion problems is possible via a documented application programming interface, accompanied by example scripts and integration tests. We demonstrate the software using three different types of seismic events: (1) a 2009 intraslab earthquake near Anchorage, Alaska; (2) an episode of the 2021 Barry Arm landslide in Alaska; and (3) the 2017 Democratic People’s Republic of Korea underground nuclear test. With these events, we illustrate the well-known complementary character of body waves, surface waves, and polarities for constraining source parameters. We also convey the distinct misfit patterns that arise from each individual data type, the importance of uncertainty quantification for detecting multimodal or otherwise poorly constrained solutions, and the software’s flexible, modular design.

58 GEOSCIENCES↗
DOE OSTI2025

Pursuing theranostics: a multimodal architecture approach

Theranostics is a field of nuclear medicine which uses the same targeting vector and chelating system for both a diagnostic and therapeutic radionuclide, allowing for uniformity in imaging and treatment.

Bender, Aidan A. [Nuclear Engineering Program, Uni↗
DOE OSTI2025

Resilient information and inference networks under mixed-trust sensing

With ubiquitous digitization, sensing, and computational intelligence deployed in increasingly more and broader domains, including critical infrastructure, potentially misleading and destabilizing effects of multimodal anomalies and adversarial behavior are growing in importance. Here, we develop randomized and reinforcement learning-based strategies for strategically recruiting and utilizing deployed (and, thus, vulnerable and potentially faulty and/or compromised) nodes from information and inference networks, while defending against adversaries that attempt to misguide assessments of inferred variables. Recognizing that, besides communication and other costs, sampling from any observable node can either provide true data or dangerously expose our inference to misinformation (without being easily distinguishable what actually happens), the proposed strategies proceed by progressively recruiting nodes and cautiously scaling their information contribution based on assumed, or, in our reinforcement learning approach, intelligently weighed trustworthiness, with the learning approach also considering network-wide, threat-inclusive risk/value tradeoffs. While avoiding the hardware, communication, analytical and computational burden of explicit redundancy, the proposed defensive schemes enable on-the-fly assessments of underlying processes, and system-wide situational awareness with demonstrable resilience against adversarial activities.

97 - MATHEMATICS AND COMPUTING↗
DOE OSTI2023

A Multimodal Event Catalog and Waveform Data Set That Supports Explosion Monitoring from Nevada, U.S.A.

Multimodal, curated data sets and nuisance event catalogs remain rare in the explosion monitoring community relative to curated seismic data sets. The source of this relative absence is the difficultly in deploying multimodal receivers that sense the seismic, acoustic, and other modalities from multiphysics sources. We provide such a data set in this study that delivers seismic, infrasound, and electromagnetic (magnetometer) sensor records collected over a two–week period, within 255 km of a 10 ton buried chemical explosion called DAG–4 that was located at 37.1146°, –116.0693° on 22 June 2019 21:06:19.88 UTC. This catalog includes 485 seismic, seismoacoustic, and infrasound–only events that an expert analyst manually built by reviewing waveforms from 29 seismic and infrasound sensors. Our data release includes waveforms from these 29 seismic, infrasound, and seismoacoustic stations and two magnetometer stations and their station metadata. We deliver these waveforms in NNSA KB Core CSS.w format (i4) with a corresponding wfdisc table that provides the header information. Here, we expect that this data set will provide a valuable, benchmark resource to develop signal processing algorithms and explosion monitoring methods against manual, human observations.

58 GEOSCIENCES↗
DOE OSTI2026

SULI Report - Development of a Molten Salt Circulation Loop for in-situ Spectroscopy

This project supports the development of real time optical monitoring capabilities for molten salt reactor (MSR) environments by designing, testing, and refining a molten salt circulation loop suitable for combined laser induced breakdown spectroscopy (LIBS) and ultraviolet visible (UV Vis) absorption measurements. Online spectroscopic monitoring is increasingly important for nuclear safeguards, corrosion tracking, and material accountancy, yet MSR process fluids present substantial challenges due to their chemical complexity and hazards such as high temperatures and radiation. To address these needs, this work focuses on Phase I, the development of a room temperature aqueous circulation loop that serves as a surrogate platform for evaluating flow behavior, optical access, and component performance prior to high temperature salt operation. Initial testing identified several practical issues—including leaks, obstructions, and two-phase flow through the absorption cell—that were systematically resolved through hardware replacement, flow path redesign, and venturi pressure optimization. Relocating the flow cell upstream of the primary venturi enabled periods of stable single-phase flow, demonstrating the feasibility of integrating optical diagnostics into a circulation system. The results of Phase I provide essential design insight for Phase II, which will incorporate furnace compatible materials and LiCl KCl eutectic salt. Completion of the molten salt system will deliver a reusable testbed for evaluating multimodal spectroscopic techniques, advancing nondestructive, real time monitoring tools for future MSR and nuclear fuel cycle applications.

98 - NUCLEAR DISARMAMENT, SAFEGUARDS, AND PHYSICAL↗
DOE OSTI2021

Possibility of an integrated transmission electron microscope: enabling complex in-situ experiments

Abstract Multimodal in-situ experiments are the wave of the future, as this approach will permit multispectral data collection and analysis during real-time nanoscale observation. In contrast, the evolution of technique development in the electron microscopy field has generally trended toward specialization and subsequent bifurcation into more and more niche instruments, creating a challenge for reintegration and backward compatibility for in-situ experiments on state-of-the-art microscopes. We do not believe this to be a requirement in the field; therefore, we propose an adaptive instrument that is designed to allow nearly simultaneous collection of data from aberration-corrected transmission electron microscopy (TEM), probe-corrected scanning transmission electron microscopy, ultrafast TEM, and dynamic TEM with a flexible in-situ testing chamber, where the entire instrument can be modified as future technologies are developed. The value would be to obtain a holistic understanding of the underlying physics and chemistry of the process-structure–property relationships in materials exposed to controlled extreme environments. Such a tool would permit the ability to explore, in-situ, the active reaction mechanisms in a controlled manner emulating those of real-world applications with nanometer and nanosecond resolution. If such a powerful tool is developed, it has the potential to revolutionize our materials understanding of nanoscale mechanisms and transients. Graphical Abstract

36 MATERIALS SCIENCE↗
NASA NTRS2004

Multimodality instrument for tissue characterization

A system with multimodality instrument for tissue identification includes a computer-controlled motor driven heuristic probe with a multisensory tip. For neurosurgical applications, the instrument is mounted on a stereotactic frame for the probe to penetrate the brain in a precisely controlled fashion. The resistance of the brain tissue being penetrated is continually monitored by a miniaturized strain gauge attached to the probe tip. Other modality sensors may be mounted near the probe tip to provide real-time tissue characterizations and the ability to detect the proximity of blood vessels, thus eliminating errors normally associated with registration of pre-operative scans, tissue swelling, elastic tissue deformation, human judgement, etc., and rendering surgical procedures safer, more accurate, and efficient. A neural network program adaptively learns the information on resistance and other characteristic features of normal brain tissue during the surgery and provides near real-time modeling. A fuzzy logic interface to the neural network program incorporates expert medical knowledge in the learning process. Identification of abnormal brain tissue is determined by the detection of change and comparison with previously learned models of abnormal brain tissues. The operation of the instrument is controlled through a user friendly graphical interface. Patient data is presented in a 3D stereographics display. Acoustic feedback of selected information may optionally be provided. Upon detection of the close proximity to blood vessels or abnormal brain tissue, the computer-controlled motor immediately stops probe penetration. The use of this system will make surgical procedures safer, more accurate, and more efficient. Other applications of this system include the detection, prognosis and treatment of breast cancer, prostate cancer, spinal diseases, and use in general exploratory surgery.

Mah, Robert W.↗
NASA NTRS2000

Multimodality Instrument for Tissue Characterization

A system with multimodality instrument for tissue identification includes a computer-controlled motor driven heuristic probe with a multisensory tip is discussed. For neurosurgical applications, the instrument is mounted on a stereotactic frame for the probe to penetrate the brain in a precisely controlled fashion. The resistance of the brain tissue being penetrated is continually monitored by a miniaturized strain gauge attached to the probe tip. Other modality sensors may be mounted near the probe tip to provide real-time tissue characterizations and the ability to detect the proximity of blood vessels, thus eliminating errors normally associated with registration of pre-operative scans, tissue swelling, elastic tissue deformation, human judgement, etc., and rendering surgical procedures safer, more accurate, and efficient. A neural network, program adaptively learns the information on resistance and other characteristic features of normal brain tissue during the surgery and provides near real-time modeling. A fuzzy logic interface to the neural network program incorporates expert medical knowledge in the learning process. Identification of abnormal brain tissue is determined by the detection of change and comparison with previously learned models of abnormal brain tissues. The operation of the instrument is controlled through a user friendly graphical interface. Patient data is presented in a 3D stereographics display. Acoustic feedback of selected information may optionally be provided. Upon detection of the close proximity to blood vessels or abnormal brain tissue, the computer-controlled motor immediately stops probe penetration.

Mah, Robert W.↗