Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Localized keyhole pore prediction during laser powder bed fusion via multimodal process monitoring and X-ray radiography

Systematic fault detection and control during laser powder bed fusion (L-PBF) has been a long-standing objective for system manufacturers and researchers in the additive manufacturing (AM) industry. This manuscript investigates a data fusion approach for detection of keyhole porosity formation during laser irradiation of Ti-6Al-4V substrates by concurrent recording of thermally induced optical emission measured using both off-axis and coaxial photodiode sensors, and acoustic emission. Subsurface defect formation was monitored via high-speed synchrotron X-ray imaging at 20,000 frames per second, enabling temporal registration of keyhole pore formation events to the monitoring signals at a resolution of 50 µs. We developed data fusion machine learning (ML) models for localized prediction of keyhole pore formation at various time scales ranging from 0.5 ms to 2 ms. The signal segments were featurized using two independent approaches: (1) power spectral density (PSD) and (2) highly comparative time series analysis (HCTSA) framework. The extracted features from different sensor modalities were fused together to construct a multimodal feature space and sequential feature selection was used to determine the most informative features for training the ML models. The predictive performance was evaluated for three classifying algorithms: Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Gaussian Naive Bayes (GNB). As a result, pore formation events were predicted with up to 0.95 F1-score, 1.0 recall and 0.94 accuracy. The most heavily weighted features indicate that model performance is chiefly governed by the acoustic monitoring signal, with a secondary contribution from the optical emission sensors.

36 MATERIALS SCIENCE↗

Building a Long-Lived 3D Multimode Quantum Processing Unit with TESLA Cavities

Superconducting radio-frequency (SRF) cavities, such as TESLA-shaped cavities, are excellent candidates for long-lived quantum memories. Integrating these cavities with nonlinear elements, such as transmons, enables universal quantum control; however, this integration can often introduce unwanted losses. In this talk, we present the successful integration of a long-lived 2-cell TESLA cavity with an ancillary qubit, forming a quantum processing unit (QPU) that allows for robust control while maintaining coherence. We demonstrate universal control of the QPU through various schemes, including ancilla-cavity sideband and echoed conditional displacement (ECD) methods, enabling fault-tolerant high-fidelity preparation of large Fock states and fast entanglement generation. These results illustrate the versatility and efficiency of our 3D quantum processing unit, paving the way for advanced quantum computing applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

CRCNS21 Computational Models of Multisensory Integration by Upper Limb in Humanoids and Amputees

This international collaborative research project between Johns Hopkins University (JHU) and the Technical University of Munich (TUM) investigated how the human brain processes and integrates multiple types of sensory information, such as touch and force, with the goal of improving prosthetic limbs for amputees and advancing sensory capabilities in humanoid robots. The research advanced our understanding of how the brain responds to sensory feedback in upper-limb amputees. Through experiments in which amputees received electrical stimulation while performing phantom hand movements, we demonstrated that sensory feedback activates the cortical sensorimotor and multisensory regions, and that these regions communicate dynamically during stimulation. Experiments with intact-limb participants explored the integration of visual, haptic, and force feedback, as well as in virtual reality motor training, further showing how the brain processes multimodal sensory information. In addition, this research inspired work on examining the reliability of where amputees perceive sensations over time, which contributed to a successful doctoral fellowship for continued investigation. Our collaborators at TUM improved multimodal sensor technology combining tactile and thermal feedback for humanoid robots, demonstrating the feasibility of integrating multiple sensor types into a unified system for detecting and responding to environmental stimuli. The experimental methods and analysis techniques developed across both teams, including functional network analysis and multimodal sensor integration, provide a foundation for future research in prosthetics and robotics. This research benefits the public by generating knowledge about how amputees process restored sensory information. Advances in humanoid sensing contribute to safer human-robot interaction. The project also fostered international collaboration and cross-disciplinary training: one TUM doctoral student spent a summer at JHU working on multimodal sensor integration, while two JHU students traveled to TUM to host workshops on neuromorphic sensory encoding and sensory integration.

42 ENGINEERING↗

Photon–photon chemical thermodynamics of frequency conversion processes in highly multimode systems

Abstract Frequency generation in highly multimode nonlinear optical systems is inherently a complex process, giving rise to an exceedingly convoluted landscape of evolution dynamics. While predicting and controlling the global conversion efficiencies in such nonlinear environments has long been considered impossible, here, we formally address this challenge even in scenarios involving a very large number of spatial modes. By utilizing fundamental notions from optical statistical mechanics, we develop a universal theoretical framework that effectively treats all frequency components as chemical reactants/products, capable of undergoing optical thermodynamic reactions facilitated by a variety of multi-wave mixing effects. These photon–photon reactions are governed by conservation laws that directly determine the optical temperatures and chemical potentials of the ensued chemical equilibria for each frequency species. In this context, we develop a comprehensive stoichiometric model and formally derive an expression that relates the chemical potentials to the optical stoichiometric coefficients, in a manner akin to atomic/molecular chemical reactions. This advancement unlocks new predictive capabilities that can facilitate the optimization of frequency generation in highly multimode photonic arrangements, surpassing the limitations of conventional schemes that rely exclusively on nonlinear optical dynamics. Notably, we identify a universal regime of Rayleigh–Jeans thermalization where an optical reaction at near-zero optical temperatures can promote the complete and entropically irreversible conversion of light to the fundamental mode at a target frequency. Our theoretical results are corroborated by numerical simulations in settings where second-harmonic generation, sum-frequency generation and four-wave mixing processes can manifest.

Optics↗

Numerical simulation of the beam self-cleaning process in a multimode graded-index fibre during propagation of a pump wave and a Stokes component

A model of coupled modes is proposed, which makes it possible to describe the propagation of a pump wave and a Stokes component in a multimode graded-index fibre. A decrease in the depression in the centre of the profile of the pump wave intensity distribution as a result of the influence of Kerr effects and random linear coupling of modes is demonstrated. (paper)

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Multimodal Analysis of Spatially Heterogeneous Microstructural Refinement and Softening Mechanisms in Three-Pass Friction Stir Processed Al4Si Alloy

Multiple thermally and thermomechanically induced microstructural refinement mechanisms can be activated in metallic alloys when subjected to solid phase processing methods such as friction stir processing (FSP). In this work, we provide detailed descriptions of the relationship between region-specific microstructural refinement mechanisms and the variation in microhardness, through a systematic and multimodal microstructural characterization of an FSP-processed 75% cold-rolled Al-4 at.% Si model binary alloy. Spatially resolved high-energy synchrotron X-ray diffraction, electron backscattered diffraction, and scanning transmission electron microscopy were used to understand the spatially heterogeneous microstructural evolution due to FSP. Results provide insights into how mechanisms such as static recovery, static recrystallization, dynamic recovery and recrystallization, geometric and continuous dynamic recrystallization, and particle-stimulated static or dynamic grain nucleation may occur heterogeneously in the microstructure as a function of the distance from the stir zone in processed alloys, directly influencing the degree of softening. The systematic analysis of microstructures and hardness in the FSP-processed model binary alloy given in this work highlights the rich microstructural domains that can be uniquely harnessed through solid phase processing of metallic alloys.

Al4Si, Friction Stir Processing, Geometric Dynamic↗

Unsupervised multimodal fusion of in-process sensor data for advanced manufacturing process monitoring

Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments often generate vast amounts of complementary multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, fusing and interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable or impractical to obtain. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, overcoming limitations of traditional supervised machine learning methods in manufacturing contexts. Our proposed method demonstrates the ability to handle and learn encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. The unsupervised nature of our method makes it broadly applicable across various manufacturing domains, where large volumes of unlabeled sensor data are common. We evaluate the effectiveness of our approach through a series of experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to the field of smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations. The proposed method paves the way for more robust, data-driven decision-making in complex manufacturing processes.

Contrastive Learning↗

Integrated multi-wavelength microscope combining TIRFM and IRM modalities for imaging cellulases and other processive enzymes

We describe a multimodal microscope for visualizing processive enzymes moving on immobilized substrates. The instrument combines interference reflection microscopy (IRM) with multi-wavelength total internal reflectance fluorescence microscopy (TIRFM). The microscope can localize quantum dots with a precision of 2.8 nm at 100 frames/s, and was used to image the dynamics of the cellulase, Cel7a interacting with surface-immobilized cellulose. The instrument, which was built with off-the-shelf components and is controlled by custom software, is suitable for tracking other degradative enzymes such as collagenases, as well as motor proteins moving along immobilized tracks.

09 BIOMASS FUELS↗

Shape optimization for maximum stability and dynamic stiffness

Any optimization of structures for maximum stability or for maximum dynamic stiffness deals with an eigenvalue problem. The goal of this optimization is to raise the lowest eigenvalue (or eigenvalues) of the problem to its highest (optimal) level at a constant volume of the structure. Likely the lowest eigenvalue may be either inherently multi-modal or it can become multi-modal as a result of the optimization process. The multimodeness introduces some ambiguity to the eigenvalue problem and make the optimization difficult to handle. Thus far, only the simplest cases of multi-modal structures have been effectively optimized using rather elaborate analytical methods. Numerous publications report design of a minimum volume structure with different eigenvalues constraints, in which, however, the modality of the problem is assumed a priori. The method presented here utilizes a multi-modal optimality criteria and allows for inclusion of an arbitrary number of buckling or vibrations modes which might influence the optimization process. The real multi-modality of the problem, that is the number of modes participating in the final optimal design is determined iteratively. Because of a natural use of the FEM technique the method is easy to program and might be helpful in design of large flexible space structures.

Szyszkowski, W.↗

Detection of photon statistics and multimode field correlations by Raman processes

Glauber’s g (2) -function provides a common measure of quantum field statistics through two-photon coincidence counting in Hanbury Brown–Twiss measurements. Here, we propose to use nonlinear optical signals as a tool for the characterization of quantum light. In particular, we show that Raman measurements provide an alternative direct probe for a different component of the four-point correlation function underlying the g (2) -function. We illustrate this capacity for a specific quantum state obtained from a frequency conversion process. Our work points out how the analysis of controlled optical nonlinear processes can provide an alternative window toward the analysis of quantum light sources.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deciphering between enhanced light emission and absorption in multi-mode porphyrin cavity polariton samples

Abstract It remains unclear how the collective strong coupling of cavity-confined photons to the electronic transitions of molecular chromophore leverages the distinct properties of the polaritonic constituents for future technologies. In this study, we design, fabricate, and characterize multiple types of Fabry-Pérot (FP) mirco-resonators containing copper(II) tetraphenyl porphyrin (CuTPP) to show how cavity polariton formation affects radiative relaxation processes in the presence of substantial non-Condon vibronic coupling between two of this molecule’s excited electronic states. Unlike the prototypical enhancement of Q state radiative relaxation of CuTPP in a FP resonator incapable of forming polaritons, we find the light emission processes in multimode cavity polariton samples become enhanced for cavity-exciton energy differences near those of vibrations known to mediate non-Condon vibronic coupling. We propose the value of this detuning is consistent with radiative relaxation of Herzberg-Teller polaritons into collective molecular states coupled to the cavity photon coherently. We contrast the feature stemming from light emission from the HT polariton state with those that occur due to polariton-enhanced light absorption. Our results demonstrate the landscape of molecular and photonic interactions enabled by cavity polariton formation using complex chromophores and how researchers can design resonators to leverage these interactions to characterize and control polaritonic properties.

Odewale, Elizabeth O.↗

Visual-vestibular integration motion perception reporting

Self-orientation and self/surround-motion perception derive from a multimodal sensory process that integrates information from the eyes, vestibular apparatus, proprioceptive and somatosensory receptors. Results from short and long duration spaceflight investigations indicate that: (1) perceptual and sensorimotor function was disrupted during the initial exposure to microgravity and gradually improved over hours to days (individuals adapt), (2) the presence and/or absence of information from different sensory modalities differentially affected the perception of orientation, self-motion and surround-motion, (3) perceptual and sensorimotor function was initially disrupted upon return to Earth-normal gravity and gradually recovered to preflight levels (individuals readapt), and (4) the longer the exposure to microgravity, the more complete the adaptation, the more profound the postflight disturbances, and the longer the recovery period to preflight levels. While much has been learned about perceptual and sensorimotor reactions and adaptation to microgravity, there is much remaining to be learned about the mechanisms underlying the adaptive changes, and about how intersensory interactions affect perceptual and sensorimotor function during voluntary movements. During space flight, SMS and perceptual disturbances have led to reductions in performance efficiency and sense of well-being. During entry and immediately after landing, such disturbances could have a serious impact on the ability of the commander to land the Orbiter and on the ability of all crew members to egress from the Orbiter, particularly in a non-nominal condition or following extended stays in microgravity. An understanding of spatial orientation and motion perception is essential for developing countermeasures for Space Motion Sickness (SMS) and perceptual disturbances during spaceflight and upon return to Earth. Countermeasures for optimal performance in flight and a successful return to Earth require the development of preflight and in-flight training to help astronauts acquire and maintain a dual adaptive state. Despite the considerable experience with, and use of, an extensive set of countermeasures in the Russian space program, SMS and perceptual disturbances remain an unresolved problem on long-term flights. Reliable, valid perceptual reports are required to develop and refine stimulus rearrangements presented in the PAT devices currently being developed as countermeasures for the prevention of motion sickness and perceptual disturbances during spaceflight, and to ensure a less hazardous return to Earth. Prior to STS-8, crew member descriptions of their perceptual experiences were, at best, anecdotal. Crew members were not schooled in the physiology or psychology of sensory perception, nor were they exposed to the appropriate professional vocabulary. However, beginning with the STS-8 Shuttle flight, a serious effort was initiated to teach astronauts a systematic method to classify and quantify their perceptual responses in space, during entry, and after flight. Understanding, categorizing, and characterizing perceptual responses to spaceflight has been greatly enhanced by implementation of that training system.

Harm, Deborah L.↗

Continuous multimodal technologies in industrial microbiology: potential for achieving high process performance and agility

The field of industrial microbiology has great potential to tackle the most pressing challenges facing society, but today, this entails much more than classical fermentation technology followed by product upgrading. Methods are needed to confer unit-operation-level functionality and whole-system-level robustness, as well as flexibility. As examples of this concept, we focus here on methodologies often deployed at industrial scale as discrete processes—such as biological, chemical, and electrochemical unit operations—that are also capable of substantial complementarity when combined sequentially and operated continuously. A key and relatively new type of conversion step is made possible by catalyst immobilization methods at both the enzymatic and cellular level. Cases drawn from electro-fermentation for acetoin production and syngas-mediated microbial electrosynthesis of ethanol exemplify the synergy of combined operational approaches. Because modular processes can be treated as a series of reactor elements, considerable flexibility is possible. As discussed, however, not all processes are amenable to this intensification.

09 BIOMASS FUELS↗

Universal liquid sample device and process for high resolution transmission electron microscope imaging and multimodal analyses of liquid sample materials

Liquid sample imaging devices and processes are disclosed for high resolution TEM imaging and multimodal analyses of liquid sample materials in situ under high vacuum that are compatible with standard type TEM chip membranes and TEM sample holders allowing TEM liquid sample imaging to be performed wherever a TEM instrument is accessible and at a substantially reduced cost compared to prior art systems and approaches.

36 MATERIALS SCIENCE↗

Parallel Information Processing (Image Transmission Via Fiber Bundle and Multimode Fiber

Growing demand for visual, user-friendly representation of information inspires search for the new methods of image transmission. Currently used in-series (sequential) methods of information processing are inherently slow and are designed mainly for transmission of one or two dimensional arrays of data. Conventional transmission of data by fibers requires many fibers with array of laser diodes and photodetectors. In practice, fiber bundles are also used for transmission of images. Image is formed on the fiber-optic bundle entrance surface and each fiber transmits the incident image to the exit surface. Since the fibers do not preserve phase, only 2D intensity distribution can be transmitted in this way. Each single mode fiber transmit only one pixel of an image. Multimode fibers may be also used, so that each mode represent different pixel element. Direct transmission of image through multimode fiber is hindered by the mode scrambling and phase randomization. To overcome these obstacles wavelength and time-division multiplexing have been used, with each pixel transmitted on a separate wavelength or time interval. Phase-conjugate techniques also was tested in, but only in the unpractical scheme when reconstructed image return back to the fiber input end. Another method of three-dimensional imaging over single mode fibers was demonstrated in, using laser light of reduced spatial coherence. Coherence encoding, needed for a transmission of images by this methods, was realized with grating interferometer or with the help of an acousto-optic deflector. We suggest simple practical holographic method of image transmission over single multimode fiber or over fiber bundle with coherent light using filtering by holographic optical elements. Originally this method was successfully tested for the single multimode fiber. In this research we have modified holographic method for transmission of laser illuminated images over commercially available fiber bundle (fiber endoscope, or fiberscope).

Kukhtarev, Nicholai↗

Evaluating Effects of Shear Processing on 2D Crystalline Materials in 3D Metal Matrices: Atomistic Understanding of High Shear Deformation of Copper Graphene Composites

Composites made of copper and graphene demonstrate high strength, lubricity and enhanced electrical and thermal conductivity compared pure copper. However, manufacturing these composites at bulk volumes for industrial applications has been a big challenge. Shear deformation assisted processing is an effective method for manufacturing materials such as copper-graphene composites demonstrating ultra-fine grain structures and compositional homogeneity. Nevertheless, microstructural evolution of the composites and their property development under such conditions is not clearly understood currently. To rectify this gap in literature, high strain shear deformation of copper coated graphene foils was performed using a tribometer pin in this study. Changes in microstructure of the composite as well as the constituent components under shear deformation was correlated to process conditions. A sharp increase in the coefficient of friction attributed to rupture and smearing of graphene layer into copper substrate was observed during the shear processing. The coefficient of friction of the sheared copper/graphene composite was lower than that of pure copper, suggesting that partially worn graphene is effectively lubricious at the macroscale. A multimodal characterization of the processed region further revealed a shear deformation-induced ultrafine two-phase grain-structure consisting of copper and graphitic domains. Shear deformation reduced the copper grain size from around 50 – 100 µm to ~200 nm on an average and ~2 – 5 nm in some locations. The semicrystalline graphene films were observed to fracture into 10 – 50 µm size flakes. Oxygen enrichment was observed in the processed region. Finally, graphitic domain were identified for the first time in the copper matrix and not just at the grain boundaries providing evidence for a metastable composite microstructure as a result of solid phase processing at room temperature.

36 MATERIALS SCIENCE↗