Search NASA⌕ Search

SEARCH · Search NASA

Results for “multimodal processing”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Localized keyhole pore prediction during laser powder bed fusion via multimodal process monitoring and X-ray radiography

Systematic fault detection and control during laser powder bed fusion (L-PBF) has been a long-standing objective for system manufacturers and researchers in the additive manufacturing (AM) industry. This manuscript investigates a data fusion approach for detection of keyhole porosity formation during laser irradiation of Ti-6Al-4V substrates by concurrent recording of thermally induced optical emission measured using both off-axis and coaxial photodiode sensors, and acoustic emission. Subsurface defect formation was monitored via high-speed synchrotron X-ray imaging at 20,000 frames per second, enabling temporal registration of keyhole pore formation events to the monitoring signals at a resolution of 50 µs. We developed data fusion machine learning (ML) models for localized prediction of keyhole pore formation at various time scales ranging from 0.5 ms to 2 ms. The signal segments were featurized using two independent approaches: (1) power spectral density (PSD) and (2) highly comparative time series analysis (HCTSA) framework. The extracted features from different sensor modalities were fused together to construct a multimodal feature space and sequential feature selection was used to determine the most informative features for training the ML models. The predictive performance was evaluated for three classifying algorithms: Support Vector Machine (SVM), K-Nearest Neighbor (KNN), and Gaussian Naive Bayes (GNB). As a result, pore formation events were predicted with up to 0.95 F1-score, 1.0 recall and 0.94 accuracy. The most heavily weighted features indicate that model performance is chiefly governed by the acoustic monitoring signal, with a secondary contribution from the optical emission sensors.

36 MATERIALS SCIENCE↗

Building a Long-Lived 3D Multimode Quantum Processing Unit with TESLA Cavities

Superconducting radio-frequency (SRF) cavities, such as TESLA-shaped cavities, are excellent candidates for long-lived quantum memories. Integrating these cavities with nonlinear elements, such as transmons, enables universal quantum control; however, this integration can often introduce unwanted losses. In this talk, we present the successful integration of a long-lived 2-cell TESLA cavity with an ancillary qubit, forming a quantum processing unit (QPU) that allows for robust control while maintaining coherence. We demonstrate universal control of the QPU through various schemes, including ancilla-cavity sideband and echoed conditional displacement (ECD) methods, enabling fault-tolerant high-fidelity preparation of large Fock states and fast entanglement generation. These results illustrate the versatility and efficiency of our 3D quantum processing unit, paving the way for advanced quantum computing applications.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

CRCNS21 Computational Models of Multisensory Integration by Upper Limb in Humanoids and Amputees

This international collaborative research project between Johns Hopkins University (JHU) and the Technical University of Munich (TUM) investigated how the human brain processes and integrates multiple types of sensory information, such as touch and force, with the goal of improving prosthetic limbs for amputees and advancing sensory capabilities in humanoid robots. The research advanced our understanding of how the brain responds to sensory feedback in upper-limb amputees. Through experiments in which amputees received electrical stimulation while performing phantom hand movements, we demonstrated that sensory feedback activates the cortical sensorimotor and multisensory regions, and that these regions communicate dynamically during stimulation. Experiments with intact-limb participants explored the integration of visual, haptic, and force feedback, as well as in virtual reality motor training, further showing how the brain processes multimodal sensory information. In addition, this research inspired work on examining the reliability of where amputees perceive sensations over time, which contributed to a successful doctoral fellowship for continued investigation. Our collaborators at TUM improved multimodal sensor technology combining tactile and thermal feedback for humanoid robots, demonstrating the feasibility of integrating multiple sensor types into a unified system for detecting and responding to environmental stimuli. The experimental methods and analysis techniques developed across both teams, including functional network analysis and multimodal sensor integration, provide a foundation for future research in prosthetics and robotics. This research benefits the public by generating knowledge about how amputees process restored sensory information. Advances in humanoid sensing contribute to safer human-robot interaction. The project also fostered international collaboration and cross-disciplinary training: one TUM doctoral student spent a summer at JHU working on multimodal sensor integration, while two JHU students traveled to TUM to host workshops on neuromorphic sensory encoding and sensory integration.

42 ENGINEERING↗

Photon–photon chemical thermodynamics of frequency conversion processes in highly multimode systems

Abstract Frequency generation in highly multimode nonlinear optical systems is inherently a complex process, giving rise to an exceedingly convoluted landscape of evolution dynamics. While predicting and controlling the global conversion efficiencies in such nonlinear environments has long been considered impossible, here, we formally address this challenge even in scenarios involving a very large number of spatial modes. By utilizing fundamental notions from optical statistical mechanics, we develop a universal theoretical framework that effectively treats all frequency components as chemical reactants/products, capable of undergoing optical thermodynamic reactions facilitated by a variety of multi-wave mixing effects. These photon–photon reactions are governed by conservation laws that directly determine the optical temperatures and chemical potentials of the ensued chemical equilibria for each frequency species. In this context, we develop a comprehensive stoichiometric model and formally derive an expression that relates the chemical potentials to the optical stoichiometric coefficients, in a manner akin to atomic/molecular chemical reactions. This advancement unlocks new predictive capabilities that can facilitate the optimization of frequency generation in highly multimode photonic arrangements, surpassing the limitations of conventional schemes that rely exclusively on nonlinear optical dynamics. Notably, we identify a universal regime of Rayleigh–Jeans thermalization where an optical reaction at near-zero optical temperatures can promote the complete and entropically irreversible conversion of light to the fundamental mode at a target frequency. Our theoretical results are corroborated by numerical simulations in settings where second-harmonic generation, sum-frequency generation and four-wave mixing processes can manifest.

Optics↗

Numerical simulation of the beam self-cleaning process in a multimode graded-index fibre during propagation of a pump wave and a Stokes component

A model of coupled modes is proposed, which makes it possible to describe the propagation of a pump wave and a Stokes component in a multimode graded-index fibre. A decrease in the depression in the centre of the profile of the pump wave intensity distribution as a result of the influence of Kerr effects and random linear coupling of modes is demonstrated. (paper)

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Multimodal Analysis of Spatially Heterogeneous Microstructural Refinement and Softening Mechanisms in Three-Pass Friction Stir Processed Al4Si Alloy

Multiple thermally and thermomechanically induced microstructural refinement mechanisms can be activated in metallic alloys when subjected to solid phase processing methods such as friction stir processing (FSP). In this work, we provide detailed descriptions of the relationship between region-specific microstructural refinement mechanisms and the variation in microhardness, through a systematic and multimodal microstructural characterization of an FSP-processed 75% cold-rolled Al-4 at.% Si model binary alloy. Spatially resolved high-energy synchrotron X-ray diffraction, electron backscattered diffraction, and scanning transmission electron microscopy were used to understand the spatially heterogeneous microstructural evolution due to FSP. Results provide insights into how mechanisms such as static recovery, static recrystallization, dynamic recovery and recrystallization, geometric and continuous dynamic recrystallization, and particle-stimulated static or dynamic grain nucleation may occur heterogeneously in the microstructure as a function of the distance from the stir zone in processed alloys, directly influencing the degree of softening. The systematic analysis of microstructures and hardness in the FSP-processed model binary alloy given in this work highlights the rich microstructural domains that can be uniquely harnessed through solid phase processing of metallic alloys.

Al4Si, Friction Stir Processing, Geometric Dynamic↗

Unsupervised multimodal fusion of in-process sensor data for advanced manufacturing process monitoring

Effective monitoring of manufacturing processes is crucial for maintaining product quality and operational efficiency. Modern manufacturing environments often generate vast amounts of complementary multimodal data, including visual imagery from various perspectives and resolutions, hyperspectral data, and machine health monitoring information such as actuator positions, accelerometer readings, and temperature measurements. However, fusing and interpreting this complex, high-dimensional data presents significant challenges, particularly when labeled datasets are unavailable or impractical to obtain. This paper presents a novel approach to multimodal sensor data fusion in manufacturing processes, inspired by the Contrastive Language-Image Pre-training (CLIP) model. We leverage contrastive learning techniques to correlate different data modalities without the need for labeled data, overcoming limitations of traditional supervised machine learning methods in manufacturing contexts. Our proposed method demonstrates the ability to handle and learn encoders for five distinct modalities: visual imagery, audio signals, laser position (x and y coordinates), and laser power measurements. By compressing these high-dimensional datasets into low-dimensional representational spaces, our approach facilitates downstream tasks such as process control, anomaly detection, and quality assurance. The unsupervised nature of our method makes it broadly applicable across various manufacturing domains, where large volumes of unlabeled sensor data are common. We evaluate the effectiveness of our approach through a series of experiments, demonstrating its potential to enhance process monitoring capabilities in advanced manufacturing systems. This research contributes to the field of smart manufacturing by providing a flexible, scalable framework for multimodal data fusion that can adapt to diverse manufacturing environments and sensor configurations. The proposed method paves the way for more robust, data-driven decision-making in complex manufacturing processes.

Contrastive Learning↗

Integrated multi-wavelength microscope combining TIRFM and IRM modalities for imaging cellulases and other processive enzymes

We describe a multimodal microscope for visualizing processive enzymes moving on immobilized substrates. The instrument combines interference reflection microscopy (IRM) with multi-wavelength total internal reflectance fluorescence microscopy (TIRFM). The microscope can localize quantum dots with a precision of 2.8 nm at 100 frames/s, and was used to image the dynamics of the cellulase, Cel7a interacting with surface-immobilized cellulose. The instrument, which was built with off-the-shelf components and is controlled by custom software, is suitable for tracking other degradative enzymes such as collagenases, as well as motor proteins moving along immobilized tracks.

09 BIOMASS FUELS↗

Detection of photon statistics and multimode field correlations by Raman processes

Glauber’s g (2) -function provides a common measure of quantum field statistics through two-photon coincidence counting in Hanbury Brown–Twiss measurements. Here, we propose to use nonlinear optical signals as a tool for the characterization of quantum light. In particular, we show that Raman measurements provide an alternative direct probe for a different component of the four-point correlation function underlying the g (2) -function. We illustrate this capacity for a specific quantum state obtained from a frequency conversion process. Our work points out how the analysis of controlled optical nonlinear processes can provide an alternative window toward the analysis of quantum light sources.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Deciphering between enhanced light emission and absorption in multi-mode porphyrin cavity polariton samples

Abstract It remains unclear how the collective strong coupling of cavity-confined photons to the electronic transitions of molecular chromophore leverages the distinct properties of the polaritonic constituents for future technologies. In this study, we design, fabricate, and characterize multiple types of Fabry-Pérot (FP) mirco-resonators containing copper(II) tetraphenyl porphyrin (CuTPP) to show how cavity polariton formation affects radiative relaxation processes in the presence of substantial non-Condon vibronic coupling between two of this molecule’s excited electronic states. Unlike the prototypical enhancement of Q state radiative relaxation of CuTPP in a FP resonator incapable of forming polaritons, we find the light emission processes in multimode cavity polariton samples become enhanced for cavity-exciton energy differences near those of vibrations known to mediate non-Condon vibronic coupling. We propose the value of this detuning is consistent with radiative relaxation of Herzberg-Teller polaritons into collective molecular states coupled to the cavity photon coherently. We contrast the feature stemming from light emission from the HT polariton state with those that occur due to polariton-enhanced light absorption. Our results demonstrate the landscape of molecular and photonic interactions enabled by cavity polariton formation using complex chromophores and how researchers can design resonators to leverage these interactions to characterize and control polaritonic properties.

Odewale, Elizabeth O.↗

Continuous multimodal technologies in industrial microbiology: potential for achieving high process performance and agility

The field of industrial microbiology has great potential to tackle the most pressing challenges facing society, but today, this entails much more than classical fermentation technology followed by product upgrading. Methods are needed to confer unit-operation-level functionality and whole-system-level robustness, as well as flexibility. As examples of this concept, we focus here on methodologies often deployed at industrial scale as discrete processes—such as biological, chemical, and electrochemical unit operations—that are also capable of substantial complementarity when combined sequentially and operated continuously. A key and relatively new type of conversion step is made possible by catalyst immobilization methods at both the enzymatic and cellular level. Cases drawn from electro-fermentation for acetoin production and syngas-mediated microbial electrosynthesis of ethanol exemplify the synergy of combined operational approaches. Because modular processes can be treated as a series of reactor elements, considerable flexibility is possible. As discussed, however, not all processes are amenable to this intensification.

09 BIOMASS FUELS↗

Universal liquid sample device and process for high resolution transmission electron microscope imaging and multimodal analyses of liquid sample materials

Liquid sample imaging devices and processes are disclosed for high resolution TEM imaging and multimodal analyses of liquid sample materials in situ under high vacuum that are compatible with standard type TEM chip membranes and TEM sample holders allowing TEM liquid sample imaging to be performed wherever a TEM instrument is accessible and at a substantially reduced cost compared to prior art systems and approaches.

36 MATERIALS SCIENCE↗

Evaluating Effects of Shear Processing on 2D Crystalline Materials in 3D Metal Matrices: Atomistic Understanding of High Shear Deformation of Copper Graphene Composites

Composites made of copper and graphene demonstrate high strength, lubricity and enhanced electrical and thermal conductivity compared pure copper. However, manufacturing these composites at bulk volumes for industrial applications has been a big challenge. Shear deformation assisted processing is an effective method for manufacturing materials such as copper-graphene composites demonstrating ultra-fine grain structures and compositional homogeneity. Nevertheless, microstructural evolution of the composites and their property development under such conditions is not clearly understood currently. To rectify this gap in literature, high strain shear deformation of copper coated graphene foils was performed using a tribometer pin in this study. Changes in microstructure of the composite as well as the constituent components under shear deformation was correlated to process conditions. A sharp increase in the coefficient of friction attributed to rupture and smearing of graphene layer into copper substrate was observed during the shear processing. The coefficient of friction of the sheared copper/graphene composite was lower than that of pure copper, suggesting that partially worn graphene is effectively lubricious at the macroscale. A multimodal characterization of the processed region further revealed a shear deformation-induced ultrafine two-phase grain-structure consisting of copper and graphitic domains. Shear deformation reduced the copper grain size from around 50 – 100 µm to ~200 nm on an average and ~2 – 5 nm in some locations. The semicrystalline graphene films were observed to fracture into 10 – 50 µm size flakes. Oxygen enrichment was observed in the processed region. Finally, graphitic domain were identified for the first time in the copper matrix and not just at the grain boundaries providing evidence for a metastable composite microstructure as a result of solid phase processing at room temperature.

36 MATERIALS SCIENCE↗

Deciphering electrocatalysts with multimodal operando approaches

Here, the electrocatalytic processes of a copper catalyst during nitrate electroreduction are unveiled by correlated operando microscopy and spectroscopy. Catalysts are vital to the modern chemical industry, yet their development has largely relied on trial-and-error approaches. Optimizing catalysts requires a fundamental understanding of their structure–chemical property behaviour under operational conditions. However, operando characterization remains challenging, especially for reactions occurring in the complex and dynamic liquid environments of electrochemical systems. Over the past decade, the rapid development of in situ and operando environmental transmission electron microscopy (ETEM) and microelectromechanical system (MEMS)-based closed-cell holders, enabling environmental studies within the vacuum environment of transmission electron microscopy (TEM), has substantively advanced understanding of heterogeneous catalysis, notably for gas-phase reactions. In situ ETEM enables the direct observation of the catalyst evolution in terms of structure, morphology and chemical state at the nano-to-atomic scale, providing insights into their correlation with catalytic performance.

36 MATERIALS SCIENCE↗

Scalable in situ non-destructive evaluation of additively manufactured components using process monitoring, sensor fusion, and machine learning

Laser Powder Bed Fusion (L-PBF) Additive Manufacturing (AM) is among the metal 3D printing technologies most broadly adopted by the manufacturing industry. However, the current industry qualification paradigm for critical-application L-PBF parts relies heavily on expensive non-destructive inspection techniques, which significantly limits the use-cases of L-PBF. In situ monitoring of the process promises a less expensive alternative to ex situ testing, but existing sensor technologies and data analysis techniques struggle to detect sub-surface flaws (e.g., porosity and cracking) on production-scale L-PBF printers. In this work, an in situ NDE (INDE) system was engineered to detect subsurface flaws detected in X-Ray Computed Tomography (XCT) directly from process monitoring data. A multilayer, multimodal data input allowed the INDE system to detect numerous subsurface flaws in the size range of 200–1000µm using a novel human-in-the-loop annotation procedure. Furthermore, a framework was established for generating probability-of-detection (POD) and probability-of-false-alarm (PFA) curves compliant with NDE standards by systematically comparing instances of detected subsurface flaws to post-build XCT data. Here, we also introduce for the first time in the AM in situ sensing literature the a 90/95 – the flaw size corresponding to a 90% detection rate on the lower 95% confidence interval of the POD curve. The INDE system successfully demonstrated POD capabilities commensurate with traditional NDE methods. Traditional ML performance metrics were also shown to be inadequate for assessing the ability of the INDE system’s flaw detection performance. It is the hope of the authors that future studies will adopt the POD and PFA approach outlined here to provide better insight into the utility of process monitoring for AM.

36 MATERIALS SCIENCE↗

Facile fabrication of 2D material multilayers and vdW heterostructures with multimodal microscopy and AFM characterization

Reliable transfer processes that enable manipulation of two-dimensional (2D) materials, e.g., transition metal dichalcogenides (TMDCs) and MXenes, from one substrate to another has been a necessity for successful device fabrication. Here, with both mechanical exfoliation and chemical vapor deposition (CVD) widely used, a versatile, clean, deterministic, and yet simple transfer technique is highly needed. To address such need, we developed a transfer method that takes advantage of wettability contrast between interfaces without the use of sacrificial layers or chemical processes. More importantly, a setup was developed to carry out this transfer method with high sample selectivity and fine control of the position and orientation of transferred TMDC crystals, a feature required for fabrication of the devices based on vertical 2D heterostructures. Using both exfoliated and CVD grown materials and subsequent atomic force microscopy (AFM), photoluminescence (PL), confocal Raman and tip enhanced Raman spectroscopy (TERS) characterization, we ascertained the quality of interfaces resulting from the transfer process while preserving excellent 2D material integrity. PL and TERS maps revealed nanometer-scale heterogeneities in the interfaces of fabricated heterostructures, which should enable further perfection of the transfer technique. TERS/TEPL information were employed to identify areas suitable for nanodevice fabrication, making the reported transfer and characterization methods ideal for making high quality assembly of 2D heterostructure more accessible, which should facilitate exploration of vertical 2D heterostructures for applications in electronics, batteries, solar cells, and twistronics.

2D material transfer↗