SEARCH · Search NASA

Results for “Multi-modal integration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2003

Development of a Dynamically Configurable, Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation

The following reports are presented on this project:A first year progress report on: Development of a Dynamically Configurable,Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation; A second year progress report on: Development of a Dynamically Configurable, Object-Oriented Framework for Distributed, Multi-modal Computational Aerospace Systems Simulation; An Extensible, Interchangeable and Sharable Database Model for Improving Multidisciplinary Aircraft Design; Interactive, Secure Web-enabled Aircraft Engine Simulation Using XML Databinding Integration; and Improving the Aircraft Design Process Using Web-based Modeling and Simulation.

Afjeh, Abdollah A.↗

DOE OSTI2025

Multi-modality deep learning for pulse prediction in homogeneous nonlinear systems via parametric conversion

In this Letter, we introduce FusionNet, a multi-modality deep learning framework designed to predict and analyze output pulses in high-power rare-earth-doped laser systems driving parametric conversion in homogeneous guided nonlinear media. FusionNet integrates temporal, spectral, and physical experimental conditions to model ultrafast nonlinear phenomena, including parametric nonlinear frequency conversion, self-phase modulation, and cross-phase modulation in homogeneous guided systems such as gas-filled hollow-core fibers. These systems bridge physical models with experimental data, advancing our understanding of light-guiding principles and nonlinear interactions while expediting the design and optimization of on-demand high-power, high-brightness systems. Our results demonstrate a 73% reduction in prediction error and an 83% improvement in computational efficiency compared to conventional neural networks. This work establishes a new paradigm for accelerating parametric simulations and optimizing experimental designs in high-power laser systems, with further implications for high-precision spectroscopy, quantum information science, and distributed entangled interconnects.

47 OTHER INSTRUMENTATION↗

DOE OSTI2025

Energy-resolved neutron imaging and diffraction including grain orientation mapping using event camera technology

Time-of-flight neutron diffraction and energy-resolved imaging each provide unique perspectives into material properties. Neutron diffraction is useful for assessing microstructural parameters such as phase composition, texture, and dislocation densities, though it typically provides averaged data over the sampled volume. Energy-resolved imaging, on the other hand, offers both spatial and spectral information by detecting Bragg edges and neutron absorption resonances, which enables detailed mapping of microstructure and isotopic composition. When combined, these techniques have the potential to enrich our understanding of material behavior across different scales, enhancing our understanding of complex materials. Traditionally, these modalities are conducted on separate instruments, which is time-consuming and poses challenges for data integration. Here, we report the integration of the LumaCam, an event-mode energy-resolved neutron imaging camera with the HIPPO time-of-flight diffractometer at LANSCE. This integration enables simultaneous diffraction and imaging across the full spectrum, with analysis optimized for diffraction and Bragg-edge imaging in the thermal range (0.45–10 Å) and resonance imaging in the epithermal range (0.5–3000 eV), facilitating comprehensive multi-modal analysis. We demonstrate its capabilities through case studies, including spatial mapping of grain orientations in a steel sample and accurate thickness estimations for irregular samples including a depleted uranium cylinder and a natural silver-containing mineral specimen. The combined setup enhances real-time sample alignment and provides comprehensive data for crystal structure, texture, and isotopic composition analysis. This approach opens new possibilities for advanced applications in nuclear engineering, archaeology, and materials science.

36 MATERIALS SCIENCE↗

DOE OSTI2026

Thermal cycling-driven microstructural changes of eutectic Al–Si phase change materials in SS304 containers revealed by multi-modal imaging

Aluminum-based Al–Si alloys are widely used as phase change materials (PCMs) in thermal energy storage (TES) systems owing to their high volumetric latent heat and superior thermal conductivity. However, their long-term reliability is limited by degradation processes that remain insufficiently understood. In this work, we employ a multimodal, correlative characterization framework to systematically resolve the degradation behavior of eutectic Al–Si PCMs in contact with SS304 containers under repeated thermal cycling. By integrating high-resolution electron microscopy, three-dimensional X-ray fluorescence (3D XRF) imaging, and differential scanning calorimetry (DSC), we directly link spatially resolved compositional and microstructural evolution to changes in thermophysical properties. The correlative analysis reveals that elemental leaching of Fe, Cr, and Ni from the stainless-steel container into the PCM drives the formation of intermetallic compounds (IMCs) both at the interface and within the bulk PCM, leading to pronounced compositional heterogeneity. These interfacial reactions and diffusion-induced transformations progressively destabilize the Al–Si eutectic, reducing the effective phase-transforming fraction. Consistent with these observations, DSC measurements show a decrease in melting temperature and latent heat of fusion with thermal cycling. These results underscore the critical influence of interfacial reactions and material compatibility on the stability, durability, and overall performance of Al–Si-based TES systems.

25 ENERGY STORAGE↗

NASA NTRS2005

Inductive Learning Approaches for Improving Pilot Awareness of Aircraft Faults

Neural network flight controllers are able to accommodate a variety of aircraft control surface faults without detectable degradation of aircraft handling qualities. Under some faults, however, the effective flight envelope is reduced; this can lead to unexpected behavior if a pilot performs an action that exceeds the remaining control authority of the damaged aircraft. The goal of our work is to increase the pilot s situational awareness by informing him of the type of damage and resulting reduction in flight envelope. Our methodology integrates two inductive learning systems with novel visualization techniques. One learning system, the Inductive Monitoring System (IMS), learns to detect when a simulation includes faulty controls, while two others, Inductive Classification System (INCLASS) and multiple binary decision tree system (utilizing C4.5), determine the type of fault. In off-line training using only non-failure data, IMS constructs a characterization of nominal flight control performance based on control signals issued by the neural net flight controller. This characterization can be used to determine the degree of control augmentation required in the pitch, roll, and yaw command channels to counteract control surface failures. This derived information is typically sufficient to distinguish between the various control surface failures and is used to train both INCLASS and C4.5. Using data from failed control surface flight simulations, INCLASS and C4.5 independently discover and amplify features in IMS results that can be used to differentiate each distinct control surface failure situation. In real-time flight simulations, distinguishing features learned during training are used to classify control surface failures. Knowledge about the type of failure can be used by an additional automated system to alter its approach for planning tactical and strategic maneuvers. The knowledge can also be used directly to increase the pilot s situational awareness and inform manual maneuver decisions. Our multi-modal display of this information provides speech output to issue control surface failure warnings to a lesser-used communication channel and provides graphical displays with pilot-selectable !eve!s of details to issues additional information about the failure. We also describe a potential presentation for flight envelope reduction that can be viewed separately or integrated with an existing attitude indicator instrument. Preliminary results suggest that the inductive approach is capable of detecting that a control surface has failed and determining the type of fault. Furthermore, preliminary evaluations suggest that the interface discloses a concise summary of this information to the pilot.

Spikovska, Lilly↗

DOE OSTI2025

Integration and Demonstration of Monitoring, Modeling, and Prediction of DV-1 Amendment Performance at the Bench Scale: DV-1 Amendment Demonstration

During fiscal years 2024 and 2025, the U.S. Department of Energy’s Hanford Field Office commissioned Pacific Northwest National Laboratory to conduct applied research aimed at reducing the cost, time, and uncertainty associated with in situ treatment of vadose zone contaminants at the Hanford Site. This report outlines the integration of three key research efforts into a meso-scale demonstration designed to advance field-scale solutions that aim to (1) optimize the delivery of chemical amendments to contaminated soils, (2) reduce uncertainty in amendment delivery performance assessment using advanced monitoring techniques, and (3) provide real-time insights into when and where amendment-induced precipitation reactions occur in the subsurface. To achieve these objectives, the tank-scale (~ 1 cubic meter) Geophysical Imaging of Flow and Transport (GIFT) system was developed. GIFT enables experimental testing of amendment delivery while incorporating automated multi-modal monitoring approaches, including pressure measurements, direct fluid sampling, and remote time-lapse geophysical imaging. The data generated from these monitoring techniques will serve as inputs for a generative artificial-intelligence-driven digital twin – a numerical simulation model designed to honor observed data while quantifying uncertainty in simulation accuracy. Using this simulator, researchers will refine an amendment injection strategy to maximize delivery efficiency within a low-permeability soil zone. Monitoring data will be interpreted through simulated outputs to enhance understanding of the injection process. The efficacy of this integrated approach will be evaluated through direct sampling at the conclusion of the experiment.

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

DOE OSTI2025

A Scalable Multi-Modal Framework for High-Fidelity Distributed Human Mobility Simulations

The development of data-driven models for human mobility in urban settings requires access to substantial and diverse real-world data. However, existing historical data often presents challenges such as limited volume, variety, and veracity, as well as missing data and privacy preservation concerns. Also, urban mobility modeling is inherently time-variant, complex, and multi-modal, encompassing everything from individual walking and running to private road travel and large-scale public transportation. These challenges call for innovative solutions to overcome data limitations and compute needs to model mobility behaviors accurately. To address these challenges, we propose a distributed, co-simulation-based architecture DURMOSim that integrates real-world data with scalable, high-fidelity simulations, demonstrating distributed co-simulation feasibility with existing mobility models. DURMOSim underpins a modular integration that would enable using any available mobility simulators for greater extensibility and scalability in performing various urban scenarios. In this paper, we present the design, implementation, and performance evaluation of DURMOSim, highlighting its capability to model population-scale mobility patterns. Our initial results show its ability to dynamically synchronize multiple simulation models at runtime with negligible computational overhead. We believe DURMOSim could be a robust tool for advancing urban mobility research and intelligent transportation systems.

Yoginath, Srikanth [ORNL] (ORCID:0000000184236050)↗

NASA NTRS1988

Presentation planning using an integrated knowledge base

A description is given of user interface research aimed at bringing together multiple input and output modes in a way that handles mixed mode input (commands, menus, forms, natural language), interacts with a diverse collection of underlying software utilities in a uniform way, and presents the results through a combination of output modes including natural language text, maps, charts and graphs. The system, Integrated Interfaces, derives much of its ability to interact uniformly with the user and the underlying services and to build its presentations, from the information present in a central knowledge base. This knowledge base integrates models of the application domain (Navy ships in the Pacific region, in the current demonstration version); the structure of visual displays and their graphical features; the underlying services (data bases and expert systems); and interface functions. The emphasis is on a presentation planner that uses the knowledge base to produce multi-modal output. There has been a flurry of recent work in user interface management systems. (Several recent examples are listed in the references). Existing work is characterized by an attempt to relieve the software designer of the burden of handcrafting an interface for each application. The work has generally focused on intelligently handling input. This paper deals with the other end of the pipeline - presentations.

Arens, Yigal↗

NASA NTRS1995

Multi-modal virtual environment research at Armstrong Laboratory

One mission of the Paul M. Fitts Human Engineering Division of Armstrong Laboratory is to improve the user interface for complex systems through user-centered exploratory development and research activities. In support of this goal, many current projects attempt to advance and exploit user-interface concepts made possible by virtual reality (VR) technologies. Virtual environments may be used as a general purpose interface medium, an alternative display/control method, a data visualization and analysis tool, or a graphically based performance assessment tool. An overview is given of research projects within the division on prototype interface hardware/software development, integrated interface concept development, interface design and evaluation tool development, and user and mission performance evaluation tool development.

Eggleston, Robert G.↗

NASA NTRS1980

Multi-modal information processing for visual workload relief

The simultaneous performance of two single-dimensional compensatory tracking tasks, one with the left hand and one with the right hand, is discussed. The tracking performed with the left hand was considered the primary task and was performed with a visual display or a quickened kinesthetic-tactual (KT) display. The right-handed tracking was considered the secondary task and was carried out only with a visual display. Although the two primary task displays had afforded equivalent performance in a critical tracking task performed alone, in the dual-task situation the quickened KT primary display resulted in superior secondary visual task performance. Comparisons of various combinations of primary and secondary visual displays in integrated or separated formats indicate that the superiority of the quickened KT display is not simply due to the elimination of visual scanning. Additional testing indicated that quickening per se also is not the immediate cause of the observed KT superiority.

Burke, M. W.↗

DOE OSTI2026

Forensic characterization of surrogate nuclear explosion debris: radiochemical and spectroscopic strategies for method validation

Surrogate nuclear explosion debris (SNED) has emerged as a critical platform for advancing post-detonation nuclear forensic analysis in the absence of readily accessible historic materials. SNED enables controlled investigation and validation of analytical methodologies used to interrogate the chemical, isotopic, radiological, and microstructural signatures preserved in nuclear explosion debris. This review presents an integrated assessment of destructive and non-destructive analytical techniques commonly employed within decision-driven nuclear forensic workflows. Each technique is discussed individually while highlighting how it contributes to different stages of post-detonation analysis. Core methods – including gamma and alpha spectrometry, ICP-MS, TIMS, SIMS, SEM-EDS, XRF, LIBS, vibrational spectroscopy, and X-ray absorption spectroscopy – are critically evaluated with respect to forensic maturity, information content, and matrix limitations. Emphasis is placed on the role of SNED in benchmarking multi-modal workflows and identifying gaps in reproducing heterogeneity, fractionation, and radiation-driven evolution relevant to forensic attribution.

X-ray spectroscopic methods↗

DOE OSTI2025

SANE: strategic autonomous non-smooth exploration for multiple optima discovery in multi-modal and non-differentiable black-box functions

Both computational and experimental material discovery bring forth the challenge of exploring multidimensional and multimodal parameter spaces, such as phase diagrams of Hamiltonians with multiple interactions, composition spaces of combinatorial libraries, material structure image spaces, and molecular embedding spaces. Often these systems are black-boxes and time-consuming to evaluate, which resulted in strong interest towards active learning methods such as Bayesian optimization (BO). However, these systems are often noisy which make the black box function severely multi-modal and non-differentiable, where a vanilla BO can get overly focused near a single or faux optimum, deviating from the broader goal of scientific discovery. To address these limitations, here we developed Strategic Autonomous Non-Smooth Exploration (SANE) to facilitate an intelligent Bayesian optimized navigation with a proposed cost-driven probabilistic acquisition function to find multiple global and local optimal regions, avoiding the tendency to becoming trapped in a single optimum. To distinguish between a true and false optimal region due to noisy experimental measurements, a human (domain) knowledge driven dynamic surrogate gate is integrated with SANE. We implemented the gate-SANE into pre-acquired piezoresponse spectroscopy data of a ferroelectric combinatorial library with high noise levels in specific regions, and piezoresponse force microscopy (PFM) hyperspectral data. SANE demonstrated better performance than classical BO to facilitate the exploration of multiple optimal regions and thereby prioritized learning with higher coverage of scientific values in autonomous experiments. Our work showcases the potential application of this method to real-world experiments, where such combined strategic and human intervening approaches can be critical to unlocking new discoveries in autonomous research.

Biswas, Arpan [University of Tennessee, Knoxville,↗

DOE OSTI2026

Data-based filtered dissipation rate modelling for multi-modal turbulent combustion: evaluating a priori model generalizability

Manifold-based models offer a computationally efficient alternative to directly transporting the thermochemical state in computational simulations of turbulent reacting flows, projecting the high-dimensional thermochemical state-space onto a low-dimensional manifold. Recent efforts have yielded a manifold-based model applicable to multi-modal combustion, enabling reconstruction of the thermochemical state from solutions to two-dimensional manifold equations in mixture fraction and generalized progress variable that are parameterised by three scalar dissipation rates. In coarse-grained simulations such as Large Eddy Simulation (LES), closure of the multi-modal manifold equations and subfilter variances/covariance requires closure of three filtered scalar dissipation rates. Here, the present work adopts a data-based approach, providing closure for the three filtered scalar dissipation rates via deep neural networks (DNNs). High-fidelity datasets corresponding to an autoigniting n-dodecane jet flame and a bluff body swirl-stabilized confined lifted spray flame of two aviation fuels (Jet-A and C1) with different ignition propensities are leveraged to generate training data that spans a diverse range of thermodynamic conditions and combustion modes, including low- and high-temperature ignition regimes in addition to premixed and nonpremixed behaviour. A final DNN model is trained to enforce inherent physical constraints by learning nonlinear functional transformations of the three filtered scalar dissipation rates. The generalizability of this constrained DNN model is demonstrated a priori via conditional statistics evaluated on the lifted spray flame with C1–a configuration that had not been included in the training data. Excellent DNN agreement with conditional DNS statistics is observed, and integrated gradients are computed to identify the most sensitive input variables. The similarity of the marginal PDFs of the most informative input variables and outputs across configurations are quantified via the Wasserstein metric, demonstrating that data-based models may successfully generalize to unseen parametric conditions so long as the most informative input variables share similar distributions across training and testing datasets.

Data-based modelling↗

DOE OSTI2025

A case study in contrastive learning information combination: Application to technical forensics of additive manufacturing filament source identification

Combination of information from disparate data sources into a single decision is a core challenge in many fields, including the field of technical forensics. Technical forensics (TF) utilizes technical characterization of questioned samples to determine properties of that sample; these properties are then used to infer information of forensic interest, such as provenance, age, or attribution. TF is utilized in traditional forensic applications, such as the attribution of material fragments from an explosive, and in nuclear forensic applications, such as the attribution of actinides which have been interdicted out of regulatory control. The challenge of combining information from disparate sources, described alternately by many terms including “Data Fusion” and “Data Integration”, is exacerbated in the technical forensics domain due to at least two factors: the challenge of interpreting each information source singularly, and the relatively small data set sizes available. Extensive literature exists attempting to combine technical forensics information sources, both in manual and automated processes. These attempts are often bespoke to the specific information sources (such as the bi-, tri-, or quad-isotope chart (Moody, Grant, and Hutcheon 2005)), with some emerging examples of simple early- and late- fusion (, respectively). Simultaneous to the information combination efforts described in the previous paragraph, the field of natural language processing attempted (and largely succeeded) in combining information from multiple non-technical information sources. The ecosystem of “multi-modal” language models, which can take text and images as input, and generate text and images as output, became large and diverse by 2025 (Khan et al. 2025). In a generalized sense, many of these methods are trained by learning neural networks which can convert raw text or images into a vector of numbers describing the text or image, hereafter called “embeddings” and the neural networks performing the conversion are called “embedders”. By using a separate embedder for text and images, finding coincident text and images (such as images with their captions), and optimizing the parameters of the embedders such that the embeddings for the text and the image are similar, the field has found a bridge between text and images (Girdhar et al. 2023). It is the contention of the authors of this report that this insight is not limited to text and images but instead can be extended to any modality which can be found coincidently. The subject of the rest of this report is the application of this method to example multi-modal technical forensic data. Some details about the data used in this report are not appropriate for this report, and are included in a companion report (PNNL-38669).

36 MATERIALS SCIENCE↗

DOE OSTI2026

Decoding THz‐Driven Dynamic Fingerprints of Ferroelectric Nanotwin Networks

Ultrafast polarization dynamics in ferroelectrics are of considerable interest for high-speed tunable dielectrics and electro-optics. Extended domain wall networks formed in ferroelectric twin nanodomains can support collective dynamics in the terahertz regime but require techniques that track polarization and strain evolution driven by ultrafast stimulus. Here, we use multi-modal probing of THz-pulse-driven excitations in PbTiO 3 /SrTiO 3 superlattices by combining X-ray free electron laser measurements that directly tracks lattice changes, with optical second harmonic generation that tracks the electronic potential coupled with the lattice potential. Dynamical phase-field modeling enables fingerprinting of these collective modes as superpositions of domain “breathing” through wall oscillations and polarization “rotations” with still walls. Ultrafast domain wall motion at 0.1–0.5 THz is observed at practical fields of 100 kV/cm with wall velocities of >4000 m/s, approaching typical speed of sound in PbTiO 3 . A unique “charging” mode is discovered that can electrically charge and discharge domain walls on ∼4 ps time scale thus dynamically tuning wall conductivity. Integrated experimental and theoretical fingerprinting of the dynamical landscape presented here enables ultrafast control of ferroics for high-speed microelectronics and optical applications.

THz dynamics↗

DOE OSTI2024

The influence of the shock-to-reshock time on the Richtmyer–Meshkov instability in reshock

Experiments on the Richtmyer–Meshkov instability (RMI) in a dual driver vertical shock tube (DDVST) are described. An initially planar, stably stratified membraneless interface is formed by flowing air from above and sulfur hexafluoride from below the interface location using the method of Jones & Jacobs ( Phys. Fluids , vol. 9, issue 1997, 1997, pp. 3078–3085). A random three-dimensional, multi-modal initial perturbation is imposed by vertically oscillating the gas column to produce Faraday waves. The DDVST design generates two shock waves, one originating above and one below the interface, with these shocks having independently controllable strengths and interface arrival times. The shock waves have nominal strengths of $M_L=1.17$ and $M_H=1.18$ for the shock wave originating in the light and heavy gas, respectively, with these strengths chosen to result in arrested bulk interface motion following reshock. The influence of the length of the shock-to-reshock time, as well as the order of shock arrival, on the post-reshock RMI is examined. The mixing layer width grows according to $h\propto t^\theta$ , where $\theta _H=0.36\pm 0.018$ (95 %) and $\theta _L=0.38\pm 0.02$ (95 %) for heavy and light shock first experiments, respectively, indicating no strong dependence on the order of shock wave arrival. Volume integrated specific turbulent kinetic energy (TKE) in the mixing layer versus time is found to decay according to $E_{tot}/\bar {\rho }\propto t^p$ with $p_H=-0.823\pm 0.06$ (95 %) and $p_L=-1.061\pm 0.032$ (95 %) for heavy and light shock first experiments, respectively. Notably, the 95 % confidence intervals do not overlap. Analysis on the influence of the shock-to-reshock time on turbulent length scales, transition criteria, spectra and mixing layer anisotropy are also presented.

Ferguson, Kevin (ORCID:0000000166537482)↗

DOE OSTI2025

Scalable Multi-Facility Workflows for Artificial Intelligence Applications in Climate Research

Earth observation satellites and earth system models are sources of vast, multi-modal datasets that are invaluable for advancing climate and environmental research. However, their scale and complexity pose significant challenges for processing and analysis. In this paper we discuss our experiences in developing and using a scientific research application using an automated multi-facility workflow that orchestrates data collection, preprocessing, artificial intelligence (AI) inferencing, and data movement across diverse computational resources, leveraging the Advanced Computing Ecosystem Testbed at the Oak Ridge Leadership Computing Facility (OLCF). We demonstrate that our workflow can be seamlessly integrated and orchestrated across research facilities managed by different federal agencies, thus allowing users to extract new scientific insights from climate datasets. The experimental results indicate that the multi-facility workflow significantly reduces processing time, enhances scalability, and maintains high efficiency across varying workloads. Notably, our workflow processes 12,000 high-resolution satellite images in just 44 seconds using 80 workers distributed across 10 nodes on the OLCF systems. Such high throughput is essential for dynamic tokenization and sharding of petascale satellite data for distributed AI model training and inferencing at scale across thousands of GPUs.

Kurihana, Takuya [ORNL] (ORCID:0000000156698565)↗

NASA NTRS1989

A layered abduction model of perception: Integrating bottom-up and top-down processing in a multi-sense agent

A layered-abduction model of perception is presented which unifies bottom-up and top-down processing in a single logical and information-processing framework. The process of interpreting the input from each sense is broken down into discrete layers of interpretation, where at each layer a best explanation hypothesis is formed of the data presented by the layer or layers below, with the help of information available laterally and from above. The formation of this hypothesis is treated as a problem of abductive inference, similar to diagnosis and theory formation. Thus this model brings a knowledge-based problem-solving approach to the analysis of perception, treating perception as a kind of compiled cognition. The bottom-up passing of information from layer to layer defines channels of information flow, which separate and converge in a specific way for any specific sense modality. Multi-modal perception occurs where channels converge from more than one sense. This model has not yet been implemented, though it is based on systems which have been successful in medical and mechanical diagnosis and medical test interpretation.

Josephson, John R.↗