Search NASA⌕ Search

SEARCH · Search NASA

Results for “spatiotemporal fusion”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS2021

A Robust Hybrid Deep Learning Model for Spatiotemporal Image Fusion

Dense time-series remote sensing data with detailed spatial information are highly desired for the monitoring of dynamic earth systems. Due to the sensor tradeoff, most remote sensing systems cannot provide images with both high spatial and temporal resolutions. Spatiotemporal image fusion models provide a feasible solution to generate such a type of satellite imagery, yet existing fusion methods are limited in predicting rapid and/or transient phenological changes. Additionally, a systematic approach to assessing and understanding how varying levels of temporal phenological changes affect fusion results is lacking in spatiotemporal fusion research. The objective of this study is to develop an innovative hybrid deep learning model that can effectively and robustly fuse the satellite imagery of various spatial and temporal resolutions. The proposed model integrates two types of network models: super-resolution convolutional neural network (SRCNN) and long short-term memory (LSTM). SRCNN can enhance the coarse images by restoring degraded spatial details, while LSTM can learn and extract the temporal changing patterns from the time-series images. To systematically assess the effects of varying levels of phenological changes, we identify image phenological transition dates and design three temporal phenological change scenarios representing rapid, moderate, and minimal phenological changes. The hybrid deep learning model, alongside three benchmark fusion models, is assessed in different scenarios of phenological changes. Results indicate the hybrid deep learning model yields significantly better results when rapid or moderate phenological changes are present. It holds great potential in generating high-quality time-series datasets of both high spatial and temporal resolutions, which can further benefit terrestrial system dynamic studies. The innovative approach to understanding phenological changes’ effect will help us better comprehend the strengths and weaknesses of current and future fusion models.

spatiotemporal fusion↗
DOE OSTI2024

STFM: Accurate Spatio-Temporal Fusion Model for Weather Forecasting

Meteorological prediction is crucial for various sectors, including agriculture, navigation, daily life, disaster prevention, and scientific research. However, traditional numerical weather prediction (NWP) models are constrained by their high computational resource requirements, while the accuracy of deep learning models remains suboptimal. In response to these challenges, we propose a novel deep learning-based model, the Spatiotemporal Fusion Model (STFM), designed to enhance the accuracy of meteorological predictions. Our model leverages Fifth-Generation ECMWF Reanalysis (ERA5) data and introduces two key components: a spatiotemporal encoder module and a spatiotemporal fusion module. The spatiotemporal encoder integrates the strengths of convolutional neural networks (CNNs) and recurrent neural networks (RNNs), effectively capturing both spatial and temporal dependencies. Meanwhile, the spatiotemporal fusion module employs a dual attention mechanism, decomposing spatial attention into global static attention and channel dynamic attention. This approach ensures comprehensive extraction of spatial features from meteorological data. The combination of these modules significantly improves prediction performance. Experimental results demonstrate that STFM excels in extracting spatiotemporal features from reanalysis data, yielding predictions that closely align with observed values. In comparative studies, STFM outperformed other models, achieving a 7% improvement in ground and high-altitude temperature predictions, a 5% enhancement in the prediction of the u/v components of 10 m wind speed, and an increase in the accuracy of potential height and relative humidity predictions by 3% and 1%, respectively. This enhanced performance highlights STFM’s potential to advance the accuracy and reliability of meteorological forecasting.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Assessing methods in fusion and fitting for time series construction in remote sensing-based earth observations

This study evaluates the comparative performance of spatiotemporal fusion and time-series fitting methods for constructing high-spatiotemporal-resolution remote sensing time-series data. Due to in-class similarity of fusion methods and fitting methods, we employ the Fit-FC (Fitting, spatial Filtering, and residual Compensation) model as a representative fusion method and the linear harmonic fitting model as a representative fitting method. Both Fit-FC and the linear harmonic fitting are widely used for high-spatiotemporal-resolution time-series data construction, and we modify the original Fit-FC model to enable automatic time-series fusion. To ensure data representativeness, we use 3 years (2019–2021) of Harmonized Landsat and Sentinel-2 surface reflectance datasets and Terra MCD43A4 products. Eight experimental regions are selected worldwide to guarantee generalization of the comparative performance between fusion and fitting methods, covering diverse land-use types (cropland, developed land, forest, and grassland) and varying climatological conditions. Time-series of NDVI and surface reflectance are analyzed under both actual observations and simulated data-missing scenarios. The constructed time-series data reveals that (1) the modified Fit-FC and linear harmonic fitting model achieve excellent performance in constructing high-resolution time-series images; (2) the fusion method outperforms the fitting method in constructing time-series of NDVI and surface reflectance images in cropland-, forest-, and grassland-dominated regions; (3) both methods achieve comparable performance in developed-dominated regions; (4) the fusion method is more robust to missing data, and better captures abrupt phenological transitions under conditions of continuous missing data; (5) the fitting method is computationally more efficient, making it suitable for large-scale time-series image reconstruction. This study provides valuable insights for selecting optimal strategies to generate high-resolution time-series images across diverse application scenarios and lays a foundation for extensions to other vegetation indices or land surface variables.

54 ENVIRONMENTAL SCIENCES↗
NASA NTRS2017

Daily Ambient Air Pollution Metrics for Five Cities: Evaluation of Data Fusion-Based Estimates and Uncertainties

Spatiotemporal characterization of ambient air pollutant concentrations is increasingly relying on the combination of observations and air quality models to provide well-constrained, spatially and temporally complete pollutant concentration fields. Air quality models, in particular, are attractive, as they characterize the emissions, meteorological, and physiochemical process linkages explicitly while providing continuous spatial structure. However, such modeling is computationally intensive and has biases. The limitations of spatially sparse and temporally incomplete observations can be overcome by blending the data with estimates from a physically and chemically coherent model, driven by emissions and meteorological inputs. We recently developed a data fusion method that blends ambient ground observations and chemical transport-modeled (CTM) data to estimate daily, spatially resolved pollutant concentrations and associatedcorrelations. In this study, we assess the ability of the data fusion method to produce daily metrics (i.e., 1-hr max, 8-hr max, and 24-hr average) of ambient air pollution that capture spatiotemporal air pollution trends for 12 pollutants (CO, NO2, NOx, O3, SO2, PM (sub10), PM (sub 2.5), and five PM (sub 2.5) components) across five metropolitan areas (Atlanta, Birmingham, Dallas, Pittsburgh, and St. Louis), from 2002 to 2008. Three sets of comparisons are performed: (1) the CTM concentrations are evaluated for each pollutant and metropolitan domain, (2) the data fusion concentrations are compared with the monitor data, (3) a comprehensive cross-validation analysis against observed data evaluates the quality of the data fusion model simulations across multiple metropolitan domains. The resulting daily spatial field estimates of air pollutant concentrations and uncertainties are not only consistent with observations, emissions, andmeteorology, but substantially improve CTM-derived results for nearly all pollutants and all cities, with the exception of NO2 for Birmingham. The greatest improvements occur for O3 and PM (sub 2.5). Squared spatiotemporal correlation coefficients range between simulations and observations determined using cross-validation across all cities for air pollutants of secondary and mixed origins are R-squared equal to 0.88-0.93 (O3), 0.81-0.89 (SO4), 0.67-0.83 (PM (sub 2.5)), 0.52-0.72 (NO3), 0.43-0.80 (NH4), 0.32-0.51 (OC), and 0.14-0.71 (PM (sub 10)). Results for relatively homogeneous pollutants of secondary origin, tend to be better than those for more spatially heterogeneous (larger spatial gradients) pollutants of primary origin (NOx, CO, SO2 and EC). Generally, background concentrations and spatial concentration gradients reflect interurban airshed complexity and the effects of regional transport, whereas daily spatial pattern variability shows intra urban consistency in the fused data. With sufficiently high CTM spatial resolution, traffic-related pollutants exhibit gradual concentration gradients that peak toward the urban centers. Ambient pollutant concentration uncertainty estimates for the fused data are both more accurate and smaller than those for either the observations or the model simulations alone.

spatiotemporal fusion↗
DOE OSTI2021

Quantifying experimental edge plasma evolution via multidimensional adaptive Gaussian process regression

The edge density and temperature of tokamak plasmas are strongly correlated with energy and particle confinement and their quantification is fundamental to understanding edge dynamics. These quantities exhibit behaviours ranging from sharp plasma gradients and fast transient phenomena (e.g. transitions between low and high confinement regimes) to nominal stationary phases. Analysis of experimental edge measurements therefore require robust fitting techniques to capture potentially stiff spatiotemporal evolution. Additionally, fusion plasma diagnostics inevitably involve measurement errors and data analysis requires a statistical framework to accurately quantify uncertainties. This paper outlines a generalized multidimensional adaptive Gaussian process routine capable of automatically handling noisy data and spatiotemporal correlations. We focus on the edge-pedestal region in order to underline advancements in quantifying time-dependent plasma profiles including transport barrier formation on the Alcator C-Mod tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2024

PANDA-FES: Portable and Adaptable Neutron Diagnostics for Advancing Fusion Energy Science

Nuclear fusion is a potential source of carbon-free electricity with many concepts in development. The Portable and Adaptable Neutron Diagnostics for Advancing Fusion Energy Science (PANDA-FES) suite has been deployed since 2021 to measure neutron yield, energy, and spatiotemporal source location at two different Z-pinch fusion devices. This diagnostic can be used at a variety of facilities pursuing fusion in the magnetic, inertial, and magneto-inertial regimes. These different regimes have a wide range of time scales from less than 100 ns to a few μ s, neutron yields from 10 6 to 10 11 , and noise environments. Neutron yield is measured through activation of 79 Br and 89 Y with calibrated detectors. Temporal, spatial, and energy dependence of neutrons is measured with scintillators coupled to photomultiplier tubes (PMTs). Experimental setups and data analysis methods have been developed for these conditions. Finally, neutron yield, neutron energy anisotropy, and spatiotemporal evolution of the source have been measured.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2023

Spatiotemporal features of traffic help reduce automatic accident detection time

Quick and reliable automatic detection of traffic accidents is of paramount importance to save human lives in transportation systems. However, automatically detecting when accidents occur has proven challenging, and minimizing the time to detect accidents (TTDA) by using traditional features in machine learning (ML) classifiers has plateaued. We hypothesize that accidents affect traffic farther from the accident location than previously reported. Therefore, leveraging traffic signatures from neighboring sensors that are adjacent to accidents should help improve their detection. We confirm this hypothesis by using verified ground-truth accident data, traffic data from radar detection system sensors, and light and weather conditions and show that we can minimize the TTDA while maximizing classification performance by considering spatiotemporal features of traffic. Specifically, we compare the performance of different ML classifiers (i.e, logistic regression, random forest, and XGBoost) when controlling for different numbers of neighboring sensors and TTDA horizons. We use data from interstates 75 and 24 in the metropolitan area that surrounds Chattanooga, TN. Our results show that the XGBoost classifier produces the best results by detecting accidents as quickly as 1.0 min after their occurrence with an area under the receiver operating characteristic curve of up to 83% and an average precision of up to 49%. We describe limitations, open challenges, and how the proposed framework can be used for quicker operational accident detection.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2024

Did the exposure of coacervate droplets to rain make them the first stable protocells?

Membraneless coacervate microdroplets have long been proposed as model protocells as they can grow, divide, and concentrate RNA by natural partitioning. However, the rapid exchange of RNA between these compartments, along with their rapid fusion, both within minutes, means that individual droplets would be unable to maintain their separate genetic identities. Hence, Darwinian evolution would not be possible, and the population would be vulnerable to collapse due to the rapid spread of parasitic RNAs. In this study, we show that distilled water, mimicking rain/freshwater, leads to the formation of electrostatic crosslinks on the interface of coacervate droplets that not only suppress droplet fusion indefinitely but also allow the spatiotemporal compartmentalization of RNA on a timescale of days depending on the length and structure of RNA. We suggest that these nonfusing membraneless droplets could potentially act as protocells with the capacity to evolve compartmentalized ribozymes in prebiotic environments.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗
DOE OSTI2022

AI-Enabled Robots for Automated Nondestructive Evaluation and Repair of Power Plant Boilers. Final Report

Boiler failure could cause loss of life and safety issues, cost hundreds of thousands of dollars in equipment repairs, property damage and production losses, and drive up the cost of electric power. Boiler maintenance is challenging and risky for inspectors working on scaffolding in confined hazardous spaces inside of a boiler and sometimes the space is hard to access. The operation is also time-consuming due to the large area of vertical structures for inspection and the tremendous effort needed for scaffolding. Recently, the use of robotics (e.g., drones and crawlers) in power plants for maintenance is growing rapidly. However, the existing robotics solutions show two notable technological gaps: no live repair capability, and no Artificial Intelligence (AI) for smart autonomy. The objective of this project is to develop an integrated autonomous robotic platform that is equipped with compact non-destructive evaluation (NDE) sensors to perform live inspection, operates onboard repair devices to perform live repair, and uses AI for intelligent data fusion and predictive analysis for automated and smart spatiotemporal inspection, analysis and repair of the furnace walls in coal-fired boilers. The approach to achieve the objective includes developing NDE sensors with signal processing techniques, designing and evaluating repair devices for robots based on fusion and solid-state technologies, and an autonomous robotic platform that can attach to and navigate on boiler furnace walls using magnetic drive tracks. The robot is also powered by AI to automate data gathering (e.g., 3D mapping and damage localization) and predictive analysis. This project has advanced the state-of-the-art by providing technological breakthroughs including compact NDE and repair tools for robots, AI capabilities for smart autonomy, and a robotic platform for automated boiler maintenance. This project has great potential to result in significant benefits including limiting or eliminating the need to send operators to assess difficult-to-access or hazardous areas, enabling automated live inspection and repair, avoiding time consuming scaffolding (especially for partial maintenance during unplanned outage), collecting comprehensive and well-organized data smartly, and avoiding or limiting the need for onsite or remote piloting technicians. The impacts can be tremendous in terms of the time and cost savings, reducing the risk for human operators, and increasing boiler reliability, usability, and efficiency. In addition, by developing the new technologies on the autonomous inspection and repair robot, by involving multiple undergraduate and graduate students working together with the faculty members on this project, and by generating knowledge and building up collaborations with industrial partners, this effort will significantly update the education capabilities, support long-term fundamental research, and maintain the leadership of Colorado School of Mines and Michigan State University in energy fields.

20 FOSSIL-FUELED POWER PLANTS↗
DOE OSTI2025

Multi-messenger dynamic imaging of laser-driven shocks in water using a plasma wakefield accelerator

Understanding dense matter hydrodynamics is critical for predicting plasma behavior in environments relevant to laser-driven inertial confinement fusion. Traditional diagnostic sources face limitations in brightness, spatiotemporal resolution, and in their ability to detect relevant electromagnetic fields. In this work, we present a dual-probe, multi-messenger laser wakefield accelerator platform combining ultrafast X-rays and relativistic electron beams at 1 Hz, to interrogate a free-flowing water target in vacuum, heated by an intense 200 ps laser pulse. This scheme enables high-repetition-rate tracking the evolution of the interaction using both particle types. Betatron X-rays reveal a cylindrically symmetric shock compression morphology assisted by low-density vapor, resembling foam-layer-assisted fusion targets. The synchronized electron beam detects time-evolving electromagnetic fields, uncovering charge separation and ion species differentiation during plasma expansion – phenomena not captured by photons or hydrodynamic simulations. We show that combining both probes provides complementary insights spanning kinetic to hydrodynamic regimes, highlighting the need for hybrid physics models to accurately predict fusion-relevant plasma behavior.

imaging techniques↗
DOE OSTI2022

Few-femtosecond resolved imaging of laser-driven nanoplasma expansion

Abstract The free expansion of a planar plasma surface is a fundamental non-equilibrium process relevant for various fields but as-yet experimentally still difficult to capture. The significance of the associated spatiotemporal plasma motion ranges from astrophysics and controlled fusion to laser machining, surface high-harmonic generation, plasma mirrors, and laser-driven particle acceleration. Here, we show that x-ray coherent diffractive imaging can surpass existing approaches and enables the quantitative real-time analysis of the sudden free expansion of laser-heated nanoplasmas. For laser-ionized SiO 2 nanospheres, we resolve the formation of the emerging nearly self-similar plasma profile evolution and expose the so far inaccessible shell-wise expansion dynamics including the associated startup delay and rarefaction front velocity. Our results establish time-resolved diffractive imaging as an accurate quantitative diagnostic platform for tracing and characterizing plasma expansion and indicate the possibility to resolve various laser-driven processes including shock formation and wave-breaking phenomena with unprecedented resolution.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

4D Multimodal Co-attention Fusion Network with Latent Contrastive Alignment for Alzheimer’s Diagnosis

Multimodal neuroimaging provides complementary structural and functional insights into both human brain organization and disease-related dynamics. Recent studies demonstrate enhanced diagnostic sensitivity for Alzheimer’s disease (AD) through synergistic integration of neuroimaging data (e.g., sMRI, fMRI) with tabular data (e.g., behavioral and cognitive tests). However, the intrinsic heterogeneity across modalities (e.g., 4D spatiotemporal fMRI dynamics vs. 3D anatomical sMRI structure) presents critical challenges for discriminative feature fusion, often leading to information loss or biased fusion. To bridge this gap, we propose M2M-AlignNet: a multimodal co-attention network with latent alignment for early AD diagnosis using sMRI and fMRI. At the core of our approach is a multi-patch-to-multi-patch (M2M) contrastive loss function that quantifies and reduces representational discrepancies via weighted patch correspondence, explicitly aligning fMRI components across brain regions with their sMRI structural substrates without one-to-one constraints. Additionally, we propose a latent-as-query co-attention module to autonomously discover fusion patterns, circumventing modality prioritization biases while minimizing feature redundancy. We conduct extensive experiments to confirm the effectiveness of our method and highlight the correspondence between fMRI and sMRI as AD biomarkers.

Wei, Yuxiang [Georgia Institute of Technology]↗
DOE OSTI2025

Spatiotemporal Automatic Calibration of Infrastructure Lidar, Radar, and Camera with a Global Navigation Satellite System

Robust and accurate perception is important for modern intelligent transportation systems (ITS), which use sensors of various modalities for data fusion to create a digital twin of an intersection. Sensor calibration is an important process that creates a unified coordinate frame for the sensor output data so that it can be used for data fusion. Classical approaches for sensor calibration are time-consuming, require an overlapping field of view for feature matching, and are not feasible for ITS application as they cause disruptions in the flow of traffic. In this paper, we present a spatiotemporal automatic calibration approach to calibrate multiple infrastructure lidar, radar, and cameras installed at a traffic intersection. The approach uses global navigation satellite system (GNSS) positioning information shared by connected vehicles, and when the vehicle is detected by the sensor, we match the sensor detections with the GNSS coordinates. The proposed algorithm is evaluated with a real-world dataset utilizing detections from two radars, cameras, and lidars with a test vehicle instrumented with a post-processing kinematic (PPK)-corrected GNSS driving past the sensors installed at a four-way traffic intersection. The experimental results show that the proposed automatic calibration approach can achieve the transformation with a root mean squared error of less than 0.5 for radar and lidar and less than 2 for camera detections. The ability to rapidly calibrate sensors not only benefits initial installations, but can also be used for system health monitoring, while utilizing available connected vehicle data to test the real-time sensor fidelity and operational status.

ADVANCED PROPULSION SYSTEMS,ENERGY CONSERVATION, C↗
NASA NTRSDate not supplied

A Spectral Fingerprinting Method for Deriving Consistent Climate Data Records from Multiple Satellite IR Sounders

Deriving Climate Data Records (CDRs) from multiple IR sounders such AIRS on Aqua, CrIS on S-NPP, NOAA20 and JPSS-2, IASI on Metop A, B, and C requires from current operational products are challenging due to different radiative transfer models and retireavl algorithms used for processing level 2 data. We developed a Climate Fingerprinting Sounder Product (ClimFiSP) algorithm, which uses a single set of radiative kernels a robust spectral fingerprinting method to performs retrievals using spatiotemporally averaged L1 hyperspectral radiances directly. The ClimFiSP algorithm provides accurate data fusion CDR products from multiple satellite sensors. We have applied this method to both AIRS and CrIS (on SNPP and on NOAA 20) data and generated two decades climate data records for atmospheric temperature, water vapor, cloud, trace gases, and surface skin temperature. We plan to add IASI to the CDR data set in the future. The ClimFiSP is being transitioned to NASA GES DISC data center for public access.

Xu Liu↗
DOE OSTI2022

Multimodal sensor fusion framework for residential building occupancy detection

For several years now, smart building energy systems have been a research area of intensive activity. In light of the increasing need for sustainable buildings and energy systems, this trend motivates an increasing need for a solution to reduce carbon dioxide emissions and improve energy efficiency. This work proposes a high-performing and transferable occupancy detection framework that combines sensor data from different data modalities, including time series environmental data (temperature, humidity, and illuminance), image data, and acoustic energy data using ensemble method. To draw out the best prediction performance in each modality, the proposed framework was developed, including various models that were designed to learn the occupancy patterns reflected in the physical data streams. To tackle the time series environmental data, we designed two variants of an occupancy detection spatiotemporal pattern network (Occ-STPN) that performs both feature level and decision level fusion, respectively. We also propose a new metric; the fading memory mean square error (FMMSE), that provides a fair evaluation and penalization of delayed occupancy predictions. Multiple open-sourced datasets, including the Electricity Consumption and Occupancy and the University of California, Irvine's (UCI) building occupancy detection dataset, along with our own real data collected from six different houses, were used to validate the algorithms' performance. The experimental results presented herein break down the performance for each sensing modality, and a detailed analysis of the performance is also discussed.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗
DOE OSTI2025

Spatiotemporal forecasting of the edge localized modes in tokamak plasmas using neural networks

Artificial intelligence techniques have been increasingly adopted by the plasma and fusion science to address problems like plasma reconstruction, surrogate modeling, and tokamak/stellarator optimization. A key focus in sustained fusion research is the prediction and mitigation of edge-localized-modes (ELMs), instabilities that occur in short, periodic bursts and can cause erosion to the tokamak vessel wall. Recent research has demonstrated the power of neural networks in approximating continuous functions. In this work, we build spatiotemporal forecasting models that can predict the onset of ELMs and their evolution at early stages. We leverage recent advances in generative modeling, sequence-to-sequence modeling, and Fourier neural operators to propose architectures and training strategies that can learn to forecast short to long term dynamics of the noisy signals due to ELMs. We benchmark the developed model against a state-of-the-art foundation model using the beam emission spectroscopy (BES) data that captures the plasma fluctuations due to ELMs over a 8 x 8 spatial grid. Our models demonstrate high accuracy, outperforming the baselines, in predicting the evolution of BES signals during ELM events. Furthermore, the developed models exhibit high accuracy in predicting the rapid rise and relaxation of the signals due to ELMs within 30–80 µs.

edge localized modes↗