Search NASA⌕ Search

SEARCH · Search NASA

Results for “temporal fusion transformer”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

17 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

Airport Delay Prediction with Temporal Fusion Transformers

Since flight delay hurts passengers, airlines, and airports, its prediction becomes crucial for the decision-making of all stakeholders in the aviation industry and thus has been attempted by various previous research. However, previous delay predictions are often categorical and at a highly aggregated level. To improve that, this study proposes to apply the novel Temporal Fusion Transformer model and predict numerical airport arrival delays at quarter hour level for U.S. top 30 airports. Inputs to our model include airport demand and capacity forecasts, historic airport operation efficiency information, airport wind and visibility conditions, as well as en-route weather and traffic conditions. The results show that our model achieves satisfactory performance measured by small prediction errors on the test set. In addition, the interpretability analysis of the model outputs identifies the important input factors for delay prediction.

Liu, Ke [University of California Berkeley]↗
DOE OSTI2025

Explainable AI for Multivariate Time Series Pattern Exploration: Latent Space Visual Analytics With Temporal Fusion Transformer and Variational Autoencoders in Power Grid Event Diagnosis

Detecting and analyzing complex patterns in multivariate time-series data is crucial for decision-making in urban and environmental system operations. However, challenges arise from the high dimensionality, intricate complexity, and interconnected nature of complex patterns, which hinder the understanding of their underlying physical processes. Existing AI methods often face limitations in interpretability, computational efficiency, and scalability, reducing their applicability in real-world scenarios. This paper proposes a novel visual analytics framework that integrates two generative AI models, Temporal Fusion Transformer (TFT) and Variational Autoencoders (VAEs), to reduce complex patterns into lower-dimensional latent spaces and visualize them in 2D using dimensionality reduction techniques such as PCA, t-SNE, and UMAP with DBSCAN. These visualizations, presented through coordinated and interactive views and tailored glyphs, enable intuitive exploration of complex multivariate temporal patterns, identifying patterns’ similarities and uncover their potential correlations for a better interpretability of the AI outputs. The framework is demonstrated through a case study on power grid signal data, where it identifies multi-label grid event signatures, including faults and anomalies with diverse root causes. Additionally, novel metrics and visualizations are introduced to validate the models and assess the performance, efficiency, and consistency of latent maps generated by VAE, which have been utilized in prior studies for latent space cartography and used as a benchmark in this study, and the emerging TFT architecture under various configurations. These analyses provide actionable insights for model parameter tuning and reliability improvements. Comparative results highlight that TFT achieves shorter run times and superior scalability to diverse time-series data shapes compared to VAE. This work advances fault diagnosis in multivariate time series, fostering explainable AI to support critical system operations.

Explainable AI↗
DOE OSTI2023

A scalable transformer model for real-time decision making in neutron scattering experiments

The U.S. Department of Energy's (DOE's) neutron research facilities at Oak Ridge National Laboratory (ORNL), including the High Flux Isotope Reactor (HFIR) and the Spallation Neutron Source (SNS), are a state-of-the-art neutron scattering facility that allows researchers to study the structure and dynamics of materials at the atomic scale. At the SNS, neutrons are measured using the time-of-flight (TOF) technique as they move through a neutron beamline to interact with a sample. Large volumes of neutron scattering data are collected and recorded in neutron event mode. Optimal productivity of the TOF instrument is limited due to the lack of real-time data analysis tools. The large amount of data generated by the experiments can be challenging to process and analyze in real time, particularly for experiments that require rapid feedback and adjustment of experimental parameters. The regular computer/workstation cannot keep up with the experiment speed to provide real-time feedback to adjust experimental parameters, so connecting the supercomputers available to the neutron facility is necessary to achieve real-time data analysis and experiment steering. To address this challenge, we exploit the Frontier supercomputer at Oak Ridge Leadership Computing Facility (OLCF) to train a scalable temporal fusion transformer model for real-time decision making of TOF neutron scattering experimentation. Here, in this paper, we present the results using Frontier to provide the processing power needed to rapidly process and analyze large volumes of single-crystal diffraction data collected at TOPAZ, a neutron time-of-flight Laue single-crystal diffractometer at the SNS.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Integrated edge-to-exascale workflow for real-time steering in neutron scattering experiments

We introduce a computational framework that integrates artificial intelligence (AI), machine learning, and high-performance computing to enable real-time steering of neutron scattering experiments using an edge-to-exascale workflow. Focusing on time-of-flight neutron event data at the Spallation Neutron Source, our approach combines temporal processing of four-dimensional neutron event data with predictive modeling for multidimensional crystallography. At the core of this workflow is the Temporal Fusion Transformer model, which provides voxel-level precision in predicting 3D neutron scattering patterns. The system incorporates edge computing for rapid data preprocessing and exascale computing via the Frontier supercomputer for large-scale AI model training, enabling adaptive, data-driven decisions during experiments. This framework optimizes neutron beam time, improves experimental accuracy, and lays the foundation for automation in neutron scattering. Although real-time experiment steering is still in the proof-of-concept stage, the demonstrated potential of this system offers a substantial reduction in data processing time from hours to minutes via distributed training, and significant improvements in model accuracy, setting the stage for widespread adoption across neutron scattering facilities and more efficient exploration of complex material systems.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Data-driven modeling of power generation for a coal power plant under cycling

Increased penetration of renewables for power generation has negatively impacted the dynamics of conventional fossil fuel-based power plants. The power plants operating on the base load are forced to cycle, to adjust to the fluctuating power demands. This results in an inefficient operation of the coal power plants, which leads up to higher operating losses. To overcome such operational challenge associated with cycling and to develop an optimal process control, this work analyzes a set of models for predicting power generation. Moreover, the power generation is intrinsically affected by the state of the power plant components, and therefore our model development also incorporates additional power plant process variables while forecasting the power generation. We present and compare multiple state-of-the-art forecasting data-driven methods for power generation to determine the most adequate and accurate model. We also develop an interpretable attention-based transformer model to explain the importance of process variables during training and forecasting. The trained deep neural network (DNN) LSTM model has good accuracy in predicting gross power generation under various prediction horizons with/without cycling events and outperforms the other models for long-term forecasting. The DNN memory-based models show significant superiority over other state-of-the-art machine learning models for short, medium and long range predictions. The transformer-based model with attention enhances the selection of historical data for multi-horizon forecasting, and also allows to interpret the significance of internal power plant components on the power generation. This newly gained insights can be used by operation engineers to anticipate and monitor the health of power plant equipment during high cycling periods.

01 COAL, LIGNITE, AND PEAT↗
DOE OSTI2025

A robust fourth-order finite-difference discretization for the strongly anisotropic transport equation in magnetized plasmas

We propose a second-order temporally implicit, fourth-order-accurate spatial discretization scheme for the strongly anisotropic heat transport equation characteristic of hot, fusion-grade plasmas. Following Du Toit et al. (2018), the scheme transforms mixed-derivative diffusion fluxes (which are responsible for the lack of a discrete maximum principle) into nonlinear advective fluxes, amenable to nonlinear-solver-friendly monotonicity-preserving limiters. The scheme enables accurate multi-dimensional heat transport simulations with up to seven orders of magnitude of heat-transport-coefficient anisotropies with low cross-field numerical error pollution and excellent algorithmic performance, with the number of linear iterations scaling very weakly with grid resolution and grid anisotropy, and scaling with the square-root of the implicit timestep. We propose a multigrid preconditioning strategy based on a lower-order approximation that renders the scheme efficient and scalable under grid refinement. Several numerical tests are presented that display the expected spatial convergence rates and strong algorithmic performance, including fully nonlinear magnetohydrodynamics simulations of kink instabilities in a Bennett pinch in 2D helical geometry and of ITER in 3D toroidal geometry.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2026

Automatic Calibration and Health Monitoring of Infrastructure Sensors

Smart transportation infrastructure relies on networks of heterogeneous sensors - cameras, radars, and lidars - continuously monitoring traffic conditions. However, executing the initial spatial calibration of multiple sensors and the subsequent health monitoring presents significant operational challenges. Environmental factors, mechanical vibrations, and gradual drift cause spatial misalignment, degrading fusion performance and tracking accuracy. Traditional calibration approaches require manual intervention with specialized targets or survey equipment, resulting in service interruptions and high maintenance costs. This work presents an automated framework for initial calibration and continuous health monitoring without human intervention or service disruption. Our approach addresses two critical problems: (1) detecting when sensors become miscalibrated during operation, and (2) automatically re-establishing spatial alignment using only operational traffic data. The health monitoring component analyzes measurement innovations - differences between sensor observations and predicted object states - to detect systematic biases indicative of calibration drift. By computing bias magnitude, directional consistency, and rejection rates, the system identifies miscalibrations as small as 0.5 meters. Unlike traditional methods requiring known calibration targets, our diagnostic operates continuously on live traffic observations, enabling early detection before fusion quality degrades. The automatic recalibration algorithm leverages overlapping sensor fields-of-view and temporal correlation of vehicle observations. Using graph-based optimization, the system automatically discovers which sensor pairs observe common regions, estimates pairwise spatial transformations using RANSAC-based robust estimation, and jointly optimizes all sensor poses through bundle adjustment. The framework handles practical deployment challenges, including different sensor sampling rates (1-10 Hz), varying installation positions, unknown orientations, and limited overlap regions (>10%). When approximate sensor positions are available from installation surveys (+/-1m accuracy), the algorithm additionally estimates sensor orientations, refining both position and rotation to sub-meter and sub-degree accuracy. We validate the framework on multi-hour traffic datasets from six heterogeneous sensors with sampling rates ranging from 1 Hz to 10 Hz. Results demonstrate successful calibration even with sparse overlap (<20%) and automatic detection of miscalibrations exceeding 0.8 meters. This work enables a "deploy-and-forget" sensor infrastructure that maintains calibration autonomously, reducing maintenance costs while improving tracking accuracy. The techniques generalize beyond transportation to any multi-sensor monitoring application requiring robust spatial alignment, including smart cities, industrial monitoring, and surveillance systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2024

ThermoPore: Predicting part porosity based on thermal images using deep learning

Part qualification is often a critical and labor-intensive process in additive manufacturing, particularly in the detection of defects such as porosity, which stands to benefit significantly from advancements in machine learning. We present a deep learning approach for quantifying and localizing ex-situ porosity within Laser Powder Bed Fusion fabricated samples utilizing in-situ thermal image monitoring data. Our goal is to build the real time porosity map of parts based on thermal images acquired during the build. The quantification task builds upon the established Convolutional Neural Network model architecture to predict pore count and the localization task leverages the spatial and temporal attention mechanisms of the novel Video Vision Transformer model to indicate areas of expected porosity. Our model for porosity quantification achieved a R 2 score of 0.57 and our model for porosity localization produced an average Intersection over Union (IoU) score of 0.32 and a maximum of 1.0. This work is setting the foundations of part porosity “Digital Twins” based on additive manufacturing monitoring data and can be applied downstream to reduce time-intensive post-inspection and testing activities during part qualification and certification. In addition, we seek to accelerate the acquisition of crucial insights normally only available through ex-situ part evaluation by means of machine learning analysis of in-situ process monitoring data.

Deep learning↗
DOE OSTI2021

An Eulerian Vlasov-Fokker–Planck algorithm for spherical implosion simulations of inertial confinement fusion capsules

Here, we present a numerical algorithm that enables a phase-space adaptive Eulerian Vlasov–Fokker–Planck (VFP) simulation of inertial confinement fusion (ICF) capsule implosions. The approach relies on extending a recent mass, momentum, and energy conserving phase-space moving-mesh adaptivity strategy to spherical geometry. In configuration space, we employ a mesh motion partial differential equation (MMPDE) strategy while, in velocity space, the mesh is expanded/contracted and shifted with the plasma’s evolving temperature and drift velocity. The mesh motion is dealt with by transforming the underlying VFP equations into a computational (logical) coordinate, with the resulting inertial terms carefully discretized to ensure conservation. To deal with the spatial and temporally varying dynamics in a spherically imploding system, we have developed a novel nonlinear stabilization strategy for MMPDE in the configuration space. The strategy relies on a nonlinear optimization procedure that optimizes between mesh quality and the volumetric rate change of the mesh to ensure both accuracy and stability of the solution. Implosions of ICF capsules are driven by several boundary conditions: (1) an elastic moving wall boundary; (2) a time-dependent Maxwellian Dirichlet boundary; and (3) a pressure-driven Lagrangian boundary. Of these, the pressure-driven Lagrangian boundary driver is new to our knowledge. The implementation of our strategy is verified through a set of test problems, including the Guderley and Van-Dyke implosion problems — the first-ever reported using a Vlasov–Fokker–Planck model.

1D2V↗
DOE OSTI2024

Extracting Vehicle Trajectories from Partially Overlapping Roadside Radar

This work presents a methodology for extracting vehicle trajectories from six partially-overlapping roadside radars through a signalized corridor. The methodology incorporates radar calibration, transformation to the Frenet space, Kalman filtering, short-term prediction, lane-classification, trajectory association, and a covariance intersection-based approach to track fusion. The resulting dataset contains 79,000 fused radar trajectories over a 26-h period, capturing diverse driving scenarios including signalized intersections, merging behavior, and a wide range of speeds. Compared to popular trajectory datasets such as NGSIM and highD, this dataset offers extended temporal coverage, a large number of vehicles, and varied driving conditions. The filtered leader–follower pairs from the dataset provide a substantial number of trajectories suitable for car-following model calibration. The framework and dataset presented in this work has the potential to be leveraged broadly in the study of advanced traffic management systems, autonomous vehicle decision-making, and traffic research.

33 ADVANCED PROPULSION SYSTEMS↗
DOE OSTI2023

Modification of Lie's transform perturbation theory for charged particle motion in a magnetic field

It is pointed out that the conventional Lie transform perturbation theory for the guiding center motion of charged particles in a magnetic field needs to be modified for ordering inconsistency. There are two reasons. First, the ordering difference between the temporal variation of gyrophase and that of the other phase space coordinates needs to be taken into account. Second, it is also important to note that the parametric limit of the derivative of a function is not equivalent to the derivative of the limit function. When these facts are taken into account, the near identity transformation rule for one form related to the Lagrangian is modified. With the modified near identity transformation rule, the drift motion of charged particles can be described in the first order, instead of the second order and beyond through a tedious expansion process as in the conventional formulation. Here, this resolves the discrepancy between the direct and Lie transform treatments in the Lagrangian perturbation theory for charged particle motion in a magnetic field.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Electric Field Measurements at the Surface of a Piezoelectric Transformer for Plasma Jet Formation using the Plasma Research Facility at Sandia National Laboratory (Final Report)

The overall goal of this research is to understand how low-temperature non-equilibrium plasmas form directly off the surface of piezoelectric crystals. Specifically, this project aims to temporally and spatially resolve how the electric field evolves on and near the surface of a piezoelectric transformer (PT) at conditions that produce an atmospheric-pressure air plasma jet emanating from the corner of the PT. In this work, we used the Plasma Research Facility (PRF) at Sandia National Laboratory to conduct temporally- and spatially-resolved electric field induced second harmonic (E-FISH) measurements of the electric field at and near the surface of a piezoelectric transformer, providing a fundamental understanding of how piezoelectric crystals can be used for plasma generation. We carried out two Objectives: Objective 1 – The first objective was to measure the temporal and spatial evolution of the electric field along the surfaces approaching the corner of a rectangular PT just below conditions that lead to plasma jet formation. Such a measurement can be used to explain when and where the electric field peaks, leading to plasma formation; Objective 2 – The second objective was to measure the temporal and spatial evolution of the electric field along the surfaces approaching the corner of a rectangular PT just at conditions of plasma jet formation. This, in principle, gives a more accurate determination of how the electric field leads to plasma formation but is more difficult due to the relatively stochastic plasma jet behavior of a PT. This work was complemented by an award from the Sandia PRF, and we worked with Dr. Edward Barnat at the Sandia PRF to carry out these objectives.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2025

Calibration and characterization of the line-VISAR diagnostic at the HED-HIBEF instrument at the European XFEL

In dynamic-compression experiments, the line-imaging Velocity Interferometer System for Any Reflector (VISAR) is a well-established diagnostic used to probe the velocity history, including wave profiles derived from dynamically compressed interfaces and wavefronts, depending on material optical properties. Knowledge of the velocity history allows for the determination of the pressure achieved during compression. Such a VISAR analysis is often based on Fourier transform techniques and assumes that the recorded interferograms are free from image distortions. In this paper, we describe the VISAR diagnostic installed at the HED-HIBEF instrument located at the European XFEL along with its calibration and characterization. It comprises a two-color (532, 1064 nm), three-arm (with three velocity sensitivities) line imaging system. We provide a procedure to correct VISAR images for geometric distortions and evaluate the performance of the system using Fourier analysis. We finally discuss the spatial and temporal calibrations of the diagnostic. As an example, we compare the pressure extracted from the VISAR analysis of shock-compressed polyimide and silicon.

47 OTHER INSTRUMENTATION↗
DOE OSTI2023

Estimates of the wavenumber wavelet power spectrum of magnetic fluctuations during magnetic reconnection

Fluctuation analyses of experimental observations generally lack high temporal resolution and are in frequency-space f, contrary to theoretical efforts in wavenumber-space k. This is due to the inherent limits of the Fourier transform, though it is prominent due to the ease of diagnostic implementation. Advances in wavelet-based analysis have provided relief due to its temporal resolution, but in its common use, is still hard to compare to theoretical models. By using the two-point correlation technique in conjunction with large data sets, a wavelet power spectrum in wavenumber-space can be created. Dubbed the wavenumber wavelet power spectrum, this spectrum relates wavenumber to power in time. Further, this analysis technique more closely connects characterizations of experimentally observed fluctuations with other system parameters and theoretical predictions. In this article, we develop the wavenumber wavelet power spectrum using magnetic fluctuations caused by tearing instability driven magnetic reconnection in reproducible, high temperature laboratory plasmas. These dynamic magnetic fluctuations generated in reversed field pinch plasmas are broadband, ranging from the low frequency, 10's of kHz, up to the ion gyroradii frequencies, 100's of kHz. The dominant fluctuations have poloidal and toroidal mode numbers (m,n)=(1,6−10) and can grow to 2%–3% of the mean magnetic field. During these reconnection events, ions, and electrons are energized, magnetic fluctuation amplitudes increase, plasma flow is halted, and the toroidal magnetic flux increases, all on a semi-periodic basis. The newly developed spectrum provides better temporal resolution of spectrum characteristics to correlate with these particle energization phenomena.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2021

Performance of Compact Pulsed Thermal Imaging System for In-Service Applications. Pulsed thermal tomography nondestructive examination of additively manufactured reactor materials and components

Additive manufacturing (AM) is an emerging method for cost-efficient fabrication of complex topology nuclear reactor parts from high-strength corrosion resistance alloys, such as stainless steel and Inconel. AM of metallic structures for nuclear energy applications is currently based on laser powder bed fusion (LPBF) process, which has the capability of melting metallic powder and net shaping the structures with relatively high precision. Some of the challenges with using LPBF method for nuclear manufacturing include the possibility of introducing pores into metallic structures. Integrity of AM structures needs to be evaluated nondestructively because material flaws could lead to premature failures due to creep in high temperature nuclear reactor environment. Currently, there exist limited capabilities to evaluate actual AM structures nondestructively. Pulsed Thermography (PT) imaging provides a capability for non-destructive evaluation (NDE) of sub-surface defects in arbitrary size structures. The PT method is based on recording material surface temperature transients with infrared (IR) camera following thermal pulse delivered on material surface with flash light. The PT method has advantages for NDE of actual AM structures because the method involves one-sided non-contact measurements and fast processing of large sample areas captured in one image. The data cube of PT measurements consists of surface temperature taken at sequential time intervals T(x,y,t). Material defects can be detected either by analyzing the thermograms T(x,y,t) data cube, or by using thermal tomography (TT) algorithm to obtain 3D spatial reconstruction of thermal effusivity e(x,y,z). To reduce the cost and enable in-service NDE in spatially constrained environment, it is highly desirable to develop PT with compact and inexpensive IR camera. Following initial qualification of an AM component for deployment in a nuclear reactor, a compact PT system can also be used for in-service nondestructive evaluation (NDE) applications. However, data cube obtained with PT based on compact IR camera suffers from strong thermal noises and loss of features due to relatively low sampling rate. In this report we describe two unsupervised machine learning (ML) algorithms for enhancement of PT images obtained with compact IR camera. In one approach, we introduce Sparse Coding Discrete Cosine Transform (SC/DCT) algorithm to remove additive white Gaussian noise (AWGN) from spatial thermal effusivity reconstructions. In another approach we introduce a Spatial Temporal Denoised Thermal Source Separation (STDTSS) ML algorithm to process thermograms. The STDTSS algorithm consists of spatial and temporal denoising using Gaussian and Savitzky–Golay filtering, followed by the matrix decomposition using Principal Component Analysis (PCA), and Independent Component Analysis (ICA) to automatically detect flaws. In the work described in this report, we constructed a compact PT system using a relatively small and low-cost FLIR A65 camera, consisting on uncooled microbolometer detector. Performance of SC/DCT algorithm was demonstrated on enhancing TT images of Inconel 718 AM plate. Performance of the STDTSS methods was investigated using thermography data obtained from imaging stainless steel 316L specimens produced with LPBF method with imprinted calibrated porosity defects.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗
DOE OSTI2025

Spatially and temporally resolved plasma parameter estimations of laser heated MagLIF relevant gas pipes at NIF

The ability to control laser pre-heat is an integral part of the inertial confinement fusion concept known as Magnetized Liner Inertial Fusion. This process is studied at the National Ignition Facility (NIF) where 4 of the 192 laser beams are propagated through a 1-cm long gas cell where they deposit >20 kJ of energy into the gaseous fuel via inverse bremsstrahlung absorption. This process ionizes the gas, producing a plasma that follows behind the laser front and expands over the radius of the cell. Emission from this plasma, as viewed by a gated x-ray detector, can be used to build spatially and temporally resolved estimations of the pre-heat plasma's density and temperature profiles. This can then be used to estimate the plasma pressure, internal energy, and radiation losses. Estimations show the evolution of the plasma in magnetized and unmagnetized gas cells filled with ambient temperature neopentane (C5H12) +1% Ar, as well as unmagnetized cryogenically cooled (32 K) deuterium +1% Ne filled targets. This analysis shows the effects of initial gas-fill density, composition, and axial magnetization on the time-dependent plasma parameters. Previously, these parameters at the NIF had not been experimentally characterized, and these estimations provided a potential new means of testing radiation magneto-hydrodynamic predictive capability models. Results in unmagnetized targets have strong agreement with simulations. However, in targets with a 19 T applied axial magnetic field, this method yields electron temperatures up to 100% hotter than those predicted by HYDRA codes.

Bremsstrahlung↗