Search NASA⌕ Search

SEARCH · Search NASA

Results for “state reconstruction and forecasting”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records

Reduced-order autodifferentiable ensemble Kalman filters

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system. Our reduced-order autodifferentiable ensemble Kalman filters (ROAD-EnKFs) learn a latent low-dimensional surrogate model for the dynamics and a decoder that maps from the latent space to the state space. The learned dynamics and decoder are then used within an EnKF to reconstruct and forecast the state. Numerical experiments show that if the state dynamics exhibit a hidden low-dimensional structure, ROAD-EnKFs achieve higher accuracy at lower computational cost compared to existing methods. If such structure is not expressed in the latent state dynamics, ROAD-EnKFs achieve similar accuracy at lower cost, making them a promising approach for surrogate state reconstruction and forecasting.

Mathematics↗

A Hybrid Framework Combining Model-Based and Data-Driven Methods for Hierarchical Decentralized Robust Dynamic State Estimation

This paper combines model-based and data-driven methods to develop a hierarchical, decentralized, robust dynamic state estimator (DSE). A two-level hierarchy is proposed, where the lower level consists of robust, model-based, decentralized DSEs. The state estimates sent from the lower level are received at the upper level, where they are filtered by a robust data-driven DSE after a principled sparse selection. This selection allows us to shrink the dimension of the problem at the upper level and hence significantly speed up the computational time. The proposed hybrid framework does not depend on the centralized infrastructure of the control centers; thus it can be completely embedded into the wide-area measurement systems. This feature will ultimately facilitate the placement of hierarchical decentralized control schemes at the phasor data concentrator locations. Also, the network model is not necessary; thus, a topology processor is not required. Finally, there is no assumption on the dynamics of the electric loads. The proposed framework is tested on the 2,000-bus synthetic Texas system, and shown to be capable of reconstructing the dynamic states of the generators with high accuracy, and of forecasting in the advent of missing data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Learning turbulent flows with generative models for super resolution and sparse flow reconstruction

Neural operators are promising surrogates for dynamical systems but when trained with standard L 2 losses they tend to oversmooth fine-scale turbulent structures. Here, we show that combining operator learning with generative modeling overcomes this limitation. We consider three practical turbulent-flow challenges where conventional neural operators fail: spatio-temporal super-resolution, forecasting, and sparse flow reconstruction. For Schlieren jet super-resolution, an adversarially trained neural operator (adv-NO) reduces the energy-spectrum error by 15 × while preserving sharp gradients at neural operator-like inference cost. For 3D homogeneous isotropic turbulence, adv-NO trained on only 160 timesteps from a single trajectory forecasts accurately for five eddy-turnover times and offers 114 × wall-clock speed-up at inference than the baseline diffusion-based forecasters, enabling near-real-time rollouts. For reconstructing cylinder wake flows from highly sparse Particle Tracking Velocimetry-like inputs, a conditional generative model infers full 3D velocity and pressure fields with correct phase alignment and statistics. These advances enable accurate reconstruction and forecasting at low compute cost, bringing near-real-time analysis and control within reach in experimental and computational fluid mechanics.

Fluid dynamics↗

Toward Drilling the Perfect Geothermal Well: An International Research Coordination Network for Geothermal Drilling Optimization Supported by Deep Machine Learning and Cloud Based Data Aggregation

The EDGE project, supported by the U.S. Department of Energy Geothermal Technologies Office under award DE-EE0008793, established a data-driven framework for improving the efficiency, cost-effectiveness, and reliability of geothermal well drilling. The project focused on developing scalable data infrastructure, advanced machine learning and probabilistic models, and integrated analytics tools to support continuous drilling optimization. A central objective was to reduce geothermal drilling costs by up to seventy percent while minimizing the risk of well failure through predictive diagnostics and adaptive planning. Over the project period, a comprehensive data repository was designed and deployed, incorporating records from over one hundred geothermal wells across varied geological settings. This repository supported both structured and unstructured data and adhered to FAIR data principles, enabling provenance tracking, quality control, and standardized metadata. The project introduced automated ingestion pipelines and a cloud-hosted platform that facilitated access to raw, processed, and derived datasets. This infrastructure served as the foundation for model development and analysis. Machine learning workflows were developed to predict key drilling metrics including rate of penetration, non-productive time, and total drilling costs. Self-organizing maps and dimensionality reduction methods were used to uncover operational patterns and outliers, while supervised learning algorithms such as random forests and deep neural networks were applied to forecast performance outcomes. The models were validated on heterogeneous datasets from both U.S. and Icelandic fields, demonstrating variable but significant predictive accuracy. The results indicated that finer temporal resolution, inclusion of lithological data, and consistency in operational annotations could substantially improve model performance. The project also implemented process mining techniques to reconstruct state-transition models from drilling event logs. These models enabled the identification of deviations from optimal workflows and provided insights into recurring failure modes. Analysis of non-productive time highlighted the impact of equipment failures, geological challenges, and human factors, offering opportunities for targeted mitigation strategies. The EDGE Dashboard was developed as a web-based expert system integrating data visualization, model outputs, and user-driven queries. It provided an accessible interface for operators to explore historical data, evaluate predicted outcomes, and compare drilling scenarios. Initial feedback from project partners suggested that the dashboard could serve as a foundation for more advanced advisory and optimization tools. Overall, the EDGE project demonstrated the feasibility and value of applying modern data science techniques to geothermal drilling. It delivered a set of interoperable tools and models that can support more efficient, lower-risk well development. The findings point toward a viable path for transitioning from advisory analytics to semi-autonomous drilling systems, contingent on continued collaboration, expanded datasets, and field validation. The project results have immediate relevance for drilling operations, data management practices, and future geothermal R&D efforts aimed at achieving reliable, cost-competitive geothermal energy at scale.

15 GEOTHERMAL ENERGY↗

Abundance and properties of dark radiation from the cosmic microwave background

We study the cosmological signatures of new light relics that are collisionless like standard neutrinos or are strongly interacting. We provide a simple and succinct rephrasing of their physical effects in the cosmic microwave background, as well as the resulting parameter degeneracies with other cosmological parameters, in terms of the total radiation abundance and the fraction thereof that freely streams. In these more general terms, interacting and noninteracting light relics are differentiated by their respective decrease and increase of the free-streaming fraction, and, moreover, the scale-dependent interplay thereof with a common, correlated reduction of the fraction of matter in baryons. We then derive updated constraints on various dark-radiation scenarios with the latest cosmological observations, employing this language to identify the physical origin of the impact of each dataset. The “PR4” reanalyses of Planck CMB data prefer a larger primordial helium yield and therefore also slightly more radiation than the 2018 analysis; we investigate the differences between the two releases that drives these shifts. Smaller free-streaming fractions are disfavored by the excess lensing of the CMB measured in lensing reconstruction data from Planck and the Atacama Cosmology Telescope. On the other hand, baryon acoustic oscillation measurements from the Dark Energy Spectroscopic Instrument drive marginal detections of new, strongly interacting light relics due to that data's preference for lower matter fractions. Finally, we forecast measurements from the CMB-S4 experiment.

cosmological parameters from CMBR↗

Latent Twins

Over the past decade, scientific machine learning has transformed the development of mathematical and computational frameworks for analyzing, modeling, and predicting complex systems. From inverse problems to numerical partial differential equations (PDEs), dynamical systems, and model reduction, these advances have pushed the boundaries of what can be simulated. Yet they have often progressed in parallel, with representation learning and algorithmic solution methods evolving largely as separate pipelines. With Latent Twins, we propose a unifying mathematical framework that creates a hidden surrogate in latent space for the underlying equations. Whereas digital twins mirror physical systems in the digital world, Latent Twins mirror mathematical systems in a learned latent space governed by operators. Through this lens, classical modeling, inversion, model reduction, and operator approximation all emerge as special cases of a single principle. We establish the fundamental approximation properties of Latent Twins for both ordinary differential equations (ODEs) and PDEs and demonstrate the framework across three representative settings: (i) canonical ODEs, capturing diverse dynamical regimes; (ii) a PDE benchmark using the shallow-water equations, contrasting Latent Twin simulations with deep operator network and forecasts with a four-dimensional variational method baseline; and (iii) a challenging real-data geopotential reanalysis dataset, reconstructing and forecasting from sparse, noisy observations. Latent Twins provide a compact, interpretable surrogate for solution operators that evaluate across arbitrary time gaps in a single-shot, while remaining compatible with scientific pipelines such as assimilation, control, and uncertainty quantification. Looking forward, this framework offers scalable, theory-grounded surrogates that bridge data-driven representation learning and classical scientific modeling across disciplines.

Latent Twins↗

Moving lens effect: Simulations, forecasts, and foreground mitigation

The peculiar motion of massive objects across the line of sight imprints a dipolar temperature anisotropy pattern on the cosmic microwave background known as the moving lens effect. This effect provides a unique probe of the transverse components of the peculiar velocity field, but has not yet been detected due to its small size. We implement and validate a stacking estimator for the moving lens signal using a galaxy catalog as a tracer of massive haloes combined with reconstructed velocities from the galaxy number density field. Using simulations, we forecast detection prospects for the moving lens signal from current and upcoming microwave background and galaxy surveys. Here, we demonstrate a new foreground mitigation strategy likely sufficient for current datasets, and discuss various sources of systematic error and noise. Upcoming galaxy surveys will provide high-significance statistical detections of the moving lens effect.

Astrophysical & cosmological simulations↗

A deep learning-based workflow for fast prediction of 3D state variables in geological carbon storage: A dimension reduction approach

Deep learning (DL) models are extensively used as surrogate models for high-fidelity simulations of multiphase fluid flow in porous media at large scales, enabling fast forecasts of the spatial–temporal evolution of three-dimensional (3D) state variables in geological carbon storage (GCS). However, training these models in high-dimensional space remains computationally demanding and prone to overfitting because of limited training data. This paper presents a novel workflow to address these challenges by integrating dimension reduction (DR) methods. Here, the proposed workflow employed pre-trained DR models to extract the latent variables of geological models and state variables and utilized the multi-layer perceptron (MLP) for constructing mapping functions between the input and output variables in latent spaces. Subsequently, the pre-trained reconstruction models converted the MLP-predicted latent state variables to their original high-dimensional form. Furthermore, we proposed a novel strategy for the DR and reconstruction of 3D saturation fields to account for the unique data characteristics of sparsity, nonuniformity, and discontinuity. The proposed strategy applied PCA and inverse PCA for 2D average saturation fields and developed a DL-based 3D reconstruction model, leveraging three 2D average saturation fields as input to produce a 3D saturation field as output. The pre-training of DR and reconstruction models and training of MLP models were conducted on 84 Gulf of Mexico (GoM) simulations and evaluated on 12 testing simulations. Each simulation contained 720 monthly time steps, with the first 360 months as the injection period and the rest as the post-injection period. The proposed workflow, incorporating DR and DL models, accurately predicts the normalized 3D pressure fields, achieving mean square error (MSE) of 2.92 × 10 -7 compared to the ground truth obtained from a full-physics simulator. Furthermore, the proposed strategy outperformed PCA and convolutional autoencoder (CAE) models on 3D saturation fields, resulting in minor workflow prediction errors with an MSE of 2.93 × 10 -5 . The results suggest the proposed workflow provides sufficient predictive fidelity across temporal and spatial scales, and enables a speedup of 160 times compared to the full-physics simulator, facilitating improved decision-making and risk assessment for large-scale GCS management in real-time scenarios.

3D reconstruction model↗

How do North American weather regimes drive wind energy at the sub-seasonal to seasonal timescales?

Abstract There has been an increasing need for forecasting power generation at the subseasonal to seasonal (S2S) timescales to support the operation, management, and planning of the wind-energy system. At the S2S timescales, atmospheric variability is largely related to recurrent and persistent weather patterns, referred to as weather regimes (WRs). In this study, we identify four WRs that influence wind resources over North America using a universal two-stage procedure approach. These WRs are responsible for large-scale wind and power production anomalies over the CONUS at the S2S timescales. The WR-based reconstruction explains up to 40% of the monthly variance of power production over the western United States, and the explanatory power of WRs generally increases with the increase of timescales. The identified relationship between WRs and power production reveals the potential and limitations of the regional WR-based wind resource assessment over different regions of the CONUS across multiple timescales.

54 ENVIRONMENTAL SCIENCES↗

Spatiotemporal forecasting of the edge localized modes in tokamak plasmas using neural networks

Artificial intelligence techniques have been increasingly adopted by the plasma and fusion science to address problems like plasma reconstruction, surrogate modeling, and tokamak/stellarator optimization. A key focus in sustained fusion research is the prediction and mitigation of edge-localized-modes (ELMs), instabilities that occur in short, periodic bursts and can cause erosion to the tokamak vessel wall. Recent research has demonstrated the power of neural networks in approximating continuous functions. In this work, we build spatiotemporal forecasting models that can predict the onset of ELMs and their evolution at early stages. We leverage recent advances in generative modeling, sequence-to-sequence modeling, and Fourier neural operators to propose architectures and training strategies that can learn to forecast short to long term dynamics of the noisy signals due to ELMs. We benchmark the developed model against a state-of-the-art foundation model using the beam emission spectroscopy (BES) data that captures the plasma fluctuations due to ELMs over a 8 x 8 spatial grid. Our models demonstrate high accuracy, outperforming the baselines, in predicting the evolution of BES signals during ELM events. Furthermore, the developed models exhibit high accuracy in predicting the rapid rise and relaxation of the signals due to ELMs within 30–80 µs.

edge localized modes↗

Data and code repository for "How do the weather regimes drive wind speed and power production at the sub-seasonal to seasonal timescales over the CONUS?"

There has been an increasing need for forecasting power generation at the sub-seasonal to seasonal (S2S) timescales to support the operation, management, and planning of the wind-energy system. At the S2S timescales, atmospheric variability is largely related to recurrent and persistent weather patterns, referred to as weather regimes (WRs). In the study "How do the weather regimes drive wind speed and power production at the sub-seasonal to seasonal timescales over the CONUS?", we identify four WRs that influence wind resources over North America using a self-organizing map (SOM) algorithm. These WRs are responsible for large-scale wind and power production anomalies over the CONUS at the S2S timescales. The WR-based reconstruction explains up to 50% of the monthly variance of power production over the western United States, and the explanatory power generally increases with the increase of timescales. The identified relationship between WRs and power production reveals the potential and limitations of the regional WR-based wind resource assessment over different regions of the CONUS across multiple timescales. This repository includes all the data and codes we use for analyses in this study. Users may use them to reproduce the results of this study on their end.

17 WIND ENERGY↗

A nudged hybrid analysis and modeling approach for realtime wake-vortex transport and decay prediction

Here, we put forth a long short-term memory (LSTM) nudging framework for the enhancement of reduced order models (ROMs) of fluid flows utilizing noisy measurements for air traffic improvements. Toward emerging applications of digital twins in aviation, the proposed approach allows for constructing a realtime predictive tool for wake-vortex transport and decay systems. We build on the fact that in realistic application, there are uncertainties in initial and boundary conditions, model parameters, as well as measurements. Moreover, conventional nonlinear ROMs based on Galerkin projection (GROMs) suffer from imperfection and solution instabilities, especially for advection-dominated flows with slow decay in the Kolmogorov n-width. In the presented LSTM nudging (LSTM-N) approach, we fuse forecasts from a combination of imperfect GROM and uncertain state estimates, with sparse Eulerian sensor measurements to provide more reliable predictions in a dynamical data assimilation framework. We illustrate our concept by solving the two-dimensional vorticity transport equation. We investigate the effects of measurements noise and state estimate uncertainty on the performance of the LSTM-N behavior. We also demonstrate that it can sufficiently handle different levels of temporal and spatial measurement sparsity, and offer a huge potential in developing next-generation digital twin technologies for aerospace applications.

42 ENGINEERING↗

Avoidance of disruptions on KSTAR due to vertical displacement events via novel real-time stability assessment

Disruption avoidance via the DECAF approach has been achieved on KSTAR using a novel real-time vertical stability assessment and a multiactuator feedback control strategy. The development of disruption avoidance strategies with reactor-relevant reliability is an urgent activity, enabling future fusion power plants. The stability metric employed is based on a new formulation of a vertical force gradient balance metric evaluated across the poloidal cross section of the plasma, with parameters tuned using historical data. Evaluation of this metric on a validation set of 400 recent KSTAR shots indicates >82% of Vertical displacement events can be avoided via feedback control. Essential to its calculation is the two-dimensional toroidal current density distribution in the plasma. Measurement of this profile faster than fully-converged equilibrium reconstructions can deliver is found to improve forecaster performance and is achieved with a surrogate model that takes as input magnetic diagnostic measurements and outputs the current profile on a basis comprising the top principal components of historical current profiles (from past equilibrium reconstructions). This method solves the non-uniqueness problem typically faced when reconstructing current profiles directly from diagnostics, while improving computational time and accuracy. On average, profiles produced by this model reach coefficients of determination of >0.99 with respect to those from equilibrium reconstructions. The avoidance actuators employed include poloidal field coils and an electron cyclotron current drive system. The multiactuator approach, as shown in this first demonstration, allows disruption avoidance while minimizing impact to operational performance. This ability, along with its flexibility and speed, makes this new approach an attractive option for avoiding these types of disruptions in reactors.

Tobin, Matthew [Columbia Univ., New York, NY (Unit↗

How extreme rainfall and failing dams unleashed the Derna flood disaster

On September 11, 2023, Storm Daniel unleashed unprecedented rainfall over the Wadi Derna watershed, triggering one of the most devastating floods in modern history, striking Derna, a coastal city in Libya. This study reconstructs the disaster using an integrated modeling approach that combines satellite imagery, hydrologic, hydraulic, and geotechnical simulations, machine learning, eyewitness accounts, and digital elevation data to assess the impact of cascading dam failures. Our findings reveal that the region’s dams, even if structurally sound, would have provided minimal protection against the extreme runoff. However, their failure unleashed a destructive surge wave, amplifying the disaster’s magnitude and devastation. Here, we show that the collapse of aging flood control infrastructures, compounded by inadequate risk assessment and emergency preparedness, dramatically escalated the disaster’s impact. Our findings underscore the urgent need for systematic dam safety evaluations, enhanced flood forecasting, and adaptive risk management strategies that address climate extremes and infrastructure vulnerabilities.

Hydrology↗

The Atacama Cosmology Telescope: Large-scale velocity reconstruction with the kinematic Sunyaev-Zel'dovich effect and DESI LRGs

The kinematic Sunyaev-Zel'dovich (kSZ) effect induces a non-zero density-density-temperature bispectrum, which we can use to reconstruct the large-scale velocity field from a combination of cosmic microwave background (CMB) and galaxy density measurements, in a procedure known as “kSZ velocity reconstruction”. This method has been forecast to constrain large-scale modes with future galaxy and CMB surveys, improving their measurement beyond what is possible with the galaxy surveys alone. Such measurements will enable tighter constraints on large-scale signals such as primordial non-Gaussianity, deviations from homogeneity, and modified gravity. In this work, we demonstrate a statistically significant measurement of kSZ velocity reconstruction for the first time, by applying quadratic estimators to the combination of the ACT DR6 CMB+kSZ map and the DESI LRG galaxies (with photometric redshifts) in order to reconstruct the velocity field. We do so using a formalism appropriate for the 2-dimensional projected galaxy fields that we use, which naturally incorporates the curved-sky effects important on the largest scales. We find evidence for the signal by cross-correlating with an external estimate of the velocity field from the spectroscopic BOSS survey and rejecting the null (no-kSZ) hypothesis at 3.8σ. Our work presents a first step towards the use of this observable for cosmological analyses.

Sunyaev-Zeldovich effect↗

Forecasting high-dimensional spatio-temporal systems from sparse measurements

This paper introduces a new neural network architecture designed to forecast high-dimensional spatio-temporal data using only sparse measurements. The architecture uses a two-stage end-to-end framework that combines neural ordinary differential equations (NODEs) with vision transformers. Initially, our approach models the underlying dynamics of complex systems within a low-dimensional space; and then it reconstructs the corresponding high-dimensional spatial fields. Many traditional methods involve decoding high-dimensional spatial fields before modeling the dynamics, while some other methods use an encoder to transition from high-dimensional observations to a latent space for dynamic modeling. In contrast, our approach directly uses sparse measurements to model the dynamics, bypassing the need for an encoder. This direct approach simplifies the modeling process, reduces computational complexity, and enhances the efficiency and scalability of the method for large datasets. We demonstrate the effectiveness of our framework through applications to various spatio-temporal systems, including fluid flows and global weather patterns. Although sparse measurements have limitations, our experiments reveal that they are sufficient to forecast system dynamics accurately over long time horizons. Our results also indicate that the performance of our proposed method remains robust across different sensor placement strategies, with further improvements as the number of sensors increases. This robustness underscores the flexibility of our architecture, particularly in real-world scenarios where sensor data is often sparse and unevenly distributed.

97 MATHEMATICS AND COMPUTING↗

Idealized simulations of wind farm interactions with intermittent turbulence in stable boundary layer conditions

Stable atmospheric boundary layer conditions typically correspond to weak turbulence levels, but intermittent periods of elevated turbulence can occur during otherwise quiescent conditions. The interaction between intermittent turbulence and wind turbines is not well understood because of sparse observations, as well as the difficulty in realistically resolving small-scale turbulence during strongly stable conditions with numerical simulations. In this study, an explicit filtering and reconstruction approach for large-eddy simulation (LES) is used to simulate weakly and strongly stable conditions, with surface cooling rates of −0.2 and −2.0 K h −1 , respectively. This approach can sustain resolved background turbulence at relatively coarse grid spacing and stronger stratification compared to conventional closures, permitting more realistic intermittent stable boundary layer (SBL) turbulence. The idealized LES capability of the Weather Research and Forecasting model is employed with turbine rotors parameterized using generalized actuator disks to examine (1) how the presence of turbine wakes affects SBL evolution and (2) the effect of intermittent turbulence on power production and wake recovery. Wakes increase mixing and deepen the SBL, with a stronger effect under strongly stable conditions, primarily because the SBL is shallower and closer to the top of the wind turbine rotor layer. Intermittent turbulence does not have a significant impact on mean power generation and wake recovery because the relevant intermittent turbulent structures in this study only affect the bottom half of the rotor disk. Power production is, however, more variable during periods of elevated turbulence, demonstrating the impact of SBL intermittency. This study uses an idealized configuration, focusing on LES model performance and physical understanding, with the goal of informing future simulations of the conditions observed during the American Wake Experiment.

Energy - Wind↗

Bayesian Physics Informed Spatio-Temporal Network for Streamflow Data Imputation

Reliable reconstruction of incomplete streamflow records is critical for improving hydrological forecasting, flood preparedness, and water resource management. However, large observational gaps and uncertainties in governing physical parameters limit the accuracy of traditional statistical and machinelearning imputation frameworks. To address these challenges, we develop a Bayesian Physics-Informed Spatio-Temporal Network (BPI-STNet) that jointly captures spatial and temporal dependencies while enforcing hydrologic consistency through embedded physical constraints. The framework integrates a GraphSAGE-LSTM architecture to model spatial connectivity across gauges and temporal flow dynamics, coupled with a Bayesian update mechanism to estimate uncertain parameters in a simplified water-balance framework. Unlike conventional physics-informed networks that rely on sampling-based posterior estimation, BPI-STNet derives an analytic solution to the inverse problem, allowing closed-form Bayesian updates of uncertain parameters Λ={α,β,k} using Gaussian priors and likelihoods. Applied to daily observations from the Susquehanna River Basin (1980-2022), BPI-STNet achieves substantial improvements over a purely data-driven RGNN baseline, which reduced RMSE by 23 % and MAE by 9 %, and achieving an average NSE values up to 0.96. The results demonstrate that coupling Bayesian inference with physics-informed learning yields physically consistent, uncertainty-aware reconstructions that preserve the temporal persistence and statistical distribution of observed flows. The proposed framework establishes a generalizable paradigm for data-sparse hydrologic systems where both data fidelity and physical interpretability are essential.

Krishnan Kutty Ambika, Anukesh [ORNL] (ORCID:00000↗