Search NASA⌕ Search

SEARCH · Search NASA

Results for “ensemble Kalman filter”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2018

Precise Fault Location on Transmission Lines Using Ensemble Kalman Filter

Accurate pin-pointing of faults on transmission lines minimizes outage time, labor, and costs. This letter proposes a simple yet effective ensemble Kalman filter (EnKF) approach to accurately locating the faults on transmission lines within half-cycle time. The approach is easy to implement and does not require foreknowledge of either the fault type or an approximate guess of the fault location. Extensive results validate the effectiveness and accuracy of the EnKF-based fault location approach.

Fan, Rui↗
DOE OSTI2018

Gaussian Mixture Model-Based Ensemble Kalman Filter for Machine Parameter Calibration

This letter proposes a novel Gaussian mixture model-based ensemble Kalman filter (GMM-EnKF) approach to the accurate calibration of the parameters of machine dynamic models. This approach aims to overcome some practical challenges of machine parameter calibration. Results show the proposed approach can provide high-accuracy calibrated parameters even when the machine operates under unbalanced network conditions and the measurement noises are non-Gaussian.

Fan, Rui↗
DOE OSTI2021

A Multifidelity Ensemble Kalman Filter with Reduced Order Control Variates

This work develops a new multi delity ensemble Kalman lter (MFEnKF) algorithm based on linear control variate framework. The approach allows for rigorous multi delity extensions of the EnKF, where the uncertainty in coarser delities in the hierarchy of models represent control variates for the uncertainty in ner delities. Small ensembles of high delity model runs are com- plemented by larger ensembles of cheaper, lower delity runs, to obtain much improved analyses at only small additional computational costs. We investigate the use of reduced order models as coarse delity control variates in the MFEnKF, and provide analyses to quantify the improvements over the traditional ensemble Kalman lters. We apply these ideas to perform data assimilation with a quasi-geostrophic test problem, using direct numerical simulation and a corresponding POD-Galerkin reduced order model. Numerical results show that the two- delity MFEnKF provides better analyses than existing EnKF algorithms at comparable or reduced computational costs.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

Ensemble Kalman Filter for Dynamic State Estimation of Power Grids Stochastically Driven by Time-correlated Mechanical Input Power

State and parameter estimation of power transmission networks is important for monitoring power grid operating conditions and analyzing transient stability. Wind power generation depends on fluctuating input power levels, which are correlated in time and contribute to uncertainty in turbine dynamical models. The ensemble Kalman filter (EnKF), a standard state estimation technique, uses a deterministic forecast and does not explicitly model time-correlated noise in parameters such as mechanical input power. However, this uncertainty affects the probability of fault-induced transient instability and increased prediction bias. Here a novel approach is to model input power noise with time-correlated stochastic fluctuations, and integrate them with the network dynamics during the forecast. While the EnKF has been used to calibrate constant parameters in turbine dynamical models, the calibration of a statistical model for a time-correlated parameter has not been investigated. In this study, twin experiments on a standard transmission network test case are used to validate our time-correlated noise model framework for state estimation of unsteady operating conditions and transient stability analysis, and a methodology is proposed for the inference of the mechanical input power time-correlation length parameter using time-series data from PMUs monitoring power dynamics at generator buses.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1999

Massively Parallel Assimilation of TOGA/TAO and Topex/Poseidon Measurements into a Quasi Isopycnal Ocean General Circulation Model Using an Ensemble Kalman Filter

A massively parallel ensemble Kalman filter (EnKF)is used to assimilate temperature data from the TOGA/TAO array and altimetry from TOPEX/POSEIDON into a Pacific basin version of the NASA Seasonal to Interannual Prediction Project (NSIPP)ls quasi-isopycnal ocean general circulation model. The EnKF is an approximate Kalman filter in which the error-covariance propagation step is modeled by the integration of multiple instances of a numerical model. An estimate of the true error covariances is then inferred from the distribution of the ensemble of model state vectors. This inplementation of the filter takes advantage of the inherent parallelism in the EnKF algorithm by running all the model instances concurrently. The Kalman filter update step also occurs in parallel by having each processor process the observations that occur in the region of physical space for which it is responsible. The massively parallel data assimilation system is validated by withholding some of the data and then quantifying the extent to which the withheld information can be inferred from the assimilation of the remaining data. The distributions of the forecast and analysis error covariances predicted by the ENKF are also examined.

Keppenne, Christian L.↗
DOE OSTI2021

Penalized ensemble Kalman filters for high dimensional non-linear systems

The ensemble Kalman filter (EnKF) is a data assimilation technique that uses an ensemble of models, updated with data, to track the time evolution of a usually non-linear system. It does so by using an empirical approximation to the well-known Kalman filter. However, its performance can suffer when the ensemble size is smaller than the state space, as is often necessary for computationally burdensome models. This scenario means that the empirical estimate of the state covariance is not full rank and possibly quite noisy. To solve this problem in this high dimensional regime, we propose a computationally fast and easy to implement algorithm called the penalized ensemble Kalman filter (PEnKF). Under certain conditions, it can be theoretically proven that the PEnKF will be accurate (the estimation error will converge to zero) despite having fewer ensemble members than state dimensions. Further, as contrasted to localization methods, the proposed approach learns the covariance structure associated with the dynamical system. These theoretical results are supported with simulations of several non-linear and high dimensional systems.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Autodifferentiable Ensemble Kalman Filters

Data assimilation is concerned with sequentially estimating a temporally evolving state. This task, which arises in a wide range of scientific and engineering applications, is particularly challenging when the state is high-dimensional and the state-space dynamics are unknown. This paper introduces a machine learning framework for learning dynamical systems in data assimilation. Here, our auto-differentiable ensemble Kalman filters (AD-EnKFs) blend ensemble Kalman filters for state recovery with machine learning tools for learning the dynamics. In doing so, AD-EnKFs leverage the ability of ensemble Kalman filters to scale to high-dimensional states and the power of automatic differentiation to train high-dimensional surrogate models for the dynamics. Numerical results using the Lorenz-96 model show that AD-EnKFs outperform existing methods that use expectation-maximization or particle filters to merge data assimilation and machine learning. In addition, AD-EnKFs are easy to implement and require minimal tuning.

autodifferentiation↗
NASA NTRS2001

Design and Implementation of a Parallel Multivariate Ensemble Kalman Filter for the Poseidon Ocean General Circulation Model

A multivariate ensemble Kalman filter (MvEnKF) implemented on a massively parallel computer architecture has been implemented for the Poseidon ocean circulation model and tested with a Pacific Basin model configuration. There are about two million prognostic state-vector variables. Parallelism for the data assimilation step is achieved by regionalization of the background-error covariances that are calculated from the phase-space distribution of the ensemble. Each processing element (PE) collects elements of a matrix measurement functional from nearby PEs. To avoid the introduction of spurious long-range covariances associated with finite ensemble sizes, the background-error covariances are given compact support by means of a Hadamard (element by element) product with a three-dimensional canonical correlation function. The methodology and the MvEnKF configuration are discussed. It is shown that the regionalization of the background covariances; has a negligible impact on the quality of the analyses. The parallel algorithm is very efficient for large numbers of observations but does not scale well beyond 100 PEs at the current model resolution. On a platform with distributed memory, memory rather than speed is the limiting factor.

Keppenne, Christian L.↗
DOE OSTI2023

A Stochastic Covariance Shrinkage Approach in Ensemble Transform Kalman Filtering

The Ensemble Kalman Filters (EnKF) employ a Monte-Carlo approach to represent covariance information, and are affected by sampling errors in operational settings where the number of model realizations is much smaller than the model state dimension. To alleviate the effects of these errors EnKF relies on model-specific heuristics such as covariance localization, which takes advantage of the spatial locality of correlations among the model variables. This work proposes an approach to alleviate sampling errors that utilizes a locally averaged-in-time dynamics of the model, described in terms of a climatological covariance of the dynamical system. We use this covariance as the target matrix in covariance shrinkage methods, and develop a stochastic covariance shrinkage approach where synthetic ensemble members are drawn to enrich both the ensemble subspace and the ensemble transformation. We additionally provide for a way in which this methodology can be localized similar to the state-of-the-art LETKF method, and that for a certain model setup, our methodology significantly outperforms it.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2019

ENSEMBLE KALMAN FILTERS FOR RELIABILITY ESTIMATION IN PERFUSION INFERENCE

We consider the solution of inverse problems in dynamic contrast–enhanced imaging by means of ensemble Kalman filters. Our quantity of interest is blood perfusion, i.e., blood flow rates in tissue. While existing approaches to compute blood perfusion parameters for given time series of radiological measurements mainly rely on deterministic, deconvolution–based methods, we aim at recovering probabilistic solution information for given noisy measurements. To this end, we model radiological image capturing as a sequential data assimilation process and solve it by an ensemble Kalman filter. Thereby, we recover deterministic results as an ensemble–based mean and are able to compute reliability information such as probabilities for the perfusion to be in a given range. Our target application is the inference of blood perfusion parameters in the human brain. A numerical study shows promising results for artificial measurements generated by a digital perfusion phantom.

Engineering↗
DOE OSTI2016

Data assimilation in a solar dynamo model using Ensemble Kalman Filters: sensitivity and robustness in reconstruction of meridional flow speed

We implement an Ensemble Kalman Filter procedure using the Data Assimilation Research Testbed for assimilating “synthetic” meridional flow-speed data in a Babcock–Leighton-type flux-transport solar dynamo model. By performing several “observing system simulation experiments,” we reconstruct time variation in meridional flow speed and analyze sensitivity and robustness of reconstruction. Using 192 ensemble members including 10 observations, each with 4% error, we find that flow speed is reconstructed best if observations of near-surface poloidal fields from low latitudes and tachocline toroidal fields from midlatitudes are assimilated. If observations include a mixture of poloidal and toroidal fields from different latitude locations, reconstruction is reasonably good for ⩽40% error in low-latitude data, even if observational error in polar region data becomes 200%, but deteriorates when observational error increases in low- and midlatitude data. Solar polar region observations are known to contain larger errors than those in low latitudes; our forward operator (a flux-transport dynamo model here) can sustain larger errors in polar region data, but is more sensitive to errors in low-latitude data. An optimal reconstruction is obtained if an assimilation interval of 15 days is used; 10- and 20-day assimilation intervals also give reasonably good results. Assimilation intervals <5 days do not produce faithful reconstructions of flow speed, because the system requires a minimum time to develop dynamics to respond to flow variations. Reconstruction also deteriorates if an assimilation interval >45 days is used, because the system’s inherent memory interferes with its short-term dynamics during a substantially long run without updating.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Multifidelity Ensemble Kalman Filtering Using Surrogate Models Defined by Theory-Guided Autoencoders

Data assimilation is a Bayesian inference process that obtains an enhanced understanding of a physical system of interest by fusing information from an inexact physics-based model, and from noisy sparse observations of reality. The multifidelity ensemble Kalman filter (MFEnKF) recently developed by the authors combines a full-order physical model and a hierarchy of reduced order surrogate models in order to increase the computational efficiency of data assimilation. The standard MFEnKF uses linear couplings between models, and is statistically optimal in case of Gaussian probability densities. This work extends the MFEnKF into to make use of a broader class of surrogate model such as those based on machine learning methods such as autoencoders non-linear couplings in between the model hierarchies. We identify the right-invertibility property for autoencoders as being a key predictor of success in the forecasting power of autoencoder-based reduced order models. We propose a methodology that allows us to construct reduced order surrogate models that are more accurate than the ones obtained via conventional linear methods. Numerical experiments with the canonical Lorenz'96 model illustrate that nonlinear surrogates perform better than linear projection-based ones in the context of multifidelity ensemble Kalman filtering. We additionality show a large-scale proof-of-concept result with the quasi-geostrophic equations, showing the competitiveness of the method with a traditional reduced order model-based MFEnKF.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Reduced-order autodifferentiable ensemble Kalman filters

This paper introduces a computational framework to reconstruct and forecast a partially observed state that evolves according to an unknown or expensive-to-simulate dynamical system. Our reduced-order autodifferentiable ensemble Kalman filters (ROAD-EnKFs) learn a latent low-dimensional surrogate model for the dynamics and a decoder that maps from the latent space to the state space. The learned dynamics and decoder are then used within an EnKF to reconstruct and forecast the state. Numerical experiments show that if the state dynamics exhibit a hidden low-dimensional structure, ROAD-EnKFs achieve higher accuracy at lower computational cost compared to existing methods. If such structure is not expressed in the latent state dynamics, ROAD-EnKFs achieve similar accuracy at lower cost, making them a promising approach for surrogate state reconstruction and forecasting.

Mathematics↗
NASA NTRS2021

Geomagnetic Secular Variation Forecast using the NASA GEMS Ensemble Kalman filter: A Candidate SV Model for IGRF‑13

We have produced a 5-year mean secular variation (SV ) of the geomagnetic field for the period 2020–2025. We use the NASA Geomagnetic Ensemble Modeling System (GEMS), which consists of the NASA Goddard geodynamo model and ensemble Kalman filter (EnKF) with 400 ensemble members. Geomagnetic field models are used as observations for the assimilation, including gufm1 (1590–1960), CM4 (1961–2000) and CM6 (2001–2019). The forecast involves a bias correction scheme that assumes that the model bias changes on timescales much longer than the forecast period, so that they can be removed by successive forecast series. The algorithm was validated on the time period 2010-2015 by comparing with CM6 before being applied to the 2020–2025 time period. This forecast has been sub-mitted as a candidate predictive model of IGRF-13 for the period 2020–2025.

Andrew Tangborn↗
NASA NTRS1999

Reconstruction of the 1997/1998 El Nino from TOPEX/POSEIDON and TOGA/TAO Data Using a Massively Parallel Pacific-Ocean Model and Ensemble Kalman Filter

Two massively parallel data assimilation systems in which the model forecast-error covariances are estimated from the distribution of an ensemble of model integrations are applied to the assimilation of 97-98 TOPEX/POSEIDON altimetry and TOGA/TAO temperature data into a Pacific basin version the NASA Seasonal to Interannual Prediction Project (NSIPP)ls quasi-isopycnal ocean general circulation model. in the first system, ensemble of model runs forced by an ensemble of atmospheric model simulations is used to calculate asymptotic error statistics. The data assimilation then occurs in the reduced phase space spanned by the corresponding leading empirical orthogonal functions. The second system is an ensemble Kalman filter in which new error statistics are computed during each assimilation cycle from the time-dependent ensemble distribution. The data assimilation experiments are conducted on NSIPP's 512-processor CRAY T3E. The two data assimilation systems are validated by withholding part of the data and quantifying the extent to which the withheld information can be inferred from the assimilation of the remaining data. The pros and cons of each system are discussed.

Keppenne, C. L.↗
DOE OSTI2025

Covariance operator estimation: Sparsity, lengthscale, and ensemble Kalman filters

This paper investigates covariance operator estimation via thresholding. For Gaussian random fields with approximately sparse covariance operators, we establish non-asymptotic bounds on the estimation error in terms of the sparsity level of the covariance and the expected supremum of the field. We prove that thresholded estimators enjoy an exponential improvement in sample complexity compared with the standard sample covariance estimator if the field has a small correlation lengthscale. As an application of the theory, we study thresholded estimation of covariance operators within ensemble Kalman filters.

Covariance operator estimation↗
NASA NTRS2019

Application of Synoptic Magnetograms for Prediction of Solar Activity Using Ensemble Kalman Filter

Solar activity predictions using the data assimilation approach have demonstrated great potential to build reliable long-term forecasts of solar activity. In particular, it has been shown that the Ensemble Kalman Filter (EnKF) method applied to a non-linear dynamo model is capable of predicting solar activity up to one sunspot cycle ahead in time, as well as estimating the properties of the next cycle a few years before it begins. These developments assume an empirical relationship between the mean toroidal magnetic field flux and the sunspot number. Estimated from the sunspot number series, variations of the toroidal field have been used to assimilate the data into the Parker-Kleeorin-Ruzmakin (PKR) dynamo model by applying the EnKF method. The dynamo model describes the evolution of the toroidal and poloidal components of the magnetic field and the magnetic helicity. Full-disk magnetograms provide more accurate and complete input data by constraining both the toroidal and poloidal global field components, but these data are available only for the last four solar cycles. In this presentation, using the available magnetogram data, we discuss development of the methodology and forecast quality criteria (including forecast uncertainties and sources of errors). We demonstrate the influence of limited time series observations on the accuracy of solar activity predictions. We present EnKF predictions of the upcoming Solar Cycle 25 based on both the sunspot number series and observed magnetic fields and discuss the uncertainties and potential of the data assimilation approach.

Kitiashvili, Irina N.↗