Search NASA⌕ Search

SEARCH · Search NASA

Results for “Sparse inverse covariance estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

12 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

On the Solution of ℓ 0 -Constrained Sparse Inverse Covariance Estimation Problems

The sparse inverse covariance matrix is used to model conditional dependencies between variables in a graphical model to fit a multivariate Gaussian distribution. Estimating the matrix from data are well known to be computationally expensive for large-scale problems. Sparsity is employed to handle noise in the data and to promote interpretability of a learning model. Although the use of a convex ℓ 1 regularizer to encourage sparsity is common practice, the combinatorial ℓ 0 penalty often has more favorable statistical properties. In this paper, we directly constrain sparsity by specifying a maximally allowable number of nonzeros, in other words, by imposing an ℓ 0 constraint. Here, we introduce an efficient approximate Newton algorithm using warm starts for solving the nonconvex ℓ 0 -constrained inverse covariance learning problem. Numerical experiments on standard data sets show that the performance of the proposed algorithm is competitive with state-of-the-art methods.

$\ell_0$-Constrained↗
DOE OSTI2018

Communication-avoiding optimization methods for distributed massive-scale sparse inverse covariance estimation

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ≈819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2018

Communication-Avoiding Optimization Methods for Distributed Massive-Scale Sparse Inverse Covariance Estimation.

Across a variety of scientific disciplines, sparse inverse covariance estimation is a popular tool for capturing the underlying dependency relationships in multivariate data. Unfortunately, most estimators are not scalable enough to handle the sizes of modern high-dimensional data sets (often on the order of terabytes), and assume Gaussian samples. To address these deficiencies, we introduce HP-CONCORD, a highly scalable optimization method for estimating a sparse inverse covariance matrix based on a regularized pseudolikelihood framework, without assuming Gaussianity. Our parallel proximal gradient method uses a novel communication-avoiding linear algebra algorithm and runs across a multi-node cluster with up to 1k nodes (24k cores), achieving parallel scalability on problems with up to ~819 billion parameters (1.28 million dimensions); even on a single node, HP-CONCORD demonstrates scalability, outperforming a state-of-the-art method. We also use HP-CONCORD to estimate the underlying dependency structure of the brain from fMRI data, and use the result to identify functional regions automatically. The results show good agreement with a clustering from the neuroscience literature.

Koanantakool, Penporn↗
DOE OSTI2014

A convex pseudolikelihood framework for high dimensional partial correlation estimation with convergence guarantees

Sparse high dimensional graphical model selection is a topic of much interest in modern day statistics. A popular approach is to apply l1-penalties to either parametric likelihoods, or regularized regression/pseudolikelihoods, with the latter having the distinct advantage that they do not explicitly assume Gaussianity. As none of the popular methods proposed for solving pseudolikelihood-based objective functions have provable convergence guarantees, it is not clear whether corresponding estimators exist or are even computable, or if they actually yield correct partial correlation graphs. We propose a new pseudolikelihood-based graphical model selection method that aims to overcome some of the shortcomings of current methods, but at the same time retain all their respective strengths. In particular, we introduce a novel framework that leads to a convex formulation of the partial covariance regression graph problem, resulting in an objective function comprised of quadratic forms. The objective is then optimized via a coordinate wise approach. The specific functional form of the objective function facilitates rigorous convergence analysis leading to convergence guarantees; an important property that cannot be established by using standard results, when the dimension is larger than the sample size, as is often the case in high dimensional applications. These convergence guarantees ensure that estimators are well defined under very general conditions and are always computable. In addition, the approach yields estimators that have good large sample properties and also respect symmetry. Furthermore, application to simulated and real data, timing comparisons and numerical convergence is demonstrated. We also present a novel unifying framework that places all graphical pseudolikelihood methods as special cases of a more general formulation, leading to important insights.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2017

HP-CONCORD v0.1

High performance parallel computation of sparse inverse covariance matrix estimation.

Koanantakool, Penporn↗
DOE OSTI2022

Physics-informed Karhunen-Loeve and Neural Network Approximations for Solving Inverse Differential Equation Problems

Here we present the PI-CKL-NN method for parameter estimation in differential equation (DE) models given sparse measurements of the parameters and states. In the proposed approach, the space- or time-dependent parameters are approximated by Karhunen-Loeve (KL) expansions that are conditioned on the parameters’ measurements, and the states are approximated by deep neural networks (DNNs). The unknown weights in the KL expansions and DNNs are found my minimizing the cost function that enforces the measurements of the states the DE constraint. Regularization is achieved by adding the l2 norm of the conditional KL coefficients into the loss function. Our approach assumes that the parameter fields are correlated in space or time and enforces the statistical knowledge (the mean and the covariance function) in addition to the DE constraints and measurements as opposed to the physics-informed neural network (PINN) and other similar physics-informed machine learning methods where only DE constraints and data are used for parameter estimation. We use the PI-CKL-NN method for parameter estimation in an ordinary differential equation with an unknown time-dependent parameter and the one- and two-dimensional partial differential diffusion equations with unknown space-dependent diffusion coefficients. We also demonstrate that PI-CKL-NN is more accurate than the PINN method, especially when the observations of the parameters are very sparse

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2020

Towards real-time monitoring: data assimilated time-lapse full waveform inversion for seismic velocity and uncertainty estimation

SUMMARY Rapid development of time-lapse seismic monitoring instrumentations has made it possible to collect dense time-lapse data for tomographically retrieving time-lapse (even continuous) images of subsurface changes. While traditional time-lapse full waveform inversion (TLFWI) algorithms are designed for sparse time-lapse surveys, they lack of effective temporal constraint on time-lapse data, and, more importantly, lack of the uncertainty estimation of the TLFWI results that is critical for further interpretation. Here, we propose a new data assimilation TLFWI method, using hierarchical matrix powered extended Kalman filter (HiEKF) to quantify the image uncertainty. Compared to existing Kalman filter algorithms, HiEKF allows to store and update a data-sparse representation of the cross-covariance matrices and propagate model errors without expensive operations involving covariance matrices. Hence, HiEKF is computationally efficient and applicable to 3-D TLFWI problems. Then, we reformulate TLFWI in the framework of HiEKF (termed hereafter as TLFWI-HiEKF) to predict time-lapse images of subsurface spatiotemporal velocity changes and simultaneously quantify the uncertainty of the inverted velocity changes over time. We demonstrate the validity and applicability of TLFWI–HiEKF with two realistic CO2 monitoring models derived from Frio-II and Cranfield CO2 injection sites, respectively. In both 2-D and 3-D examples, the inverted high-resolution time-lapse velocity results clearly reveal a continuous velocity reduction due to the injection of CO2. Moreover, the accuracy of the model is increasing over time by assimilating more time-lapse data while the standard deviation is decreasing over lapsed time. We expect TLFWI-HiEKF to be equipped with real-time seismic monitoring systems for continuously imaging the distribution of subsurface gas and fluids in the future large-scale CO2 sequestration experiments and reservoir management.

58 GEOSCIENCES↗
DOE OSTI2022

Scaled Vecchia Approximation for Fast Computer-Model Emulation

Many scientific phenomena are studied using computer experiments consisting of multiple runs of a computer model while varying the input settings. Gaussian processes (GPs) are a popular tool for the analysis of computer experiments, enabling interpolation between input settings, but direct GP inference is computationally infeasible for large datasets. We adapt and extend a powerful class of GP methods from spatial statistics to enable the scalable analysis and emulation of large computer experiments. Specifically, we apply Vecchia’s ordered conditional approximation in a transformed input space, with each input scaled according to how strongly it relates to the computer-model response. The scaling is learned from the data by estimating parameters in the GP covariance function using Fisher scoring. Our methods are highly scalable, enabling estimation, joint prediction, and simulation in near-linear time in the number of model runs. In several numerical examples, our approach substantially outperformed existing methods.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2023

Non-asymptotic analysis of ensemble Kalman updates: effective dimension and localization

Many modern algorithms for inverse problems and data assimilation rely on ensemble Kalman updates to blend prior predictions with observed data. Ensemble Kalman methods often perform well with a small ensemble size, which is essential in applications where generating each particle is costly. This paper develops a non-asymptotic analysis of ensemble Kalman updates, which rigorously explains why a small ensemble size suffices if the prior covariance has moderate effective dimension due to fast spectrum decay or approximate sparsity. Here, we present our theory in a unified framework, comparing everal implementations of ensemble Kalman updates that use perturbed observations, square root filtering and localization. As part of our analysis, we develop new dimension-free covariance estimation bounds for approximately sparse matrices that may be of independent interest.

Mathematics↗
NASA NTRS1989

The inverse gravimetric problem in gravity modelling

One of the main purposes of geodesy is to determine the gravity field of the Earth in the space outside its physical surface. This purpose can be pursued without any particular knowledge of the internal density even if the exact shape of the physical surface of the Earth is not known, though this seems to entangle the two domains, as it was in the old Stoke's theory before the appearance of Molodensky's approach. Nevertheless, even when large, dense and homogeneous data sets are available, it was always recognized that subtracting from the gravity field the effect of the outer layer of the masses (topographic effect) yields a much smoother field. This is obviously more important when a sparse data set is bad so that any smoothing of the gravity field helps in interpolating between the data without raising the modeling error, this approach is generally followed because it has become very cheap in terms of computing time since the appearance of spectral techniques. The mathematical description of the Inverse Gravimetric Problem (IGP) is dominated mainly by two principles, which in loose terms can be formulated as follows: the knowledge of the external gravity field determines mainly the lateral variations of the density; and the deeper the density anomaly giving rise to a gravity anomaly, the more improperly posed is the problem of recovering the former from the latter. The statistical relation between rho and n (and its inverse) is also investigated in its general form, proving that degree cross-covariances have to be introduced to describe the behavior of rho. The problem of the simultaneous estimate of a spherical anomalous potential and of the external, topographic masses is addressed criticizing the choice of the mixed collection approach.

Sanso, F.↗
NASA NTRS2014

Carbon Monitoring System Flux Estimation and Attribution: Impact of ACOS-GOSAT X(CO2) Sampling on the Inference of Terrestrial Biospheric Sources and Sinks

Using an Observing System Simulation Experiment (OSSE), we investigate the impact of JAXA Greenhouse gases Observing SATellite 'IBUKI' (GOSAT) sampling on the estimation of terrestrial biospheric flux with the NASA Carbon Monitoring System Flux (CMS-Flux) estimation and attribution strategy. The simulated observations in the OSSE use the actual column carbon dioxide (X(CO2)) b2.9 retrieval sensitivity and quality control for the year 2010 processed through the Atmospheric CO2 Observations from Space algorithm. CMS-Flux is a variational inversion system that uses the GEOS-Chem forward and adjoint model forced by a suite of observationally constrained fluxes from ocean, land and anthropogenic models. We investigate the impact of GOSAT sampling on flux estimation in two aspects: 1) random error uncertainty reduction and 2) the global and regional bias in posterior flux resulted from the spatiotemporally biased GOSAT sampling. Based on Monte Carlo calculations, we find that global average flux uncertainty reduction ranges from 25% in September to 60% in July. When aggregated to the 11 land regions designated by the phase 3 of the Atmospheric Tracer Transport Model Intercomparison Project, the annual mean uncertainty reduction ranges from 10% over North American boreal to 38% over South American temperate, which is driven by observational coverage and the magnitude of prior flux uncertainty. The uncertainty reduction over the South American tropical region is 30%, even with sparse observation coverage. We show that this reduction results from the large prior flux uncertainty and the impact of non-local observations. Given the assumed prior error statistics, the degree of freedom for signal is approx.1132 for 1-yr of the 74 055 GOSAT X(CO2) observations, which indicates that GOSAT provides approx.1132 independent pieces of information about surface fluxes. We quantify the impact of GOSAT's spatiotemporally sampling on the posterior flux, and find that a 0.7 gigatons of carbon bias in the global annual posterior flux resulted from the seasonally and diurnally biased sampling when using a diagonal prior flux error covariance.

biased sampling↗