Search NASA⌕ Search

SEARCH · Search NASA

Results for “noise covariance”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2
All sourcesNASA NTRSDOE OSTI
DOE OSTI2021

Cosmic shear power spectra in practice

Cosmic shear is one of the most powerful probes of Dark Energy, targeted by several current and future galaxy surveys. Lensing shear, however, is only sampled at the positions of galaxies with measured shapes in the catalog, making its associated sky window function one of the most complicated amongst all projected cosmological probes of inhomogeneities, as well as giving rise to inhomogeneous noise. Partly for this reason, cosmic shear analyses have been mostly carried out in real-space, making use of correlation functions, as opposed to Fourier-space power spectra. Since the use of power spectra can yield complementary information and has numerical advantages over real-space pipelines, it is important to develop a complete formalism describing the standard unbiased power spectrum estimators as well as their associated uncertainties. Building on previous work, this paper contains a study of the main complications associated with estimating and interpreting shear power spectra, and presents fast and accurate methods to estimate two key quantities needed for their practical usage: the noise bias and the Gaussian covariance matrix, fully accounting for survey geometry, with some of these results also applicable to other cosmological probes. Here, we demonstrate the performance of these methods by applying them to the latest public data releases of the Hyper Suprime-Cam and the Dark Energy Survey collaborations, quantifying the presence of systematics in our measurements and the validity of the covariance matrix estimate. We make the resulting power spectra, covariance matrices, null tests and all associated data necessary for a full cosmological analysis publicly available.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Coherent manipulation of graph states composed of finite-energy Gottesman-Kitaev-Preskill-encoded qubits

Graph states are a central resource in measurement-based quantum information processing. In the photonic qubit architecture based on Gottesman-Kitaev-Preskill (GKP) encoding, the generation of high-fidelity graph states composed of realistic, finite-energy approximate GKP-encoded qubits thus constitutes a key task. We consider the finite-energy approximation of GKP-qubit states given by a coherent superposition of shifted finite-squeezed vacuum states, where the displacements are Gaussian distributed. We present an exact description of graph states composed of such approximate GKP qubits as a coherent superposition of a Gaussian ensemble of randomly displaced ideal GKP-qubit graph states. Using standard Gaussian dynamics, we track the transformation of the covariance matrix and the mean-displacement vector elements of the Gaussian distribution of the ensemble under tools such as GKP-Steane error-correction and fusion operations that can be used to grow large high-fidelity GKP-qubit graph states. The covariance matrix elements capture the noise in the graph state due to the finite-energy approximation of GKP qubits, while the mean displacements relate to the possible absolute shift errors on the individual qubits arising conditionally from the homodyne measurements that are a part of these tools. Our work thus pins down an exact coherent error model for graph states generated from truly finite-energy GKP qubits, which can shed light on their error-correction properties.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2021

Systematic and statistical uncertainties of the hilbert-transform based high-precision FID frequency extraction method

Pulsed nuclear magnetic resonance (NMR) is widely used in high-precision magnetic field measurements. The absolute value of the magnetic field is determined from the precession frequency of nuclear magnetic moments. The Hilbert transform is widely used to extract the phase function from the observed free induction decay (FID) signal and then its frequency. In this paper, a detailed implementation of a Hilbert-transform based FID frequency extraction method is described. How artifacts and noise level in the FID signal affect the extracted phase function are derived analytically. A method of mitigating the artifacts in the extracted phase function of an FID is discussed. Correlations between noises of the phase function samples are studied for different noise spectra. We discovered that the error covariance matrix for the extracted phase function is nearly singular and improper for constructing the $\chi^2$ used in the fitting routine. A down-sampling method for fixing the singular covariance matrix has been developed, so that the minimum $\chi^2$-fit yields properly the statistical uncertainty of the extracted frequency. Other practical methods of obtaining the statistical uncertainty are also discussed.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2021

Novel 2D velocity estimation method for large transient events in plasmas

Dynamics of fast transient events are challenging to be analyzed with high time resolution. Such events can occur in fusion plasmas such as the filaments during edge-localized modes (ELMs). Here, we present a robust method—the spatial displacement estimation—for estimating the displacements of structures with fast dynamics from high spatial and time resolution imaging diagnostics [e.g., gas-puff imaging (GPI)] with sampling time temporal resolution. First, a background suppression method is shown, which suppresses the slowly time-evolving and spatially non-uniform background in the signal. In the second step, a two-dimensional polynomial trend subtraction method is presented to tackle the remaining polynomial order trend in the signal. After performing these pre-processing steps, the spatial displacement of the propagating structure is estimated from the two-dimensional spatial cross-correlation coefficient function calculated between consecutive frames. The method is tested for its robustness and accuracy by simulated Gaussian events and spatially displaced random noise. An example application of the method is presented on propagating ELM filaments measured by the GPI system on the National Spherical Torus Experiment spherical tokamak.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2022

Likelihood Maximization and Moment Matching in Low SNR Gaussian Mixture Models

We derive an asymptotic expansion for the log-likelihood of Gaussian mixture models (GMMs) with equal covariance matrices in the low signal-to-noise regime. The expansion reveals an intimate connection between two types of algorithms for parameter estimation: the method of moments and likelihood optimizing algorithms such as Expectation-Maximization (EM). We show that likelihood optimization in the low SNR regime reduces to a sequence of least squares optimization problems that match the moments of the estimate to the ground truth moments one by one. This connection is a stepping stone towards the analysis of EM and maximum likelihood estimation in a wide range of models. A motivating application for the study of low SNR mixture models is cryo-electron microscopy data, which can be modeled as a GMM with algebraic constraints imposed on the mixture centers. We discuss the application of our expansion to algebraically constrained GMMs, among other example models of interest. © 2022 The Authors. Communications on Pure and Applied Mathematics published by Wiley Periodicals LLC.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Numerical characterization of support recovery in sparse regression with correlated design

Sparse regression is employed in diverse scientific settings as a feature selection method. A pervasive aspect of scientific data is the presence of correlations between predictive features. These correlations hamper both feature selection and estimation and jeopardize conclusions drawn from estimated models. On the other hand, theoretical results on sparsity-inducing regularized regression have largely addressed conditions for selection consistency via asymptotics, and disregard the problem of model selection, whereby regularization parameters are chosen. In this numerical study, we address these issues through exhaustive characterization of the performance of several regression estimators, coupled with a range of model selection strategies. These estimators and selection criteria were examined across correlated regression problems with varying degrees of signal to noise, distributions of non-zero model coefficients, and model sparsity. Our results reveal a fundamental tradeoff between false positive and false negative control in all regression estimators and model selection criteria examined. Additionally, we numerically explore a transition point modulated by the signal-to-noise ratio and spectral properties of the design covariance matrix at which the selection accuracy of all considered algorithms degrades. Overall, we find that SCAD coupled with BIC or empirical Bayes model selection performs the best feature selection across the regression problems considered.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

JWST Noise Floor. I. Random Error Sources in JWST NIRCam Time Series

James Webb Space Telescope (JWST) transmission and emission spectra will provide invaluable glimpses of transiting exoplanet atmospheres, including possible biosignatures. This promising science from JWST, however, will require exquisite precision and understanding of systematic errors that can impact the time series of planets crossing in front of and behind their host stars. Here, we provide estimates of the random noise sources affecting JWST Near-Infrared Camera (NIRCam) time-series data on the integration-to-integration level. We find that 1/f noise can limit the precision of grism time series for two groups (230–1000 ppm depending on the extraction method and extraction parameters), but will average down like the square root of N frames/reads. The current NIRCam grism time-series mode is especially affected by 1/f noise because its GRISMR dispersion direction is parallel to the detector fast-read direction, but could be alleviated in the GRISMC direction. Care should be taken to include as many frames as possible per visit to reduce this 1/f noise source: thus, we recommend the smallest detector subarray sizes one can tolerate, four output channels, and readout modes that minimize the number of skipped frames (RAPID or BRIGHT2). We also describe a covariance-weighting scheme that can significantly lower the contributions from 1/f noise as compared to sum extraction. We evaluate the noise introduced by pre-amplifier offsets, random telegraph noise, and high dark current resistor capacitor (RC) pixels and find that these are correctable below 10 ppm once background subtraction and pixel masking are performed. We explore systematic error sources in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2021

Dark Energy Survey year 3 results: covariance modelling and its impact on parameter estimation and quality of fit

ABSTRACT We describe and test the fiducial covariance matrix model for the combined two-point function analysis of the Dark Energy Survey Year 3 (DES-Y3) data set. Using a variety of new ansatzes for covariance modelling and testing, we validate the assumptions and approximations of this model. These include the assumption of Gaussian likelihood, the trispectrum contribution to the covariance, the impact of evaluating the model at a wrong set of parameters, the impact of masking and survey geometry, deviations from Poissonian shot noise, galaxy weighting schemes, and other sub-dominant effects. We find that our covariance model is robust and that its approximations have little impact on goodness of fit and parameter estimation. The largest impact on best-fitting figure-of-merit arises from the so-called fsky approximation for dealing with finite survey area, which on average increases the χ2 between maximum posterior model and measurement by $3.7{{\ \rm per\ cent}}$ (Δχ2 ≈ 18.9). Standard methods to go beyond this approximation fail for DES-Y3, but we derive an approximate scheme to deal with these features. For parameter estimation, our ignorance of the exact parameters at which to evaluate our covariance model causes the dominant effect. We find that it increases the scatter of maximum posterior values for Ωm and σ8 by about $3{{\ \rm per\ cent}}$ and for the dark energy equation-of-state parameter by about $5{{\ \rm per\ cent}}$.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2024

Testing the ΛCDM Cosmological Model with Forthcoming Measurements of the Cosmic Microwave Background with SPT-3G

We forecast constraints on cosmological parameters enabled by three surveys conducted with SPT-3G, the third-generation camera on the South Pole Telescope. The surveys cover separate regions of 1500, 2650, and 6000 deg$^{2}$ to different depths, in total observing 25% of the sky. These regions will be measured to white noise levels of roughly 2.5, 9, and 12μK-armin, respectively, in cosmic microwave background (CMB) temperature units at 150 GHz by the end of 2024. The survey also includes measurements at 95 and 220 GHz, which have noise levels a factor of ∼1.2 and 3.5 times higher than 150 GHz, respectively, with each band having a polarization noise level ∼2times higher than the temperature noise. We use a novel approach to obtain the covariance matrices for jointly and optimally estimated gravitational lensing potential band powers and unlensed CMB temperature and polarization band powers. We demonstrate the ability to test the ΛCDM model via the consistency of cosmological parameters constrained independently from SPT-3G and Planck data, and consider the improvement in constraints on ΛCDM extension parameters from a joint analysis of SPT-3G and Planck data. The ΛCDM cosmological parameters are typically constrained with uncertainties up to ∼2 times smaller with SPT-3G data, compared to Planck, with the two data sets measuring significantly different angular scales and polarization levels, providing additional tests of the standard cosmological model.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2022

Strong Correspondence in Evapotranspiration and Carbon Dioxide Fluxes Between Different Eddy Covariance Systems Enables Quantification of Landscape Heterogeneity in Dryland Fluxes

Abstract The eddy covariance method is widely used to investigate fluxes of energy, water, and carbon dioxide at landscape scales, providing important information on how ecological systems function. Flux measurements quantify ecosystem responses to environmental perturbations and management strategies, including nature‐based climate‐change mitigation measures. However, due to the high cost of conventional instrumentation, most eddy covariance studies employ a single system, limiting spatial representation to the flux footprint. Insufficient replication may be limiting our understanding of ecosystem behavior. To address this limitation, we deployed eight lower‐cost eddy covariance systems in two clusters around two conventional eddy covariance systems in the Chihuahuan Desert of North America for a period of 2 years. These dryland settings characterized by large temperature variations and relatively low carbon dioxide fluxes represented a challenging setting for eddy covariance. We found very good closure of energy and water balance across all systems (within ±9% of unity). We found very good correspondence between the lower‐cost and conventional systems' fluxes of sensible heat (with concordance correlation coefficient (CCC) of ≥0.87), latent energy (evapotranspiration; CCC ≥ 0.89), and useful correspondence in the net ecosystem exchange ((NEE); with CCC ≥ 0.4) at the daily temporal resolution. Relative to the conventional systems, the low‐frequency systems were characterized by a higher level of random error, particularly in the NEE fluxes. Lower‐cost systems can enable wider deployment affording better replication and sampling of spatiotemporal variability at the expense of greater measurement noise that might be limiting for certain applications. Replicated eddy covariance observations may be useful when addressing gaps in the existing monitoring of critical and underrepresented ecosystems and for measuring areas larger than a single flux footprint.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Using optimal estimation to retrieve winds from velocity-azimuth display (VAD) scans by a Doppler lidar

Abstract. Low-powered commercially available coherent Doppler lidar (CDL) wind profilers provide continuous measurement of vertical profiles of wind in the lower troposphere, usually close to or up to the top of the planetary boundary layer. The vertical extent of these wind profiles is limited by the availability of scatterers and thus varies substantially throughout the day and from one day to the next. This makes it challenging to develop continuous products that rely on CDL-observed wind profiles. In order to overcome this problem, we have developed a new method for wind profile retrievals from CDL that combines the traditional velocity-azimuth display (VAD) technique with optimal estimation (OE) to provide continuous wind profiles up to 3 km. The new method exploits the level-to-level covariance present in the wind profile to fill in the gaps where the signal-to-noise ratio of the CDL return is too low to provide reliable results using the traditional VAD method. Another advantage of the new method is that it provides the full error covariance matrix of the solution and profiles of information content, which more easily facilitates the assimilation of the observed wind profiles into numerical weather prediction models. This method was tested using yearlong CDL measurements at the Atmospheric Radiation Measurement (ARM) Southern Great Plains (SGP) Central Facility in 2019. Comparison with the ARM operational CDL wind profile product and collocated radiosonde wind measurements shows excellent agreement (R2>0.99) with no degradation in results where the traditional VAD provided a valid solution. In the region where traditional VAD does not provide results, the OE wind speed and wind vector have uncertainties of 3.44 and 4.33 m s−1, respectively. As a result, the new method provides additional information over the standard technique and increases the effective range of existing CDL systems without the need for additional hardware.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Inferring Quantum Network Topology Using Local Measurements

Statistical correlations that can be generated across the nodes in a quantum network depend crucially on its topology. However, this topological information might not be known a priori, or it may need to be verified. In this paper, we propose an efficient protocol for distinguishing and inferring the topology of a quantum network. We leverage entropic quantities-namely, the von Neumann entropy and the measured mutual information-as well as measurement covariance to uniquely characterize the topology. We show that the entropic quantities are sufficient to distinguish two networks that prepare GHZ states. Moreover, if qubit measurements are available, both entropic quantities and covariance can be used to infer the network topology without state-preparation assumptions. We show that the protocol can be entirely robust to noise and can be implemented via quantum variational optimization. Numerical experiments on both classical simulators and quantum hardware show that covariance is generally more reliable for accurately and efficiently inferring the topology, whereas entropy-based methods are often better at identifying the absence of entanglement in the low-shot regime.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2022

Quantitative assessment of fitting errors associated with streak camera noise in Thomson scattering data analysis

Thomson scattering measurements in high energy density experiments are often recorded using optical streak cameras. In the low-signal regime, noise introduced by the streak camera can become an important and sometimes the dominant source of measurement uncertainty. In this paper, we present a formal method of accounting for the presence of streak camera noise in our measurements. We present a phenomenological description of the noise generation mechanisms and present a statistical model that may be used to construct the covariance matrix associated with a given measurement. This model is benchmarked against simulations of streak camera images. We demonstrate how this covariance may then be used to weight fitting of the data and provide quantitative assessments of the uncertainty in the fitting parameters determined by the best fit to the data and build confidence in the ability to make statistically significant measurements in the low-signal regime, where spatial correlations in the noise become apparent. These methods will have general applicability to other measurements made using optical streak cameras.

47 OTHER INSTRUMENTATION↗
DOE OSTI2020

Perturbation theory approach to predict the covariance matrices of the galaxy power spectrum and bispectrum in redshift space

In this paper, we predict the covariance matrices of both the power spectrum and the bispectrum, including full non-Gaussian contributions, redshift space distortions, linear bias effects, and shot-noise corrections, using perturbation theory (PT). To quantify the redshift-space distortion effect, we focus mainly on the monopole and quadrupole components of both the power and bispectra. We, for the first time, compute the 5- and 6-point spectra to predict the cross-covariance between the power and bispectra, and the autocovariance of the bispectrum in redshift space. Further, we test the validity of our calculations by comparing them with the covariance matrices measured from the MultiDark-Patchy mock catalogues that are designed to reproduce the galaxy clustering measured from the Baryon Oscillation Spectroscopic Survey Data Release 12. We argue that the simple, leading-order PT works because the shot-noise corrections for the Patchy mocks are more dominant than other higher order terms we ignore. In the meantime, we confirm some discrepancies in the comparison, especially of the cross-covariance. We discuss potential sources of such discrepancies. We also show that our PT model reproduces well the cumulative signal-to-noise ratio of the power spectrum and the bispectrum as a function of maximum wavenumber, implying that our PT model captures successfully essential contributions to the covariance matrices.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2023

The two-point correlation function covariance with fewer mocks

We present FITCOV an approach for accurate estimation of the covariance of two-point correlation functions that requires fewer mocks than the standard mock-based covariance. This can be achieved by dividing a set of mocks into jackknife regions and fitting the correction term first introduced in Mohammad & Percival (2022), such that the mean of the jackknife covariances corresponds to the one from the mocks. This extends the model beyond the shot-noise limited regime, allowing it to be used for denser samples of galaxies. We test the performance of our fitted jackknife approach, both in terms of accuracy and precision, using lognormal mocks with varying densities and approximate EZmocks mimicking the Dark Energy Spectroscopic Instrument LRG and ELG samples in the redshift range of z = [0.8, 1.1]. We find that the Mohammad–Percival correction produces a bias in the two-point correlation function covariance matrix that grows with number density and that our fitted jackknife approach does not. We also study the effect of the covariance on the uncertainty of cosmological parameters by performing a full-shape analysis. We demonstrate that our fitted jackknife approach based on 25 mocks can recover unbiased and as precise cosmological parameters as the ones obtained from a covariance matrix based on 1000 or 1500 mocks, while the Mohammad–Percival correction produces uncertainties that are twice as large.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Cosmology From CMB Lensing and Delensed EE Power Spectra Using 2019-2020 SPT-3G Polarization Data

From CMB polarization data alone we reconstruct the CMB lensing power spectrum, comparable in overall constraining power to previous temperature-based reconstructions, and an unlensed E -mode power spectrum, with clear detections of the third through tenth acoustic peaks. The observations, taken in 2019 and 2020 with the South Pole Telescope (SPT) and the SPT-3G camera, cover 1500 deg 2 at 95, 150, and 220 GHz with arcminute resolution and roughly 4.9 µ K-arcmin coadded noise in polarization. The power spectrum estimates, together with systematic parameter estimates and a joint covariance matrix, follow from a Bayesian analysis using the Marginal Unbiased Score Expansion (MUSE) method. The E -mode spectrum at ℓ > 2000 and lensing spectrum at L > 350 are the most precise to date. Assuming the ΛCDM model, and using only these SPT data and priors on τ and absolute calibration from Planck, we find H 0 = 66.81 ± 0.81 km/s/Mpc, comparable in precision to the Planck determination and in 5.4 σ tension with the most precise H 0 inference derived via the distance ladder. We also find S 8 ≡ σ 8 (Ω m /0.3) 0.5 = 0.850 ± 0.017, providing further independent evidence of a slight tension with low-redshift structure probes. The ΛCDM model provides a good simultaneous fit to the combined Planck, ACT, and SPT data, and thus passes a powerful test. Combining these CMB datasets with BAO observations, we explore extensions to the ΛCDM model. We find that the effective number of neutrino species, spatial curvature, and primordial helium fraction are consistent with standard model values, and that the 95% confidence upper limit on the neutrino mass sum is 0.075 eV, close to the minimum sum expected from observations of solar and atmospheric neutrino oscillations. The SPT data are consistent with the somewhat weak (< 3 σ ) preference for excess lensing power seen in Planck and ACT data relative to predictions of the ΛCDM model given the combined Planck, ACT, and BAO data sets. Finally, we also detect at greater than 3 σ the influence of non-linear evolution in the CMB lensing power spectrum and discuss it in the context of the S 8 tension. Forthcoming SPT-3G analyses will feature deeper and wider observations in temperature and polarization, providing even tighter constraints and more powerful tests of the ΛCDM model.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Semi-analytical covariance matrices for two-point correlation function for DESI 2024 data

We present an optimized way of producing the fast semi-analytical covariance matrices for the Legendre moments of the two-point correlation function, taking into account survey geometry and mimicking the non-Gaussian effects. We validate the approach on simulated (mock) catalogs for different galaxy types, representative of the Dark Energy Spectroscopic Instrument (DESI) Data Release 1, used in 2024 analyses. We find only a few percent differences between the mock sample covariance matrix and our results, which can be expected given the approximate nature of the mocks, although we do identify discrepancies between the shot-noise properties of the DESI fiber assignment algorithm and the faster approximation (emulator) used in the mocks. Importantly, we find a close agreement (≤ 8% relative differences) in the projected errorbars for distance scale parameters for the baryon acoustic oscillation measurements. This confirms our method as an attractive alternative to simulation-based covariance matrices, especially for non-standard models or galaxy sample selections, making it particularly relevant to the broad current and future analyses of DESI data.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2025

Applying Gaussian Process Machine Learning and Modern Probabilistic Programming to Satellite Data to Infer CO 2 Emissions

Satellite data provides essential insights into the spatiotemporal distribution of CO 2 concentrations. However, many atmospheric inverse models fail to adequately incorporate the spatial and temporal correlations inherent in satellite observations and often lack rigorous methods for estimating parameters like spatial length scales. We introduce an inference model that processes the spatiotemporal covariance in satellite data and estimates hyperparameters such as covariance length scales. Our approach uses the Gaussian process (GP) machine learning (ML) and modern probabilistic programming languages (PPLs) to perform atmospheric inversions of emissions from satellite data. We develop a GP ML inversion system based on modern PPLs and the GEOS-Chem chemical transport model, simulating atmospheric CO 2 concentrations corresponding to the Orbiting Carbon Observatory-2/3 (OCO-2/3) data for July 2020. In our supervised learning framework, we treat the GEOS-Chem simulated data set as the target, with predictors derived by scaling the target with sector-specific factors hidden from the GP machine. Our results show that the GP model, combined with GPU-enabled PPLs, effectively retrieves true emission scaling factors and infers noise levels concealed within the data. This suggests that our method could be applied over larger areas with more complex covariance structures, enabling comprehensive analysis of the spatiotemporal patterns observed in OCO-2/3 and similar satellite data sets.

54 ENVIRONMENTAL SCIENCES↗