SEARCH · Search NASA

Results for “correlated noise analysis”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

All sources NASA NTRS DOE OSTI

DOE OSTI2025

Loss Landscape Analysis for Reliable Quantized ML Models for Scientific Sensing

In this paper, we propose a method to perform empirical analysis of the loss landscape of machine learning (ML) models. The method is applied to two ML models for scientific sensing, which necessitates quantization to be deployed and are subject to noise and perturbations due to experimental conditions. Our method allows assessing the robustness of ML models to such effects as a function of quantization precision and under different regularization techniques -- two crucial concerns that remained underexplored so far. By investigating the interplay between performance, efficiency, and robustness by means of loss landscape analysis, we both established a strong correlation between gently-shaped landscapes and robustness to input and weight perturbations and observed other intriguing and non-obvious phenomena. Our method allows a systematic exploration of such trade-offs a priori, i.e., without training and testing multiple models, leading to more efficient development workflows. This work also highlights the importance of incorporating robustness into the Pareto optimization of ML models, enabling more reliable and adaptive scientific sensing systems.

Baldi, Tommaso [Pisa, Scuola Normale Superiore]↗

DOE OSTI2024

Water reflection analysis of encapsulated photovoltaic modules

A method for moisture testing of a fully assembled photovoltaic (PV) module. An assembled PV module is probed with short wave IR probe energy in the range of 1700-2000 nm. Energy reflected from the assembled PV module is collected and directed to a sensor. Noise is removed from a signal of the sensor with reference to the probe energy. Absorption is of the probe energy is determined. The absorption is correlated to moisture in the PV module. A preferred system that carries out the method provides a signal-to-noise ratio (as defined by standard deviation/mean of measured reflectance) of at least 3800.

Fenning, David↗

DOE OSTI2023

BEYONDPLANCK XI. Bayesian CMB analysis with sample-based end-to-end error propagation

We present posterior sample-based cosmic microwave background (CMB) constraints from Planck LFI and WMAP observations as derived through global end-to-end Bayesian processing within the BeyondPlanck framework. We first use these samples to study correlations between CMB, foreground, and instrumental parameters, and we identify a particularly strong degeneracy between CMB temperature fluctuations and free-free emission on intermediate angular scales (400 ≲ ℓ ≲ 600), which is mitigated through model reduction, masking, and resampling. We compare our posterior-based CMB results with previous Planck products, and find generally good agreement, although with notably higher noise due to our exclusion of HFI data. We find a best-fit CMB dipole amplitude of 3362.7 ± 1.4 µK, in excellent agreement with previous Planck results. The quoted dipole uncertainty is derived directly from the sampled posterior distribution, and does not involve any ad hoc contributions for Planck instrumental systematic effects. Similarly, we find a temperature quadrupole amplitude of ${σ}_{2}^{TT}$ = 229 ± 97 µK 2 , which is in good agreement with previous results in terms of the amplitude, but the uncertainty is an order of magnitude larger than the naive diagonal Fisher uncertainty. Relatedly, we find lower evidence for a possible alignment between the quadrupole and octopole than previously reported due to a much larger scatter in the individual quadrupole coefficients, caused both by marginalizing over a more complete set of systematic effects, but also by our more conservative analysis mask required to mitigate the free-free degeneracy. For higher multipoles, we find that the angular temperature power spectrum is generally in good agreement with both Planck and WMAP. At the same time, we note that this is the first time the sample-based asymptotically exact Blackwell-Rao estimator has been successfully established for multipoles up to ℓ ≤ 600, and it now accounts for the majority of the cosmologically important information. Overall, this analysis demonstrates the unique capabilities of the Bayesian approach with respect to end-to-end systematic uncertainty propagation, and we believe it can and should play an important role in the analysis of future CMB experiments. Cosmological parameter constraints are presented in a companion paper.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Information content of higher order galaxy correlation functions

The shapes of galaxy N-point correlation functions can be used as standard rulers to constrain the distance–redshift relationship. The cosmological density fields traced by late-time galaxy formation are initially nearly Gaussian, and hence, all the cosmological information can be extracted from their two-point correlation function. Subsequent non-linear evolution under gravity, as well as halo and then galaxy formation, generates higher order correlation functions. Since the mapping of the initial to the final density field is, on large scales, invertible, it is often claimed that the information content of the initial field’s power spectrum is equal to that of all the higher order functions of the final, non-linear field. This claim implies that reconstruction of the initial density field from the non-linear field renders analysis of higher order correlation functions of the latter superfluous. We show that this claim is false when the N-point functions are used as standard rulers. Constraints available from joint analysis of the two and three-point correlation functions can, in some cases, exceed those offered by the initial power spectrum. We provide a mathematical justification for this claim and demonstrate it using a large suite of N-body simulations. In particular, we show that for the z = 0 real-space matter field in the limit of vanishing shot-noise, taking modes up to k max = 0.2 h Mpc −1 , using the bispectrum alone offers a factor of 2 reduction in the variance on the cosmic distance scale relative to that available from the linear power spectrum.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2026

Dark Energy Survey Year 6 results: Clustering redshifts and importance sampling of self-organized-maps 𝑛⁡(𝑧) realizations for 3 × 2 ⁢pt samples

This work is part of a series establishing the redshift framework for the 3 × 2 ⁢pt analysis of the Dark Energy Survey Year 6 (DES Y6). For DES Y6, photometric redshift distributions are estimated using self-organizing maps (SOMs), calibrated with spectroscopic and many-band photometric data. To overcome limitations from color-redshift degeneracies and incomplete spectroscopic coverage, we enhance this approach by incorporating clustering-based redshift constraints (clustering-z, or WZ) from angular cross-correlations with BOSS and eBOSS galaxies and eBOSS quasar samples. We define a WZ likelihood and apply importance sampling to a large ensemble of SOM-derived 𝑛⁡(𝑧) realizations, selecting those consistent with the clustering measurements to produce a posterior sample for each lens and source bin. The analysis uses angular scales corresponding to 1.5–5 Mpc to optimize signal-to-noise ratio while mitigating modeling uncertainties and marginalizes over redshift-dependent galaxy bias and other systematics informed by the N-body simulation CARDINAL . While a sparser spectroscopic reference sample limits WZ constraining power at 𝑧 >1.1, particularly for source bins, we demonstrate that combining SOM with WZ improves redshift accuracy and enhances the overall cosmological constraining power of DES Y6. As a result, we estimate an improvement in 𝑆 8 of approximately 10% for cosmic shear and 3 ×2⁢pt analysis, primarily due to the WZ calibration of the source samples.

Cosmological parameters↗

DOE OSTI2022

Dark Energy Survey Year 3 results: Cosmology from cosmic shear and robustness to data calibration

This work, together with its companion paper, Secco, Samuroff et al., presents the Dark Energy Survey Year 3 cosmic shear measurements and cosmological constraints based on an analysis of over 100 million source galaxies. With the data spanning 4143 deg 2 on the sky, divided into four redshift bins, we produce the highest significance measurement of cosmic shear to date, with a signal-to-noise of 40. We conduct a blind analysis in the context of the ΛCDM model and find a 3% constraint of the clustering amplitude, S 8 ≡ σ 8 (Ω m /0.3) 0.5 = 0.759$^{+0.025}_{-0.023}$. A ΛCDM-Optimized analysis, which safely includes smaller scale information, yields a 2% precision measurement of S 8 = 0.772$^{+0.018}_{-0.017}$ that is consistent with the fiducial case. The two low-redshift measurements are statistically consistent with the Planck Cosmic Microwave Background result, however, both recovered S 8 values are lower than the high-redshift prediction by 2.3σ and 2.1σ (ρ-values of 0.02 and 0.05), respectively. The measurements are shown to be internally consistent across redshift bins, angular scales and correlation functions. The analysis is demonstrated to be robust to calibration systematics, with the S 8 posterior consistent when varying the choice of redshift calibration sample, the modeling of redshift uncertainty and methodology. Similarly, we find that the corrections included to account for the blending of galaxies shifts our best-fit S 8 by 0.5σ without incurring a substantial increase in uncertainty. We examine the limiting factors for the precision of the cosmological constraints and find observational systematics to be subdominant to the modeling of astrophysics. Specifically, we identify the uncertainties in modeling baryonic effects and intrinsic alignments as the limiting systematics.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Cosmology from clustering, cosmic shear, CMB lensing, and cross correlations: combining Rubin observatory and Simons Observatory

ABSTRACT In the near future, the overlap of the Rubin Observatory Legacy Survey of Space and Time (LSST) and the Simons Observatory (SO) will present an ideal opportunity for joint cosmological data set analyses. In this paper, we simulate the joint likelihood analysis of these two experiments using six two-point functions derived from galaxy position, galaxy shear, and CMB lensing convergence fields. Our analysis focuses on realistic noise and systematics models and we find that the dark energy Figure of merit (FoM) increases by 53 per cent (92 per cent) from LSST-only to LSST+SO in Year 1 (Year 6). We also investigate the benefits of using the same galaxy sample for both clustering and lensing analyses, and find the choice improves the overall signal to noise by ${\sim}30{-}40{{\ \rm per\ cent}}$, which significantly improves the photo-z calibration and mildly improves the cosmological constraints. Finally, we explore the effects of catastrophic photo-z outliers finding that they cause significant parameter biases when ignored. We develop a new mitigation approach termed ‘island model’, which corrects a large fraction of the biases with only a few parameters while preserving the constraining power.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2019

Correlating results from high resolution EBSD with TEM- and ECCI-based dislocation microscopy: Approaching single dislocation sensitivity via noise reduction

High resolution electron backscatter diffraction (HREBSD), an SEM-based diffraction technique, may be used to measure the lattice distortion of a crystalline material and to infer the geometrically necessary dislocation content. Uncertainty in the image correlation process used to compare diffraction patterns leads to an uneven distribution of measurement noise in terms of the lattice distortion, which results in erroneous identification of dislocation type and density. This work presents a method of reducing noise in HREBSD dislocation measurements by removing the effect of the most problematic components of the measured distortion. The method is then validated by comparing with TEM analysis of dislocation pile-ups near a twin boundary in austenitic stainless steel and with ECCI analysis near a nano-indentation on a tantalum oligocrystal. Finally, the HREBSD dislocation microscopy technique is able to resolve individual dislocations visible in TEM and ECCI and correctly identify their Burgers vectors.

36 MATERIALS SCIENCE↗

DOE OSTI2021

Noise and error analysis and optimization in particle-based kinetic plasma simulations

In this paper we analyze the noise in macro-particle methods used in plasma physics and fluid dynamics, leading to approaches for minimizing the total error, focusing on electrostatic models in one dimension. We begin by describing kernel density estimation for continuous values of the spatial variable x, expressing the kernel in a form in which its shape and width are represented separately. The covariance matrix of the noise in the density is computed, first for uniform true density. The bandwidth of the covariance matrix C(x,y) is related to the width of the kernel. A feature that stands out is the presence of constant negative terms in the elements of the covariance matrix both on and off-diagonal. These negative correlations are related to the fact that the total number of particles is fixed at each time step; they also lead to the property ∫C(x,y)dy = 0. We investigate the effect of these negative correlations on the electric field computed by Gauss's law, finding that the noise in the electric field is related to a process called the Ornstein-Uhlenbeck bridge, leading to a covariance matrix of the electric field with variance significantly reduced relative to that of a Brownian process. For non-constant density, p(x), still with continuous x, we analyze the total error in the density estimation and discuss it in terms of bias-variance optimization (BVO). For some characteristic length l, determined by the density and its second derivative, and kernel width h, having too few particles within h leads to too much variance; for h that is large relative to l, there is too much smoothing of the density. The optimum between these two limits is found by BVO. For kernels of the same width, it is shown that this optimum (minimum) is weakly sensitive to the kernel shape. Next, we repeat the analysis for x discretized on a grid. In this case the charge deposition rule is determined by a particle shape. An important property to be respected in the discrete system is the exact preservation of total charge on the grid; this property is necessary to ensure that the electric field is equal at both ends, consistent with periodic boundary conditions. We find that if the particle shapes satisfy a partition of unity property, the particle charge deposited on the grid is conserved exactly. Further, if the particle shape is expressed as the convolution of a kernel with another kernel that satisfies the partition of unity, then the particle shape obeys the partition of unity. This property holds for kernels of arbitrary width, including widths that are not integer multiples of the grid spacing. Furthermore, we show results relaxing the approximations used to do BVO optimization analytically, by doing numerical computations of the total error as a function of the kernel width, on a grid in x. The comparison between numerical and analytical results shows good agreement over a range of particle shapes. We discuss the practical implications of our results, including the criteria for design and implementation of computationally efficient particle shapes that take advantage of the developed theory.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

DOE OSTI2025

Stable Symmetry-Protected Topological Phases in Systems with Heralded Noise

Here, we present a family of local quantum channels whose steady-states exhibit stable mixed-state symmetry-protected topological (SPT) order. Motivated by recent experimental progress on "erasure conversion" techniques that allow one to identify ($\textit{herald}$) decoherence processes, we consider open systems with biased erasure noise, which leads to strongly symmetric heralded errors. We utilize this heralding to construct a local correction protocol that effectively confines errors into short-ranged pairs in the steady-state. Using a combination of numerical simulations and mean-field analysis, we show that our protocol stabilizes SPT order against a sufficiently low rate of decoherence. As the rate of heralded noise increases, SPT order is eventually lost through a directed percolation transition. We further find that while introducing unheralded errors destroys SPT order in the limit of long length- and time-scales, the correction protocol is sufficient for ensuring that local SPT order persists, with a correlation length that diverges as $\xi \sim (1-f_e)^{-1/2}$, where $f_e$ is the fraction of errors that are heralded.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2023

3D correlations in the Lyman-α forest from early DESI data

We present the first measurements of Lyman-α (Lyα) forest correlations using early data from the Dark Energy Spectroscopic Instrument (DESI). We measure the auto-correlation of Lyα absorption using 88 509 quasars at z > 2, and its cross-correlation with quasars using a further 147 899 tracer quasars at z ≳ 1.77. Then, we fit these correlations using a 13-parameter model based on linear perturbation theory and find that it provides a good description of the data across a broad range of scales. We detect the BAO peak with a signal-to-noise ratio of 3.8σ, and show that our measurements of the auto- and cross-correlations are fully-consistent with previous measurements by the Extended Baryon Oscillation Spectroscopic Survey (eBOSS). Even though we only use here a small fraction of the final DESI dataset, our uncertainties are only a factor of 1.7 larger than those from the final eBOSS measurement. As a result, we validate the existing analysis methods of Lyα correlations in preparation for making a robust measurement of the BAO scale with the first year of DESI data.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2021

Cosmological cross-correlations and nearest neighbour distributions

ABSTRACT Cross-correlations between data sets are used in many different contexts in cosmological analyses. Recently, k-nearest neighbour cumulative distribution functions (kNN-CDF) were shown to be sensitive probes of cosmological (auto) clustering. In this paper, we extend the framework of NN measurements to describe joint distributions of, and correlations between, two data sets. We describe the measurement of joint kNN-CDFs, and show that these measurements are sensitive to all possible connected N-point functions that can be defined in terms of the two data sets. We describe how the cross-correlations can be isolated by combining measurements of the joint kNN-CDFs and those measured from individual data sets. We demonstrate the application of these measurements in the context of Gaussian density fields, as well as for fully non-linear cosmological data sets. Using a Fisher analysis, we show that measurements of the halo-matter cross-correlations, as measured through NN measurements are more sensitive to the underlying cosmological parameters, compared to traditional two-point cross-correlation measurements over the same range of scales. Finally, we demonstrate how the NN cross-correlations can robustly detect cross-correlations between sparse samples – the same regime where the two-point cross-correlation measurements are dominated by noise.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2023

Observing signals of spectral features in the cosmic-ray positrons and electrons from Milky Way pulsars

The Alpha Magnetic Spectrometer (AMS-02) has provided unprecedented precision measurements of the electron and positron cosmic-ray fluxes and the positron fraction spectrum. At the higher energies, sources as energetic local pulsars, may contribute to both cosmic-ray species. The discreteness of the source population, can result in features both on the positron fraction measurement and in the respective electron and positron spectra. For the latter, those would coincide in energy and would contrast predictions of smooth spectra as from particle dark matter. In this work, using a library of pulsar population models for the local part of the Milky Way, we perform a power-spectrum analysis on the cosmic-ray positron fraction. We also develop a technique to cross-correlate the electron and positron fluxes. We show that both such analyses, can be used to search statistically for the presence of spectral wiggles in the cosmic-ray data. For a significant fraction of our pulsar simulations, those techniques are already sensitive enough to give a signal for the presence of those features above the regular noise, with forthcoming observations making them even more sensitive. Lastly, by cross-correlating the AMS-02 electron and positron spectra, we find an intriguing first hint for a positive correlation between them, of the kind expected by a population of local pulsars.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2022

Skewing the CMB×LSS: a fast method for bispectrum analysis

Upcoming cosmic microwave background (CMB) lensing measurements and tomographic galaxy surveys are expected to provide us with high-precision data sets in the coming years, thus paving the way for fruitful cross-correlation analyses. Here, in this paper we study the information content of the weighted skew-spectrum, a nearly-optimal estimator of the angular bispectrum amplitude, as a means to extract non-Gaussian information on both bias and cosmological parameters from the bispectra of galaxies cross-correlated with CMB lensing, while gaining significantly on speed. Our results show that for the combination of the Planck satellite and the Dark Energy Spectroscopic Instrument (DESI), the difference in the constraints on bias and cosmological parameters from the skew-spectrum and the bispectrum is at most 17%. We further compare and find agreement between our theoretical skew-spectra and those estimated from N-body simulations, for which it is important to include gravitational non-linearities beyond perturbation theory and the post-Born effect for CMB lensing. We define an algorithm to apply the skew-spectrum estimator to the data and, as a preliminary step, we use the skew-spectra to constrain bias parameters and the amplitude of shot noise from the simulations through a Markov chain Monte Carlo likelihood analysis, finding that it may be possible to reach percent-level estimates for the linear bias parameter b 1 .

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2025

Analysis strategy for ultrafast X-ray photon correlation spectroscopy

We explain an analysis strategy for ultrafast X-ray photon correlation spectroscopy, a technique enabled by X-ray free electron lasers to probe nano- and atomic-scale dynamics in complex systems on nanosecond timescales or faster. Central to the technique is the methodology for extracting contrast in coherent X-ray diffraction patterns, known as speckle patterns. Guided by simulations, we analyze common challenges and examine errors encountered in contrast extraction. A method for accurately determining contrast amid shot noise across a wide range of count rates is presented.

36 MATERIALS SCIENCE↗

DOE OSTI2020

Extracting a mixing parameter from 2D radiographic imaging of variable-density turbulent flow

We extract a suitably averaged fluctuating density from the two-dimensional radiographic image of a flow. The X-ray attenuation is given by the Beer–Lambert law which exponentially damps the incident beam intensity by a factor proportional to the density, opacity and thickness of the target. By making reasonable assumptions for the mean density, opacity and effective thickness of the target flow, we estimate the density fluctuation contribution to the attenuation. The extracted density fluctuations averaged across the thickness of the flow in the direction of the beam may be used to form the density–specific-volume correlation $b$. In a statistical description of variable-density turbulence, $b$ quantifies the degree of mixedness. The ability to extract a measure of mixedness from experimental data would be a powerful tool that could be used in the validation of mix models. The scheme proposed is tested for DNS data computed for variable density buoyancy-driven mixing. We quantify the deficits in the extracted value of $b$ due to target thickness, Atwood number and modeled signal noise. This analysis justifies using the proposed scheme to infer the mix parameter from thin targets at moderate to low Atwood numbers. Furthermore, to illustrate how the scheme might be used in a practical problem, we demonstrate its application to a radiographic image of counter-shear flow obtained from experiments at the National Ignition Facility.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

DOE OSTI2020

Towards a self-consistent analysis of the anisotropic galaxy two- and three-point correlation functions on large scales: application to mock galaxy catalogues

We establish a practical method for the joint analysis of anisotropic galaxy two- and three-point correlation functions (2PCF and 3PCF, respectively) on the basis of the decomposition formalism of the 3PCF using tripolar spherical harmonics. We perform such an analysis with MultiDark-Patchy mock catalogues to demonstrate and understand the benefit of the anisotropic 3PCF. We focus on scales above $80\, h^{-1}\, {\rm Mpc}$, and use information from the shape and the baryon acoustic oscillation (BAO) signals of the 2PCF and 3PCF. We also apply density field reconstruction to increase the signal-to-noise ratio of BAO in the 2PCF measurement, but not in the 3PCF measurement. In particular, we study in detail the constraints on the angular diameter distance and the Hubble parameter. We build a model of the bispectrum or 3PCF that includes the non-linear damping of the BAO signal in redshift space. We carefully account for various uncertainties in our analysis including theoretical models of the 3PCF, window function corrections, biases in estimated parameters from the fiducial values, the number of mock realizations to estimate the covariance matrix, and bin size. The joint analysis of the 2PCF and 3PCF monopole and quadrupole components shows a $30{{\ \rm per\ cent}}$ and $20{{\ \rm per\ cent}}$ improvement in Hubble parameter constraints before and after reconstruction of the 2PCF measurements, respectively, compared to the 2PCF analysis alone. Furthermore, this study clearly shows that the anisotropic 3PCF increases cosmological information from galaxy surveys and encourages further development of the modelling of the 3PCF on smaller scales than we consider.

79 ASTRONOMY AND ASTROPHYSICS↗

DOE OSTI2024

Periodicity significance testing with null-signal templates: reassessment of PTF’s SMBH binary candidates

Periodograms are widely employed for identifying periodicity in time series data, yet they often struggle to accurately quantify the statistical significance of detected periodic signals when the data complexity precludes reliable simulations. We develop a data-driven approach to address this challenge by introducing a null-signal template (NST). The NST is created by carefully randomizing the period of each cycle in the periodogram template, rendering it non-periodic. It has the same frequentist properties as a periodic signal template, and we show with simulations that the distribution of false positives is the same as with the original periodic template, regardless of the underlying data. Thus, performing a periodicity search with the NST acts as an effective simulation of the null (no-signal) hypothesis, without having to simulate the noise properties of the data. We apply the NST method to the supermassive black hole binaries (SMBHB) search in the Palomar Transient Factory (PTF), where Charisi et al. had previously proposed 33 high signal-to-noise candidates utilizing simulations to quantify their significance. Our approach reveals that these simulations do not capture the complexity of the real data. There are no statistically significant periodic signal detections above the non-periodic background. To improve the search sensitivity, we introduce a Gaussian quadrature based algorithm for the Bayes Factor with correlated noise as a test statistic. We show with simulations that this improves sensitivity to true signals by more than an order of magnitude. However, the Bayes Factor approach also results in no statistically significant detections in the PTF data.

79 ASTRONOMY AND ASTROPHYSICS↗