Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian parameter estimation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 469 records · Page 26

Bayesian inference and predictive performance of soil respiration models in the presence of model discrepancy

Bayesian inference of microbial soil respiration models is often based on the assumptions that the residuals are independent (i.e., no temporal or spatial correlation), identically distributed (i.e., Gaussian noise), and have constant variance (i.e., homoscedastic). In the presence of model discrepancy, as no model is perfect, this study shows that these assumptions are generally invalid in soil respiration modeling such that residuals have high temporal correlation, an increasing variance with increasing magnitude of CO 2 efflux, and non-Gaussian distribution. Relaxing these three assumptions stepwise results in eight data models. Data models are the basis of formulating likelihood functions of Bayesian inference. This study presents a systematic and comprehensive investigation of the impacts of data model selection on Bayesian inference and predictive performance. We use three mechanistic soil respiration models with different levels of model fidelity (i.e., model discrepancy) with respect to the number of carbon pools and the explicit representations of soil moisture controls on carbon degradation; therefore, we have different levels of model complexity with respect to the number of model parameters. The study shows that data models have substantial impacts on Bayesian inference and predictive performance of the soil respiration models such that the following points are true: (i) the level of complexity of the best model is generally justified by the cross-validation results for different data models; (ii) not accounting for heteroscedasticity and autocorrelation might not necessarily result in biased parameter estimates or predictions, but will definitely underestimate uncertainty; (iii) using a non-Gaussian data model improves the parameter estimates and the predictive performance; and (iv) accounting for autocorrelation only or joint inversion of correlation and heteroscedasticity can be problematic and requires special treatment. Although the conclusions of this study are empirical, the analysis may provide insights for selecting appropriate data models for soil respiration modeling.

58 GEOSCIENCES↗

Quantifying uncertainty in physics-based predictions of rare-isotope production cross sections via Bayesian-inspired model averaging across nuclear mass tables

Accurate prediction of fragmentation cross sections is essential for rare-isotope beam production, planning new-isotope searches, and designing experiments to study the most exotic regions of the nuclear chart. However, existing reaction models and phenomenological cross-section parametrizations often exhibit significant deviations over broad regions of mass and charge. In this work, a Bayesian-inspired model-averaging framework is developed to combine abrasion-ablation (AA) calculations based on multiple nuclear mass tables into a single statistically weighted estimate. For the calibrated systems, the model weights are assigned empirically according to the relative quality of fit to measured cross sections, thereby reducing systematic model bias while preserving the underlying physics content of the AA description. The weights are constrained using proton-rich fragmentation data for the 78 Kr and 124 Xe projectiles. The resulting parameter trends are then propagated to the 92 Mo and 144 Sm systems through a controlled scaling procedure. In the present implementation, the excitation-energy prescription is fixed, while the averaging is performed across nuclear-mass inputs; the framework provides both weighted cross sections and associated uncertainty estimates. Applied to proton-rich fragmentation, the present approach provides a practical basis for interpolation and limited extrapolation in regions relevant to rare-isotope production. The resulting predictions are used to assess the production of very proton-rich nuclei, and candidate new isotopes are discussed.

Bayesian methods↗

Adaptive hyperparameter updating for training restricted Boltzmann machines on quantum annealers

Restricted Boltzmann Machines (RBMs) have been proposed for developing neural networks for a variety of unsupervised machine learning applications such as image recognition, drug discovery, and materials design. The Boltzmann probability distribution is used as a model to identify network parameters by optimizing the likelihood of predicting an output given hidden states trained on available data. Training such networks often requires sampling over a large probability space that must be approximated during gradient based optimization. Quantum annealing has been proposed as a means to search this space more efficiently which has been experimentally investigated on D-Wave hardware. D-Wave implementation requires selection of an effective inverse temperature or hyperparameter (β) within the Boltzmann distribution which can strongly influence optimization. Here, we show how this parameter can be estimated as a hyperparameter applied to D-Wave hardware during neural network training by maximizing the likelihood or minimizing the Shannon entropy. We find both methods improve training RBMs based upon D-Wave hardware experimental validation on an image recognition problem. Neural network image reconstruction errors are evaluated using Bayesian uncertainty analysis which illustrate more than an order magnitude lower image reconstruction error using the maximum likelihood over manually optimizing the hyperparameter. The maximum likelihood method is also shown to out-perform minimizing the Shannon entropy for image reconstruction.

97 MATHEMATICS AND COMPUTING↗

NONPARAMETRIC BAYESIAN ESTIMATION OF PERIODIC LIGHT CURVES

Many astronomical phenomena exhibit patterns that have periodic behavior. An important step when analyzing data from such processes is the problem of identifying the period: estimating the period of a periodic function based on noisy observations made at irregularly spaced time points. This problem is still a difficult challenge despite extensive study in different disciplines. This paper makes several contributions toward solving this problem. First, we present a nonparametric Bayesian model for period finding, based on Gaussian Processes (GPs), that does not make assumptions on the shape of the periodic function. As our experiments demonstrate, the new model leads to significantly better results in period estimation especially when the light curve does not exhibit sinusoidal shape. Second, we develop a new algorithm for parameter optimization for GP which is useful when the likelihood function is very sensitive to the parameters with numerous local minima, as in the case of period estimation. The algorithm combines gradient optimization with grid search and incorporates several mechanisms to overcome the high computational complexity of GP. Third, we develop a novel approach for using domain knowledge, in the form of a probabilistic generative model, and incorporate it into the period estimation algorithm. Experimental results validate our approach showing significant improvement over existing methods.

79 ASTRONOMY AND ASTROPHYSICS↗

THE DYNAMICS OF MERGING CLUSTERS: A MONTE CARLO SOLUTION APPLIED TO THE BULLET AND MUSKET BALL CLUSTERS

Merging galaxy clusters have become one of the most important probes of dark matter, providing evidence for dark matter over modified gravity and even constraints on the dark matter self-interaction cross-section. To properly constrain the dark matter cross-section it is necessary to understand the dynamics of the merger, as the inferred cross-section is a function of both the velocity of the collision and the observed time since collision. While the best understanding of merging system dynamics comes from N-body simulations, these are computationally intensive and often explore only a limited volume of the merger phase space allowed by observed parameter uncertainty. Simple analytic models exist but the assumptions of these methods invalidate their results near the collision time, plus error propagation of the highly correlated merger parameters is unfeasible. To address these weaknesses I develop a Monte Carlo method to discern the properties of dissociative mergers and propagate the uncertainty of the measured cluster parameters in an accurate and Bayesian manner. I introduce this method, verify it against an existing hydrodynamic N-body simulation, and apply it to two known dissociative mergers: 1ES 0657-558 (Bullet Cluster) and DLSCL J0916.2+2951 (Musket Ball Cluster). I find that this method surpasses existing analytic models-providing accurate (10% level) dynamic parameter and uncertainty estimates throughout the merger history. This, coupled with minimal required a priori information (subcluster mass, redshift, and projected separation) and relatively fast computation ({approx}6 CPU hours), makes this method ideal for large samples of dissociative merging clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

Ensemble models for circuit topology estimation, fault detection and classification in distribution systems

This paper presents a methodology for simultaneous fault detection, classification, and topology estimation for adaptive protection of distribution systems. The methodology estimates the probability of the occurrence of each one of these events by using a hybrid structure that combines three sub-systems, a convolutional neural network for topology estimation, a fault detection based on predictive residual analysis, and a standard support vector machine with probabilistic output for fault classification. The input to all these sub-systems is the local voltage and current measurements. A convolutional neural network uses these local measurements in the form of sequential data to extract features and estimate the topology conditions. The fault detector is constructed with a Bayesian stage (a multitask Gaussian process) that computes a predictive distribution (assumed to be Gaussian) of the residuals using the input. Since the distribution is known, these residuals can be transformed into a Standard distribution, whose values are then introduced into a one-class support vector machine. The structure allows using a one-class support vector machine without parameter cross-validation, so the fault detector is fully unsupervised. Finally, a support vector machine uses the input to perform the classification of the fault types. All three sub-systems can work in a parallel setup for both performance and computation efficiency. In conclusion, we test all three sub-systems included in the structure on a modified IEEE123 bus system, and we compare and evaluate the results with standard approaches.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Mapping Quasar Light Echoes in 3D with Ly α Forest Tomography

The intense radiation emitted by luminous quasars dramatically alters the ionization state of their surrounding IGM. This so-called proximity effect extends out to tens of Mpc, and manifests as large coherent regions of enhanced Ly (Ly) forest transmission in absorption spectra of background sightlines. In this work, we present a novel method based on Ly forest tomography, which is capable of mapping these quasar "light echoes" in three dimensions. Using a dense grid (10-100) of faint (m r ≈ 24.7 mag) background galaxies as absorption probes, one can measure the ionization state of the IGM in the vicinity of a foreground quasar, yielding detailed information about the quasar's radiative history and emission geometry. An end-to-end analysis-combining cosmological hydrodynamical simulations post-processed with a quasar emission model, realistic estimates of galaxy number densities, and instrument + telescope throughput-is conducted to explore the feasibility of detecting quasar light echoes. We present a new, fully Bayesian statistical method that allows one to reconstruct quasar light echoes from thousands of individual low-S/N transmission measurements. Armed with this tool, we undertake an exhaustive parameter study and show that light echoes can be convincingly detected for luminous (M 1450 <-27.5 mag, corresponding to m 1450 < 18.4 mag at z≃3.6) quasars at redshifts 3 < z QSO < 5, and that a relative precision better than 20% on the quasar age can be achieved for individual objects in the expected range of ages between 1 and 100 Myr. The observational requirements are relatively modest: Moderate-resolution (R ≳ 750), multiobject spectroscopy at a low signal-to-noise ratio (S/N > 5) is sufficient, requiring three-hour integrations using existing instruments on 8 m class telescopes.

79 ASTRONOMY AND ASTROPHYSICS↗

Physics-based, Bayesian sequential detection method and system for radioactive contraband

A distributed sequential method and system for detecting and identifying radioactive contraband from highly uncertain (noisy) low-count, radionuclide measurements, i.e. an event mode sequence (EMS), using a statistical approach based on Bayesian inference and physics-model-based signal processing based on the representation of a radionuclide as a monoenergetic decomposition of monoenergetic sources. For a given photon event of the EMS, the appropriate monoenergy processing channel is determined using a confidence interval condition-based discriminator for the energy amplitude and interarrival time and parameter estimates are used to update a measured probability density function estimate for a target radionuclide. A sequential likelihood ratio test is then used to determine one of two threshold conditions signifying that the EMS is either identified as the target radionuclide or not, and if not, then repeating the process for the next sequential photon event of the EMS until one of the two threshold conditions is satisfied.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Modeling and Measurement Constraints in Fault Diagnostics for HVAC Systems

Many studies have shown that energy savings of five to fifteen percent are achievable in commercial buildings by detecting and correcting building faults, and optimizing building control systems. However, in spite of good progress in developing tools for determining HVAC diagnostics, methods to detect faults in HVAC systems are still generally undeveloped. Most approaches use numerical filtering or parameter estimation methods to compare data from energy meters and building sensors to predictions from mathematical or statistical models. They are effective when models are relatively accurate and data contain few errors. In this paper, we address the case where models are imperfect and data are variable, uncertain, and can contain error. We apply a Bayesian updating approach that is systematic in managing and accounting for most forms of model and data errors. The proposed method uses both knowledge of first principle modeling and empirical results to analyze the system performance within the boundaries defined by practical constraints. We demonstrate the approach by detecting faults in commercial building air handling units. We find that the limitations that exist in air handling unit diagnostics due to practical constraints can generally be effectively addressed through the proposed approach.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Decoding Golden Eagle Movement Behavior from High-Resolution, Variable-Rate Telemetry Data Through Bayesian Filtering

The recent advances in animal tracking technology have enabled the collection of a vast amount of in situ data regarding the movement of wildlife at high spatiotemporal resolution. These data are usually available at variable time resolutions and contains noise (error) originating from GPS fixes. Decoding movement characteristics, particularly of flying animals, from telemetry data while handling these factors is a challenging yet important task for conservation purposes. Typically, this task is broken into two subtasks: resampling, and model calibration. The resampling subtask converts the variable rate positional data into a constant time interval data, while the model calibration subtask uses the resampled data to tune time-invariant parameters of the proposed models. For telemetry data at high temporal resolutions (order of 1 second), it is very challenging to decouple noise from actual movements using interpolation-based resampling techniques. Any errors introduced during resampling can significantly alter the the calibration and prediction attributes of the movement model. We address this problem through a unified Bayesian state-space framework that can handle both the resampling and calibration tasks in a single step. In addition, we use the speed and heading of the bird from telemetry data to regularize the position information of the bird. We use a Kalman filtering approach to include these nonlinearly related motion parameters within the state space framework. We cross-validated to quantify how this inclusion affects the model performance in estimating true bird movements. The relationship between the true state of the bird and environmental and topographical covariates is then represented parametrically. These parameters are then tuned using stochastic sampling strategies like Markov Chain Monte Carlo (MCMC). We use the telemetry data collected from golden eagles in the western USA to demonstrate the applicability of this approach to build a predictive, probabilistic movement model. Our preliminary results show that this approach provides improved predictive performance in terms of capturing higher-order motion parameters such as angular and horizontal accelerations, which may have simpler and more direct relationships with environmental covariates than corresponding speeds. In this talk, we will demonstrate how this state-space approach benefits the prediction capabilities of a movement model in simulating golden eagle paths through a wind power plant in Wyoming given certain atmospheric conditions. The model outcomes are aimed at informing mitigation strategies that can minimize the potential for collisions of golden eagles with wind turbines.

Bayesian methods↗

determining the mass of kepler-78b with nonparametric Gaussian process estimation

Kepler-78b is a transiting planet that is 1.2 times the radius of Earth and orbits a young, active K dwarf every 8 hr. The mass of Kepler-78b has been independently reported by two teams based on radial velocity (RV) measurements using the HIRES and HARPS-N spectrographs. Due to the active nature of the host star, a stellar activity model is required to distinguish and isolate the planetary signal in RV data. Whereas previous studies tested parametric stellar activity models, we modeled this system using nonparametric Gaussian process (GP) regression. We produced a GP regression of relevant Kepler photometry. We then use the posterior parameter distribution for our photometric fit as a prior for our simultaneous GP + Keplerian orbit models of the RV data sets. We tested three simple kernel functions for our GP regressions. Based on a Bayesian likelihood analysis, we selected a quasi-periodic kernel model with GP hyperparameters coupled between the two RV data sets, giving a Doppler amplitude of 1.86 ± 0.25 m s{sup −1} and supporting our belief that the correlated noise we are modeling is astrophysical. The corresponding mass of 1.87{sub −0.26}{sup +0.27} M{sub ⊕} is consistent with that measured in previous studies, and more robust due to our nonparametric signal estimation. Based on our mass and the radius measurement from transit photometry, Kepler-78b has a bulk density of 6.0{sub −1.4}{sup +1.9} g cm{sup −3}. We estimate that Kepler-78b is 32% ± 26% iron using a two-component rock-iron model. This is consistent with an Earth-like composition, with uncertainty spanning Moon-like to Mercury-like compositions.

79 ASTRONOMY AND ASTROPHYSICS↗

Probabilistic Photometric Redshifts in the Era of Petascale Astronomy

With the growth of large photometric surveys, accurately estimating photometric redshifts, preferably as a probability density function (PDF), and fully understanding the implicit systematic uncertainties in this process has become increasingly important. These surveys are expected to obtain images of billions of distinct galaxies. As a result, storing and analyzing all of these photometric redshift PDFs will be non-trivial, and this challenge becomes even more severe if a survey plans to compute and store multiple different PDFs. In this thesis, we have developed an end-to-end framework that will compute accurate and robust photometric redshift PDFs for massive data sets by using two new, state-of-the-art machine learning techniques that are based on a random forest and a random atlas, respectively. By using data from several photometric surveys, we demonstrate the applicability of these new techniques, and we demonstrate that our new approach is among the best techniques currently available. We also show how different techniques can be combined by using novel Bayesian techniques to improve the photometric redshift precision to unprecedented levels while also presenting new approaches to better identify outliers. In addition, our framework provides supplementary information regarding the data being analyzed, including unbiased estimates of the accuracy of the technique without resorting to a validation data set, identification of poor photometric redshift areas within the parameter space occupied by the spectroscopic training data, and a quantification of the relative importance of the variables used during the estimation process. Furthermore, we present a new approach to represent and store photometric redshift PDFs by using a sparse representation with outstanding compression and reconstruction capabilities. We also demonstrate how this framework can also be directly incorporated into cosmological analyses. The new techniques presented in this thesis are crucial to enable the development of precision cosmology in the era of petascale astronomical surveys.

79 ASTRONOMY AND ASTROPHYSICS↗

BeyondPlanck: VII. Bayesian estimation of gain and absolute calibration for cosmic microwave background experiments

We present a Bayesian calibration algorithm for cosmic microwave background (CMB) observations as implemented within the global end-to-end BEYONDPLANCK framework and applied to the Planck Low Frequency Instrument (LFI) data. Following the most recent Planck analysis, we decomposed the full time-dependent gain into a sum of three nearly orthogonal components: one absolute calibration term, common to all detectors, one time-independent term that can vary between detectors, and one time-dependent component that was allowed to vary between one-hour pointing periods. Each term was then sampled conditionally on all other parameters in the global signal model through Gibbs sampling. The absolute calibration is sampled using only the orbital dipole as a reference source, while the two relative gain components were sampled using the full sky signal, including the orbital and Solar CMB dipoles, CMB fluctuations, and foreground contributions. We discuss various aspects of the data that influence gain estimation, including the dipole-polarization quadrupole degeneracy and processing masks. Comparing our solution to previous pipelines, we find good agreement in general, with relative deviations of -0.67% (-0.84%) for 30 GHz, 0.12% (-0.04%) for 44 GHz and -0.03% (-0.64%) for 70 GHz, compared to Planck PR4 and Planck 2018, respectively. We note that the BEYONDPLANCK calibration was performed globally, which results in better inter-frequency consistency than previous estimates. Additionally, WMAP observations were used actively in the BEYONDPLANCK analysis, which both breaks internal degeneracies in the Planck data set and results in an overall better agreement with WMAP. Finally, we used a Wiener filtering approach to smoothing the gain estimates. We show that this method avoids artifacts in the correlated noise maps as a result of oversmoothing the gain solution, which is difficult to avoid with methods like boxcar smoothing, as Wiener filtering by construction maintains a balance between data fidelity and prior knowledge. Although our presentation and algorithm are currently oriented toward LFI processing, the general procedure is fully generalizable to other experiments, as long as the Solar dipole signal is available to be used for calibration.

79 ASTRONOMY AND ASTROPHYSICS↗

Characterization of a qubit Hamiltonian using adaptive measurements in a fixed basis

We investigate schemes for Hamiltonian parameter estimation of a two-level system using repeated measurements in a fixed basis. The simplest (Fourier based) schemes yield an estimate with a mean-square error (MSE) that decreases at best as a power law {approx}N{sup -2} in the number of measurements N. By contrast, we present numerical simulations indicating that an adaptive Bayesian algorithm, where the time between measurements can be adjusted based on prior measurement results, yields a MSE which appears to scale close to exp(-0.3N). That is, measurements in a single fixed basis are sufficient to achieve exponential scaling in N.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Bayesian calibration of the Community Land Model using surrogates

We present results from the Bayesian calibration of hydrological parameters of the Community Land Model (CLM), which is often used in climate simulations and Earth system models. A statistical inverse problem is formulated for three hydrological parameters, conditional on observations of latent heat surface fluxes over 48 months. Our calibration method uses polynomial and Gaussian process surrogates of the CLM, and solves the parameter estimation problem using a Markov chain Monte Carlo sampler. Posterior probability densities for the parameters are developed for two sites with different soil and vegetation covers. Our method also allows us to examine the structural error in CLM under two error models. We find that surrogate models can be created for CLM in most cases. The posterior distributions are more predictive than the default parameter values in CLM. Climatologically averaging the observations does not modify the parameters' distributions significantly. The structural error model reveals a correlation time-scale which can be used to identify the physical process that could be contributing to it. While the calibrated CLM has a higher predictive skill, the calibration is under-dispersive.

54 ENVIRONMENTAL SCIENCES↗

Planck 2013 results. XV. CMB power spectra and likelihood

This paper presents the Planck 2013 likelihood, a complete statistical description of the two-point correlation function of the CMB temperature fluctuations that accounts for all known relevant uncertainties, both instrumental and astrophysical in nature. We use this likelihood to derive our best estimate of the CMB angular power spectrum from Planck over three decades in multipole moment, &ell;, covering 2 &le; &ell; &le; 2500. The main source of uncertainty at &ell; &le; 1500 is cosmic variance. Uncertainties in small-scale foreground modelling and instrumental noise dominate the error budget at higher &ell;s. For &ell; < 50, our likelihood exploits all Planck frequency channels from 30 to 353 GHz, separating the cosmological CMB signal from diffuse Galactic foregrounds through a physically motivated Bayesian component separation technique. At &ell; &le; 50, we employ a correlated Gaussian likelihood approximation based on a fine-grained set of angular cross-spectra derived from multiple detector combinations between the 100, 143, and 217 GHz frequency channels, marginalising over power spectrum foreground templates. We validate our likelihood through an extensive suite of consistency tests, and assess the impact of residual foreground and instrumental uncertainties on the final cosmological parameters. We find good internal agreement among the high-&ell; cross-spectra with residuals below a few &mu;K 2 at &ell; ≲ 1000, in agreement with estimated calibration uncertainties. We compare our results with foreground-cleaned CMB maps derived from all Planck frequencies, as well as with cross-spectra derived from the 70 GHz Planck map, and find broad agreement in terms of spectrum residuals and cosmological parameters. We further show that the best-fit CDM cosmology is in excellent agreement with preliminary Planck EE and TE polarisation spectra. We find that the standard ΛCDM cosmology is well constrained by Planck from the measurements at &ell; ≲ 1500. One specific example is the spectral index of scalar perturbations, for which we report a 5.4σ deviation from scale invariance, n s = 1. Increasing the multipole range beyond &ell; ≃ 1500 does not increase our accuracy for the ΛCDM parameters, but instead allows us to study extensions beyond the standard model. We find no indication of significant departures from the ΛCDM framework. Finally, we report a tension between the Planck best-fit ΛCDM model and the low-&ell; spectrum in the form of a power deficit of 5&ndash;10% at &ell; ≲ 40, with a statistical significance of 2.5&ndash;3σ. Without a theoretically motivated model for this power deficit, we do not elaborate further on its cosmological implications, but note that this is our most puzzling finding in an otherwise remarkably consistent data set.

79 ASTRONOMY AND ASTROPHYSICS↗

Improved Nested Sampling and Surrogate-Enabled Comparison With Other Marginal Likelihood Estimators

Estimating marginal likelihood is of central importance to Bayesian model selection and/or model averaging. The nested sampling method has been recently used together with the Metropolis-Hasting (M-H) sampling algorithm for estimating marginal likelihood. This study develops a new implementation of nested sampling by using the DiffeRential Evolution Adaptive Metropolis (DREAMzs) sampling algorithm. The two implementations of nested sampling are evaluated for four models of a synthetic groundwater flow modeling. The DREAMzs-based nested sampling outperforms the M-H-based nested sampling in terms of their accuracy, convergence, efficiency, and stability, which is attributed to the fact that DREAMzs is more robust than M-H for parameter sampling. The nested sampling method is also compared with four other methods (arithmetic mean, harmonic mean, stabilized harmonic mean, and thermodynamic integration) commonly used for estimating marginal likelihood and posterior probability of the four groundwater models. The comparative study requires hundreds of millions of model executions, which would not be possible without using surrogate models to replace the original models. Using the arithmetic mean estimator as the reference, the comparison reveals that thermodynamic integration outperforms nested sampling in terms of accuracy and stability, whereas nested sampling is more computationally efficient to reach to convergence. Here, the harmonic mean and stabilized harmonic mean methods give the worst marginal likelihood estimation. Accurate marginal likelihood estimation is important for accurate estimation of posterior model probability and better predictive performance of Bayesian model averaging.

54 ENVIRONMENTAL SCIENCES↗

Optimal Estimation Retrievals and Their Uncertainties: What Every Atmospheric Scientist Should Know

Remote sensing instruments are heavily used to provide observations for both the operational and research communities. These sensors do not provide direct observations of the desired atmospheric variables, but instead, retrieval algorithms are necessary to convert the indirect observations into the variable of interest. It is critical to be aware of the underlying assumptions made by many retrieval algorithms, including that the retrieval problem is often ill posed and that there are various sources of uncertainty that need to be treated properly. In short, the retrieval challenge is to invert a set of noisy observations to obtain estimates of atmospheric quantities. The problem is often complicated by imperfect forward models, by imperfect prior knowledge, and by the existence of nonunique solutions. Optimal estimation (OE) is a widely used physical retrieval method that combines measurements, prior information, and the corresponding uncertainties based on Bayes’s theorem to find an optimal solution for the atmospheric state. Furthermore, OE also allows the relative contributions of the different sources of error to the uncertainty in the final retrieved atmospheric state to be understood. Here, we provide a novel Python library to illustrate the use of OE for inverse problems in the atmospheric sciences. We introduce two example problems: how to retrieve drop size distribution parameters from radar observations and how to retrieve the temperature profile from ground-based microwave sensors. Using these examples, we discuss common pitfalls, how the various error sources impact the retrieval, and how the quality of the retrieval results can be quantified.

54 ENVIRONMENTAL SCIENCES↗