Search NASA⌕ Search

SEARCH · Search NASA

Results for “constrained Bayesian optimization”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

Jensen–Shannon divergence based novel loss functions for Bayesian neural networks

Bayesian neural networks (BNNs) are state-of-the-art machine learning methods that can naturally regularize and systematically quantify uncertainties using their stochastic parameters. Kullback–Leibler (KL) divergence-based variational inference used in BNNs suffer from unstable optimization and challenges in approximating light-tailed posteriors due to the unbounded nature of the KL divergence. To resolve these issues, we formulate a novel loss function for BNNs based on a new modification to the generalized Jensen–Shannon (JS) divergence, which is bounded. In addition, we propose a Geometric JS divergence-based loss, which is computationally efficient since it can be evaluated analytically. We found that the JS divergence-based variational inference is intractable, and hence employed a constrained optimization framework to formulate these losses. Our theoretical analysis and empirical experiments on multiple regression and classification data sets suggest that the proposed losses perform better than the KL divergence-based loss, especially when the data sets are noisy or biased. Specifically, there are approximately 5% and 8% improvements in accuracy for a noise-added CIFAR-10 dataset and a regression dataset, respectively. There is about 13% reduction in false negative predictions of a biased histopathology dataset. Additionally, we quantify and compare the uncertainty metrics for the regression and classification tasks.

97 MATHEMATICS AND COMPUTING↗

Parameter estimation from spontaneous imbibition into volcanic tuff

Two-phase fluid flow properties underlie quantitative prediction of water and gas movement, but constraining these properties typically requires multiple time-consuming laboratory methods. The estimation of two-phase flow properties (van Genuchten parameters, porosity, and intrinsic permeability) is illustrated in cores of vitric nonwelded volcanic tuff using Bayesian parameter estimation that fits numerical models to observations from spontaneous imbibition experiments. The uniqueness and correlation of the estimated parameters is explored using different modeling assumptions and subsets of the observed data. The resulting estimation process is sensitive to both moisture retention and relative permeability functions, thereby offering a comprehensive method for constraining both functions. The data collected during this relatively simple laboratory experiment, used in conjunction with a numerical model and a global optimizer, result in a viable approach for augmenting more traditional capillary pressure data obtained from hanging water column, membrane plate extractor, or mercury intrusion methods. This method may be useful when imbibition rather than drainage parameters are sought, when larger samples (e.g., including heterogeneity or fractures) need to be tested that cannot be accommodated in more traditional methods, or when in educational laboratory settings.

58 GEOSCIENCES↗

BEYONDPLANCK VI. Noise characterization and modeling

We present a Bayesian method for estimating instrumental noise parameters and propagating noise uncertainties within the global BeyondPlanck Gibbs sampling framework, and apply this to Planck Low Frequency Instrument (LFI) time-ordered data. Following previous literature, we initially adopt a 1/ f model for the noise power spectral density (PSD), but find the need for an additional lognormal component in the noise model for the 30 and 44 GHz bands. We implement an optimal Wiener-filter (or constrained realization) gap-filling procedure to account for masked data. We then use this procedure to both estimate the gapless correlated noise in the time-domain, n corr , and to sample the noise PSD parameters, ξ n = {σ 0 , f knee , α, A p }. In contrast to previous Planck analyses, we assume piecewise stationary noise only within each pointing period (PID), not throughout the full mission, but we adopt the LFI Data Processing Center (DPC) results as priors on α and f knee . On average, we find best-fit correlated noise parameters that are mostly consistent with previous results, with a few notable exceptions. However, a detailed inspection of the time-dependent results reveals many important findings. First and foremost, we find strong evidence for statistically significant temporal variations in all noise PSD parameters, many of which are directly correlated with satellite housekeeping data. Second, while the simple 1/ f model appears to be an excellent fit for the LFI 70 GHz channel, there is evidence for additional correlated noise not described by a 1/ f model in the 30 and 44 GHz channels, including within the primary science frequency range of 0.1–1 Hz. In general, most 30 and 44 GHz channels exhibit deviations from 1/ f at the 2–3σ level in each one hour pointing period, motivating the addition of the lognormal noise component for these bands. For some periods of time, we also find evidence of strong common mode noise fluctuations across the entire focal plane. Overall, we conclude that a simple 1/ f profile is not adequate to fully characterize the Planck LFI noise, even when fitted hour-by-hour, and a more general model is required. These findings have important implications for large-scale CMB polarization reconstruction with the Planck LFI data, and the current work is a first attempt at understanding and mitigating these issues.

79 ASTRONOMY AND ASTROPHYSICS↗

Bayesian optimization of the Community Land Model simulated biosphere-atmosphere exchange using CO 2 observations from a dense tower network and aircraft campaigns over Oregon

Here, the vast forests and natural areas of the Pacific Northwest comprise one of the most productive ecosystems in the northern hemisphere. The heterogeneous landscape of Oregon poses a particular challenge to ecosystem models. We present a framework using a scaling factor Bayesian inversion to improve the modeled atmosphere-biosphere exchange of carbon dioxide. Observations from 5 CO/CO 2 towers, eddy covariance towers, and airborne campaigns were used to constrain the Community Land Model CLM4.5 simulated terrestrial CO 2 exchange at a high spatial and temporal resolution (1/24°, 3-hourly). To balance aggregation errors and the degrees of freedom in the inverse modeling system, we applied an unsupervised clustering approach for the spatial structuring of our model domain. Data from flight campaigns were used to quantify the uncertainty introduced by the Lagrangian particle dispersion model that was applied for the inversions. The average annual statewide net ecosystem productivity (NEP) was increased by 32% to 29.7 TgC per year by assimilating the tropospheric mixing ratio data. The associated uncertainty was decreased by 28.4% to 29%, on average over the entire Oregon model domain with the lowest uncertainties of 11% in western Oregon. The largest differences between posterior and prior CO 2 fluxes were found for the Coast Range ecoregion of Oregon that also exhibits the highest availability of atmospheric observations and associated footprints. In this area, covered by highly productive Douglas-fir forest, the differences between the prior and posterior estimate of NEP averaged 3.84 TgC per year during the study period from 2012 through 2014.

54 ENVIRONMENTAL SCIENCES↗

Recent advances in the quantification of uncertainties in reaction theory

Uncertainty quantification has become increasingly more prominent in nuclear physics over the past several years. In few-body reaction theory, there are four main sources that contribute to the uncertainties in the calculated observables: the effective potentials, approximations made to the few-body problem, structure functions, and degrees of freedom left out of the model space. In this work, we illustrate some of the features that can be obtained when modern statistical tools are applied in the context of nuclear reactions. This work consists of a summary of the progress that has been made in quantifying theoretical uncertainties in this domain, focusing primarily on those uncertainties coming from the effective optical potential as well as their propagation within various reaction theories. We use, as the central example, reactions on the doubly-magic stable nucleus 40 Ca, namely neutron and proton elastic scattering and single-nucleon transfer 40 Ca(d,p) 41 Ca. First, we show different optimization schemes used to constrain the optical potential from differential cross sections and other experimental constraints; we then discuss how these uncertainties propagate to the transfer cross section, comparing two reaction theories. Finally, we finish by laying out our future plans.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Bayesian optimization with active learning of design constraints using an entropy-based approach

Abstract The design of alloys for use in gas turbine engine blades is a complex task that involves balancing multiple objectives and constraints. Candidate alloys must be ductile at room temperature and retain their yield strength at high temperatures, as well as possess low density, high thermal conductivity, narrow solidification range, high solidus temperature, and a small linear thermal expansion coefficient. Traditional Integrated Computational Materials Engineering (ICME) methods are not sufficient for exploring combinatorially-vast alloy design spaces, optimizing for multiple objectives, nor ensuring that multiple constraints are met. In this work, we propose an approach for solving a constrained multi-objective materials design problem over a large composition space, specifically focusing on the Mo-Nb-Ti-V-W system as a representative Multi-Principal Element Alloy (MPEA) for potential use in next-generation gas turbine blades. Our approach is able to learn and adapt to unknown constraints in the design space, making decisions about the best course of action at each stage of the process. As a result, we identify 21 Pareto-optimal alloys that satisfy all constraints. Our proposed framework is significantly more efficient and faster than a brute force approach.

36 MATERIALS SCIENCE↗

Using a surrogate-assisted Bayesian framework to calibrate the runoff-generation scheme in the Energy Exascale Earth System Model (E3SM) v1

Abstract. Runoff is a critical component of the terrestrial water cycle, and Earth system models (ESMs) are essential tools to study its spatiotemporal variability. Runoff schemes in ESMs typically include many parameters so that model calibration is necessary to improve the accuracy of simulated runoff. However, runoff calibration at a global scale is challenging because of the high computational cost and the lack of reliable observational datasets. In this study, we calibrated 11 runoff relevant parameters in the Energy Exascale Earth System Model (E3SM) Land Model (ELM) using a surrogate-assisted Bayesian framework. First, the polynomial chaos expansion machinery with Bayesian compressed sensing is used to construct computationally inexpensive surrogate models for ELM-simulated runoff at 0.5∘ × 0.5∘ for 1991–2010. The error metric between the ELM simulations and the benchmark data is selected to construct the surrogates, which facilitates efficient calibration and avoids the more conventional, but challenging, construction of high-dimensional surrogates for the ELM simulated runoff. Second, the Sobol' index sensitivity analysis is performed using the surrogate models to identify the most sensitive parameters, and our results show that, in most regions, ELM-simulated runoff is strongly sensitive to 3 of the 11 uncertain parameters. Third, a Bayesian method is used to infer the optimal values of the most sensitive parameters using an observation-based global runoff dataset as the benchmark. Our results show that model performance is significantly improved with the inferred parameter values. Although the parametric uncertainty of simulated runoff is reduced after the parameter inference, it remains comparable to the multimodel ensemble uncertainty represented by the global hydrological models in ISMIP2a. Additionally, the annual global runoff trend during the simulation period is not well constrained by the inferred parameter values, suggesting the importance of including parametric uncertainty in future runoff projections.

58 GEOSCIENCES↗

Bayesian optimization of generalized data

Direct application of Bayes' theorem to generalized data yields a posterior probability distribution function (PDF) that is a product of a prior PDF of generalized data and a likelihood function, where generalized data consists of model parameters, measured data, and model defect data. The prior PDF of generalized data is defined by prior expectation values and a prior covariance matrix of generalized data that naturally includes covariance between any two components of generalized data. A set of constraints imposed on the posterior expectation values and covariances of generalized data via a given model is formally solved by the method of Lagrange multipliers. Posterior expectation values of the constraints and their covariance matrix are conventionally set to zero, leading to a likelihood function that is a Dirac delta function of the constraining equation. It is shown that setting constraints to values other than zero is analogous to introducing a model defect. Since posterior expectation values of any function of generalized data are integrals of that function over all generalized data weighted by the posterior PDF, all elements of generalized data may be viewed as nuisance parameters marginalized by this integration. One simple form of posterior PDF is obtained when the prior PDF and the likelihood function are normal PDFs. For linear models without a defect this PDF becomes equivalent to constrained least squares (CLS) method, that is, the χ2 minimization method.

Arbanas, Goran↗

Robust scalable initialization for Bayesian variational inference with multi-modal Laplace approximations

Predictive modeling typically relies on Bayesian model calibration to provide uncertainty quantification. Variational inference utilizing fully independent (“mean-field”) Gaussian distributions are often used as approximate probability density functions. This simplification is attractive since the number of variational parameters grows only linearly with the number of unknown model parameters. However, the resulting diagonal covariance structure and unimodal behavior can be too restrictive to provide useful approximations of intractable Bayesian posteriors that exhibit highly non-Gaussian behavior, including multimodality. High-fidelity surrogate posteriors for these problems can be obtained by considering the family of Gaussian mixtures. Gaussian mixtures are capable of capturing multiple modes and approximating any distribution to an arbitrary degree of accuracy, while maintaining some analytical tractability. Unfortunately, variational inference using Gaussian mixtures with full-covariance structures suffers from a quadratic growth in variational parameters with the number of model parameters. The existence of multiple local minima due to strong nonconvex trends in the loss functions often associated with variational inference present additional complications, These challenges motivate the need for robust initialization procedures to improve the performance and computational scalability of variational inference with mixture models. In this work, we propose a method for constructing an initial Gaussian mixture model approximation that can be used to warm-start the iterative solvers for variational inference. The procedure begins with a global optimization stage in model parameter space. In this step, local gradient-based optimization, globalized through multistart, is used to determine a set of local maxima, which we take to approximate the mixture component centers. Around each mode, a local Gaussian approximation is constructed via the Laplace approximation. Finally, the mixture weights are determined through constrained least squares regression. The robustness and scalability of the proposed methodology is demonstrated through application to an ensemble of synthetic tests using high-dimensional, multimodal probability density functions. Here, the practical aspects of the approach are demonstrated with inversion problems in structural dynamics.

97 MATHEMATICS AND COMPUTING↗

Leveraging 35 years of Pinus taeda research in the southeastern US to constrain forest carbon cycle predictions: regional data assimilation using ecosystem experiments

Predicting how forest carbon cycling will change in response to climate change and management depends on the collective knowledge from measurements across environmental gradients, ecosystem manipulations of global change factors, and mathematical models. Formally integrating these sources of knowledge through data assimilation, or model–data fusion, allows the use of past observations to constrain model parameters and estimate prediction uncertainty. Data assimilation (DA) focused on the regional scale has the opportunity to integrate data from both environmental gradients and experimental studies to constrain model parameters. Here, we introduce a hierarchical Bayesian DA approach (Data Assimilation to Predict Productivity for Ecosystems and Regions, DAPPER) that uses observations of carbon stocks, carbon fluxes, water fluxes, and vegetation dynamics from loblolly pine plantation ecosystems across the southeastern US to constrain parameters in a modified version of the Physiological Principles Predicting Growth (3-PG) forest growth model. The observations included major experiments that manipulated atmospheric carbon dioxide (CO 2 ) concentration, water, and nutrients, along with nonexperimental surveys that spanned environmental gradients across an 8.6 × 10 5 km 2 region. We optimized regionally representative posterior distributions for model parameters, which dependably predicted data from plots withheld from the data assimilation. While the mean bias in predictions of nutrient fertilization experiments, irrigation experiments, and CO 2 enrichment experiments was low, future work needs to focus modifications to model structures that decrease the bias in predictions of drought experiments. Predictions of how growth responded to elevated CO 2 strongly depended on whether ecosystem experiments were assimilated and whether the assimilated field plots in the CO 2 study were allowed to have different mortality parameters than the other field plots in the region. We present predictions of stem biomass productivity under elevated CO 2 , decreased precipitation, and increased nutrient availability that include estimates of uncertainty for the southeastern US. Overall, we (1) demonstrated how three decades of research in southeastern US planted pine forests can be used to develop DA techniques that use multiple locations, multiple data streams, and multiple ecosystem experiment types to optimize parameters and (2) developed a tool for the development of future predictions of forest productivity for natural resource managers that leverage a rich dataset of integrated ecosystem observations across a region.

54 ENVIRONMENTAL SCIENCES↗

Simple model to investigate jet quenching and correlated errors for centrality-dependent nuclear modification factors in relativistic heavy-ion collisions

Here, we apply Bayesian techniques to compare a simple, empirical model for jet quenching in heavy-ion collisions to centrality-dependent jet R AA measured by ATLAS for Pb + Pb collisions at $\sqrt{s_{NN}}$ = 5.02 TeV. We find that the R AA values for central collisions are adequately described with a model for the mean p T -dependent jet energy loss using only two parameters. This model is extended by incorporating two-dimensional initial geometry information from TRENTo and compared to centrality-dependent R AA values. We find that the results are sensitive to the value of the jet-quenching formation time, τ ƒ , and that the optimal value of τ ƒ varies with the assumed path-length dependence of the energy loss. We construct a covariance error matrix for the data from the p T -dependent contributions to the ATLAS systematic errors and perform Bayesian calibrations for several different assumptions for the systematic error correlations. We show that the most-probable functions and $χ^2_d$ values are sensitive to assumptions made when fitting to correlated errors. This work demonstrates the utility of a simple model that can quickly demonstrate the constraining power of jet-quenching observables with corresponding uncertainties and guide future studies using more sophisticated models.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

A multitower measurement network estimate of California's methane emissions

In this paper, we present an analysis of methane (CH 4 ) emissions using atmospheric observations from five sites in California's Central Valley across different seasons (September 2010 to June 2011). CH 4 emissions for spatial regions and source sectors are estimated by comparing measured CH 4 mixing ratios with transport model (Weather Research and Forecasting and Stochastic Time-Inverted Lagrangian Transport) predictions based on two 0.1° CH 4 (seasonally varying “California-specific” (California Greenhouse Gas Emission Measurements, CALGEM) and a static global (Emission Database for Global Atmospheric Research, release version 42, EDGAR42)) prior emission models. Region-specific Bayesian analyses indicate that for California's Central Valley, the CALGEM- and EDGAR42-based inversions provide consistent annual total CH 4 emissions (32.87 ± 2.09 versus 31.60 ± 2.17 Tg CO 2 eq yr -1 ; 68% confidence interval (CI), assuming uncorrelated errors between regions). Summing across all regions of California, optimized CH 4 emissions are only marginally consistent between CALGEM- and EDGAR42-based inversions (48.35 ± 6.47 versus 64.97 ± 11.85 Tg CO 2 eq), because emissions from coastal urban regions (where landfill and natural gas emissions are much higher in EDGAR than CALGEM) are not strongly constrained by the measurements. Combining our results with those from a recent study of the South Coast Air Basin narrows the range of estimates to 43–57 Tg CO2eq yr -1 (1.3–1.8 times higher than the current state inventory). Finally, these results suggest that the combination of rural and urban measurements will be necessary to verify future changes in California's total CH 4 emissions.

54 ENVIRONMENTAL SCIENCES↗

Optimal bispectrum constraints on single-field models of inflation

We use WMAP 9-year bispectrum data to constrain the free parameters of an 'effective field theory' describing fluctuations in single-field inflation. The Lagrangian of the theory contains a finite number of operators associated with unknown mass scales. Each operator produces a fixed bispectrum shape, which we decompose into partial waves in order to construct a likelihood function. Based on this likelihood we are able to constrain four linearly independent combinations of the mass scales. As an example of our framework we specialize our results to the case of 'Dirac-Born-Infeld' and 'ghost' inflation and obtain the posterior probability for each model, which in Bayesian schemes is a useful tool for model comparison. Our results suggest that DBI-like models with two or more free parameters are disfavoured by the data by comparison with single-parameter models in the same class.

79 ASTRONOMY AND ASTROPHYSICS↗

Inferring Type II-P Supernova Progenitor Masses from Plateau Luminosities

Abstract Connecting observations of core-collapse supernova explosions to the properties of their massive star progenitors is a long-sought, and challenging, goal of supernova science. Recently, Barker et al. presented bolometric light curves for a landscape of progenitors from spherically symmetric neutrino-driven core-collapse supernova (CCSN) simulations using an effective model. They find a tight relationship between the plateau luminosity of the Type II-P CCSN light curve and the terminal iron-core mass of the progenitor. Remarkably, this allows us to constrain progenitor properties with photometry alone. We analyze a large observational sample of Type II-P CCSN light curves and estimate a distribution of iron-core masses using the relationship of Barker et al. The inferred distribution matches extremely well with the distribution of iron-core masses from stellar evolutionary models and namely, contains high-mass iron cores that suggest contributions from very massive progenitors in the observational data. We use this distribution of iron-core masses to infer minimum and maximum masses of progenitors in the observational data. Using Bayesian inference methods to locate optimal initial mass function parameters, we find M min = 9.8 − 0.27 + 0.37 and M max = 24.0 − 1.9 + 3.9 solar masses for the observational data.

79 ASTRONOMY AND ASTROPHYSICS↗

CORRECTING VELOCITY DISPERSIONS OF DWARF SPHEROIDAL GALAXIES FOR BINARY ORBITAL MOTION

We show that the measured velocity dispersions of dwarf spheroidal galaxies from about 4 to 10 km s{sup -1} are unlikely to be inflated by more than 30% due to the orbital motion of binary stars and demonstrate that the intrinsic velocity dispersions can be determined to within a few percent accuracy using two-epoch observations with 1-2 yr as the optimal time interval. The crucial observable is the threshold fraction-the fraction of stars that show velocity changes larger than a given threshold between measurements. The threshold fraction is tightly correlated with the dispersion introduced by binaries, independent of the underlying binary fraction and distribution of orbital parameters. We outline a simple procedure to correct the velocity dispersion to within a few percent accuracy by using the threshold fraction and provide fitting functions for this method. We also develop a methodology for constraining properties of binary populations from both single- and two-epoch velocity measurements by including the binary velocity distribution in a Bayesian analysis.

79 ASTRONOMY AND ASTROPHYSICS↗

Standing on the shoulders of dwarfs: the kepler asteroseismic legacy sample. I. Oscillation mode parameters

The advent of space-based missions like Kepler has revolutionized the study of solar-type stars, particularly through the measurement and modeling of their resonant modes of oscillation. Here we analyze a sample of 66 Kepler main-sequence stars showing solar-like oscillations as part of the Kepler seismic LEGACY project. We use Kepler short-cadence data, of which each star has at least 12 months, to create frequency-power spectra optimized for asteroseismology. For each star, we identify its modes of oscillation and extract parameters such as frequency, amplitude, and line width using a Bayesian Markov chain Monte Carlo “peak-bagging” approach. We report the extracted mode parameters for all 66 stars, as well as derived quantities such as frequency difference ratios, the large and small separations Δν and δν{sub 02}; the behavior of line widths with frequency and line widths at ν{sub max} with T{sub eff}, for which we derive parametrizations; and behavior of mode visibilities. These average properties can be applied in future peak-bagging exercises to better constrain the parameters of the stellar oscillation spectra. The frequencies and frequency ratios can tightly constrain the fundamental parameters of these solar-type stars, and mode line widths and amplitudes can test models of mode damping and excitation.

79 ASTRONOMY AND ASTROPHYSICS↗

Cosmic Inference: Constraining Parameters with Observations and a Highly Limited Number of Simulations

We look at cosmological probes that pose an inverse problem where the measurement result is obtained through observations, and the objective is to infer values of model parameters that characterize the underlying physical system-our universe, from these observations and theoretical forward-modeling. The only way to accurately forward-model physical behavior on small scales is via expensive numerical simulations, which are further “emulated” due to their high cost. Emulators are commonly built with a set of simulations covering the parameter space with Latin hypercube sampling and an interpolation procedure; the aim is to establish an approximately constant prediction error across the hypercube. In this paper, we provide a description of a novel statistical framework for obtaining accurate parameter constraints. The proposed framework uses multi-output Gaussian process emulators that are adaptively constructed using Bayesian optimization methods with the goal of maintaining a low emulation error in the region of the hypercube preferred by the observational data. In this paper, we compare several approaches for constructing multi-output emulators that enable us to take possible inter-output correlations into account while maintaining the efficiency needed for inference. Using a Lyα forest flux power spectrum, we demonstrate that our adaptive approach requires considerably fewer-by a factor of a few in the Lyα P(k) case considered here-simulations compared to the emulation based on Latin hypercube sampling, and that the method is more robust in reconstructing parameters and their Bayesian credible intervals.

79 ASTRONOMY AND ASTROPHYSICS↗

A Logarithmic Bayesian Approach to Quantum Error Detection

We consider the problem of continuous quantum error correction from a Bayesian perspective, proposing a pair of digital filters using logarithmic probabilities that are able to achieve near-optimal performance on a three-qubit bit-flip code, while still being reasonable to implement on low-latency hardware. These practical filters are approximations of an optimal filter that we derive explicitly for finite time steps, in contrast with previous work that has relied on stochastic differential equations such as the Wonham filter. By utilizing logarithmic probabilities, we are able to eliminate the need for explicit normalization and can reduce the Gaussian noise distribution to a simple quadratic expression. The state transitions induced by the bit-flip errors are modeled using a Markov chain, which for log-probabilities must be evaluated using a LogSumExp function. We develop the two versions of our filter by constraining this LogSumExp to have either one or two inputs, which favors either simplicity or accuracy, respectively. Using simulated data, we demonstrate that the single-term and two-term filters are able to significantly outperform both a double threshold scheme and a linearized version of the Wonham filter in tests of error detection under a wide variety of error rates and time steps.

97 MATHEMATICS AND COMPUTING↗