Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bayesian model selection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

Comprehensive compartmental model and calibration algorithm for the study of clinical implications of the population-level spread of COVID-19: a study protocol

The complex dynamics of the coronavirus disease 2019 (COVID-19) pandemic has made obtaining reliable long-term forecasts of the disease progression difficult. Simple mechanistic models with deterministic parameters are useful for short-term predictions but have ultimately been unsuccessful in extrapolating the trajectory of the pandemic because of unmodelled dynamics and the unrealistic level of certainty that is assumed in the predictions. We propose a 22-compartment epidemiological model that includes compartments not previously considered concurrently, to account for the effects of vaccination, asymptomatic individuals, inadequate access to hospital care, post-acute COVID-19 and recovery with long-term health complications. Additionally, new connections between compartments introduce new dynamics to the system and provide a framework to study the sensitivity of model outputs to several concurrent effects, including temporary immunity, vaccination rate and vaccine effectiveness. Subject to data availability for a given region, we discuss a means by which population demographics (age, comorbidity, socioeconomic status, sex and geographical location) and clinically relevant information (different variants, different vaccines) can be incorporated within the 22-compartment framework. Considering a probabilistic interpretation of the parameters allows the model's predictions to reflect the current state of uncertainty about the model parameters and model states. We propose the use of a sparse Bayesian learning algorithm for parameter calibration and model selection. This methodology considers a combination of prescribed parameter prior distributions for parameters that are known to be essential to the modelled dynamics and automatic relevance determination priors for parameters whose relevance is questionable. This is useful as it helps prevent overfitting the available epidemiological data when calibrating the parameters of the proposed model. Population-level administrative health data will serve as partial observations of the model states.

59 BASIC BIOLOGICAL SCIENCES↗

Uncertainty-aware mixed-variable machine learning for materials design

Abstract Data-driven design shows the promise of accelerating materials discovery but is challenging due to the prohibitive cost of searching the vast design space of chemistry, structure, and synthesis methods. Bayesian optimization (BO) employs uncertainty-aware machine learning models to select promising designs to evaluate, hence reducing the cost. However, BO with mixed numerical and categorical variables, which is of particular interest in materials design, has not been well studied. In this work, we survey frequentist and Bayesian approaches to uncertainty quantification of machine learning with mixed variables. We then conduct a systematic comparative study of their performances in BO using a popular representative model from each group, the random forest-based Lolo model (frequentist) and the latent variable Gaussian process model (Bayesian). We examine the efficacy of the two models in the optimization of mathematical functions, as well as properties of structural and functional materials, where we observe performance differences as related to problem dimensionality and complexity. By investigating the machine learning models’ predictive and uncertainty estimation capabilities, we provide interpretations of the observed performance differences. Our results provide practical guidance on choosing between frequentist and Bayesian uncertainty-aware machine learning models for mixed-variable BO in materials design.

36 MATERIALS SCIENCE↗

Process damping identification using Bayesian learning and time domain simulation

Process damping can provide improved machining productivity by increasing the stability limit at low spindle speeds. However, existing methods for identifying process damping models experimentally require specialized setups and/or multiple cutting tests. While the phenomenon is well known, the modeling challenges limit pre-process parameter selection that leverages the potential increases in material removal rates. This paper proposes a physics-informed Bayesian method that can identify the cutting force and process damping models from a limited set of test cuts without requiring direct measurements of cutting force or vibration. The method uses time domain simulation to incorporate process damping and provide a basis for test selection. Here, new strategies for efficient sampling and dimensionality reduction are applied to lower computation time and minimize the effect of model error. The proposed method is demonstrated and the identified cutting and damping force coefficients are compared to values obtained using machining tests and least-squares fitting.

42 ENGINEERING↗

Automated and efficient local adaptive regression for principal component-based reduced-order modeling of turbulent reacting flows

Principal Component Analysis can be used to reduce the cost of Computational Fluid Dynamics simulations of turbulent reacting flows by reducing the dimensionality of the transported variables through projection of the thermochemical state onto a lower-dimensional manifold. However, because of the nonlinearity of the principal component source terms, nonlinear regression techniques must be utilized for the source terms in terms of the principal components. Unfortunately, widely available and utilized nonlinear regression techniques can have prohibitive computational requirements and/or accuracy that is highly dependent on user experience in ad hoc tuning of model architecture and hyperparameters. Here, in this work, a new nonlinear regression approach is proposed that is both computationally efficient and automated so does not require any user input. The approach is evaluated through a priori prediction of principal component source terms using data from a Direct Numerical Simulation of a turbulent nonpremixed n-heptane/air jet flame. In particular, the proposed framework consists of local regressions whose complexity is adapted according to the local nonlinearity of the data: local linear regression when accurate enough and local Artificial Neural Networks when nonlinear regression is required. The number of local clusters for local regression is determined automatically using the Davies-Bouldin index. In addition, Bayesian optimization is utilized for model training (i.e., to select the best architectures and hyperparameters of the nonlinear regressions in an unsupervised fashion), eliminating ad hoc hand-tuning and/or expensive grid searches. Overall, compared to a single, global neural network, the new local adaptive regression approach is shown to have comparable accuracy but 69% less training time due to the utilization of local linear regression and faster training of local neural networks.

42 ENGINEERING↗

A Bayesian approach to evaluation of soil biogeochemical models

Abstract. To make predictions about the carbon cycling consequences of rising global surface temperatures, Earth system scientists rely on mathematical soil biogeochemical models (SBMs). However, it is not clear which models have better predictive accuracy, and a rigorous quantitative approach for comparing and validating the predictions has yet to be established. In this study, we present a Bayesian approach to SBM comparison that can be incorporated into a statistical model selection framework. We compared the fits of linear and nonlinear SBMs to soil respiration data compiled in a recent meta-analysis of soil warming field experiments. Fit quality was quantified using Bayesian goodness-of-fit metrics, including the widely applicable information criterion (WAIC) and leave-one-out cross validation (LOO). We found that the linear model generally outperformed the nonlinear model at fitting the meta-analysis data set. Both WAIC and LOO computed higher overfitting risk and effective numbers of parameters for the nonlinear model compared to the linear model, conditional on the data set. Goodness of fit for both models generally improved when they were initialized with lower and more realistic steady-state soil organic carbon densities. Still, testing whether linear models offer definitively superior predictive performance over nonlinear models on a global scale will require comparisons with additional site-specific data sets of suitable size and dimensionality. Such comparisons can build upon the approach defined in this study to make more rigorous statistical determinations about model accuracy while leveraging emerging data sets, such as those from long-term ecological research experiments.

54 ENVIRONMENTAL SCIENCES↗

Bayesian evidence and predictivity of the inflationary paradigm

In this paper we consider the issue of paradigm evaluation by applying Bayes' theorem along the following nested hierarchy of progressively more complex structures: i) parameter estimation (within a model), ii) model selection and comparison (within a paradigm), iii) paradigm evaluation. In such a hierarchy the Bayesian evidence works both as the posterior's normalization at a given level and as the likelihood function at the next level up. Whilst raising no objections to the standard application of the procedure at the two lowest levels, we argue that it should receive a considerable modification when evaluating paradigms, when testability and fitting data are equally important. By considering toy models we illustrate how models and paradigms that are difficult to falsify are always favoured by the Bayes factor. We argue that the evidence for a paradigm should not only be high for a given dataset, but exceptional with respect to what it would have been, had the data been different. With this motivation we propose a measure which we term predictivity , as well as a prior to be incorporated into the Bayesian framework, penalising unpredictivity as much as not fitting data. We apply this measure to inflation seen as a whole, and to a scenario where a specific inflationary model is hypothetically deemed as the only one viable as a result of information alien to cosmology (e.g. Solar System gravity experiments, or particle physics input). We conclude that cosmic inflation is currently hard to falsify, but that this could change were external/additional information to cosmology to select one of its many models. We also compare this state of affairs to bimetric varying speed of light cosmology.

79 ASTRONOMY AND ASTROPHYSICS↗

Improvement of two-phase closure models in CTF using Bayesian inference

Under the Consortium for Advanced Simulation of Light Water Reactors (CASL) program, extensive capabilities have been developed in CTF to analyze light-water reactors (LWRs) for normal operating conditions, departure from nucleate boiling (DNB), and system transients. However, further improvements are required in the modeling and simulation of boiling water reactors (BWRs), which is a focus of the Nuclear Energy Advanced Modeling and Simulation (NEAMS) program. In this work, CTF validation results were used to optimize selected modeling coefficients by calibrating to experimental data using a Bayesian inference approach. Here, calibration studies were conducted to improve (vapor) void fraction prediction without worsening the two-phase pressure drop prediction, as well as to improve the two-phase pressure drop prediction. Calibration was performed for interfacial drag and wall shear models. Surrogates were developed to alleviate the computational expense required for sampling the parameter space using Markov chain Monte Carlo (MCMC). An assessment performed with calibrated models demonstrated an improvement of CTF in its prediction of key parameters such as void fraction and two-phase pressure drop.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Descriptor Aided Bayesian Optimization for Many-Level Qualitative Variables With Materials Design Applications

Abstract Engineering design often involves qualitative and quantitative design variables, which requires systematic methods for the exploration of these mixed-variable design spaces. Expensive simulation techniques, such as those required to evaluate optimization objectives in materials design applications, constitute the main portion of the cost of the design process and underline the need for efficient search strategies—Bayesian optimization (BO) being one of the most widely adopted. Although recent developments in mixed-variable Bayesian optimization have shown promise, the effects of dimensionality of qualitative variables have not been well studied. High-dimensional qualitative variables, i.e., with many levels, impose a large design cost as they typically require a larger dataset to quantify the effect of each level on the optimization objective. We address this challenge by leveraging domain knowledge about underlying physical descriptors, which embody the physics of the underlying physical phenomena, to infer the effect of unobserved levels that have not been sampled yet. We show that physical descriptors can be intuitively embedded into the latent variable Gaussian process approach—a mixed-variable GP modeling technique—and used to selectively explore levels of qualitative variables in the Bayesian optimization framework. This physics-informed approach is particularly useful when one or more qualitative variables are high dimensional (many-level) and the modeling dataset is small, containing observations for only a subset of levels. Through a combination of mathematical test functions and materials design applications, our method is shown to be robust to certain types of incomplete domain knowledge and significantly reduces the design cost for problems with high-dimensional qualitative variables.

Engineering↗

Uncertainty quantification of mass models using ensemble Bayesian model averaging

Developments in the description of the masses of atomic nuclei have led to various nuclear mass models that provide predictions for masses across the whole chart of nuclides. These mass models play an important role in understanding the synthesis of heavy elements in the rapid neutron capture ( r ) process. However, it is still a challenging task to estimate the size of uncertainty associated with the predictions of each mass model. In this work, a method called ensemble Bayesian model averaging (EBMA) is introduced to quantify the uncertainty of one-neutron separation energies (S 1 n ) which are directly relevant in the calculations of r -process observables. Here, this Bayesian method provides a natural way to perform model averaging, selection, and uncertainty quantification, by combining the mass models as a mixture of normal distributions whose parameters are optimized against the experimental data, employing the Markov chain Monte Carlo method using the no-u-turn sampler. The EBMA model optimized with all the experimental S 1 n from the AME2003 nuclides are shown to provide reliable uncertainty estimates when tested with the new data in the AME2020.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

eDNAjoint: An R package for interpreting paired or semi‐paired environmental DNA and traditional survey data in a Bayesian framework

Abstract Environmental DNA (eDNA) sampling is increasingly used in surveys of species distribution as a potentially sensitive and efficient monitoring method. Yet access to modelling tools designed specifically for interpreting this new data type lags behind its ubiquity. While occupancy modelling software has dominated the analytical landscape for eDNA data analysis of single species, this type of model may not always be the most appropriate. The rate of eDNA detection often corresponds to species density, rather than just occupancy, and researchers often have access to observations from non‐genetic sampling methods at the same sites. To provide users access to a modelling framework designed to maximize the use of all available data, we developed an R package, eDNAjoint . The package provides an easy‐to‐use interface for fitting a ‘joint’ model that integrates data from paired or semi‐paired eDNA and traditional surveys in a Bayesian framework. The model can be used to estimate parameters like the probability of a false positive eDNA detection and mean catch rate at a site, and the package allows access to multiple model variations and Bayesian prior customization. Additional functionality can be used for model selection, summarising posteriors and comparing the relative sensitivities of the two survey methods. We demonstrate the use of eDNAjoint by fitting a variation of the model with site‐level covariates that scale the sensitivity of eDNA sampling relative to traditional sampling. The example workflow uses binary eDNA and seine count data for the endangered tidewater goby ( Eucyclogobius newberryi ) from a study by Schmelzle and Kinziger (2016). This use case includes a prior sensitivity analysis and an evaluation of the relationship between detection rates and environmental variables. eDNAjoint has the potential to greatly increase the range of users who will be able to rigorously analyse eDNA and traditional survey data in a Bayesian framework, understand if and how eDNA can improve monitoring practices, and gain confidence in the interpretability of eDNA data.

Keller, Abigail G. [Department of Environment Scie↗

Automated integration gate selection for Gaussian mixture model pulse shape discrimination

Pulse shapes differ between neutron and gamma particles when measured with detector devices employing pulse shape discriminating (PSD) scintillators. Digitized waveforms can be used in detection systems to perform pulse shape discrimination for this application. Prior Gaussian Mixture Model (GMM) methods require access to the pulse full-waveform. Reducing the waveform to a smaller set of combined samples reduces computational cost while affecting PSD performance. In this work, we develop a method for selecting the best performing combination of integration gates, or contiguous summed segments of the digitized pulse for PSD. The method uses a discrimination score based on the GMM PSD approach. Furthermore, the final selection is performed using Bayesian Optimization. PSD detection results are compared with varying numbers of selected gates on time-of-flight (TOF) data. This method can be used to fully automate the selection of gates in an unsupervised (without ground truth labels) setting.

42 ENGINEERING↗

Hydrogeological Model Selection Among Complex Spatial Priors

Hydrogeological field studies rely often on a single conceptual representation of the subsurface. This is problematic since the impact of a poorly chosen conceptual model on predictions might be significantly larger than the one caused by parameter uncertainty. Furthermore, conceptual models often need to incorporate geological concepts and patterns in order to provide meaningful uncertainty quantification and predictions. Consequently, several geologically realistic conceptual models should ideally be considered and evaluated in terms of their relative merits. Here, we propose a full Bayesian methodology based on Markov chain Monte Carlo to enable model selection among 2-D conceptual models that are sampled using training images and concepts from multiple-point statistics. More precisely, power posteriors for the different conceptual subsurface models are sampled using sequential geostatistical resampling and Graph Cuts. To demonstrate the methodology, we compare and rank five alternative conceptual geological models that have been proposed in the literature to describe aquifer heterogeneity at the MAcroDispersion Experiment site in Mississippi, USA. We consider a small-scale tracer test for which the spatial distribution of hydraulic conductivity impacts multilevel solute concentration data observed along a 2-D transect. The thermodynamic integration and the stepping-stone sampling methods were used to compute the evidence and associated Bayes factors using the computed power posteriors. We find that both methods are compatible with multiple-point statistics-based inversions and provide a consistent ranking of the competing conceptual models considered.

58 GEOSCIENCES↗

Synthetic Streamflow Datasets Derived from DOE 9505 for Select Texas Basins

This dataset is generated using a Bayesian Hidden Markov Model trained on the DOE 9505 streamflow projection ensemble. A set of 21,000 streamflow realizations are generated for the Colorado, Sabine, and Trinity river basins in Texas. Separate models are trained either using the full 9505 ensemble or a subset based on three hyperparameters: bias correction, downscaling, and hydrological model.

drought↗

Detection and Modeling of High-Dimensional Thresholds for Fault Detection and Diagnosis

Many Fault Detection and Diagnosis (FDD) systems use discrete models for detection and reasoning. To obtain categorical values like oil pressure too high, analog sensor values need to be discretized using a suitablethreshold. Time series of analog and discrete sensor readings are processed and discretized as they come in. This task isusually performed by the wrapper code'' of the FDD system, together with signal preprocessing and filtering. In practice,selecting the right threshold is very difficult, because it heavily influences the quality of diagnosis. If a threshold causesthe alarm trigger even in nominal situations, false alarms will be the consequence. On the other hand, if threshold settingdoes not trigger in case of an off-nominal condition, important alarms might be missed, potentially causing hazardoussituations. In this paper, we will in detail describe the underlying statistical modeling techniques and algorithm as well as the Bayesian method for selecting the most likely shape and its parameters. Our approach will be illustrated by several examples from the Aerospace domain.

Computer Systems↗

TESTING NONSTANDARD COSMOLOGICAL MODELS WITH SNLS3 SUPERNOVA DATA AND OTHER COSMOLOGICAL PROBES

We investigate the implications for some nonstandard cosmological models using data from the first three years of the Supernova Legacy Survey (SNLS3), assuming a spatially flat universe. A comparison between the constraints from the SNLS3 and those from other SN Ia samples, such as the ESSENCE, Union2, SDSS-II, and Constitution samples, is given and the effects of different light-curve fitters are considered. We find that analyzing SNe Ia with SALT2 or SALT or SiFTO can give consistent results and the tensions between different data sets and different light-curve fitters are obvious for fewer-free-parameters models. At the same time, we also study the constraints from SNLS3 along with data from the cosmic microwave background and the baryonic acoustic oscillations (CMB/BAO), and the latest Hubble parameter versus redshift (H(z)). Using model selection criteria such as {chi}{sup 2}/dof, goodness of fit, Akaike information criterion, and Bayesian information criterion, we find that, among all the cosmological models considered here ({Lambda}CDM, constant w, varying w, Dvali-Gabadadze-Porrati (DGP), modified polytropic Cardassian, and the generalized Chaplygin gas), the flat DGP is favored by SNLS3 alone. However, when additional CMB/BAO or H(z) constraints are included, this is no longer the case, and the flat {Lambda}CDM becomes preferred.

79 ASTRONOMY AND ASTROPHYSICS↗

Object-Oriented Bayesian Networks (OOBN) for Aviation Accident Modeling and Technology Portfolio Impact Assessment

The concern for reducing aviation safety risk is rising as the National Airspace System in the United States transforms to the Next Generation Air Transportation System (NextGen). The NASA Aviation Safety Program is committed to developing an effective aviation safety technology portfolio to meet the challenges of this transformation and to mitigate relevant safety risks. The paper focuses on the reasoning of selecting Object-Oriented Bayesian Networks (OOBN) as the technique and commercial software for the accident modeling and portfolio assessment. To illustrate the benefits of OOBN in a large and complex aviation accident model, the in-flight Loss-of-Control Accident Framework (LOCAF) constructed as an influence diagram is presented. An OOBN approach not only simplifies construction and maintenance of complex causal networks for the modelers, but also offers a well-organized hierarchical network that is easier for decision makers to exploit the model examining the effectiveness of risk mitigation strategies through technology insertions.

Shih, Ann T.↗

The SRG/eROSITA All-Sky Survey: Dark Energy Survey year 3 weak gravitational lensing by eRASS1 selected galaxy clusters

Context. Number counts of galaxy clusters across redshift are a powerful cosmological probe if a precise and accurate reconstruction of the underlying mass distribution is performed – a challenge called mass calibration. With the advent of wide and deep photometric surveys, weak gravitational lensing (WL) by clusters has become the method of choice for this measurement. Aims. We measured and validated the WL signature in the shape of galaxies observed in the first three years of the Dark Energy Survey (DES Y3) caused by galaxy clusters and groups selected in the first all-sky survey performed by SRG (Spectrum Roentgen Gamma)/eROSITA (eRASS1). These data were then used to determine the scaling between the X-ray photon count rate of the clusters and their halo mass and redshift. Methods. We empirically determined the degree of cluster member contamination in our background source sample. The individual cluster shear profiles were then analyzed with a Bayesian population model that self-consistently accounts for the lens sample selection and contamination and includes marginalization over a host of instrumental and astrophysical systematics. To quantify the accuracy of the mass extraction of that model, we performed mass measurements on mock cluster catalogs with realistic synthetic shear profiles. This allowed us to establish that hydrodynamical modeling uncertainties at low lens redshifts (z < 0.6) are the dominant systematic limitation. At high lens redshift, the uncertainties of the sources’ photometric redshift calibration dominate. Results. With regard to the X-ray count rate to halo mass relation, we determined its amplitude, its mass trend, the redshift evolution of the mass trend, the deviation from self-similar redshift evolution, and the intrinsic scatter around this relation. Conclusions. The mass calibration analysis performed here sets the stage for a joint analysis with the number counts of eRASS1 clusters to constrain a host of cosmological parameters. We demonstrate that WL mass calibration of galaxy clusters can be performed successfully with source galaxies whose calibration was performed primarily for cosmic shear experiments, opening the way for the cluster cosmological exploitation of future optical and NIR surveys like Euclid and LSST.

79 ASTRONOMY AND ASTROPHYSICS↗

Developing and applying quantifiable metrics for diagnostic and experiment design on Z

This project applies methods in Bayesian inference and modern statistical methods to quantify the value of new experimental data, in the form of new or modified diagnostic configurations and/or experiment designs. We demonstrate experiment design methods that can be used to identify the highest priority diagnostic improvements or experimental data to obtain in order to reduce uncertainties on critical inferred experimental quantities and select the best course of action to distinguish between competing physical models. Bayesian statistics and information theory provide the foundation for developing the necessary metrics, using two high impact experimental platforms on Z as exemplars to develop and illustrate the technique. We emphasize that the general methodology is extensible to new diagnostics (provided synthetic models are available), as well as additional platforms. We also discuss initial scoping of additional applications that began development in the last year of this LDRD.

97 MATHEMATICS AND COMPUTING↗