Search NASA⌕ Search

SEARCH · Search NASA

Results for “model selection”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2025

LASSO for CALPHAD Model Selection Enables Data-Efficient Thermodynamic Modeling: An Application in Thermochemical Hydrogen Production Materials

Phenomenological CALPHAD (CALculation of PHAse Diagrams) models, widely used for multicomponent materials, often contain a considerable number of parameters and require fitting using data from a relatively small number of experimental measurements or theoretical calculations. Sometimes these parameters are introduced for the purpose of improving model fits but without clear physical justification, which leads to overparametrized models with poor generalization performance. Automated approaches for optimal model selection based on the available data therefore become critical. Here, in this work, a least absolute shrinkage and selection operator (LASSO)-based approach is developed for model selection by leveraging the linearity of the CALPHAD model with respect to its parameters to convert the model selection and fitting to a LASSO minimization problem. We demonstrate its utility for thermodynamic modeling of thermochemical hydrogen (TCH) production materials using lanthanum strontium manganite (LSM) as an example. Various TCH-relevant properties, including oxygen stoichiometry as a function of oxygen partial pressure, enthalpy of reduction, and entropy of reduction, are successfully predicted with reasonable accuracy using a minimal set of model parameters. Importantly, the model selection and fitting involve minimal human decision; it can therefore be applied to high-throughput DFT defect calculations and yield efficient workflows for TCH material modeling and optimization.

CALPHAD↗
DOE OSTI2020

CMIP5 model selection for ISMIP6 ice sheet model forcing: Greenland and Antarctica

Abstract. The ice sheet model intercomparison project for CMIP6 (ISMIP6) effort brings together the ice sheet and climate modeling communities to gain understanding of the ice sheet contribution to sea level rise. ISMIP6 conducts stand-alone ice sheet experiments that use space- and time-varying forcing derived from atmosphere–ocean coupled global climate models (AOGCMs) to reflect plausible trajectories for climate projections. The goal of this study is to recommend a subset of CMIP5 AOGCMs (three core and three targeted) to produce forcing for ISMIP6 stand-alone ice sheet simulations, based on (i) their representation of current climate near Antarctica and Greenland relative to observations and (ii) their ability to sample a diversity of projected atmosphere and ocean changes over the 21st century. The selection is performed separately for Greenland and Antarctica. Model evaluation over the historical period focuses on variables used to generate ice sheet forcing. For stage (i), we combine metrics of atmosphere and surface ocean state (annual- and seasonal-mean variables over large spatial domains) with metrics of time-mean subsurface ocean temperature biases averaged over sectors of the continental shelf. For stage (ii), we maximize the diversity of climate projections among the best-performing models. Model selection is also constrained by technical limitations, such as availability of required data from RCP2.6 and RCP8.5 projections. The selected top three CMIP5 climate models are CCSM4, MIROC-ESM-CHEM, and NorESM1-M for Antarctica and HadGEM2-ES, MIROC5, and NorESM1-M for Greenland. This model selection was designed specifically for ISMIP6 but can be adapted for other applications.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

Bayesian Model Selection as a Continuous-Variable Helmholtz Machine

We show that Bayesian model selection is equivalent to optimization of a suitably defined free energy. The entropy term in the free energy is defined over the space of possible model selections and model parameters for a given model. Bayesian model selection thus follows a minimum description length principle.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2022

Contextual Active Online Model Selection with Expert Advice

How can we collect the most useful labels to learn a model selection policy, when presented with arbitrary heterogeneous data streams? In this paper, we formulate this task as a contextual active model selection problem, where at each round the learner receives an unlabeled data point along with a context. The goal is to output the best model for any given context without obtaining an excessive amount of labels. In particular, we focus on the task of selecting pre-trained classifiers, and propose a contextual active model selection algorithm (CAMS), which relies on a novel uncertainty sampling query criterion defined on a given policy class for adaptive model selection. In comparison to prior art, our algorithm does not assume a globally optimal model. We provide rigorous theoretical analysis for the regret and query complexity under both adversarial and stochastic settings. Our experiments on several benchmark classification datasets demonstrate the algorithm’s effectiveness in terms of both regret and query complexity. Notably, to achieve the same accuracy, CAMS incurs less than 10% of the label cost when compared to the best online model selection baselines on CIFAR10.

Liu, Xuefeng↗
DOE OSTI2023

An empirical approach to model selection: weak lensing and intrinsic alignments

ABSTRACT In cosmology, we routinely choose between models to describe our data, and can incur biases due to insufficient models or lose constraining power with overly complex models. In this paper, we propose an empirical approach to model selection that explicitly balances parameter bias against model complexity. Our method uses synthetic data to calibrate the relation between bias and the χ2 difference between models. This allows us to interpret χ2 values obtained from real data (even if catalogues are blinded) and choose a model accordingly. We apply our method to the problem of intrinsic alignments – one of the most significant weak lensing systematics, and a major contributor to the error budget in modern lensing surveys. Specifically, we consider the example of the Dark Energy Survey Year 3 (DES Y3), and compare the commonly used non-linear alignment (NLA) and tidal alignment and tidal torque (TATT) models. The models are calibrated against bias in the Ωm–S8 plane. Once noise is accounted for, we find that it is possible to set a threshold Δχ2 that guarantees an analysis using NLA is unbiased at some specified level Nσ and confidence level. By contrast, we find that theoretically defined thresholds (based on, e.g. p-values for χ2) tend to be overly optimistic, and do not reliably rule out cosmological biases up to ∼1–2σ. Considering the real DES Y3 cosmic shear results, based on the reported difference in χ2 from NLA and TATT analyses, we find a roughly $30{{\ \rm per\ cent}}$ chance that were NLA to be the fiducial model, the results would be biased (in the Ωm–S8 plane) by more than 0.3σ. More broadly, the method we propose here is simple and general, and requires a relatively low level of resources. We foresee applications to future analyses as a model selection tool in many contexts.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
NASA NTRS1977

'Chain pooling' model selection as developed for the statistical analysis of a rotor burst protection experiment

A statistical decision procedure called chain pooling had been developed for model selection in fitting the results of a two-level fixed-effects full or fractional factorial experiment not having replication. The basic strategy included the use of one nominal level of significance for a preliminary test and a second nominal level of significance for the final test. The subject has been reexamined from the point of view of using as many as three successive statistical model deletion procedures in fitting the results of a single experiment. The investigation consisted of random number studies intended to simulate the results of a proposed aircraft turbine-engine rotor-burst-protection experiment. As a conservative approach, population model coefficients were chosen to represent a saturated 2 to the 4th power experiment with a distribution of parameter values unfavorable to the decision procedures. Three model selection strategies were developed.

Holms, A. G.↗
DOE OSTI2021

A meta-learning based distribution system load forecasting model selection framework

This paper presents a meta-learning based, automatic distribution system load forecasting model selection framework. Furthermore, the framework includes the following processes: feature extraction, candidate model preparation and labeling, offline training, and online model recommendation. Using load forecasting needs and data characteristics as input features, multiple metalearners are used to rank the candidate load forecast models based on their forecasting accuracy. Then, a scoring-voting mechanism is proposed to weights recommendations from each meta-leaner and make the final recommendations. Heterogeneous load forecasting tasks with different temporal and technical requirements at different load aggregation levels are set up to train, validate, and test the performance of the proposed framework. Simulation results demonstrate that the performance of the meta-learning based approach is satisfactory in both seen and unseen forecasting tasks.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2023

Multiverse: Bayesian model selection for neural networks

Multiverse is a code repository for a set of tools for Bayesian model selection for neural networks. The goal of the tools is to provide capabilities for selecting among prior and model specifications for Bayesian neural networks. Multiverse will include tools for creating Bayesian neural networks, evaluating the Bayesian model evidence, and performing inference in Bayesian neural networks. These components are written in Python, a high-level programming language that takes advantage of the Python ecosystem of high-quality open-source packages for machine learning and signal processing.

Klein, Natalie↗
NASA NTRS1980

'Chain pooling' model selection for two-level fixed effects factorial experiments

As many as three iterated statistical model deletion procedures are considered for an experiment. Population model coefficients were chosen to simulate a saturated factorial experiment having an unfavorable distribution of parameter values. Using random number studies, three model selection strategies were developed, namely, (1) a strategy to be used in anticipation of large coefficients of variation (neighborhood of 65 percent), (2) strategy to be used in anticipation of small coefficients of variation (4 percent or less), and (3) a security regret strategy to be used in the absence of such prior knowledge.

Holms, A. G.↗
NASA NTRSDate not supplied

Improving Multi-Model Trajectory Simulation Estimators using Model Selection and Tuning

Multi-model Monte Carlo methods have been demonstrated to be an efficient and accurate alternative to standard Monte Carlo (MC) in the model-based propagation of uncertainty in entry, descent, and landing (EDL) applications. These multi-model MC methods fuse predictions from low-fidelity models with the high-fidelity EDL model of interest to produce unbiased statistics with a fraction of the computational cost. The accuracy and efficiency of the multi-model MC methods are dependent upon the magnitude of correlations of the low-fidelity models with the high-fidelity model, but also upon the correlation amongst the low-fidelity models, and their relative computational cost. Because of this layer of complexity, the question of how to optimally select the set of low-fidelity models has remained open. In this work, methods for optimal model construction and tuning are investigated as a means to increase the speed and precision of trajectory simulation for EDL. Specifically, the focus is on the inclusion of low-fidelity model tuning within the sample allocation optimization that accompanies multi-model MC methods. Preliminary results indicate that low-fidelity model tuning can significantly improve efficiency and precision of trajectory simulations and provide an increased edge to multi-model MC methods when compared to standard MC. The challenges and potential benefits to exploring a fully iterative and comprehensive optimization strategy in future work are highlighted.

uncertainty quantification↗
NASA NTRSDate not supplied

Improving Multi-Model Trajectory Simulation Estimators using Model Selection and Tuning

Multi-model Monte Carlo methods have been demonstrated to be an efficient and accurate alternative to standard Monte Carlo (MC) in the model-based propagation of uncertainty in entry, descent, and landing (EDL) applications. These multi-model MC methods fuse predictions from low-fidelity models with the high-fidelity EDL model of interest to produce unbiased statistics with a fraction of the computational cost. The accuracy and efficiency of the multi-model MC methods are dependent upon the magnitude of correlations of the low-fidelity models with the high-fidelity model, but also upon the correlation among the low-fidelity models, and their relative computational cost. Because of this layer of complexity, the question of how to optimally select the set of low-fidelity models has remained open. In this work, methods for optimal model construction and tuning are investigated as a means to increase the speed and precision of trajectory simulation for EDL. Specifically, the focus is on the inclusion of low-fidelity model tuning within the sample allocation optimization that accompanies multi-model MC methods. Preliminary results indicate that low-fidelity model tuning can significantly improve efficiency and precision of trajectory simulations and provide an increased edge to multi-model MC methods when compared to standard MC. The challenges and potential benefits to exploring a fully iterative and comprehensive optimization strategy in future work are highlighted.

uncertainty quantification↗
NASA NTRS1977

Chain Pooling modeling selection as developed for the statistical analysis of a rotor burst protection experiment

As many as three iterated statistical model deletion procedures were considered for an experiment. Population model coefficients were chosen to simulate a saturated 2 to the 4th power experiment having an unfavorable distribution of parameter values. Using random number studies, three model selection strategies were developed, namely, (1) a strategy to be used in anticipation of large coefficients of variation, approximately 65 percent, (2) a strategy to be sued in anticipation of small coefficients of variation, 4 percent or less, and (3) a security regret strategy to be used in the absence of such prior knowledge.

Holms, A. G.↗