Search NASA⌕ Search

SEARCH · Search NASA

Results for “input uncertainty”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 451 records · Page 25
All sourcesNASA NTRSDOE OSTI
DOE OSTI2020

State-of-the-Art Reactor Consequence Analysis Project: Sensitivity Analysis Using Dakota

The State-of-the-Art Reactor Consequence Analyses (SOARCA) project has focused on best estimate analyses and uncertainty analysis for postulated accidents at specific nuclear power plants. The consequences of these accidents are estimated using the simulation tools MELCOR and MACCS. To understand which uncertain input variables are important to determining these consequences, analysts have performed sensitivity analyses. The tool used to perform these sensitivity analyses in previous SOARCA work, CompModSA, is no longer supported. Therefore, the current work focuses on migrating these analyses to another tool and evaluating its performance. Dakota, which is a tool developed at Sandia National Laboratories, is used in this work. Sensitivity results are created for three analyses from the SOARCA Surry UA. Though CompModSA and Dakota vary slightly in their algorithms and implementation, their sensitivity results generally agree, which gives confidence in the Dakota approach and increases confidence in the original analyses. It is likely that this methodology is extendable to the rest of SOARCA analyses.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2020

Probabilistic Groundwater Modeling of the RDX Plume at Los Alamos National Laboratory to Support Risk Assessment - 20359

A plume of the contaminant hexahydro-1,3,5-trinitro-1,3,5-triazine (RDX) with concentrations greater than the New Mexico tap water drinking standard (9.66 ppb) is present in the regional aquifer near the southwestern boundary of Los Alamos National Laboratory (LANL). A risk assessment is performed for exposure to regional aquifer groundwater, with long-term predictions of RDX concentrations provided by a calibrated, probabilistic, numerical fate and transport model. The structure of the model is hierarchical, with the RDX Regional Aquifer (RA) groundwater model acting as the primary tool for analysis of downgradient RDX concentrations. The RA model is deeply informed by the conceptual site model (CSM) and is calibrated using site RDX concentration data and hydraulic head measurements, along with other analyses. The model is calibrated using data through December 2019. Where data are scarce other lines of evidence are used to inform inputs, including the multiphase RDX Vadose Zone (VZ) model and Pipe and Disk analytical screening tool. Model inputs are described with informative prior distributions using a robust approach to development that incorporates all available lines of evidence for every parameter used as an input to the RA model. Calibration is performed using a classical nonlinear optimization routine, which is then used to initialize a Bayesian calibration. The Bayesian calibration constrains the uncertainty in the classical calibration, ultimately providing posterior distributions for all model parameters. The challenges of the calibration include high-dimensional parameter space, including spatially heterogeneous hydraulic conductivities, comparatively sparse data, and low RDX concentrations. Posterior parameter distributions developed in the calibration process are then used for stochastic predictive model runs into the future. The result of the forward runs is spatially and temporally explicit estimates of head and concentration with uncertainty at all points in space and time. The probabilistic modeling approach presented here includes innovative computational and statistical methods that leverage high-performance computing (HPC) resources. It makes use of prior modeling work performed at the chromium plume site in the central LANL area, with extensive updates. The risk assessment will ultimately be used to support decision-making at the site, using multiple appropriately weighted sources of information, as well as uncertainty, sensitivity, and value-of-information analyses. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗
DOE OSTI2021

Exploring Sensitivity of ICF Outputs to Design Parameters in Experiments Using Machine Learning

We report building a sustainable burn platform in inertial confinement fusion (ICF) requires an understanding of the complex coupling of physical processes and the effects that key experimental design changes have on implosion performance. While simulation codes are used to model ICF implosions, incomplete physics and the need for approximations deteriorate their predictive capability. Identification of relationships between controllable design inputs and measurable outcomes can help guide the future design of experiments and development of simulation codes, which can potentially improve the accuracy of the computational models used to simulate ICF implosions. In this article, we leverage developments in machine learning (ML) and methods for ML feature importance/sensitivity analysis to identify complex relationships in ways that are difficult to process using expert judgment alone. We present work using random forest (RF) regression for prediction of yield, velocity, and other experimental outcomes given a suite of design parameters, along with an assessment of important relationships and uncertainties in the prediction model. We show that RF models are capable of learning and predicting on ICF experimental data with high accuracy, and we extract feature importance metrics that provide insight into the physical significance of different controllable design inputs for various ICF design configurations. These results can be used to augment expert intuition and simulation results for optimal design of future ICF experiments.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗
DOE OSTI2020

Deep Direct-Use Geothermal Feasibility Study for Hawthorne NV

The Hawthorne Nevada, deep direct-use geothermal study is a two-year effort funded by the U.S. Department of Energy to determine the techno-economic feasibility of implementing a large-scale, direct-use facility for the Hawthorne Army Depot (HAD) and the public facilities of the city of Hawthorne and Mineral County. The approach links a production side analysis (PSA) and a demand side analysis (DSA) into a whole-system analysis (WSA) to provide an integrated assessment of the resource and the probability of delivering economically viable direct-use energy to Hawthorne. Hawthorne, Nevada is in the western part of the Basin and Range province and has been the focus of geothermal investigations for over 40 years. Over the last 15 years, several studies completed by the U.S. Navy Geothermal Program Office (GPO) in conjunction with industry professionals quantified the existence of several low temperature geothermal prospects, the most promising of which is called Prospect A. The promise of Prospect A is based on drilling and flow testing that produced ~100 °C water at flow rates of up to 31 l/s (500 gallons per minute). Measured productivity indexes range from 40-85 l/s/MPa, suggesting a warm and productive heat source. Despite the promise of the resource, uncertainties in its spatial extent and long-term sustainability mean that techno-economic analyses must include probabilities of the sustainability of the resource under different operating scenarios. Here, the PSA is conducted by integrating a wide range of disparate data to estimate lognormal P90, P50, and P10 resource capacities. These capacities are used as input to a thermal-hydrologic (T-H) model to estimate thermal drawdown for each capacity estimate for several different DSA scenarios. Using a systems-based approach, the WSA links the dynamic T-H simulations of the PSA/DSA combinations with the techno-economic model GEOPHIRES to account for both the temporal dynamics and uncertainties in the system to produce probabilistic distributions of several performance metrics including the levelized cost of heat (LCOH) and the return on investment (ROI). This report is the final delivery for the project and documents the study's activities and results.

15 GEOTHERMAL ENERGY↗
DOE OSTI2023

Probabilistic-learning-based stochastic surrogate model from small incomplete datasets for nonlinear dynamical systems

We consider a high-dimensional nonlinear computational model of a dynamical system, parameterized by a vector-valued control parameter, in the presence of uncertainties represented by an uncontrolled parameter modeled by a vector-valued random variable, and possibly with stochastic excitation. The objective is to construct a statistical surrogate model where the input is any deterministic value of the control parameter, and the output is a vector-valued observation of the computational model, which is a random vector whose probability measure is updated using a target dataset. To construct this statistical surrogate model, the stochastic response of the computational model must be built, which is a vector-valued time-discretized stochastic process in high dimension, depending on the control parameter. It is assumed that the computational cost of a single evaluation of the deterministic model is high. For the probabilistic updating, we consider a subset of the components of the observation of the computational model, defined as the “identification observation” of the computational model, for which a small target dataset is available. Therefore, the target dataset is associated with partial observability, corresponding to an incomplete data case. Given a prior probability model of the random control and uncontrolled parameters, a training dataset is constructed, consisting of realizations of the random triplet composed of the stochastic response, the random identification observation, and the random control parameter. Since the computational cost of a single evaluation of the deterministic model is assumed to be large, the training dataset is also of small size. The main challenges in this problem are the high dimensionality, partial observability leading to incomplete data in the target dataset for the identification observation of the computational model (which is not sufficient to identify the computational stochastic responses), and the availability of a small training dataset. To address these challenges, we propose a methodology based on statistical methods for constructing necessary reduced representations, direct probabilistic learning under constraints using probabilistic learning on manifolds (PLoM) constrained by the target dataset, and the use of a weak formulation of the Fourier transform of probability measures. Statistical conditioning is also employed to explore the learned dataset. The constructed predictive statistical surrogate model can be implemented in the context of online computation. Here, we apply this approach to a problem of nonlinear stochastic dynamics in high dimensions within the framework of deformable solids mechanics.

Engineering↗
DOE OSTI2022

Translator Plan: A Coordinated Vision for Fiscal Years 2023-2025

Translators serve a unique role in the U.S. Department of Energy (DOE)’s Atmospheric Radiation Measurement (ARM) user facility, offering scientific input through various leadership and service roles. The Translators direct the creation of value-added products (VAPs) and analysis tools that make ARM measurements more accessible to the scientific community. Translators also serve as liaisons between users and the ARM infrastructure, collecting information about priorities and communicating ARM data and services. A key group focus is supporting the DOE Atmospheric System Research (ASR) program scientists and ASR’s efforts towards a process-level understanding of cloud-aerosol interactions, and in reducing uncertainty in global climate model projections. The ARM Translator Group (Table 1) consists of the five Translators, a representative of software development, and one from the Data Quality Office (DQO). Additionally, the ARM Engineering and Process Manager participates in this group and provides input and direction from ARM and its programmatic priorities.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Automated theoretical chemical kinetics: Predicting the kinetics for the initial stages of pyrolysis

Large scale implementation of high level computational theoretical chemical kinetics offers the prospect for dramatically improving the fidelity of combustion chemical modeling. To facilitate such efforts, we have developed a suite of codes, collectively referred to as AutoMech, that allow for the automatic prediction of the kinetics for large sets of reactions via ab initio transition-state-theory based master-equation calculations. The primary input is simply the mechanism, a dictionary relating chemically identifiable species descriptors (e.g., SMILES or InChIs) to species labels in the mechanism, and a specification of the electronic structure and transition state theory models to be implemented. Here we illustrate the current utility of AutoMech through a study of the initial stages of pyrolysis for 3 sets of fuels: sequences of alkanes, alcohols, and aldehydes. For simplicity, the analysis focuses on abstractions from the fuel by H, CH 3 , and OH, and the decomposition of the resulting radicals. Altogether, there are a total of 166 input channels in these sets (more than 363 forward reactions when expanded to the full set of elementary reactions). The code successfully produces high quality rate estimates (with apparent uncertainties less than a factor of two in limited comparisons with experiment) for > 95% of these. For the radical decomposition reactions, the analysis includes predictions for the pressure dependence of the kinetics. This wide-ranging exploration illustrates (i) the effect of different levels of prediction on the expected accuracy, (ii) the branching between abstractions at different sites for different abstractors, (iii) the dependence of the rates on the chemical structure, and (iv) the variation in radical stabilities across chemical families. These results, as well as the demonstrated feasibility of the methodology, should find further utility in the development of accurate rate expressions for arbitrary fuels. (c) 2020 The Combustion Institute. Published by Elsevier Inc. All rights reserved.

Elliott, Sarah N.↗
DOE OSTI2023

Experimental spectrum averaged cross sections (SACS) in 252 Cf(sf) neutron field and its impact on the evaluation of neutron standards

A new evaluation of spectrum averaged cross sections (SACS) of 235 U, 238 U, and 239 Pu measured in the 252 Cf(sf) reference neutron field is presented and found to be consistent with the original Mannhart SACS evaluation in IRDF-2002. The comprehensive vetted experimental database that includes SACS ratio and absolute SACS measurements of major actinides is being used to update the SACS database employed as input of the new GMApy code to derive the Neutron Standards. An update of the current neutron standards based on Time Projection Chamber (TPC) shape data, a new comprehensive uncertainty quantification, and revised SACS experimental database is proposed which result in a 0.7% increase of the evaluated 239 Pu(n,f)/ 235 U(n,f) crosssection ratio in the 1–5 MeV energy region. The increase is due to a 0.3% reduction of the standard 235 U(n,f) cross section and a 0.4% increase of the 239 Pu(n,f) reference cross section in the 1–5 MeV energy region. Those changes are well within estimated USU fission cross-section uncertainties of 1.2%, but are relevant for the evaluated mean values.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2022

Combined Theoretical Analysis of the Parity-Violating Asymmetry for 48 Ca and 208 Pb

The recent experimental determination of the parity violating asymmetry A PV in 48 Ca and 208 Pb at Jefferson Lab is important for our understanding on how neutrons and protons arrange themselves inside the atomic nucleus. To better understand the impact of these measurements, we present a rigorous theoretical investigation of A PV in 48 Ca and 208 Pb and assess the associated uncertainties. Here, we complement our study by inspecting the static electric dipole polarizability in these nuclei. The analysis is carried out within nuclear energy density functional theory with quantified input. We conclude that the simultaneous accurate description of A PV in 48 Ca and 208 Pb cannot be achieved by our models that accommodate a pool of global nuclear properties, such as masses and charge radii, throughout the nuclear chart, and describe—within one standard deviation—the experimental dipole polarizabilities in these nuclei.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2020

Description and Use of SCALE Sampler Parametric Capability for Engineering Analysis and Optimization

The Sampler sequence was introduced into the SCALE nuclear modeling and simulation suite in SCALE 6.2 to perform uncertainty quantification via random sampling of nuclear data, material number densities, and dimensions. Sampler was expanded with the introduction of a parametric capability in SCALE 6.2.2. This paper discusses input for the Sampler parametric sequence and presents two case studies of analyses performed using the sequence. These case studies include preconceptual design of a package for transporting high assay low-enriched uranium (HALEU) oxide and scoping calculations to support subcritical limit development for a future update of the ANSI/ANS-8.1 (ANS-8.1) standard. The parametric capability within Sampler provides many benefits to analysts. For instance, parametric sweeps are frequently used to identify optimum parameter values as part of safety analysis or system design, but such sweeps can require substantial engineering time or may rely on custom-written scripts or scripts such as Write One, Run Many (or WORM) developed outside of any software quality assurance program. With the parametric capabilities in Sampler, however, a large number of inputs can be generated automatically without recourse to scripting by individual analysts. The parametric capability can also be used in lieu of the CSAS5S search sequence to identify optimum parameters more simply with straightforward inputs and outputs. Sampler can also be used to calculate input parameters from engineering specifications. For example, diameters can be converted to radii, or masses can be used to calculate number densities. Overall, the Sampler parametric capability provides a robust feature within SCALE, eliminating the need for user-developed scripting.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2023

Considering the Variability of Soiling in Long-Term PV Performance Forecasting: Preprint

This study presents the development of a methodology for evaluating the variability associated with soiling on long-term PV forecasting. Independent engineering firms typically build P50 forecasts for large PV plants through the use of the PVsyst software, where monthly soiling losses are one of many inputs to the P50 model. Subsequently, long-term performance distributions, or Pvalues, are constructed through a Monte Carlo analysis that includes various factors such as: satellite irradiance modeling uncertainty, uncertainty in the PVsyst model, and long-term irradiance variability. Often the PVsyst model uncertainty is increased to account for sites with significant soiling concerns but no systematic method has been presented in the literature to specifically include soiling variability within Pvalues. In this work soiling information from 16 sites in the U.S. Southwest are combined with 20 years of rainfall data to generate 20 years of energy production with soiling losses and then subsequently generate Pvalues. The results show that the spread of Pvalues (P1-P99) can increase from 0-13% when interannual soiling variability is included.

interannual variability↗
DOE OSTI2025

Future Wind Energy Resources and Cost Uncertainties Across the United States

This dataset contains results estimating projections of change of annual capacity factors and levelized cost of energy for several turbine technologies in the 2024 Annual Technology Baseline (ATB). Projections of change are based on downscaled earth system model (ESM) data from Sup3rCC. There has been evidence of reductions in average wind speeds over land in North America since the 1980s, and several models project that average wind speeds will continue to decrease. Concurrently, the cost of wind energy systems in the United States has been decreasing since around 2010, a trend also projected to continue. There is considerable uncertainty in these future projections, with quantitative estimates of future wind resource and system costs varying widely. To study this, we run land-based wind energy models with a range of possible future system costs, turbine designs, and meteorological inputs from multiple downscaled earth system models over the contiguous United States to estimate critical system performance metrics such as annual energy production (AEP) and levelized cost of energy. Where multiple earth system models agree, changes in mean AEP from the time period 2000-2019 to 2040-2059 can be as high as +10% in South Texas or as low as -20% in Iowa. Several additional states in the Midwest that currently have considerable wind generation capacity show the possibility of substantial decreases in AEP by mid-century. Larger turbines and moderate reductions in system costs can offset even the largest projected decreases in wind resource, but much uncertainty remains in the extent to which wind resources will actually change into the future and to what extent wind energy systems can drive down future costs. An analysis of variance shows, in several states in the Midwest, the uncertainty in future wind resource can be almost as important for future changes in the cost of wind energy as the uncertainty in future system costs.

17 WIND ENERGY↗
DOE OSTI2024

Improved multifidelity Monte Carlo estimators based on normalizing flows and dimensionality reduction techniques

Here, we study the problem of multifidelity uncertainty propagation for computationally expensive models. In particular, we consider the general setting where the high-fidelity and low-fidelity models have a dissimilar parameterization both in terms of number of random inputs and their probability distributions, which can be either known in closed form or provided through samples. We derive novel multifidelity Monte Carlo estimators which rely on a shared subspace between the high-fidelity and low-fidelity models where the parameters follow the same probability distribution, i.e., a standard Gaussian. We build the shared space employing normalizing flows to map different probability distributions into a common one, together with linear and nonlinear dimensionality reduction techniques, active subspaces and autoencoders, respectively, which capture the subspaces where the models vary the most. We then compose the existing low-fidelity model with these transformations and construct modified models with an increased correlation with the high-fidelity model, which therefore yield multifidelity estimators with reduced variance. A series of numerical experiments illustrate the properties and advantages of our approaches.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2024

Actinide signatures in low electron fraction kilonova ejecta

ABSTRACT Neutron star (NS) mergers are known to produce heavy elements through rapid neutron capture (r-process) nucleosynthesis. Actinides are expected to be created solely by the r-process in the most neutron-rich environments. Confirming if NS mergers provide the requisite conditions for actinide creation is therefore central to determining their origin in the Universe. Actinide signatures in kilonova (KN) spectra may yield an answer, provided adequate models are available in order to interpret observational data. In this study, we investigate actinide signatures in neutron-rich merger ejecta. We use three ejecta models with different compositions and radioactive power, generated by nucleosynthesis calculations using the same initial electron fraction ($Y_e = 0.15$) but with different nuclear physics inputs and thermodynamic expansion history. These are evolved from 10 to 100 d after merger using the sumo non-local thermodynamic equilibrium (NLTE) radiative transfer code. We highlight how uncertainties in nuclear properties, as well as choices in thermodynamic trajectory, may yield entirely different outputs for equal values of $Y_e$. We consider an actinide-free model and two actinide-rich models, and find that the emergent spectra and light-curve evolution are significantly different depending on the amount of actinides present, and the overall decay properties of the models. We also present potential key actinide spectral signatures, of which doubly ionized $_{89}$Ac and $_{90}$Th may be particularly interesting as spectral indicators of actinide presence in KN ejecta.

Pognan, Quentin (ORCID:0000000152550782)↗
DOE OSTI2021

New physics in rare B decays after Moriond 2021

Abstract The anomalies in rare B decays endure. We present results of an updated global analysis that takes into account the latest experimental input – in particular the recent results on $$R_K$$ R K and BR $$(B_s \rightarrow \mu ^+\mu ^-)$$ ( B s → μ + μ - ) – and that qualitatively improves the treatment of theory uncertainties. Fit results are presented for the Wilson coefficients of four-fermion contact interactions. We find that muon specific Wilson coefficients $$C_9 \simeq -0.73$$ C 9 ≃ - 0.73 or $$C_9 = -C_{10} \simeq -0.39$$ C 9 = - C 10 ≃ - 0.39 continue to give an excellent description of the data. If only theoretically clean observables are considered, muon specific $$C_{10} \simeq 0.60$$ C 10 ≃ 0.60 or $$C_9=-C_{10} \simeq -0.35$$ C 9 = - C 10 ≃ - 0.35 improve over the Standard Model by $$\sqrt{\Delta \chi ^2} \simeq 4.7\sigma $$ Δ χ 2 ≃ 4.7 σ and $$\sqrt{\Delta \chi ^2} \simeq 4.6\sigma $$ Δ χ 2 ≃ 4.6 σ , respectively. In various new physics scenarios we provide predictions for lepton flavor universality observables and CP asymmetries that can be tested with more data. We update our previous combination of ATLAS, CMS, and LHCb data on BR $$(B_s \rightarrow \mu ^+\mu ^-)$$ ( B s → μ + μ - ) and BR $$(B^0\rightarrow \mu ^+\mu ^-)$$ ( B 0 → μ + μ - ) taking into account the full two-dimensional non-Gaussian experimental likelihoods.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗
DOE OSTI2023

Legacy Well Leakage Risk Analysis at the Farnsworth Unit Site

This paper summarizes the results of the risk analysis and characterization of the CO 2 and brine leakage potential of Farnsworth Unit (FWU) site wells. The study is part of the U.S. DOE’s National Risk Assessment Partnership (NRAP) program, which aims to quantitatively evaluate long-term environmental risks under conditions of significant geologic uncertainty and variability. To achieve this, NRAP utilizes risk assessment and computational tools specifically designed to quantify uncertainties and calculate the risk associated with geologic carbon dioxide (CO 2 ) sequestration. For this study, we have developed a workflow that utilizes physics-based reservoir simulation results as input to perform leakage calculations using NRAP Tools, specifically NRAP-IAM-CS and RROM-Gen. These tools enable us to conduct leakage risk analysis based on ECLIPSE reservoir simulation results and to characterize wellbore leakage at the Farnsworth Unit Site. We analyze the risk of leakage from both individual wells and the entire field under various wellbore integrity distribution scenarios. The results of the risk analysis for the leakage potential of FWU wells indicate that, when compared to the total amount of CO 2 injected, the highest cemented well integrity distribution scenario (FutureGen high flow rate) exhibits approximately 0.01% cumulative CO 2 leakage for a 25-year CO 2 injection duration at the end of a 50-year post-injection monitoring period. In contrast, the highest possible leakage scenario (open well) shows approximately 0.1% cumulative CO 2 leakage over the same time frame.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Estimation of the radiation budget during MOSAiC based on ground-based and satellite remote sensing observations

An accurate representation of the radiation budget is essential for investigating the impact of clouds on the climate system, especially in the Arctic, an environment highly sensitive to complex and rapid environmental changes. In this study, we analyse a unique dataset of observations from the central Arctic made during the MOSAiC (Multidisciplinary drifting Observatory for the Study of Arctic Climate) expedition in conjunction with state-of-the-art satellite products from CERES (Clouds and the Earth's Radiant Energy System) to investigate the radiative effect of clouds and radiative closure at the surface and the top of the atmosphere (TOA). We perform a series of radiative transfer simulations using derived cloud macro- and microphysical properties as inputs to the simulations for the entire MOSAiC period, comparing our results to collocated satellite products and ice-floe observations. The radiative closure biases were generally within the instrumental uncertainty, indicating that the simulations are sufficiently accurate to reproduce the radiation budget during MOSAiC. Comparisons of the simulated radiation budget relative to CERES show similar values in the terrestrial flux but relatively large differences in the solar flux, which are attributed to a lower surface albedo and a possible underestimation of atmospheric opacity by CERES. While the simulation results were consistent with the observations, more detailed analyses reveal an overestimation of simulated cloud opacity for cases involving geometrically thick ice clouds. In the annual mean, we found that, during the MOSAiC expedition, the presence of clouds leads to a loss of 5.2 W m −2 of the atmosphere–surface system to space, while the surface gains 25.0 W m −2 and the atmosphere is cooled by 30.2 W m −2 .

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2023

Extending Conformal Prediction to Hidden Markov Models with Exact Validity via de Finetti’s Theorem for Markov Chains

Conformal prediction is a widely used method to quantify uncertainty in settings where the data is independent and identically distributed (IID), or more generally, exchangeable. Conformal prediction takes in a pre-trained classifier and a calibration dataset as inputs, and returns a function which maps feature vectors to subsets of classes. The output of the returned function for a new feature vector is guaranteed to contain the true class with a pre-specified confidence. Despite its success and usefulness in IID settings, extending conformal prediction to non-exchangeable (e.g., Markovian) data in a manner that provably preserves all desirable theoretical properties has largely remained an open problem. As a solution, we extend conformal prediction to the setting of a Hidden Markov Model (HMM) with unknown parameters. The key idea behind the proposed method is to partition the non-exchangeable Markovian data from the HMM into exchangeable blocks by exploiting the de Finetti’s Theorem for Markov Chains discovered by Diaconis and Freedman (1980). The permutations of the exchangeable blocks are then viewed as randomizations of the observed Markovian data from the HMM. The proposed method provably retains all desirable theoretical guarantees offered by the classical conformal prediction framework and is general enough to be useful in many sequential prediction problems.

Nettasinghe, Don Buddhika Wijayantha↗