Search NASA⌕ Search

SEARCH · Search NASA

Results for “uncertainty assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5
All sourcesNASA NTRSDOE OSTI
DOE OSTI2022

Assessment of Crustal Leg Uncertainties for RSTT Path-dependent Uncertainty

The Regional Seismic Travel Time (RSTT) model and software were recently updated to allow for path-dependent uncertainty (PDU) estimates for travel-time (Begnaud et al. 2021; Begnaud et al. 2020). Begnaud et al. (2021) describes the process of improving the current distance-dependent uncertainty parameterization for RSTT using a random effects model (REM) to estimate slowness (inverse velocity) uncertainty as a mean squared error for each model parameter. The REM separates the error between observed slowness and model predicted slowness into bias and random components. The PDU is calculated by integrating these mean squared errors along a seismic-phase ray path.

58 GEOSCIENCES↗
DOE OSTI2021

Review of Neutron Uncertainty Types

The assessment of neutron data requires understanding the uncertainty. The uncertainty of the final quantities of interest, neutron multiplication and mass, is a combination of many other uncertainties because it is a combination of many other variables. The equations for the values of neutron multiplication and mass are quite complex. The relations between the many contributing variables and their uncertainties are complicated leading to complications in combining them. Additionally, some are statistical in nature, while others are systematic. In an effort to better understand and quantify the total uncertainty in neutron multiplication and SNM mass, they have been subdivided into four categories: setup, detector response, nuclear data, and object characterization. This paper discusses the four categories and what types of uncertainties are in each of them. This effort attempts to generalize this process for any neutron detector, but many of our examples show the Next Generation Multiplicity Detector (MC-15).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2024

Characterizing the Uncertainty of Measurement of Traceable Isotope Ratios with Bayesian Statistical Techniques

Analytical techniques such as multicollector—inductively coupled plasma—mass spectrometry (MC-ICP-MS) are routinely employed at SRNL, other National Laboratories, and in academia to determine the precise isotopic composition of diverse natural and anthropogenic samples (e.g., rocks and nuclear materials). Quantifying and reporting uncertainty in such analyses, while regularly performed, have a rigorous statistical foundation. The Guide to the Expression of Uncertainty in Measurement 4 (GUM) outlines conventional techniques used to assess such uncertainty. As the accessibility and speed of statistical computing increase, there is a need to modernize conventional techniques. For example, Supplement 1 to the 3rd to the GUM suggests the use of approximation methods as an updated approach to the GUM.

McLarty, Ellis C.↗
DOE OSTI2026

Laboratory and field assessment of mid-infrared absorption (MIRA) instrument performance for methane and ethane dry mole fractions

Concurrent measurements of methane (CH 4 ) and ethane (C 2 H 6 ) can be used to identify and separate methane sources, as ethane is present in thermogenic sources (e.g., oil and natural gas) but not in biogenic sources (e.g., agriculture). In this study, we evaluated the performance of multiple Aeris MIRA Ultra instruments (Versions 1 and 2) through controlled laboratory tests and tower-based deployments under field conditions. The systems were modified with an external pump, flow control, a Nafion dryer, and a custom-built auxiliary box to automate the system and transmit near real-time data. We determined the best calibration approach for our application, given practical limitations, to be a full calibration cycle (with ambient and high calibration cylinders) about once per day and an ambient calibration cylinder sampled hourly. Measurement uncertainty was assessed, including the uncertainty due to instrument noise as a function of calibration frequency, uncertainty in the water vapor correction, and cylinder assignment uncertainty. Instrument noise was the dominant source of uncertainty for C 2 H 6 , while the water vapor correction dominated the CH 4 uncertainty. For Version 2 systems with hourly calibrations and a Nafion dryer with counterflow, the mean total uncertainty, including both systematic errors and noise, of hourly averages was 0.8–3.0 ppb CH 4 and 0.35–0.37 ppb C 2 H 6 . Laboratory intercomparisons showed network compatibility within 1.2 ppb CH 4 and 0.23 ppb C 2 H6, and a collocated deployment with a NOAA Picarro system agreed within 1.8 ppb CH 4 . Instrument noise varied substantially amongst the instruments, with errors reaching up to 11 ppb CH 4 and 2 ppb C 2 H 6 for hourly means, with similar variability indicated in a 50 h cylinder test. With appropriate engineering and calibration, the Aeris MIRA Ultra has the potential to distinguish regional methane emission sources in many field settings.

03 NATURAL GAS↗
DOE OSTI2025

Coupled Lake‐Atmosphere‐Land Physics Uncertainties in a Great Lakes Regional Climate Model

Abstract This study develops a surrogate‐based method to assess the uncertainty within a convective permitting integrated modeling system of the Great Lakes region, arising from interacting physics parameterizations across the lake, atmosphere, and land surface. Perturbed physics ensembles of the model during the 2018 summer are used to train a neural network surrogate model to predict lake surface temperature (LST) and near‐surface air temperature (T2m). Average physics uncertainties are determined to be 1.5C for LST and T2m over land, and 1.9C for T2m over lake, but these have significant spatiotemporal variations. We find that atmospheric physics parameterizations alone are the dominant sources of uncertainty (45%–53%), while lake and land parameterizations account for 33% and 38% of the uncertainty of LST and T2m over land respectively. Interactions of atmosphere physics parameterizations with those of the land and lake contribute to an additional 13%–17% of the total variance. LST and T2m over the lake are more uncertain in the deeper northern lakes, particularly during the rapid warming phase that occurs in late spring/early summer. The LST uncertainty increases with sensitivity to the lake model's surface wind stress scheme. T2m over land is more uncertain over forested areas in the north, where it is most sensitive to the land surface model, than the more agricultural land in the south, where it is most sensitive to the atmospheric planetary boundary and surface layer scheme. Uncertainty also increases in the southwest during multiday temperature declines with higher sensitivity to the land surface model.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2021

Assessing dynamic vegetation model parameter uncertainty across Alaskan arctic tundra plant communities

Abstract As the Arctic region moves into uncharted territory under a warming climate, it is important to refine the terrestrial biosphere models (TBMs) that help us understand and predict change. One fundamental uncertainty in TBMs relates to model parameters, configuration variables internal to the model whose value can be estimated from data. We incorporate a version of the Terrestrial Ecosystem Model (TEM) developed for arctic ecosystems into the Predictive Ecosystem Analyzer (PEcAn) framework. PEcAn treats model parameters as probability distributions, estimates parameters based on a synthesis of available field data, and then quantifies both model sensitivity and uncertainty to a given parameter or suite of parameters. We examined how variation in 21 parameters in the equation for gross primary production influenced model sensitivity and uncertainty in terms of two carbon fluxes (net primary productivity and heterotrophic respiration) and two carbon (C) pools (vegetation C and soil C). We set up different parameterizations of TEM across a range of tundra types (tussock tundra, heath tundra, wet sedge tundra, and shrub tundra) in northern Alaska, along a latitudinal transect extending from the coastal plain near Utqiaġvik to the southern foothills of the Brooks Range, to the Seward Peninsula. TEM was most sensitive to parameters related to the temperature regulation of photosynthesis. Model uncertainty was mostly due to parameters related to leaf area, temperature regulation of photosynthesis, and the stomatal responses to ambient light conditions. Our analysis also showed that sensitivity and uncertainty to a given parameter varied spatially. At some sites, model sensitivity and uncertainty tended to be connected to a wider range of parameters, underlining the importance of assessing tundra community processes across environmental gradients or geographic locations. Generally, across sites, the flux of net primary productivity (NPP) and pool of vegetation C had about equal uncertainty, while heterotrophic respiration had higher uncertainty than the pool of soil C. Our study illustrates the complexity inherent in evaluating parameter uncertainty across highly heterogeneous arctic tundra plant communities. It also provides a framework for iteratively testing how newly collected field data related to key parameters may result in more effective forecasting of Arctic change.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2020

How Wrong Can the Operational AEP Uncertainty Estimate Be When We Ignore the Correlations Between the Uncertainty Components?

Calculations of wind farm annual energy production (AEP) on operational data are essential for a variety of financial transactions during the life of wind plants. The AEP estimate is associated with an uncertainty value, which is calculated by combining contributions connected to on-site measurements, long-term reference measurements, losses, regression, windiness adjustment, and wind resource interannual variability. Although very limited documentation on the topic exists, the conventional approach currently used by the wind energy community to assess the uncertainty connected to the operational AEP estimate assume that the different uncertainty components are uncorrelated and therefore calculates the overall uncertainty with a sum of squares approach. In this analysis, we contrast the traditional technique to estimate the overall AEP uncertainty by ignoring the correlation between its different components with a novel Monte Carlo based approach, which can instead take into account the correlation between different uncertainty categories. We consider monthly operational data from 472 wind farms from the Energy Information Administration (EIA) 923 database to assess the difference between the two approaches. Long-term wind speed data needed for the AEP assessment are taken from three reanalysis products: the Modern-Era Retrospective analysis for Research and Applications v2 (MERRA-2), the European Reanalysis Interim (ERA-interim), and the National Centers for Environmental Prediction v2 (NCEP-2). The results of the Monte Carlo approach show that three pairs of AEP uncertainty components do show a statistically significant correlation: the uncertainty connected with wind resource inter-annual variability is positively correlated with the one related to the windiness correction and negatively correlated with the one due to the regression, and the wind measurement uncertainty is positively correlated with the regression uncertainty. All these correlations, which are found between uncertainty components which are not only part of an operational analysis, but also of a wind resource assessment, are currently ignored in the conventional techniques used as industry standard. We further investigate the causes of these correlations, in terms of common dependencies of different uncertainty components on wind resource variability, number of data points, and quality of the regression between wind speed and energy production data. Next, we quantify the error in the current industry standard technique, in terms of the percentage difference in total uncertainty calculated with the two considered approaches, for all the analyzed wind farms. We find a mean absolute percentage difference of about 6%, with the largest differences being greater than 20%. The data clearly confirm that ignoring the actual correlation between the uncertainty components can lead to large errors in the assessment of the operational AEP uncertainty, and the proposed Monte Carlo approach should be preferred.

Monte Carlo↗
DOE OSTI2021

Data Quality of Salt Property Measurements

Data quality is a key element to assessing technical uncertainties in property values of salt mixtures that are pivotal in molten salt reactor (MSR) design and safety assessments under both normal operating conditions and accident scenarios. Key properties for the salt mixtures being evaluated for use in MSRs have not been measured and the few values for relevant salt compositions available in the literature are inconsistent and not suitable for use in licensing. The applications of established techniques commonly used for property measurements to molten salts are still experimental in nature and vetted consensus standards are not yet available. Work is in progress to develop and standardize methods through DOE and commercially funded projects. The quality of measured property values limits the current ability to reliably design equipment and model normal MSR operation and the effects of operational upsets. Insights from experimental measurements pertinent to reducing technical uncertainty for regulatory development are summarized in this report. Quality controls are recommended for key properties based on the mathematical calculations used to derive property values and methods used to measure experimental values and salt produced for use in the measurements.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗
DOE OSTI2021

The implications of uncertain renewable resource potentials for global wind and solar electricity projections

Studies exploring long-term energy system transitions rely on resource cost-supply curves derived from estimates of renewable energy (RE) potentials to generate wind and solar power projections. However, estimates of RE potentials are characterized by large uncertainties stemming from methodological assumptions that vary across studies, including factors such as the suitability of land and the performance and configuration of technology. Based on a synthesis of modeling approaches and parameter values used in prior studies, we explore the implications of these uncertain assumptions for onshore wind and solar photovoltaic electricity generation projections globally using the Global Change Analysis Model. We show that variability in parametric assumptions related to land use (e.g. land suitability) are responsible for the most substantial uncertainty in both wind and solar generation projections. Additionally, assumptions about the average turbine installation density and turbine technology are responsible for substantial uncertainty in wind generation projections. Under scenarios that account for climate impacts on wind and solar energy, we find that these parametric uncertainties are far more significant than those emerging from differences in climate models and scenarios in a global assessment, but uncertainty surrounding climate impacts (across models and scenarios) have significant effects regionally, especially for wind. Our analysis suggests the need for studies focusing on long-term energy system transitions to account for this uncertainty.

14 SOLAR ENERGY↗
DOE OSTI2022

Deployment of Multifidelity Uncertainty Quantification for Thermal Battery Assessment Part I: Algorithms and Single Cell Results

This report documents the results of an FY22 ASC V&V level 2 milestone demonstrating new algorithms for multifidelity uncertainty quantification. Part I of the report describes the algorithms, studies their performance on a simple model problem, and then deploys the methods to a thermal battery example from the open literature. Part II (restricted distribution) applies the multifidelity UQ methods to specific thermal batteries of interest to the NNSA/ASC program.

25 ENERGY STORAGE↗
DOE OSTI2022

Do Machine Learning Approaches Offer Skill Improvement for Short-Term Forecasting of Wind Gust Occurrence and Magnitude?

Abstract Wind gusts, and in particular intense gusts, are societally relevant but extremely challenging to forecast. This study systematically assesses the skill enhancement that can be achieved using artificial neural networks (ANNs) for forecasting of wind gust occurrence and magnitude. Geophysical predictors from the ERA5 reanalysis are used in conjunction with an autoregressive term in regression and ANN models with different predictors, and varying model complexity. Models are derived and assessed for the warm (April–September) and cold (October–March) seasons for three high passenger volume airports in the United States. Model uncertainty is assessed by deriving models for 1000 different randomly selected training (70%) and testing (30%) subsets. Gust prediction fidelity in independent test samples is critically dependent on inclusion of an autoregressive term. Gust occurrence probabilities derived using five-layer ANNs exhibit consistently higher fidelity than those from regression models and shallower ANNs. Inclusion of the autoregressive term and increasing the number of hidden layers in ANNs from 1 to 5 also improve the model performance for gust magnitudes (lower RMSE, increased correlation, and model standard deviations that more closely approximate observed values). Deeper ANNs (e.g., 20 hidden layers) exhibit higher skill in forecasting strong (17–25.7 m s −1 ) and damaging (≥25.7 m s −1 ) wind gusts. However, such deep networks exhibit evidence of overfitting and still substantially underestimate (by 50%) the frequency of strong and damaging wind gusts at the three airports considered herein. Significance Statement Improved short-term forecasting of wind gusts will enhance aviation safety and logistics and may offer other societal benefits. Here we present a rigorous investigation of the relative skill of models of wind gust occurrence and magnitude that employ different statistical methods. It is shown that artificial neural networks (ANNs) offer considerable skill enhancement over regression methods, particularly for strong and damaging wind gusts. For wind gust magnitudes in particular, application of deeper learning networks (e.g., five or more hidden layers) offers tangible improvements in forecast accuracy. However, deeper networks are vulnerable to overfitting and exhibit substantial variability with the specific training and testing data subset used. Also, even deep ANNs reproduce only half of strong and damaging wind gusts. These results indicate the need for future work to elucidate the dynamical mechanisms of intense wind gusts and advance solutions to their prediction.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2025

Toward shell model interactions with credible uncertainties

Background: The nuclear shell model is a powerful framework for predicting nuclear structure observables, but relies on interaction matrix elements fit to experimental data as its inputs. Extending the shell model's applicability, particularly toward dripline nuclei, requires efficient fitting methods and credible uncertainty quantification. Traditional approaches face computational challenges and may underestimate uncertainties. Purpose: We develop and test a framework combining eigenvector continuation and Markov chain Monte Carlo to efficiently fit shell model interaction matrix elements and quantify their uncertainties. Methods: Eigenvector continuation is used to emulate shell model calculations, reducing computational costs. The emulator enables Markov chain Monte Carlo sampling to optimize interaction matrix elements and rigorously assess parametric uncertainties. Here, the framework is benchmarked using the USDB interaction in the 𝑠⁢𝑑 shell. Results: The emulator reproduces the USDB interaction with negligible error, validating its use in shell model fitting applications. However, we find that to obtain credible predictive intervals, the model defect of the shell model itself, rather than experimental or emulator error, must be taken into account in order to obtain credible uncertainties. Conclusions: The proposed framework provides an efficient and rigorous approach for fitting shell model interactions and quantifying uncertainties. Further, the normality assumption used in the past appears sufficient to describe the distribution of interaction matrix elements. However, it is crucial to account for model correlations to avoid underestimating uncertainties.

Nuclear forces↗
DOE OSTI2021

Foundations of Rigorous Cyber Experimentation

This report presents the results of the “Foundations of Rigorous Cyber Experimentation” (FORCE) Laboratory Directed Research and Development (LDRD) project. This project is a companion project to the “Science and Engineering of Cyber security through Uncertainty quantification and Rigorous Experimentation” (SECURE) Grand Challenge LDRD project. This project leverages the offline, controlled nature of cyber experimentation technologies in general, and emulation testbeds in particular, to assess how uncertainties in network conditions affect uncertainties in key metrics. We conduct extensive experimentation using a Firewheel emulation-based cyber testbed model of Invisible Internet Project (I2P) networks to understand a de-anonymization attack formerly presented in the literature. Our goals in this analysis are to see if we can leverage emulation testbeds to produce reliably repeatable experimental networks at scale, identify significant parameters influencing experimental results, replicate the previous results, quantify uncertainty associated with the predictions, and apply multi-fidelity techniques to forecast results to real-world network scales. The I2P networks we study are up to three orders of magnitude larger than the networks studied in SECURE and presented additional challenges to identify significant parameters. The key contributions of this project are the application of SECURE techniques such as UQ to a scenario of interest and scaling the SECURE techniques to larger network sizes. This report describes the experimental methods and results of these studies in more detail. In addition, the process of constructing these large-scale experiments tested the limits of the Firewheel emulation-based technologies. Therefore, another contribution of this work is that it informed the Firewheel developers of scaling limitations, which were subsequently corrected.

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2020

Nuclear data uncertainty propagation and modeling uncertainty impact evaluation in neutronics core simulation

Uncertainty analysis is a critical requirement in reactor simulation as it is used to quantify the reliability of best-estimate calculation. A comprehensive uncertainty analysis should characterize all sources of uncertainties in a computationally-feasible and scientifically-defendable manner. Here we employ a well-established reduced order modeling (ROM) based uncertainty quantification methodology to propagate uncertainties throughout neutronic calculations. ROM relies on recent advances in randomized data mining techniques applied to large data streams. In our proposed implementation, the nuclear data uncertainties are first propagated from multi-group level through lattice physics calculation to generate few-group parameter uncertainties, described using a vector of mean values and a covariance matrix. Employing an ROM-based compression of the covariance matrix, the few-group uncertainties are then propagated through downstream core simulation in a computationally efficient manner. This straightforward approach, albeit efficient as compared to brute force forward and/or adjoint-based methods, often employs a number of assumptions that have been unquestioned in the literature of neutronic uncertainty analysis. This manuscript argues that these assumptions could introduce another source of uncertainty referred to as modeling uncertainties, whose magnitude needs to be quantified in tandem with nuclear data uncertainties. Thus, our primary goal is to explore the interactions between these two uncertainty sources in order to assess whether modeling uncertainties have an impact on parameter uncertainties. To explore this endeavor, the impact of a number of modeling assumptions on core attributes uncertainties is quantified. The study employs a CANDU reactor model, with Serpent and NEWT as lattice physics solvers and NESTLE-C as core simulator. The modeling assumptions investigated include those related with the uncertainty propagation method employed, e.g., deterministic vs. stochastic, the few-group energy structure employed to represent the cross-sections, the resonance treatment in lattice physics calculation, the reference values for the cross-section, and the number of samples employed to render ROM compression. Results indicate that some of the modeling assumptions could have a non-negligible impact on the core responses propagated uncertainties, highlighting the need for a more comprehensive approach to combine parameter and modeling uncertainties.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗
DOE OSTI2021

NRAP-open-IAM: A flexible open-source integrated-assessment-model for geologic carbon storage risk assessment and management

Large-scale implementation of geologic carbon storage (GCS) to help reduce atmospheric greenhouse gas emissions requires stakeholder confidence that injected CO2 will remain contained and that potential subsurface environmental risks are acceptably small and manageable. The U.S. Department of Energy’s National Risk Assessment Partnership (NRAP) has developed an open-source integrated assessment model (NRAP-Open-IAM) to help address questions about a potential GCS site’s ability to effectively contain injected CO 2 and protect groundwater and other overlying environmentally sensitive receptors. NRAP-Open-IAM allows a user to: (1) incorporate relevant site geologic and injection scenario data; (2) characterize important site features and events;(3) couple fast prediction models of various system components of the engineered geologic system; and (4) execute stochastic, dynamic simulation of whole GCS system performance, leakage risk assessment, and uncertainty quantification. NRAP-Open-IAM is available on GitLab (https://gitlab.com/NRAP/OpenIAM), and is accompanied by multiple application examples and detailed user and developer guides.

54 ENVIRONMENTAL SCIENCES↗