Search NASA⌕ Search

SEARCH · Search NASA

Results for “errors and uncertainties”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 883 records · Page 49

Evaluation of airborne lidar wind measurements

The verification of the Doppler lidar wind measurement is continued. The in situ measurements on a 500 meter tall tower are used to check the lidar determined winds (flight 4), followed by other tests applied to the extensive data sets of flights 19 and 21. After checking for possible software errors in the processing programs, the inconsistency was not solved. The source of the errors appears to be the uncertainty in the calculation of the contribution of the aircraft motion to the lidar measured line of sight speed. Possible techniques for estimating the aircraft motion errors from the aircraft data are investigated.

Carroll, J. J.↗

Performance of Trajectory Models with Wind Uncertainty

Typical aircraft trajectory predictors use wind forecasts but do not account for the forecast uncertainty. A method for generating estimates of wind prediction uncertainty is described and its effect on aircraft trajectory prediction uncertainty is investigated. The procedure for estimating the wind prediction uncertainty relies uses a time-lagged ensemble of weather model forecasts from the hourly updated Rapid Update Cycle (RUC) weather prediction system. Forecast uncertainty is estimated using measures of the spread amongst various RUC time-lagged ensemble forecasts. This proof of concept study illustrates the estimated uncertainty and the actual wind errors, and documents the validity of the assumed ensemble-forecast accuracy relationship. Aircraft trajectory predictions are made using RUC winds with provision for the estimated uncertainty. Results for a set of simulated flights indicate this simple approach effectively translates the wind uncertainty estimate into an aircraft trajectory uncertainty. A key strength of the method is the ability to relate uncertainty to specific weather phenomena (contained in the various ensemble members) allowing identification of regional variations in uncertainty.

Lee, Alan G.↗

Sparse Bayesian mass mapping with uncertainties: local credible intervals

ABSTRACT Until recently, mass-mapping techniques for weak gravitational lensing convergence reconstruction have lacked a principled statistical framework upon which to quantify reconstruction uncertainties, without making strong assumptions of Gaussianity. In previous work, we presented a sparse hierarchical Bayesian formalism for convergence reconstruction that addresses this shortcoming. Here, we draw on the concept of local credible intervals (cf. Bayesian error bars) as an extension of the uncertainty quantification techniques previously detailed. These uncertainty quantification techniques are benchmarked against those recovered via Px-MALA – a state-of-the-art proximal Markov chain Monte Carlo (MCMC) algorithm. We find that, typically, our recovered uncertainties are everywhere conservative (never underestimate the uncertainty, yet the approximation error is bounded above), of similar magnitude and highly correlated with those recovered via Px-MALA. Moreover, we demonstrate an increase in computational efficiency of $\mathcal {O}(10^6)$ when using our sparse Bayesian approach over MCMC techniques. This computational saving is critical for the application of Bayesian uncertainty quantification to large-scale stage IV surveys such as LSST and Euclid.

Price, M. A.↗

Deep Learning for Multigroup Cross-Section Representation in Two-Step Core Calculations

Here we investigate using deep learning, a type of machine-learning algorithm employing multiple layers of artificial neurons, for the mathematical representation of multigroup cross sections for use in the Griffin reactor multiphysics code for two-step deterministic neutronics calculations. A three-dimensional fuel element typical of a high-temperature gas reactor as well as a two-dimensional sodium-cooled fast reactor lattice are modeled using the Serpent Monte Carlo code, and multigroup macroscopic cross sections are generated for various state parameters to produce a training data set and a separate validation data set. A fully connected, feedforward neural network is trained using the open-source PyTorch machine-learning framework, and its accuracy is compared against the standard piecewise linear interpolation model. Additionally, we provide in this work a generic technique for propagating the cross-section model errors up to the k eff using sensitivity coefficients with the first-order uncertainty propagation rule. Quantifying the eigenvalue error due to the cross-section regression errors is especially practical for appropriately selecting the mathematical representation of the cross sections. We demonstrate that the artificial neural network model produces lower errors and therefore enables better accuracy relative to the piecewise linear model when the cross sections exhibit nonlinear dependencies; especially when a coarse grid is employed, where the errors can be halved by the artificial neural network. However, for linearly dependent multigroup cross sections as found for the sodium-cooled fast reactor case, a simpler linear regression outperforms deeper networks.

22 GENERAL STUDIES OF NUCLEAR REACTORS↗

Uncertain of uncertainties? A comparison of uncertainty quantification metrics for chemical data sets

Abstract With the increasingly more important role of machine learning (ML) models in chemical research, the need for putting a level of confidence to the model predictions naturally arises. Several methods for obtaining uncertainty estimates have been proposed in recent years but consensus on the evaluation of these have yet to be established and different studies on uncertainties generally uses different metrics to evaluate them. We compare three of the most popular validation metrics (Spearman’s rank correlation coefficient, the negative log likelihood (NLL) and the miscalibration area) to the error-based calibration introduced by Levi et al. ( Sensors 2022 , 22 , 5540). Importantly, metrics such as the negative log likelihood (NLL) and Spearman’s rank correlation coefficient bear little information in themselves. We therefore introduce reference values obtained through errors simulated directly from the uncertainty distribution. The different metrics target different properties and we show how to interpret them, but we generally find the best overall validation to be done based on the error-based calibration plot introduced by Levi et al. Finally, we illustrate the sensitivity of ranking-based methods (e.g. Spearman’s rank correlation coefficient) towards test set design by using the same toy model ferent test sets and obtaining vastly different metrics (0.05 vs. 0.65).

Rasmussen, Maria H.↗

OR14-V-Uncertainty-PD2La Uncertainty Quantification for Nuclear Safeguards and Nondestructive Assay Final Report

The various methods of nondestructive assay (NDA) of special nuclear material (SNM) have applications in nuclear nonproliferation, including detection and identification of illicit SNM at border crossings and quantifying SNM at nuclear facilities for safeguards. No assay method is complete without “error bars,” which provide one way of expressing confidence in the assay result. Consequently, NDA specialists typically provide error bars and also partition total uncertainty into “random” and “systematic” components so that, for example, an error bar can be developed for the total mass estimate in multiple items. Uncertainty Quantification (UQ) for NDA has always been important, but it is recognized that greater rigor is needed and achievable using modern statistical methods.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Statistical Characterization of Environmental Error Sources Affecting Electronically Scanned Pressure Transducers

Minimization of uncertainty is essential to extend the usable range of the 15-psid Electronically Scanned Pressure (ESP) transducer measurements to the low free-stream static pressures found in hypersonic wind tunnels. Statistical characterization of environmental error sources inducing much of this uncertainty requires a well defined and controlled calibration method. Employing such a controlled calibration system, several studies were conducted that provide quantitative information detailing the required controls needed to minimize environmental and human induced error sources. Results of temperature, environmental pressure, over-pressurization, and set point randomization studies for the 15-psid transducers are presented along with a comparison of two regression methods using data acquired with both 0.36-psid and 15-psid transducers. Together these results provide insight into procedural and environmental controls required for long term high-accuracy pressure measurements near 0.01 psia in the hypersonic testing environment using 15-psid ESP transducers.

Green, Del L.↗

Statistical Characterization of Environmental Error Sources Affecting Electronically Scanned Pressure Transducers

Minimization of uncertainty is essential to extend the usable range of the 15-psid Electronically Scanned Pressure [ESP) transducer measurements to the low free-stream static pressures found in hypersonic wind tunnels. Statistical characterization of environmental error sources inducing much of this uncertainty requires a well defined and controlled calibration method. Employing such a controlled calibration system, several studies were conducted that provide quantitative information detailing the required controls needed to minimize environmental and human induced error sources. Results of temperature, environmental pressure, over-pressurization, and set point randomization studies for the 15-psid transducers are presented along with a comparison of two regression methods using data acquired with both 0.36-psid and 15-psid transducers. Together these results provide insight into procedural and environmental controls required for long term high-accuracy pressure measurements near 0.01 psia in the hypersonic testing environment using 15-psid ESP transducers.

Green, Del L.↗

Observations of cloud liquid water path over oceans: Optical and microwave remote sensing methods

Published estimates of cloud liquid water path (LWP) from satellite-measured microwave radiation show little agreement, even about the relative magnitudes of LWP in the tropics and midlatitudes. To understand these differences and to obtain more reliable estimate, optical and microwave LWP retrieval methods are compared using the International Satellite Cloud Climatology Project (ISCCP) and special sensor microwave/imager (SSM/I) data. Errors in microwave LWP retrieval associated with uncertainties in surface, atmosphere, and cloud properties are assessed. Sea surface temperature may not produce great LWP errors, if accurate contemporaneous measurements are used in the retrieval. An uncertainty of estimated near-surface wind speed as high as 2 m/s produces uncertainty in LWP of about 5 mg/sq cm. Cloud liquid water temperature has only a small effect on LWP retrievals (rms errors less than 2 mg/sq cm), if errors in the temperature are less than 5 C; however, such errors can produce spurious variations of LWP with latitude and season. Errors in atmospheric column water vapor (CWV) are strongly coupled with errors in LWP (for some retrieval methods) causing errors as large as 30 mg/sq cm. Because microwave radiation is much less sensitive to clouds with small LWP (less than 7 mg/sq cm) than visible wavelength radiation, the microwave results are very sensitive to the process used to separate clear and cloudy conditions. Different cloud detection sensitivities in different microwave retrieval methods bias estimated LWP values. Comparing ISCCP and SSM/I LWPs, we find that the two estimated values are consistent in global, zonal, and regional means for warm, nonprecipitating clouds, which have average LWP values of about 5 mg/sq cm and occur much more frequently than precipitating clouds. Ice water path (IWP) can be roughly estimated from the differences between ISCCP total water path and SSM/I LWP for cold, nonprecipitating clouds. IWP in the winter hemisphere is about 3 times the LWP but only half the LWP in the summer hemisphere. Precipitating clouds contribute significantly to monthly, zonal mean LWP values determined from microwave, especially in the intertropical convergence zone (ITCZ), because they have almost 10 times the liquid water (cloud plus precipitation) of nonprecipitating clouds on average. There are significant differences among microwave LWP estimates associated with the treatment of precipitating clouds.

Lin, Bing↗

Evaluation of methods to assess the uncertainty in estimated energy savings

In this work we present a methodology to evaluate the accuracy of methods used to quantify the uncertainty in estimated total energy savings. We focus on savings measurement and verification (M&V) approaches that use a baseline model to characterize energy use, and that forward-project the model for a counterfactual to determine avoided energy use. These approaches are common to the International Performance Measurement and Verification Protocol's (IPMVP's) Option C and Option B. This methodology can be used to evaluate the uncertainty in savings estimates that are due to model error. It has been applied to evaluate two uncertainty estimation methods, including the industry standard ASHRAE Guideline 14 approach. The evaluation used data from 69 commercial buildings and four different baseline models that span daily and hourly granularity, as well as linear and non-linear/non-parametric forms. The findings of this work indicate that the standard methods that are widely used by the M&V community for estimating the total savings uncertainty over the post-installation period tend to underestimate the uncertainty. The tendency to underestimate the uncertainty is stronger for hourly models than for daily models, due to stronger autocorrelation in model residuals at the hourly time scale.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Assessment of Optimal Flexibility in Ensemble of Frequency Responsive Loads

Potential of electrical loads in providing grid ancillary services is often limited due to the uncertainties associated with the load behavior. A knowledge of the expected uncertainties with a load control program would invariably yield to better informed control policies, opening up the possibility of extracting the maximal load control potential without affecting grid operations. In the context of frequency responsive load control, a probabilistic uncertainty analysis framework is presented to quantify the expected error between the target and actual load response, under uncertainties in the load dynamics. A closed-form expression of an optimal demand flexibility, minimizing the expected error in actual and committed flexibility, is provided. Analytical results are validated through Monte Carlo simulations of ensembles of electric water heaters.

Kundu, Soumya↗

Statistical uncertainty of eddy covariance CO 2 fluxes inferred using a residual bootstrap approach

High-frequency eddy-covariance measurements of net ecosystem CO 2 exchange (NEE) with the atmosphere are valuable resources for model parameterization, calibration, and validation. However, uncertainties in measured data, i.e., data gaps and inherent random errors, create problems for researchers attempting to quantify uncertainties in model projections of terrestrial ecosystem carbon cycling. Here, we demonstrate that a model-data fusion method (residual bootstrap) produces defensible annual NEE sums, through mimicking the behavior of random errors, filling missing values, and simulating gap-filling biases. This study estimated annual NEE sums for 53 site-years based on nine eddy-covariance tower sites in the USA, and found that our annual estimates were, in most cases, comparable in magnitude with those obtained from AmeriFlux gap-filled data. Additionally, compared to the AmeriFlux standardized gap-filling, our approach provides better NEE estimates for moderate to longer, and more frequent, data gaps. Annual accumulated uncertainties in NEE at the 95% confidence level were ±30gCm -2 year -1 for evergreen needleleaf forests; ±60gCm -2 year -1 for deciduous broadleaf forests; and ±80gCm -2 year -1 for croplands. The residual bootstrap performed worst when gap length was greater than one month or data exclusion greater than 90% during the growing season, common to other gap-filling techniques. However, this study produced robust results for most site years when monthly data coverage during the growing season is not extremely low. We therefore suggest that the inclusion of NEE uncertainty estimates and better estimation for moderate to longer, and more frequent, data gaps as provided by the residual bootstrap approach can be beneficial for ecosystem model evaluation.

54 ENVIRONMENTAL SCIENCES↗

Spatial Predictions and Associated Uncertainty of Annual Soil Respiration at the Global Scale

Abstract Soil respiration (Rs), the soil‐to‐atmosphere CO 2 flux produced by microbes and plant roots, is a critical but uncertain component of the global carbon cycle. Our current understanding of the variability and dynamics is limited by the coarse spatial resolution of existing estimates. We predicted annual Rs and associated uncertainty across the world at 1‐km resolution using a quantile regression forest algorithm trained with observations from the global Soil Respiration Database spanning from 1961 to 2011. This model yielded a global annual Rs estimate of 87.9 Pg C/year with an associated global uncertainty of 18.6 (mean absolute error) and 40.4 (root mean square error) Pg C/year. The estimated annual heterotrophic respiration (Rh), derived from empirical relationships with Rs, was 49.7 Pg C/year over the same period. Predicted Rs rates and associated uncertainty varied widely across vegetation types, with the greatest predicted rates of Rs in evergreen broadleaf forests (accounting for 20.9% of global Rs). The greatest prediction uncertainties were in northern latitudes and arid to semiarid ecosystems, suggesting that these areas should be targeted in future measurement campaigns. This study provides predictions of Rs (and associated prediction uncertainty) at unprecedentedly high spatial resolution across the globe that could help constrain local‐to‐global process‐based models. Furthermore, it provides insights into the large variability of Rs and Rh across vegetation classes and identifies regions and vegetation types with poor model performance that should be prioritized for future data collection.

Warner, D. L.↗

Complete quantification of parametric uncertainties in ( d , p ) transfer reactions

Deuteron-induced transfer reactions are a popular probe in nuclear structure and nuclear astro physics studies. The interpretation of these transfer measurements relies on reaction theory that takes as input effective interactions between the nucleons and the target nucleus. Previous work quantified the uncertainty associated with the optical potentials between the nucleons and the target. In this study, we extend that work by also including the parameters of the mean field associated with the overlap function of the final bound state, thus obtaining the full parametric uncertainty on transfer observables. We use Bayesian Markov Chain Monte Carlo simulations to obtain parameter posterior distributions. We use elastic-scattering cross sections to constrain the optical potential parameters and use the asymptotic normalization coefficient of the final state to constrain the bound state interaction. We then propagate these posteriors to the transfer angular distributions and obtain confidence intervals for this observable. We study (d,p) reactions on 14 C, 16 O, and 48 Ca at energies in the range $E$ $d$ = 7 - 24 MeV. Our results show a strong reduction in uncertainty by using the asymptotic normalization coefficient as a constraint, particularly for those reactions most sensitive to ambiguities in the mean field. For those reactions, the importance of constraining the bound state interaction is equal to that of constraining the optical potentials. Furthermore, the case of 14 C is an outlier because the cross section is less sensitive to the nuclear interior. When minimal constraints are used on the parameters of the nucleon-target interaction, the 1σ uncertainties on the differential cross sections are large (~ 500-100%). However, if elastic-scattering data and the asymptotic normalization coefficient are used in the analysis, with an error of 10% (5%), this uncertainty reduces to ~ 30% (~ 15%).

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

Satellite Remote-Sensing Capability to Assess Tropospheric-Column Ratios of Formaldehyde and Nitrogen Dioxide: Case Study During the Long Island Sound Tropospheric Ozone Study 2018 (LISTOS 2018) Field Campaign

Satellite retrievals of tropospheric-column formaldehyde (HCHO) and nitrogen dioxide (NO 2 ) are frequently used to investigate the sensitivity of ozone (O 3 ) production to emissions of nitrogen oxides and volatile organic carbon compounds. This study inter-compared the systematic biases and uncertainties in retrievals of NO 2 and HCHO, as well as resulting HCHO–NO 2 ratios (FNRs), from two commonly applied satellite sensors to investigate O 3 production sensitivities (Ozone Monitoring Instrument, OMI, and TROPOspheric Monitoring Instrument, TROPOMI) using airborne remote-sensing data taken during the Long Island Sound Tropospheric Ozone Study 2018 between 25 June and 6 September 2018. Compared to aircraft-based HCHO and NO 2 observations, the accuracy of OMI and TROPOMI were magnitude-dependent with high biases in clean environments and a tendency towards more accurate comparisons to even low biases in moderately polluted to polluted regions. OMI and TROPOMI NO 2 systematic biases were similar in magnitude (normalized median bias, NMB = 5 %–6 %; linear regression slope ≈ 0.5–0.6), with OMI having a high median bias and TROPOMI resulting in small low biases. Campaign-averaged uncertainties in the three satellite retrievals (NASA OMI; Quality Assurance for Essential Climate Variables, QA4ECV OMI; and TROPOMI) of NO 2 were generally similar, with TROPOMI retrievals having slightly less spread in the data compared to OMI. The three satellite products differed more when evaluating HCHO retrievals. Campaign-averaged tropospheric HCHO retrievals all had linear regression slopes ∼0.5 and NMBs of 39 %, 17 %, 13 %, and 23 % for NASA OMI, QA4ECV OMI, and TROPOMI at finer (0.05° x 0.05°) and coarser (0.15° x 0.15°) spatial resolution, respectively. Campaign-averaged uncertainty values (root mean square error, RMSE) in NASA and QA4ECV OMI HCHO retrievals were ~9.0 x 10 15 molecules cm –2 (∼ 50 %–55 % of mean column abundance), and the higher-spatial-resolution retrievals from TROPOMI resulted in RMSE values ∼30 % lower. Spatially averaging TROPOMI tropospheric-column HCHO, along with NO 2 and FNRs, to resolutions similar to the OMI reduced the uncertainty in these retrievals. Systematic biases in OMI and TROPOMI NO 2 and HCHO retrievals tended to cancel out, resulting in all three satellite products comparing well to observed FNRs. However, while satellite-derived FNRs had minimal campaign-averaged median biases, unresolved errors in the indicator species did not cancel out in FNR calculations, resulting in large RMSE values compared to observations. Uncertainties in HCHO retrievals were determined to drive the unresolved biases in FNR retrievals.

Matthew S. Johnson↗

Planck 2015 results: IX. Diffuse component separation: CMB maps

In this paper, we present foreground-reduced cosmic microwave background (CMB) maps derived from the full Planck data set in both temperature and polarization. Compared to the corresponding Planck 2013 temperature sky maps, the total data volume is larger by a factor of 3.2 for frequencies between 30 and 70 GHz, and by 1.9 for frequencies between 100 and 857 GHz. In addition, systematic errors in the forms of temperature-to-polarization leakage, analogue-to-digital conversion uncertainties, and very long time constant errors have been dramatically reduced, to the extent that the cosmological polarization signal may now be robustly recovered on angular scales ℓ ≳ 40. On the very largest scales, instrumental systematic residuals are still non-negligible compared to the expected cosmological signal, and modes with ℓ< 20 are accordingly suppressed in the current polarization maps by high-pass filtering. As in 2013, four different CMB component separation algorithms are applied to these observations, providing a measure of stability with respect to algorithmic and modelling choices. Additionally, the resulting polarization maps have rms instrumental noise ranging between 0.21 and 0.27μK averaged over 55' pixels, and between 4.5 and 6.1μK averaged over 3$'\atop{.}$4 pixels. The cosmological parameters derived from the analysis of temperature power spectra are in agreement at the 1σ level with the Planck 2015 likelihood. Unresolved mismatches between the noise properties of the data and simulations prevent a satisfactory description of the higher-order statistical properties of the polarization maps. Thus, the primary applications of these polarization maps are those that do not require massive simulations for accurate estimation of uncertainties, for instance estimation of cross-spectra and cross-correlations, or stacking analyses. However, the amplitude of primordial non-Gaussianity is consistent with zero within 2σ for all local, equilateral, and orthogonal configurations of the bispectrum, including for polarization E-modes. Moreover, excellent agreement is found regarding the lensing B-mode power spectrum, both internally among the various component separation codes and with the best-fit Planck 2015 Λ cold dark matter model.

79 ASTRONOMY AND ASTROPHYSICS↗

Understanding Calibration and Error Propagation in Longitudinal and Lateral Manganin Gauge Shock Experiments

Soda-lime glass remains elastic under shock pressures up to approximately 6.5 GPa. In this paper, soda lime glass shock experiments are used to understand the calibration and error propagation in longitudinal and lateral manganin gauge experiments. The errors in the calibration process for both longitudinal and lateral manganin gauge experiments are identified and discussed. Five soda-lime glass experiments up to 3 GPa are analyzed using this refined technique. The results show good agreement between the experimental data and the published calibration by Rosenberg and Moshel. Errors are propagated from the experimental data measurements and the assumptions that are made in the calibration, where the bulk of the error seems to come from uncertainty in the dynamic yield strength of manganin. Finally, the method presented in this paper allows for analysis of dynamic shock experiments with longitudinal and lateral manganin gauges including error propagation.

36 MATERIALS SCIENCE↗