Search NASA⌕ Search

SEARCH · Search NASA

Results for “Statistical error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,009 records · Page 56

Bayesian Model Averaging for Ensemble-Based Estimates of Solvation Free Energies

This paper applies the Bayesian Model Averaging (BMA) statistical ensemble technique to estimate small molecule solvation free energies. There is a wide range methods for predicting solvation free energies, ranging from empirical statistical models to {\it ab initio} quantum mechanical approaches. Each of these methods are based on a set of conceptual assumptions that can affect a method's predictive accuracy and transferability. Using an iterative statistical process, we have selected and combined solvation energy estimates using an ensemble of 17 diverse methods from the SAMPL4 blind prediction study to form a single, aggregated solvation energy estimate. The ensemble design process evaluates the statistical information in each individual method as well as the performance of the aggregate estimate obtained from the ensemble as a whole. Methods that possess minimal or redundant information are pruned from the ensemble and the evaluation process repeats until aggregate predictive performance can no longer be improved. We show that this process results in a final aggregate estimate that outperforms all individual methods by reducing estimate errors by as much as 91% to 1.2 kcal/mol accuracy. We also compare our iterative refinement approach to other statistical ensemble approaches and demonstrate that this iterative process reduces estimate errors by as much as 61%. This work provides a new approach for accurate solvation free energy prediction and lays the foundation for future work on aggregate models that can balance computational cost with predictive accuracy.

solvation, Bayesian model averaging, Electrostatic↗

Cosmology intertwined: A review of the particle physics, astrophysics, and cosmology associated with the cosmological tensions and anomalies

The standard Λ Cold Dark Matter (ΛCDM) cosmological model provides a good description of a wide range of astrophysical and cosmological data. However, there are a few big open questions that make the standard model look like an approximation to a more realistic scenario yet to be found. In this paper, we list a few important goals that need to be addressed in the next decade, taking into account the current discordances between the different cosmological probes, such as the disagreement in the value of the Hubble constant H 0 , the σ 8 – S 8 tension, and other less statistically significant anomalies. While these discordances can still be in part the result of systematic errors, their persistence after several years of accurate analysis strongly hints at cracks in the standard cosmological scenario and the necessity for new physics or generalisations beyond the standard model. In this paper, we focus on the 5.0 σ tension between the Planck CMB estimate of the Hubble constant H 0 and the SH0ES collaboration measurements. After showing the H 0 evaluations made from different teams using different methods and geometric calibrations, we list a few interesting new physics models that could alleviate this tension and discuss how the next decade's experiments will be crucial. Moreover, we focus on the tension of the Planck CMB data with weak lensing measurements and redshift surveys, about the value of the matter energy density Ω m , and the amplitude or rate of the growth of structure ( σ 8 , f σ 8 ). We list a few interesting models proposed for alleviating this tension, and we discuss the importance of trying to fit a full array of data with a single model and not just one parameter at a time. Additionally, we present a wide range of other less discussed anomalies at a statistical significance level lower than the H 0 – S 8 tensions which may also constitute hints towards new physics, and we discuss possible generic theoretical approaches that can collectively explain the non-standard nature of these signals. Finally, we give an overview of upgraded experiments and next-generation space missions and facilities on Earth that will be of crucial importance to address all these open questions.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Practical and reliable error bars for quantum process tomography

Current techniques in quantum process tomography typically return a single point estimate of an unknown process based on a finite albeit large amount of measurement data. Due to statistical fluctuations, however, other processes close to the point estimate can also produce the observed data with near certainty. Unless appropriate error bars can be constructed, the point estimate does not carry any sound operational interpretation. Here, we provide a solution to this problem by constructing a confidence region estimator for quantum processes. Our method enables reliable estimation of essentially any figure of merit for quantum processes on few qubits, including the diamond distance to a specific noise model, the entanglement fidelity, and the worst-case entanglement fidelity, by identifying error regions which contain the true state with high probability. We also provide a software package, QPtomographer, implementing our estimator for the diamond norm and the worst-case entanglement fidelity. We illustrate its usage and performance with several simulated examples. Furthermore, our tools can be used to reliably certify the performance of, e.g., error correction codes, implementations of unitary gates, or more generally any noise process affecting a quantum system.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Possible Improvements of the ACE Diversity Interchange Methodology

North American Electric Reliability Corporation (NERC) grid is operated by about 131 balancing authorities (BA). Within each BA, operators are responsible for managing the unbalance (caused by both load and wind). As wind penetration levels increase, the challenges of managing power variation increases. Working independently, balancing area with limited regulating/load following generation and high wind power penetration faces significant challenges. The benefits of BA cooperation and consolidation increase when there is a significant wind energy penetration. To explore the benefits of BA cooperation, this paper investigates ACE sharing approach. A technology called ACE diversity interchange (ADI) is already in use in the western interconnection. A new methodology extending ADI is proposed in the paper. The proposed advanced ADI overcoming some limitations existing in conventional ADI. Simulations using real statistical data of CAISO and BPA have shown high performance of the proposed advanced ADI methodology.

17 WIND ENERGY↗

SU-F-T-252: An Investigation of Gamma Knife Frame Definition Error When Using a Pre-Planning Workflow

Purpose: To determine causal factors related to high frame definition error when treating GK patients using a pre-planning workflow. Methods: 160 cases were retrospectively reviewed. All patients received treatment using a pre-planning workflow whereby stereotactic coordinates are determined from a CT scan acquired after framing using a fiducial box. The planning software automatically detects the fiducials and compares their location to expected values based on the rigid design of the fiducial system. Any difference is reported as mean and maximum frame definition error. The manufacturer recommends these values be less than 1.0 mm and 1.5 mm. In this study, frame definition error was analyzed in comparison with a variety of factors including which neurosurgeon/oncologist/physicist was involved with the procedure, number of post used during framing (3 or 4), type of lesion, and which CT scanner was utilized for acquisition. An analysis of variance (ANOVA) approach was used to statistically evaluate the data and determine causal factors related to instances of high frame definition error. Results: Two factors were identified as significant: number of post (p=0.0003) and CT scanner (p=0.0001). Further analysis showed that one of the four scanners was significantly different than the others. This diagnostic scanner was identified as an older model with localization lasers not tightly calibrated. The average value for maximum frame definition error using this scanner was 1.48 mm (4 posts) and 1.75 mm (3 posts). For the other scanners this value was 1.13 mm (4 posts) and 1.40 mm (3 posts). Conclusion: In utilizing a pre-planning workflow the choice of CT scanner matters. Any scanner utilized for GK should undergo routine QA at a level appropriate for radiation oncology. In terms of 3 vs 4 post, it is hypothesized that three posts provide less stability during CT acquisition. This will be tested in future work.

60 APPLIED LIFE SCIENCES↗

Monte Carlo MP2-F12 for Noncovalent Interactions: The C 60 Dimer

A scalable stochastic algorithm is presented that can evaluate explicitly correlated (F12) second-order many-body perturbation (MP2) energies of weak, noncovalent, intermolecular interactions. It first transforms the formulas of the MP2 and F12 energy differences into a short sum of high-dimensional integrals of Green’s functions in real space and imaginary time. Furthermore, these integrals are then evaluated by the Monte Carlo method augmented by parallel execution, redundant-walker convergence acceleration, direct-sampling autocorrelation elimination, and control-variate error reduction. By sharing electron-pair walkers across the supermolecule and its subsystems spanned by the joint basis set, the statistical uncertainty is reduced by one to 2 orders of magnitude in the MP2 binding energy corrected for the basis-set incompleteness and superposition errors. The method predicts the MP2-F12/aug-cc-pVDZ binding energy of 19.1 ± 4.0 kcal mol –1 for the C 60 dimer at the center distance of 9.748 Å.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Balancing aggregation and smoothing errors in inverse models

Inverse models use observations of a system (observation vector) to quantify the variables driving that system (state vector) by statistical optimization. When the observation vector is large, such as with satellite data, selecting a suitable dimension for the state vector is a challenge. A state vector that is too large cannot be effectively constrained by the observations, leading to smoothing error. However, reducing the dimension of the state vector leads to aggregation error as prior relationships between state vector elements are imposed rather than optimized. Here we present a method for quantifying aggregation and smoothing errors as a function of state vector dimension, so that a suitable dimension can be selected by minimizing the combined error. Reducing the state vector within the aggregation error constraints can have the added advantage of enabling analytical solution to the inverse problem with full error characterization. We compare three methods for reducing the dimension of the state vector from its native resolution: (1) merging adjacent elements (grid coarsening), (2) clustering with principal component analysis (PCA), and (3) applying a Gaussian mixture model (GMM) with Gaussian pdfs as state vector elements on which the native-resolution state vector elements are projected using radial basis functions (RBFs). The GMM method leads to somewhat lower aggregation error than the other methods, but more importantly it retains resolution of major local features in the state vector while smoothing weak and broad features.

Turner, A. J.↗

Tracking Climate Change through the Spatiotemporal Dynamics of the Teletherms, the Statistically Hottest and Coldest Days of the Year

Instabilities and long term shifts in seasons, whether induced by natural drivers or human activities, pose great disruptive threats to ecological, agricultural, and social systems. Here, we propose, measure, and explore two fundamental markers of location-sensitive seasonal variations: the Summer and Winter Teletherms—the on-average annual dates of the hottest and coldest days of the year. We analyze daily temperature extremes recorded at 1218 stations across the contiguous United States from 1853–2012, and observe large regional variation with the Summer Teletherm falling up to 90 days after the Summer Solstice, and 50 days for the Winter Teletherm after the Winter Solstice. We show that Teletherm temporal dynamics are substantive with clear and in some cases dramatic shifts reflective of system bifurcations. We also compare recorded daily temperature extremes with output from two regional climate models finding considerable though relatively unbiased error. In conclusion, our work demonstrates that Teletherms are an intuitive, powerful, and statistically sound measure of local climate change, and that they pose detailed, stringent challenges for future theoretical and computational models.

54 ENVIRONMENTAL SCIENCES↗

Uncertainty quantification for nuclear density functional theory and information content of new measurements

Statistical tools of uncertainty quantification can be used to assess the information content of measured observables with respect to present-day theoretical models, to estimate model errors and thereby improve predictive capability, to extrapolate beyond the regions reached by experiment, and to provide meaningful input to applications and planned measurements. To showcase new opportunities offered by such tools, we make a rigorous analysis of theoretical statistical uncertainties in nuclear density functional theory using Bayesian inference methods. By considering the recent mass measurements from the Canadian Penning Trap at Argonne National Laboratory, we demonstrate how the Bayesian analysis and a direct least-squares optimization, combined with high-performance computing, can be used to assess the information content of the new data with respect to a model based on the Skyrme energy density functional approach. Employing the posterior probability distribution computed with a Gaussian process emulator, we apply the Bayesian framework to propagate theoretical statistical uncertainties in predictions of nuclear masses, two-neutron dripline, and fission barriers. Overall, we find that the new mass measurements do not impose a constraint that is strong enough to lead to significant changes in the model parameters. In addition, the example discussed in this study sets the stage for quantifying and maximizing the impact of new measurements with respect to current modeling and guiding future experimental efforts, thus enhancing the experiment-theory cycle in the scientific method.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

A digital twin for rapid qualification of 3D printed metallic components

The customized production of complex components by 3D printing has been hailed as a potentially transformative tool in manufacturing with important applications in health care, automotive and aerospace industries. However, after about a quarter of a century of research and development, only a handful of commercial alloys can be printed and the market value of all 3D printed products now amounts to a negligible portion of the manufacturing economy. This difficulty is attributable to a remarkable diversity in structure and properties of the printed components and susceptibility to defects. In addition, the current practice of qualifying components by prolonged trial and error with expensive printing equipment and feed stock material confine the printed products to a niche market where the high product cost and the delay in the qualification are not critical factors. In this report we explain how a digital twin or a digital replica of the printing machine will reduce the number of trial and error tests to obtain desired product attributes and reduce the time required for part qualification to make the printed components cost effective. It is shown that a comprehensive digital twin of 3D printing machine consisting of mechanistic, control and statistical models of 3D printing, machine learning and big data can reduce the volume of trial and error testing, reduce defects and shorten time between the design and production.

36 MATERIALS SCIENCE↗

A neural network for real-time retrievals of PWV and LWP from Arctic millimeter-wave ground-based observations.

This paper presents a new neural network (NN) algorithm for real-time retrievals of low amounts of precipitable water vapor (PWV) and integrated liquid water from millimeter-wave ground-based observations. Measurements are collected by the 183.3-GHz G-band vapor radiometer (GVR) operating at the Atmospheric Radiation Measurement (ARM) Program Climate Research Facility, Barrow, AK. The NN provides the means to explore the nonlinear regime of the measurements and investigate the physical boundaries of the operability of the instrument. A methodology to compute individual error bars associated with the NN output is developed, and a detailed error analysis of the network output is provided. Through the error analysis, it is possible to isolate several components contributing to the overall retrieval errors and to analyze the dependence of the errors on the inputs. The network outputs and associated errors are then compared with results from a physical retrieval and with the ARM two-channel microwave radiometer (MWR) statistical retrieval. When the NN is trained with a seasonal training data set, the retrievals of water vapor yield results that are comparable to those obtained from a traditional physical retrieval, with a retrieval error percentage of {approx}5% when the PWV is between 2 and 10 mm, but with the advantages that the NN algorithm does not require vertical profiles of temperature and humidity as input and is significantly faster computationally. Liquid water path (LWP) retrievals from the NN have a significantly improved clear-sky bias (mean of {approx}2.4 g/m{sup 2}) and a retrieval error varying from 1 to about 10 g/m{sup 2} when the PWV amount is between 1 and 10 mm. As an independent validation of the LWP retrieval, the longwave downwelling surface flux was computed and compared with observations. The comparison shows a significant improvement with respect to the MWR statistical retrievals, particularly for LWP amounts of less than 60 g/m{sup 2}.

47 OTHER INSTRUMENTATION↗

On the accuracy of Hipparcos using binary stars as a calibration tool

Stellar binary systems, specifically those that present the most accurate available orbital elements, are a reliable tool to test the accuracy of astrometric observations. We selected all 35 binaries with these characteristics. Our objective is to provide standard uncertainties for the positions and parallaxes measured by Hipparcos relative to this trustworthy set, as well as to check supposed correlations between several parameters (measurement residuals, positions, magnitudes, and parallaxes). In addition, using the high-confidence subset of visual–spectroscopic binaries, we implemented a validation test of the Hipparcos trigonometric parallaxes of binary systems that allowed the evaluation of their reliability. Standard and non-standard statistical analysis techniques were applied in order to achieve well-founded conclusions. In particular, errors-in-variables models such as the total least-squares method were used to validate Hipparcos parallaxes by comparison with those obtained directly from the orbital elements. Previously, we executed Thompson's τ technique in order to detect suspected outliers in the data. Furthermore, several statistical hypothesis tests were carried out to verify if our results were statistically significant. A statistically significant trend indicating larger Hipparcos angular separations with respect to the reference values in 5.2 ± 1.4 mas was found at the 10{sup −8} significance level. Uncertainties in the polar coordinates θ and ρ of 1.°8 and 6.3 mas, respectively, were estimated for the Hipparcos observations of binary systems. We also verified that the parallaxes of binary systems measured in this mission are absolutely compatible with the set of orbital parallaxes obtained from the most accurate orbits at least at the 95% confidence level. This methodology allows us to better estimate the accuracy of Hipparcos observations of binary systems. Indeed, further application to the data collected by Gaia should yield a standard procedure to compare both data sets.

79 ASTRONOMY AND ASTROPHYSICS↗

Confirmation of standard error analysis techniques applied to EXAFS using simulations

Systematic uncertainties, such as those in calculated backscattering amplitudes, crystal glitches, etc., not only limit the ultimate accuracy of the EXAFS technique, but also affect the covariance matrix representation of real parameter errors in typical fitting routines. Despite major advances in EXAFS analysis and in understanding all potential uncertainties, these methods are not routinely applied by all EXAFS users. Consequently, reported parameter errors are not reliable in many EXAFS studies in the literature. This situation has made many EXAFS practitioners leery of conventional error analysis applied to EXAFS data. However, conventional error analysis, if properly applied, can teach us more about our data, and even about the power and limitations of the EXAFS technique. Here, we describe the proper application of conventional error analysis to r-space fitting to EXAFS data. Using simulations, we demonstrate the veracity of this analysis by, for instance, showing that the number of independent dat a points from Stern's rule is balanced by the degrees of freedom obtained from a 2 statistical analysis. By applying such analysis to real data, we determine the quantitative effect of systematic errors. In short, this study is intended to remind the EXAFS community about the role of fundamental noise distributions in interpreting our final results.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

Ensuring statistical reproducibility of ocean model simulations in the age of hybrid computing

Novel high performance computing systems that feature hybrid architectures require large scale code refactoring to unravel underlying exploitable parallelism. Such redesign can often be accompanied with machine-precision changes as the order of computation cannot always be maintained. For chaotic systems like climate models, these round-off level differences can grow rapidly. Systematic errors may also manifest initially as machine-precision differences. Isolating genuine round off level differences from such errors remains a challenge. Here, we apply two-sample equality of distribution tests to evaluate statistical reproducibility of the ocean model component of US Department of Energy's Energy Exascale Earth System Model (E3SM). A 2-year control simulation ensemble is compared to a modified ensemble as a test case - after a known non-bit-for-bit change in a model component is introduced - to evaluate the null hypothesis that the two ensembles are statistically indistinguishable. To quantify the false negative rates of these tests, we conduct a formal power analysis using a targeted suite of short simulation ensembles. The ensemble suite contains several perturbed ensembles, each with a progressively different climate than the baseline ensemble - obtained by perturbing the magnitude of a single model tuning parameter, the Gent and McWilliams κ, in a controlled manner. The null hypothesis is evaluated for each of perturbed ensembles using these tests. The power analysis informs on the detection limits of the tests for given ensemble size allowing model developers to evaluate the impact of an introduced non-bit-for-bit change to the model.

Mahajan, Salil↗

IS QUASAR OPTICAL VARIABILITY A DAMPED RANDOM WALK?

The damped random walk (DRW) model is increasingly used to model the variability in quasar optical light curves, but it is still uncertain whether the DRW model provides an adequate description of quasar optical variability across all timescales. Using a sample of OGLE quasar light curves, we consider four modifications to the DRW model by introducing additional parameters into the covariance function to search for deviations from the DRW model on both short and long timescales. We find good agreement with the DRW model on timescales that are well sampled by the data (from a month to a few years), possibly with some intrinsic scatter in the additional parameters, but this conclusion depends on the statistical test employed and is sensitive to whether the estimates of the photometric errors are correct to within {approx}10%. On very short timescales (below a few months), we see some evidence of the existence of a cutoff below which the correlation is stronger than the DRW model, echoing the recent finding of Mushotzky et al. using quasar light curves from Kepler. On very long timescales (>a few years), the light curves do not constrain models well, but are consistent with the DRW model.

79 ASTRONOMY AND ASTROPHYSICS↗

Statistically Appraising Process Quality of Affinity Isolation Experiments

Quality affinity isolation experiments are necessary to identify valid protein-protein interactions. Biological error, processing error, and random variability can reduce the quality of an experiment, and thus hinder the identification of protein interaction pairs. Appraising affinity isolation assay quality is essential to inferring protein associations. An important step of the assay is the mass spectrometric identification of proteins. To evaluate this step, a known mixture of proteins is processed through a mass spectrometer as a quality control mixture. If the mass spectrometer yields unexpected results, the process is currently qualitatively evaluated, tuned, and reset. Statistical quality control (SQC) procedures, including the use of cumulative sum, the individual measurement, and moving range charts are implemented to analyze the stability of the mass spectrometric analysis. The SQC measures presented here can assist in establishing preliminary control limits to identify an out-of-control process and investigate assignable causes for shifts in the process mean in real-time.

59 BASIC BIOLOGICAL SCIENCES↗

A CRITICAL ASSESSMENT OF PHOTOMETRIC REDSHIFT METHODS: A CANDELS INVESTIGATION

We present results from the Cosmic Assembly Near-infrared Deep Extragalactic Legacy Survey (CANDELS) photometric redshift methods investigation. In this investigation, the results from 11 participants, each using a different combination of photometric redshift code, template spectral energy distributions (SEDs), and priors, are used to examine the properties of photometric redshifts applied to deep fields with broadband multi-wavelength coverage. The photometry used includes U-band through mid-infrared filters and was derived using the TFIT method. Comparing the results, we find that there is no particular code or set of template SEDs that results in significantly better photometric redshifts compared to others. However, we find that codes producing the lowest scatter and outlier fraction utilize a training sample to optimize photometric redshifts by adding zero-point offsets, template adjusting, or adding extra smoothing errors. These results therefore stress the importance of the training procedure. We find a strong dependence of the photometric redshift accuracy on the signal-to-noise ratio of the photometry. On the other hand, we find a weak dependence of the photometric redshift scatter with redshift and galaxy color. We find that most photometric redshift codes quote redshift errors (e.g., 68% confidence intervals) that are too small compared to that expected from the spectroscopic control sample. We find that all codes show a statistically significant bias in the photometric redshifts. However, the bias is in all cases smaller than the scatter; the latter therefore dominates the errors. Finally, we find that combining results from multiple codes significantly decreases the photometric redshift scatter and outlier fraction. We discuss different ways of combining data to produce accurate photometric redshifts and error estimates.

79 ASTRONOMY AND ASTROPHYSICS↗

Impact of genotype‐calling methodologies on genome‐wide association and genomic prediction in polyploids

Abstract Discovery and analysis of genetic variants underlying agriculturally important traits are key to molecular breeding of crops. Reduced representation approaches have provided cost‐efficient genotyping using next‐generation sequencing. However, accurate genotype calling from next‐generation sequencing data is challenging, particularly in polyploid species due to their genome complexity. Recently developed Bayesian statistical methods implemented in available software packages, polyRAD, EBG, and updog, incorporate error rates and population parameters to accurately estimate allelic dosage across any ploidy. We used empirical and simulated data to evaluate the three Bayesian algorithms and demonstrated their impact on the power of genome‐wide association study (GWAS) analysis and the accuracy of genomic prediction. We further incorporated uncertainty in allelic dosage estimation by testing continuous genotype calls and comparing their performance to discrete genotypes in GWAS and genomic prediction. We tested the genotype‐calling methods using data from two autotetraploid species, Miscanthus sacchariflorus and Vaccinium corymbosum , and performed GWAS and genomic prediction. In the empirical study, the tested Bayesian genotype‐calling algorithms differed in their downstream effects on GWAS and genomic prediction, with some showing advantages over others. Through subsequent simulation studies, we observed that at low read depth, polyRAD was advantageous in its effect on GWAS power and limit of false positives. Additionally, we found that continuous genotypes increased the accuracy of genomic prediction, by reducing genotyping error, particularly at low sequencing depth. Our results indicate that by using the Bayesian algorithm implemented in polyRAD and continuous genotypes, we can accurately and cost‐efficiently implement GWAS and genomic prediction in polyploid crops.

59 BASIC BIOLOGICAL SCIENCES↗