Search NASA⌕ Search

SEARCH · Search NASA

Results for “Statistical error”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 1,153 records · Page 64

TESTING TESTS ON ACTIVE GALACTIC NUCLEI MICROVARIABILITY

Literature on optical and infrared microvariability in active galactic nuclei (AGNs) reflects a diversity of statistical tests and strategies to detect tiny variations in the light curves of these sources. Comparison between the results obtained using different methodologies is difficult, and the pros and cons of each statistical method are often badly understood or even ignored. Even worse, improperly tested methodologies are becoming more and more common, and biased results may be misleading with regard to the origin of the AGN microvariability. This paper intends to point future research on AGN microvariability toward the use of powerful and well-tested statistical methodologies, providing a reference for choosing the best strategy to obtain unbiased results. Light curves monitoring has been simulated for quasars and for reference and comparison stars. Changes for the quasar light curves include both Gaussian fluctuations and linear variations. Simulated light curves have been analyzed using {chi}{sup 2} tests, F tests for variances, one-way analyses of variance and C-statistics. Statistical Type I and Type II errors, which indicate the robustness and the power of the tests, have been obtained in each case. One-way analyses of variance and {chi}{sup 2} prove to be powerful and robust estimators for microvariations, while the C-statistic is not a reliable methodology and its use should be avoided.

79 ASTRONOMY AND ASTROPHYSICS↗

Validation of sea ice models using an uncertainty-based distance metric for multiple model variables: NEW METRIC FOR SEA ICE MODEL VALIDATION

Here, we implement a variance-based distance metric (D n ) to objectively assess skill of sea ice models when multiple output variables or uncertainties in both model predictions and observations need to be considered. The metric compares observations and model data pairs on common spatial and temporal grids improving upon highly aggregated metrics (e.g., total sea ice extent or volume) by capturing the spatial character of model skill. The D n metric is a gamma-distributed statistic that is more general than the χ 2 statistic commonly used to assess model fit, which requires the assumption that the model is unbiased and can only incorporate observational error in the analysis. The D n statistic does not assume that the model is unbiased, and allows the incorporation of multiple observational data sets for the same variable and simultaneously for different variables, along with different types of variances that can characterize uncertainties in both observations and the model. This approach represents a step to establish a systematic framework for probabilistic validation of sea ice models. The methodology is also useful for model tuning by using the D n metric as a cost function and incorporating model parametric uncertainty as part of a scheme to optimize model functionality. We apply this approach to evaluate different configurations of the standalone Los Alamos sea ice model (CICE) encompassing the parametric uncertainty in the model, and to find new sets of model configurations that produce better agreement than previous configurations between model and observational estimates of sea ice concentration and thickness.

58 GEOSCIENCES↗

Statistical Process Control and Capability Study on the Water Content Measurements in NASA Glenn's Icing Research Tunnel

The Icing Research Tunnel (IRT) at NASA Glenn Research Center follows the recommended practice for icing tunnel calibration outlined in SAE's ARP5905 document. The calibration team has followed the schedule of a full calibration every five years with a check calibration done every six months following. The liquid water content of the IRT has maintained stability within in the specifications presented to customers that the variation is within +/- 10% of the calibrated, target measurement. With recent measurements and instrumentation errors, a more thorough assessment of error source was desired. By constructing statistical process control charts, the ability to determine how the instrument varies in the short term, mid term, and long term was gained. The control charts offer a view of instrument error, facility error, or installation changes. It was discovered that there was a shift from target to mean baseline thus leading to the study of the overall capability indices of the liquid water content measuring instrument to perform within specifications defined in the IRT. This presentation describes data processing procedures for the Multi-Element Sensor in the IRT, including collision efficiency corrections, canonical correlation analysis, Chauvenet's Criterion for rejection of data, distribution check of data, and mean, median and mode for construction of control charts. Further data is presented to describe the repeatability of the IRT with the Multi-Element Sensor and the ability to maintain a stable process for the defined calibration schedule.

Timko, Emily N.↗

IMFIT: A FAST, FLEXIBLE NEW PROGRAM FOR ASTRONOMICAL IMAGE FITTING

I describe a new, open-source astronomical image-fitting program called IMFIT, specialized for galaxies but potentially useful for other sources, which is fast, flexible, and highly extensible. A key characteristic of the program is an object-oriented design that allows new types of image components (two-dimensional surface-brightness functions) to be easily written and added to the program. Image functions provided with IMFIT include the usual suspects for galaxy decompositions (Sérsic, exponential, Gaussian), along with Core-Sérsic and broken-exponential profiles, elliptical rings, and three components that perform line-of-sight integration through three-dimensional luminosity-density models of disks and rings seen at arbitrary inclinations. Available minimization algorithms include Levenberg-Marquardt, Nelder-Mead simplex, and Differential Evolution, allowing trade-offs between speed and decreased sensitivity to local minima in the fit landscape. Minimization can be done using the standard χ{sup 2} statistic (using either data or model values to estimate per-pixel Gaussian errors, or else user-supplied error images) or Poisson-based maximum-likelihood statistics; the latter approach is particularly appropriate for cases of Poisson data in the low-count regime. I show that fitting low-signal-to-noise ratio galaxy images using χ{sup 2} minimization and individual-pixel Gaussian uncertainties can lead to significant biases in fitted parameter values, which are avoided if a Poisson-based statistic is used; this is true even when Gaussian read noise is present.

79 ASTRONOMY AND ASTROPHYSICS↗

New Approaches to Quantifying Transport Model Error in Atmospheric CO2 Simulations

In recent years, much progress has been made in observing CO2 distributions from space. However, the use of these observations to infer source/sink distributions in inversion studies continues to be complicated by difficulty in quantifying atmospheric transport model errors. We will present results from several different experiments designed to quantify different aspects of transport error using the Goddard Earth Observing System, Version 5 (GEOS-5) Atmospheric General Circulation Model (AGCM). In the first set of experiments, an ensemble of simulations is constructed using perturbations to parameters in the model s moist physics and turbulence parameterizations that control sub-grid scale transport of trace gases. Analysis of the ensemble spread and scales of temporal and spatial variability among the simulations allows insight into how parameterized, small-scale transport processes influence simulated CO2 distributions. In the second set of experiments, atmospheric tracers representing model error are constructed using observation minus analysis statistics from NASA's Modern-Era Retrospective Analysis for Research and Applications (MERRA). The goal of these simulations is to understand how errors in large scale dynamics are distributed, and how they propagate in space and time, affecting trace gas distributions. These simulations will also be compared to results from NASA's Carbon Monitoring System Flux Pilot Project that quantified the impact of uncertainty in satellite constrained CO2 flux estimates on atmospheric mixing ratios to assess the major factors governing uncertainty in global and regional trace gas distributions.

Ott, L.↗

Correction of Dual-PRF Doppler Velocity Outliers in the Presence of Aliasing

Abstract In Doppler weather radars, the presence of unfolding errors or outliers is a well-known quality issue for radial velocity fields estimated using the dual–pulse repetition frequency (PRF) technique. Postprocessing methods have been developed to correct dual-PRF outliers, but these need prior application of a dealiasing algorithm for an adequate correction. This paper presents an alternative procedure based on circular statistics that corrects dual-PRF errors in the presence of extended Nyquist aliasing. The correction potential of the proposed method is quantitatively tested by means of velocity field simulations and is exemplified in the application to real cases, including severe storm events. The comparison with two other existing correction methods indicates an improved performance in the correction of clustered outliers. The technique proposed is well suited for real-time applications requiring high-quality Doppler radar velocity fields, such as wind shear and mesocyclone detection algorithms, or assimilation in numerical weather prediction models.

58 GEOSCIENCES↗

Characterizing Quantum Devices Using Model Selection

Quantum information processors are a new kind of computing device with the potential to solve certain problems very quickly. The quantum bits, or "ʺqubits,"ʺ that enable such processors are being developed at Sandia and around the world. In support of this effort, Sandia’s Center for Computing Research is designing methods and algorithms to characterize and debug the behavior of experimental qubits, such as trapped ions and electron spins in silicon. This is a challenge, because qubits are hard to characterize for exactly the same reason that they are powerful. Their behavior (and misbehavior) is far richer and more complicated than that of classical bits! The most reliable models of quantum errors have infinitely many parameters, and we never have enough data to fit all of them. Until now, researchers simply threw out most of these parameters resulting in a crude approximation that makes the problem tractable, but isn’t reliable or accurate enough for Sandia’s experimental work. Our new techniques use statistical model selection to identify “significant” error parameters on the fly, applying a data-­‐‑driven information criterion to choose a Hilbert space that describes the data well without overfitting. To find a reliable criterion, we needed to evaluate many candidates on an enormous corpus of test cases–for which we relied on Red Sky’s ability to run parallel Monte Carlo simulations. Our computations demonstrate how classic model selection methods have to be modified for quantum devices. These results will enable new techniques that identify the effective dimension of as-­‐‑built qubits on the fly, and then use this information to reliably pin down error behaviors and produce successively better and more useful qubits. This project highlights the close and productive relationship between Sandia and the University of New Mexico, where Travis Scholten is working towards a Ph.D. in physics. Both the intellectual environment of UNM and the high-­‐‑powered supercomputing available at Sandia were critical ingredients in achieving our research goals.

97 MATHEMATICS AND COMPUTING↗

Tunable power law in the desynchronization events of coupled chaotic electronic circuits

We study the statistics of the amplitude of the synchronization error in chaotic electronic circuits coupled through linear feedback. Depending on the coupling strength, our system exhibits three qualitatively different regimes of synchronization: weak coupling yields independent oscillations; moderate to strong coupling produces a regime of intermittent synchronization known as attractor bubbling; and stronger coupling produces complete synchronization. In the regime of moderate coupling, the probability distribution for the sizes of desynchronization events follows a power law, with an exponent that can be adjusted by changing the coupling strength. Such power-law distributions are interesting, as they appear in many complex systems. However, most of the systems with such a behavior have a fixed value for the exponent of the power law, while here we present an example of a system where the exponent of the power law is easily tuned in real time.

71 CLASSICAL AND QUANTUM MECHANICS, GENERAL PHYSIC↗

Multibaseline gravitational wave radiometry

We present a statistic for the detection of stochastic gravitational wave backgrounds (SGWBs) using radiometry with a network of multiple baselines. We also quantitatively compare the sensitivities of existing baselines and their network to SGWBs. We assess how the measurement accuracy of signal parameters, e.g., the sky position of a localized source, can improve when using a network of baselines, as compared to any of the single participating baselines. The search statistic itself is derived from the likelihood ratio of the cross correlation of the data across all possible baselines in a detector network and is optimal in Gaussian noise. Specifically, it is the likelihood ratio maximized over the strength of the SGWB and is called the maximized-likelihood ratio (MLR). One of the main advantages of using the MLR over past search strategies for inferring the presence or absence of a signal is that the former does not require the deconvolution of the cross correlation statistic. Therefore, it does not suffer from errors inherent to the deconvolution procedure and is especially useful for detecting weak sources. In the limit of a single baseline, it reduces to the detection statistic studied by Ballmer [Classical Quantum Gravity 23, S179 (2006).] and Mitra et al.[Phys. Rev. D 77, 042002 (2008).]. Unlike past studies, here the MLR statistic enables us to compare quantitatively the performances of a variety of baselines searching for a SGWB signal in (simulated) data. Although we use simulated noise and SGWB signals for making these comparisons, our method can be straightforwardly applied on real data.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Full-stack Quantification of Variability in Predicting Ion Transport Properties using Machine-learned Interatomic Potentials

Machine-learned interatomic potentials (MLIPs) have become the state-of-the-art for performing accurate, scalable molecular dynamics (MD) simulations. It is therefore crucial to understand and quantify the reliability of MLIPs for downstream property predictions. Uncertainty in predicted properties can arise from limitations in first-principles training data, intrinsic MLIP model errors in representing the data, and the statistical noise introduced during subsequent MD simulations. Using ion transport in Li7P3S11 as a case study, we systematically assess the impact of training set size and selection, neural network stochasticity, and MD sampling statistics on predicted diffusivity and activation energy. We find that when using equivariant MLIP architectures with standard MD protocols, uncertainty arising from MD sampling dominates over model-induced errors. In contrast, MLIP errors relative to the underlying first-principles data are consistently minor. Given this, there are two main routes to improving the accuracy of predictions based on MLIP potentials: adopting higher accuracy reference data generation methods, and improving the MD sampling statistics.

36 MATERIALS SCIENCE↗

Modeling energy balance while correcting for measurement error via free knot splines

Measurements of energy balance components (energy intake, energy expenditure, changes in energy stores) are often plagued with measurement error. Doubly-labeled water can measure energy intake (EI) with negligible error, but is expensive and cumbersome. An alternative approach that is gaining popularity is to use the energy balance principle, by measuring energy expenditure (EE) and change in energy stores (ES) and then back-calculate EI. Gold standard methods for EE and ES exist and are known to give accurate measurements, albeit at a high cost. We propose a joint statistical model to assess the measurement error in cheaper, non-intrusive measures of EE and ES. We let the unknown true EE and ES for individuals be latent variables, and model them using a bivariate distribution. We try both a bivariate Normal as well as a Dirichlet Process Mixture Model, and compare the results via simulation. Our approach, is the first to account for the dependencies that exist in individuals’ daily EE and ES. We employ semiparametric regression with free knot splines for measurements with error, and linear components for error free covariates. We adopt a Bayesian approach to estimation and inference and use Reversible Jump Markov Chain Monte Carlo to generate draws from the posterior distribution. Based on the semiparameteric regression, we develop a calibration equation that adjusts a cheaper, less reliable estimate, closer to the true value. Along with this calibrated value, our method also gives credible intervals to assess uncertainty. A simulation study shows our calibration helps produce a more accurate estimate. Furthermore, our approach compares favorably in terms of prediction to other commonly used models.

60 APPLIED LIFE SCIENCES↗

Design study for future satellite microwave scatterometers, part 3

A computerized simulation analysis for a number of scatterometer antenna configuration and polarization modes including the Seasat scatterometer (SASS) is presented. The results of the simulations were expressed in terms of performance statistics. These statistics relate to the wind direction alias removal capability and to the rms sensing errors for friction velocity and wind direction X. The statistics are analyzed, and optimum scatterometer configurations are recommended. The accuracy of the SASS in measuring U* and X, and its capability to resolve wind direction aliases are assessed.

Wentz, F. J.↗

Per-beam, planar IMRT QA passing rates do not predict clinically relevant patient dose errors

Purpose: The purpose of this work is to determine the statistical correlation between per-beam, planar IMRT QA passing rates and several clinically relevant, anatomy-based dose errors for per-patient IMRT QA. The intent is to assess the predictive power of a common conventional IMRT QA performance metric, the Gamma passing rate per beam. Methods: Ninety-six unique data sets were created by inducing four types of dose errors in 24 clinical head and neck IMRT plans, each planned with 6 MV Varian 120-leaf MLC linear accelerators using a commercial treatment planning system and step-and-shoot delivery. The error-free beams/plans were used as ''simulated measurements'' (for generating the IMRT QA dose planes and the anatomy dose metrics) to compare to the corresponding data calculated by the error-induced plans. The degree of the induced errors was tuned to mimic IMRT QA passing rates that are commonly achieved using conventional methods. Results: Analysis of clinical metrics (parotid mean doses, spinal cord max and D1cc, CTV D95, and larynx mean) vs IMRT QA Gamma analysis (3%/3 mm, 2/2, 1/1) showed that in all cases, there were only weak to moderate correlations (range of Pearson's r-values: -0.295 to 0.653). Moreover, the moderate correlations actually had positive Pearson's r-values (i.e., clinically relevant metric differences increased with increasing IMRT QA passing rate), indicating that some of the largest anatomy-based dose differences occurred in the cases of high IMRT QA passing rates, which may be called ''false negatives.'' The results also show numerous instances of false positives or cases where low IMRT QA passing rates do not imply large errors in anatomy dose metrics. In none of the cases was there correlation consistent with high predictive power of planar IMRT passing rates, i.e., in none of the cases did high IMRT QA Gamma passing rates predict low errors in anatomy dose metrics or vice versa. Conclusions: There is a lack of correlation between conventional IMRT QA performance metrics (Gamma passing rates) and dose errors in anatomic regions-of-interest. The most common acceptance criteria and published actions levels therefore have insufficient, or at least unproven, predictive power for per-patient IMRT QA.

60 APPLIED LIFE SCIENCES↗

Inversion of data from diffraction-limited multiwavelength remote sensors. I - Linear case

The remote sensing inverse problem is considered in which the sensor does not necessarily view the same area on the earth at each wavelength, and spatial correlations of the geophysical parameters may be present. Under the conditions of linearity and stationary statistics, the minimum mean-square error solution to the problem of inverting such data is a spatial filter of the Wiener-Kolmogorov class. The resulting remote-sensing system can be characterized by an impulse response matrix in ordinary space or by a transfer matrix in frequency space. A signal-to-noise matrix for the geophysical parameters to be sensed is also defined; this matrix depends on the postulated a priori statistics and on the characteristics of the remote-sensing instrument. The system transfer matrix and the signal-to-noise matrix are simultaneously diagonalizable. The optimum transfer matrix filters out of the estimate vector those eigenparameters for which eigenvalues of the signal-to-noise matrix are less than unity.

Rosenkranz, P. W.↗

The effects of long delay and transmission errors on the performance of TP-4 implementations

A set of tools that allows us to measure and examine the effects of transmission delay and errors on the performance of TP-4 implementations has been developed. The tools give insight into both the large- and small-scale behaviors of an implementation. These tools have been systematically applied to a commercial implementation of TP-4. Measurements show, among other things, that a 2-second one-way transmission delay and an effective bit-error rate of 1 error per 100,000 bits can result in a 95 percent reduction in TP-4 throughput. The detailed statistics give insight into why transmission delay and errors affect this implementations so significantly and support a number of 'lessons learned' that could be applied to TP-4 implementations that operate more robustly across networks with long transmission delays and transmission errors.

Durst, Robert C.↗

Quantitative validation of carbon-fiber laminate low velocity impact simulations

Simulations of low velocity impact with a flat cylindrical indenter upon a carbon fiber fabric reinforced polymer laminate are rigorously validated. Comparison of the impact energy absorption between the model and experiment is used as the validation metric. Additionally, non-destructive evaluation, including ultrasonic scans and three-dimensional computed tomography, provide qualitative validation of the models. The simulations include delamination, matrix cracks and fiber breaks. An orthotropic damage and failure constitutive model, capable of predicting progressive damage and failure, is developed in conjunction and described. An ensemble of simulations incorporating model parameter uncertainties is used to predict a response distribution which is then compared to experimental output using appropriate statistical methods. Lastly, the model form errors are exposed and corrected for use in an additional blind validation analysis. The result is a quantifiable confidence in material characterization and model physics when simulating low velocity impact in structures of interest.

36 MATERIALS SCIENCE↗