Search NASA⌕ Search

SEARCH · Search NASA

Results for “outliers”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS2000

An Adaptive Buddy Check for Observational Quality Control

An adaptive buddy check algorithm is presented that adjusts tolerances for outlier observations based on the variability of surrounding data. The algorithm derives from a statistical hypothesis test combined with maximum-likelihood covariance estimation. Its stability is shown to depend on the initial identification of outliers by a simple background check. The adaptive feature ensures that the final quality control decisions are not very sensitive to prescribed statistics of first-guess and observation errors, nor on other approximations introduced into the algorithm. The implementation of the algorithm in a global atmospheric data assimilation is described. Its performance is contrasted with that of a non-adaptive buddy check, for the surface analysis of an extreme storm that took place in Europe on 27 December 1999. The adaptive algorithm allowed the inclusion of many important observations that differed greatly from the first guess and that would have been excluded on the basis of prescribed statistics. The analysis of the storm development was much improved as a result of these additional observations.

Dee, Dick P.↗
NASA NTRS2003

Simultaneous Ocean Wave Measurements by the JASON and TOPEX Satellites, With Buoy and Model Comparisons

The verification phase of the Jason-1 satellite altimeter mission presents a unique opportunity for comparing near-simultaneous, independent satellite measurements. We here examine simultaneous significant wave height measurements by the Jason-1 and Topex/Poseidon altimeters. These data are also compared with in-situ measurements from deep-ocean buoys and with predicted wave heights from the WaveWatch 111 operational model. The rms difference between Jason and Topex wave heights is 28 cm, and this can be lowered by half through improved outlier editing and filtering of high-frequency noise. Noise is slightly larger in the Jason dataset, exceeding Topex by about 7 cm rms at frequencies above 0.05 Hz, which is the frequency at which the coherence between Topex and Jason measurements drops to zero. Jason wave heights are more prone to outliers, especially during periods of moderate to high backscatter. Buoy comparisons confirm previous reports that Topex wave heights are roughly 5% smaller than buoy measurements for waves between 2 and 5m; Jason heights in general are 3% smaller than Topex. Spurious dips in the Topex density function for 3- and 6-meter waves, a problem that has existed since the beginning of the mission, can be solved by waveform retracking.

Ray, R. D.↗
NASA NTRS2003

Comparison of mode estimation methods and application in molecular clock analysis

BACKGROUND: Distributions of time estimates in molecular clock studies are sometimes skewed or contain outliers. In those cases, the mode is a better estimator of the overall time of divergence than the mean or median. However, different methods are available for estimating the mode. We compared these methods in simulations to determine their strengths and weaknesses and further assessed their performance when applied to real data sets from a molecular clock study. RESULTS: We found that the half-range mode and robust parametric mode methods have a lower bias than other mode methods under a diversity of conditions. However, the half-range mode suffers from a relatively high variance and the robust parametric mode is more susceptible to bias by outliers. We determined that bootstrapping reduces the variance of both mode estimators. Application of the different methods to real data sets yielded results that were concordant with the simulations. CONCLUSION: Because the half-range mode is a simple and fast method, and produced less bias overall in our simulations, we recommend the bootstrapped version of it as a general-purpose mode estimator and suggest a bootstrap method for obtaining the standard error and 95% confidence interval of the mode.

Evolution, Molecular↗
NASA NTRS2008

On the Period-Amplitude and Amplitude-Period Relationships

Examined are Period-Amplitude and Amplitude-Period relationships based on the cyclic behavior of the 12-month moving averages of monthly mean sunspot numbers for cycles 0.23, both in terms of Fisher's exact tests for 2x2 contingency tables and linear regression analyses. Concerning the Period-Amplitude relationship (same cycle), because cycle 23's maximum amplitude is known to be 120.8, the inferred regressions (90-percent prediction intervals) suggest that its period will be 131 +/- 24 months (using all cycles) or 131 +/- 18 months (ignoring cycles 2 and 4, which have the extremes of period, 108 and 164 months, respectively). Because cycle 23 has already persisted for 142 months (May 1996 through February 2008), based on the latter prediction, it should end before September 2008. Concerning the Amplitude-Period relationship (following cycle maximum amplitude versus preceding cycle period), because cycle 23's period is known to be at least 142 months, the inferred regressions (90-percent prediction intervals) suggest that cycle 24's maximum amplitude will be about less than or equal to 96.1 +/- 55.0 (using all cycle pairs) or less than or equal to 91.0 +/- 36.7 (ignoring statistical outlier cycle pairs). Hence, cycle 24's maximum amplitude is expected to be less than 151, perhaps even less than 128, unless cycle pair 23/24 proves to be a statistical outlier.

Wilson, Robert M.↗
NASA NTRS2012

Improved Cloud and Snow Screening in MAIAC Aerosol Retrievals Using Spectral and Spatial Analysis

An improved cloud/snow screening technique in the Multi-Angle Implementation of Atmospheric Correction (MAIAC) algorithm is described. It is implemented as part of MAIAC aerosol retrievals based on analysis of spectral residuals and spatial variability. Comparisons with AERONET aerosol observations and a large-scale MODIS data analysis show strong suppression of aerosol optical thickness outliers due to unresolved clouds and snow. At the same time, the developed filter does not reduce the aerosol retrieval capability at high 1 km resolution in strongly inhomogeneous environments, such as near centers of the active fires. Despite significant improvement, the optical depth outliers in high spatial resolution data are and will remain the problem to be addressed by the application-dependent specialized filtering techniques.

Lyapustin, A.↗
NASA NTRS2012

An Automated Algorithm to Screen Massive Training Samples for a Global Impervious Surface Classification

An algorithm is developed to automatically screen the outliers from massive training samples for Global Land Survey - Imperviousness Mapping Project (GLS-IMP). GLS-IMP is to produce a global 30 m spatial resolution impervious cover data set for years 2000 and 2010 based on the Landsat Global Land Survey (GLS) data set. This unprecedented high resolution impervious cover data set is not only significant to the urbanization studies but also desired by the global carbon, hydrology, and energy balance researches. A supervised classification method, regression tree, is applied in this project. A set of accurate training samples is the key to the supervised classifications. Here we developed the global scale training samples from 1 m or so resolution fine resolution satellite data (Quickbird and Worldview2), and then aggregate the fine resolution impervious cover map to 30 m resolution. In order to improve the classification accuracy, the training samples should be screened before used to train the regression tree. It is impossible to manually screen 30 m resolution training samples collected globally. For example, in Europe only, there are 174 training sites. The size of the sites ranges from 4.5 km by 4.5 km to 8.1 km by 3.6 km. The amount training samples are over six millions. Therefore, we develop this automated statistic based algorithm to screen the training samples in two levels: site and scene level. At the site level, all the training samples are divided to 10 groups according to the percentage of the impervious surface within a sample pixel. The samples following in each 10% forms one group. For each group, both univariate and multivariate outliers are detected and removed. Then the screen process escalates to the scene level. A similar screen process but with a looser threshold is applied on the scene level considering the possible variance due to the site difference. We do not perform the screen process across the scenes because the scenes might vary due to the phenology, solar-view geometry, and atmospheric condition etc. factors but not actual landcover difference. Finally, we will compare the classification results from screened and unscreened training samples to assess the improvement achieved by cleaning up the training samples. Keywords:

Tan, Bin↗
NASA NTRS2013

Coherent Uncertainty Analysis of Aerosol Measurements from Multiple Satellite Sensors

Aerosol retrievals from multiple spaceborne sensors, including MODIS (on Terra and Aqua), MISR, OMI, POLDER, CALIOP, and SeaWiFS altogether, a total of 11 different aerosol products were comparatively analyzed using data collocated with ground-based aerosol observations from the Aerosol Robotic Network (AERONET) stations within the Multi-sensor Aerosol Products Sampling System (MAPSS, http://giovanni.gsfc.nasa.gov/mapss/ and http://giovanni.gsfc.nasa.gov/aerostat/). The analysis was performed by comparing quality-screened satellite aerosol optical depth or thickness (AOD or AOT) retrievals during 2006-2010 to available collocated AERONET measurements globally, regionally, and seasonally, and deriving a number of statistical measures of accuracy. We used a robust statistical approach to detect and remove possible outliers in the collocated data that can bias the results of the analysis. Overall, the proportion of outliers in each of the quality-screened AOD products was within 12%. Squared correlation coefficient (R2) values of the satellite AOD retrievals relative to AERONET exceeded 0.6, with R2 for most of the products exceeding 0.7 over land and 0.8 over ocean. Root mean square error (RMSE) values for most of the AOD products were within 0.15 over land and 0.09 over ocean. We have been able to generate global maps showing regions where the different products present advantages over the others, as well as the relative performance of each product over different landcover types. It was observed that while MODIS, MISR, and SeaWiFS provide accurate retrievals over most of the landcover types, multi-angle capabilities make MISR the only sensor to retrieve reliable AOD over barren and snow / ice surfaces. Likewise, active sensing enables CALIOP to retrieve aerosol properties over bright-surface shrublands more accurately than the other sensors, while POLDER, which is the only one of the sensors capable of measuring polarized aerosols, outperforms other sensors in certain smoke-dominated regions, including broadleaf evergreens in Brazil and South-East Asia.

remote sensing↗
NASA NTRS2014

Interval Predictor Models with a Formal Characterization of Uncertainty and Reliability

This paper develops techniques for constructing empirical predictor models based on observations. By contrast to standard models, which yield a single predicted output at each value of the model's inputs, Interval Predictors Models (IPM) yield an interval into which the unobserved output is predicted to fall. The IPMs proposed prescribe the output as an interval valued function of the model's inputs, render a formal description of both the uncertainty in the model's parameters and of the spread in the predicted output. Uncertainty is prescribed as a hyper-rectangular set in the space of model's parameters. The propagation of this set through the empirical model yields a range of outputs of minimal spread containing all (or, depending on the formulation, most) of the observations. Optimization-based strategies for calculating IPMs and eliminating the effects of outliers are proposed. Outliers are identified by evaluating the extent by which they degrade the tightness of the prediction. This evaluation can be carried out while the IPM is calculated. When the data satisfies mild stochastic assumptions, and the optimization program used for calculating the IPM is convex (or, when its solution coincides with the solution to an auxiliary convex program), the model's reliability (that is, the probability that a future observation would be within the predicted range of outputs) can be bounded rigorously by a non-asymptotic formula.

Crespo, Luis G.↗
NASA NTRS2015

Trend Estimates of AERONET-Observed and Model-Simulated AOTs Between 1993 and 2013

Recently, temporal changes in Aerosol Optical Thickness (AOT) have been investigated based on model simulations, satellite and ground-based observations. Most AOT trend studies used monthly or annual arithmetic means that discard details of the generally right-skewed AOT distributions. Potentially, such results can be biased by extreme values (including outliers). This study additionally uses percentiles (i.e., the lowest 5%, 25%, 50%, 75% and 95% of the monthly cumulative distributions fitted to Aerosol Robotic Network (AERONET)-observed and ECHAM/MESSy Atmospheric Chemistry (EMAC)-model simulated AOTs) that are less affected by outliers caused by measurement error, cloud contamination and occasional extreme aerosol events. Since the limited statistical representativeness of monthly percentiles and means can lead to bias, this study adopts the number of observations as a weighting factor, which improves the statistical robustness of trend estimates. By analyzing the aerosol composition of AERONET-observed and EMAC-simulated AOTs in selected regions of interest, we distinguish the dominant aerosol types and investigate the causes of regional AOT trends. The simulated and observed trends are generally consistent with a high correlation coefficient (R = 0.89) and small bias (slope+/-2(sigma) = 0.75 +/- 0.19). A significant decrease in EMAC-decomposed AOTs by water-soluble compounds and black carbon is found over the USA and the EU due to environmental regulation. In particular, a clear reversal in the AERONET AOT trend percentiles is found over the USA, probably related to the AOT diurnal cycle and the frequency of wildfires. In most of the selected regions of interest, EMAC-simulated trends are mainly attributed to the significant changes of the dominant aerosols; e.g., significant decrease in sea salt and water soluble compounds over Central America, increase in dust over Northern Africa and Middle East, and decrease in black carbon and organic carbon over Australia.

AERONET↗
NASA NTRSDate not supplied

TPSAS-NF1676L-17940-DND

The NASA/GEWEX SRB (Global Energy and Water Exchanges, Surface Radiation Budget) project produces and archives shortwave and longwave radiation budget flux estimates at the top of the atmosphere and at the Earth's surface. The latest version in the archive, Release 3.0, is available as 3-hourly, 3-hourly-monthly, daily and monthly means continuously over the period from July 1983 to December 2007 on a quasi-equal-area grid system of 44016 grid boxes. SRB Release 4 with further improvements in data quality and higher spatial resolution is being developed. The SRB shortwave/longwave fluxes at the Earth's surface from the algorithm GSW(V3.0)/GLW(V3.1) have been extensively validated against high-quality ground-based observations, in particular observed data from the Baseline Surface Radiation Network (BSRN). Comparisons with nearly 6000 site-months of both shortwave and longwave data from 52 BSRN sites show generally good agreement. In addition, the GEWEX SRB data have also been found to compare favorably with the World Radiation Data Centre (WRDC) data and the Global Energy Balance Archive (GEBA) data. In spite of the fact that the BSRN sites are scattered on all seven continents, the validation of the SRB data over the vast oceans had not been done until recently. In this paper, we present comparisons of the GEWEX-SRB data shortwave/longwave data with observations made on arrays moored in tropical oceans. Specifically, we have data from 21 buoys, or moorings, from Predictions and Research Moored Array in the Atlantic (PIRATA), 14 buoys from the Research Moored Array for African-Asian-Australian Monsoon Analysis and Prediction (RAMA) in the Indian Ocean, 20 buoys from the Tropical Atmosphere Ocean (TAO) array in the Pacific, and 3 buoys from the Woods Hole Oceanographic Institute (WHOI) (2 in the Pacific and 1 in the Atlantic). The data from these buoys span 12 years from 2000 to 2011, though not necessarily continuously. It is found that except for occasional outliers, the majority of the comparable pairs of data points agree well. Further examination of the outliers is necessary to determine whether they can be attributed to wind-driven tilt and/or aerosol accumulation on the instruments. In order to see how the ocean-based validation compares with land-based validation, we made SRB-BSRN monthly mean shortwave/longwave radiation comparisons in the tropical region only, where all the buoys are moored. The results show that in the case of shortwave radiation, the bias error from the ocean-based validation is somewhat larger than its land-based counterpart, but their RMS errors are not significantly different. In the case of longwave radiation, the ocean-based and land-based validations show nearly identical comparison statistics.

Taiping Zhang↗
NASA NTRS2021

Refining the Transit Timing and Photometric Analysis of TRAPPIST-1: Masses, Radii, Densities, Dynamics, and Ephemerides

We have collected transit times for the TRAPPIST-1 system with the Spitzer Space Telescope over four years. We add to these ground-based, HST, and K2 transit-time measurements, and revisit an N-body dynamical analysis of the seven-planet system using our complete set of times from which we refine the mass ratios of the planets to the star. We next carry out a photodynamical analysis of the Spitzer light curves to derive the density of the host starand the planet densities. We find that all seven planets’ densities may be described with a single rocky mass–radius relation which is depleted in iron relative to Earth, with Fe 21 wt% versus 32 wt% for Earth, and otherwise Earth-like in composition. Alternatively, the planets may have an Earth-like composition but enhanced in light elements,such as a surface water layer or a core-free structure with oxidized iron in the mantle. We measure planet masses toa precision of 3%–5%, equivalent to a radial-velocity(RV)precision of 2.5 cm s−1, or two orders of magnitude more precise than current RV capabilities. We find the eccentricities of the planets are very small, the orbits are extremely coplanar, and the system is stable on 10 Myr timescales. We find evidence of infrequent timing outliers, which we cannot explain with an eighth planet; we instead account for the outliers using a robust likelihood function. We forecast JWST timing observations and speculate on possible implications of the planet densities forthe formation, migration, and evolution of the planet system.

Eric Agol↗
NASA NTRS2019

Directional limits on persistent gravitational waves using data from Advanced LIGO’s first two observing runs

We perform an unmodeled search for persistent, directional gravitational wave (GW) sources using data from the first and second observing runs of Advanced LIGO. We do not find evidence for any GW signals. We place limits on the broadband GW flux emitted at 25 Hz from point sources with a power law spectrum at F(α;Θ) < (0.05–25) × 10^(−8) erg/sq. cmsHz and the (normalized) energy density spectrum in GWs at 25 Hz from extended sources at Ω(α)(Θ) < (0.19–2.89) × 10^(−8) per sr where α is the spectral index of the energy density spectrum. These represent improvements of 2.5–3× over previous limits. We also consider point sources emitting GWs at a single frequency, targeting the directions of Sco X-1, SN 1987A, and the Galactic center. The best upper limits on the strain amplitude of a potential source in these three directions range from h(0) < (3.6–4.7) × 10^(−25), 1.5× better than previous limits set with the same analysis method. We also report on a marginally significant outlier at 36.06 Hz. This outlier is not consistent with a persistent gravitational-wave source as its significance diminishes when combining all of the available data.

B. P. Abbott↗
NASA NTRS2020

Automatic Feature Tracking on Small Bodies for Autonomous Approach

Abstract—The autonomous approach of a spacecraft to an asteroid or comet (a small body) relies heavily on visual feature tracking to aid in estimating relative trajectories and the properties of the small body. Feature tracking for small bodies brings several challenges, including changing lighting, poor visual texture, and a concentration of features in a small part of an image. Six existing, open-source algorithms for feature tracking were tested on a simulated dataset and compared to the ground truth in the path of features. The main finding is that none of the algorithms provide all of the desired characteristics of long feature tracks with low errors and few outliers. Instead, there is a trade-off between long feature tracks and low error. The feature-matching algorithms SIFT, and BRISK provide good error characteristics, but short feature tracks, whereas the optical flow algorithm KLT provides long feature tracks, but with many features of large error. Given the challenges in feature tracking, it is recommended to focus development on each component of a feature tracking system: detection, description, and outlier rejection.

Morrell, Benjamin J↗
NASA NTRS2018

Performance Metrics for the Assessment of Satellite Data Products: An Ocean Color Case Study

Performance assessment of ocean color satellite data has generally relied on statistical metrics chosen for their common usage and the rationale for selecting certain metrics is infrequently explained. Commonly reported statistics based on mean squared errors, such as the coefficient of determination (r2), root mean square error, and regression slopes, are most appropriate for Gaussian distributions without outliers and, therefore, are often not ideal for ocean color algorithm performance assessment, which is often limited by sample availability. In contrast, metrics based on simple deviations, such as bias and mean absolute error, as well as pair-wise comparisons, often provide more robust and straightforward quantities for evaluating ocean color algorithms with non-Gaussian distributions and outliers. This study uses a SeaWiFS chlorophyll-a validation data set to demonstrate a framework for satellite data product assessment and recommends a multimetric and user-dependent approach that can be applied within science, modeling, and resource management communities.

remote sensing↗
NASA NTRS2024

On the Use of Mahalanobis Distance in Particle Image Velocimetry Post-Processing

Particle Image Velocimetry (PIV) is a method of flow measurement that has become increasingly popular as an experimental tool. New technology has made high-speed and higher-dimension (stereoscopic, tomographic, etc) methods available to an ever-growing population of researchers. These advanced methods can provide significantly more data than traditional low-speed planar PIV, but these larger data sets also require more resources to process and store. A major time sink in the post-processing of large PIV data sets is the identification and rejection of “bad” or ”spurious” vectors that survive an initial processing step in commercial software, which attempts to identify spurious vectors on an image-by-image basis and not with respect to repeated trials. The Mahalanobis distance, an almost 100-year-old statistical function, was determined to be well suited for this task for its computational efficiency and higher-dimensional nature. This work includes a summary of the Mahalanobis distance and validation of its usefulness as a tool for outlier rejection in the post-processing of PIV data.

Experimental Methods↗
NASA NTRS1977

Smoothing of orbital tracking data: Mission planning, mission analysis and software formulation

The problem created by the presence of wild or outlying data points among orbital tracking data, is addressed. Consideration is given to the effects of such outliers on the orbit determination process, and methods for minimizing or even eliminating these effects are proposed. Some preliminary efforts implementing these new methods are described, and the results thus far obtained are summarized. Based on these ideas and results, recommendations are made for future investigation.

Vedder, J. D.↗
NASA NTRS1980

Evaluation of gravimetric ground truth soil moisture data collected for the agricultural soil moisture experiment, 1978 Colby, Kansas, aircraft mission

Soil moisture data acquired to support the development of algorithms for estimating surface soil moisture from remotely sensed backscattering of microwaves from ground surfaces are presented. Aspects of field uniformity and variability of gravimetric soil moisture measurements are discussed. Moisture distribution patterns are illustrated by frequency distributions and contour plots. Standard deviations and coefficients of variation relative to degree of wetness and agronomic features of the fields are examined. Influence of sampling depth on observed moisture content an variability are indicated. For the various sets of measurements, soil moisture values that appear as outliers are flagged. The distribution and legal descriptions of the test fields are included along with examinations of soil types, agronomic features, and sampling plan. Bulk density data for experimental fields are appended, should analyses involving volumetric moisture content be of interest to the users of data in this report.

Arya, L. M.↗
NASA NTRS1981

Recent developments with the ORSER system

Additions to the ORSER remote sensing data processing package are described. The ORSER package consists of about 35 individual programs that are grouped into preprocessing, data analysis, and display subsystems. Additional data formats and data management, data transformation, and geometric correlation programs were supplemented to the preprocessing subsystem. Enhancements to the data analysis techniques include a maximum likelihood classifier (MAXCLASS) and a new version of the STATS program which makes delineation of training areas easier and allows for detection of outlier points. Ongoing developments are also described.

Baumer, G. M.↗