Search NASA⌕ Search
Home/Archive/NASA NTRS

NASA NTRS · 20240001391

Data Quality Challenges for Analysis Ready Data (ARD)

Zhong Liu·Robert Downs·Ge Peng·David F Moroni·Hampapuram Ramapriyan·Yaxing Wei·Chung-lin Shie
View original source ↗Export metadata ↓

Abstract

Data quality plays a critical role in research and applications. The Earth Science Information Partners (ESIP) Information Quality Cluster (IQC) defines four aspects of information quality: Science, Product, Stewardship, and Services. The ESIP IQC has become internationally recognized as an authoritative and responsive resource of information and guidance to data producers and distributors on how to implement data quality standards and best practices for their science data systems, datasets, and data/metadata dissemination services. In recent years, cloud computing environments have provided scale-up capabilities such as data archives and services, enabling interdisciplinary science and applications. More value-added products are expected from data service providers, including Analysis Ready Data (ARD). ARD refers to data that has been preprocessed into a form that allows immediate analysis by the end user, processed to a minimum set of requirements and provides interoperability over time and across multiple datasets. Once a dataset has been developed from its original form to produce ARD, what quality characteristics should the derived dataset or ARD possess? Also, is it safe to assume that the quality of the ARD is consistent with the quality of the source data, or are there special attributes to an ARD that would warrant a secondary, independent quality assessment? What provenance (also called “data lineage”) information needs to be included in ARD? It is important to answer these questions, especially given the ease of use of ARD, and the consequent temptation by users to trust ARD without understanding the limitations or possible variations in quality compared to the source data. In this presentation, we will discuss data quality challenges for ARD products and services and introduce IQC for participation.

Explore related subjects

data qualityanalysis ready data

Keep this discovery

Explore connections, maps & timelines

BibTeXRIS

Zhong Liu, Robert Downs, Ge Peng, David F Moroni, Hampapuram Ramapriyan, Yaxing Wei, Chung-lin Shie. Data Quality Challenges for Analysis Ready Data (ARD). https://ntrs.nasa.gov/citations/20240001391

Cite the original work for its findings. Save a collection to share your selection of sources.

KEEP EXPLORING

Related reports

NASA NTRS2019

Uncertainty Analysis of the CE-22 Advanced Nozzle Test Facility

This paper presents methods and results of a detailed measurement uncertainty analysis that was performed for the Advanced Nozzle Test Facility, CE-22, located at the NASA Glenn Research Center. Results for the uncertainty in thrust and flow coefficients, in addition to other variables of interest, are provided. Results are presented separately as random uncertainty (characterizing errors inherent to instrument or measurement environment, impacting repeatability), systematic uncertainty (capturing inaccuracies due to measurement process, calibration, installation effects or other sources which may introduce bias), and total combined uncertainty. The statistical methods and engineering judgments used to estimate elemental uncertainties are described. The analysis tool MANTUS (Measurement Analysis Tool for Uncertainty in Systems) was used to quantify instrumentation system uncertainty, and statistical analysis and engineering judgment were used to quantify other random and systematic uncertainty sources. The Monte Carlo method was used to propagate systematic and random elemental uncertainties to determine the uncertainties of various calculated variables of interest.

data quality↗
NASA NTRS2018

ISO 191* Revisions and Amendments

ISO standards have changed recently in response to new requirements from introduced by NASA, NOAA, and other ESIP members. These changes address important uses cases related to identification, parameter metadata, and platform/instrument/sensor metadata. Each change will be described along with discussion of how they may be used.

data quality↗
NASA NTRS2016

International Metadata Standards and Enterprise Data Quality Metadata Systems

Well-documented data quality is critical in situations where scientists and decision-makers need to combine multiple datasets from different disciplines and collection systems to address scientific questions or difficult decisions. Standardized data quality metadata could be very helpful in these situations. Many efforts at developing data quality standards falter because of the diversity of approaches to measuring and reporting data quality. The one size fits all paradigm does not generally work well in this situation. I will describe these and other capabilities of ISO 19157 with examples of how they are being used to describe data quality across the NASA EOS Enterprise and also compare these approaches with other standards.

data quality↗