Data Quality Assessment Using SERI-QC
This presentation provides an overview of data quality assessment using SERI QC.
14 SOLAR ENERGY↗
SEARCH · Search NASA
Results for “data quality”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data Quality Assessment Using SERI-QC
This poster provides an overview of data quality assessment using SERI QC.
14 SOLAR ENERGY↗
Analytical methods for online data quality assessment
This chapter provides a comprehensive overview of the main steps for algorithmic sensor signal quality assessment, which can enhance the decision-making process for water resource recovery facility (WRRF) operation and optimization. It introduces the concept of redundancy as the basis for data quality assessment. It also explains the typical data processing pipeline, which consists of preliminary analysis, data pre-processing, and specific algorithmic approaches. Each of these processes is presented and discussed in three separate sections. Importantly, this chapter introduces the main approaches for data quality assessment, provides guidelines for selecting the most suitable one and the key performance indicators to evaluate them and explains how to collect metadata through such an algorithmic approach.
Aguado, Daniel↗
Spatio-Temporal Anomaly Detection with Graph Networks for Data Quality Monitoring of the Hadron Calorimeter
The Compact Muon Solenoid (CMS) experiment is a general-purpose detector for high-energy collision at the Large Hadron Collider (LHC) at CERN. It employs an online data quality monitoring (DQM) system to promptly spot and diagnose particle data acquisition problems to avoid data quality loss. In this study, we present a semi-supervised spatio-temporal anomaly detection (AD) monitoring system for the physics particle reading channels of the Hadron Calorimeter (HCAL) of the CMS using three-dimensional digi-occupancy map data of the DQM. We propose the GraphSTAD system, which employs convolutional and graph neural networks to learn local spatial characteristics induced by particles traversing the detector and the global behavior owing to shared backend circuit connections and housing boxes of the channels, respectively. Recurrent neural networks capture the temporal evolution of the extracted spatial features. We validate the accuracy of the proposed AD system in capturing diverse channel fault types using the LHC collision data sets. The GraphSTAD system achieves production-level accuracy and is being integrated into the CMS core production system for real-time monitoring of the HCAL. We provide a quantitative performance comparison with alternative benchmark models to demonstrate the promising leverage of the presented system.
43 PARTICLE ACCELERATORS↗
AVIRIS data quality for coniferous canopy chemistry
An assessment of AVIRIS data quality for studying coniferous canopy chemistry was made. Seven flightlines of AVIRIS data were acquired over a transect of coniferous forest sites in central Oregon. Both geometric and radiometric properties of the data were examined including: pixel size, swath width, spectral position and signal-to-noise ratio. A flat-field correction was applied to AVIRIS data from a coniferous forest site. Future work with this data set will exclude data from spectrometers C and D due to low signal-to-noise ratios. Data from spectrometers A and B will be used to examine the relationship between the canopy chemical composition of the forest sites and AVIRIS spectral response.
Swanberg, Nancy A.↗
Generation of housing quality data from multiband aerial photographs
Generation of housing quality data from aerial photographs
Wellar, B. S.↗
Review of Technical Photovoltaic Key Performance Indicators and the Importance of Data Quality Routines
Technical key performance indicators (KPIs) are important metrics used to assess and quantitatively summarize various aspects of photovoltaic (PV) systems, including long-term performance, economic viability, and carbon footprint. Herein, a group of experts of the International Energy Agency's Photovoltaic Power Systems Programme Task 13 collect and describ the most important technical KPIs used in the industry. Thereby, a set of best practices for reliably handling PV system data is presented and the impact of data quality and climatic variability on KPI calculation is investigated. Further, the effective use of technical KPIs allows triggering data-driven and informed decisions to optimize PV systems and providing a comprehensive overview of how PV systems operate across different conditions and climates. With the worldwide growth of the PV industry, more companies operate/own PV systems in different regions, where the climatic and seasonal profiles differ. This requires context-aware evaluation of KPIs, or the judicious application of multiple KPIs, to ensure that each asset is evaluated correctly. Beyond that, there is untapped potential in the utilization of KPIs through geospatial mapping and extrapolation of fleet KPIs. This study demonstrates that the uncertainty in KPI estimation is not well understood and depends on data quality, climatic variability, and system configuration.
14 SOLAR ENERGY↗
Developing Data Services to Provide Data Quality Information for Global Satellite-Based Precipitation Products
Global satellite-based precipitation products have been widely used in research and applications around the world. Compared to ground-based observations, satellite-based measurements provide data on a global scale, especially in remote continents and over oceans. The NASA Goddard Earth Sciences (GES) Data and Information Services Center (DISC) is home to NASA global precipitation product archives including the Tropical Rainfall Measuring Mission (TRMM), the Global Precipitation Measurement (GPM), as well as other global and regional precipitation products. Precipitation is one of the top downloaded and accessed parameters in the GES DISC data archive. Meanwhile, users want to easily locate and obtain data quality information at regional and global scales to better understand how precipitation products perform and how reliable they are. As a data service provider, it is necessary to provide easy access to data quality information. However, such information normally is not available, and when it is available, it is not in one place and difficult to locate. In this presentation, we will present such challenges and activities at the GES DISC to address precipitation data (other datasets as well) quality issues.
Liu, Z.↗
LANDSAT 4 image data quality analysis
A comparative analysis of TM and MSS data was completed and the results indicate that there are half as many separable spectral classes in the MSS data than in TM. In addition, the minimum separability between classes was also much less in MSS data. Radiometric data quality was also investigated for the TM by computing power spectrum estimates for dark-level data from Lake Michigan. Two significant coherent noise frequencies were observed, one with a wavelength of 3.12 pixels and the other with a 17 pixel wavelength. The amplitude was small (nominally .6 digital count standard deviation) and the noise appears primarily in Bands 3 and 4. No significant levels were observed in other bands. Scan angle dependent brightness effects were also evaluated.
Anuta, P. E.↗
Anomaly Detection Based on Machine Learning for the CMS Electromagnetic Calorimeter Online Data Quality Monitoring
Using a semi-supervised machine learning approach we present a real-time anomaly detection system based on an autoencoder used for online data quality monitoring of the CMS electromagnetic calorimeter operating at the CERN LHC. We introduce a novel method that maximizes the anomaly detection performance making use of the time-dependence of anomalies and the spatial variations in the detector response. The autoencoder-based system efficiently detects anomalies in real time and maintains a very low false discovery rate. We validate the performance of this novel system with anomalies from LHC collision data taken in 2018 and 2022. In addition, results are presented after deploying the autoencoder-based system in the CMS online Data Quality Monitoring workflow at the beginning of LHC Run 3 resulting in the system to detect issues that were missed by the existing system.
Harilal, Abhirami [Carnegie Mellon University, Pit↗
An Investigation Into HPLC Data Quality Problems
This report summarizes the analyses and results produced by a five-member investigative team of Government, university, and industry experts, established by NASA HQ. The team examined data quality problems associated with high performance liquid chromatography (HPLC) analyses of pigment concentrations in seawater samples produced by the San Diego State University (SDSU) Center for Hydro-Optics and Remote Sensing (CHORS). This report shows CHORS did not validate the methods used before placing them into service to analyze field samples for NASA principal investigators (PIs), even though the HPLC literature contained easily accessible method validation procedures, and the importance of implementing them, more than a decade ago. In addition, there were so many sources of significant variance in the CHORS methodologies, that the HPLC system rarely operated within performance criteria capable of producing the requisite data quality. It is the recommendation of the investigative team to a) not correct the data, b) make all the data that was temporarily sequestered available for scientific use, and c) label the affected data with an appropriate warning, e.g., "These data are not validated and should not be used as the sole basis for a scientific result, conclusion, or hypothesis--independent corroborating evidence is required."
Hooker, Stanford B.↗
Development and status of data quality assurance program at NASA Langley research center: Toward national standards
As part of a continuing effort to re-engineer the wind tunnel testing process, a comprehensive data quality assurance program is being established at NASA Langley Research Center (LaRC). The ultimate goal of the program is routing provision of tunnel-to-tunnel reproducibility with total uncertainty levels acceptable for test and evaluation of civilian transports. The operational elements for reaching such levels of reproducibility are: (1) statistical control, which provides long term measurement uncertainty predictability and a base for continuous improvement, (2) measurement uncertainty prediction, which provides test designs that can meet data quality expectations with the system's predictable variation, and (3) national standards, which provide a means for resolving tunnel-to-tunnel differences. The paper presents the LaRC design for the program and discusses the process of implementation.
Hemsch, Michael J.↗
A compilation and analysis of helicopter handling qualities data. Volume 2: Data analysis
A compilation and an analysis of helicopter handling qualities data are presented. Multiloop manual control methods are used to analyze the descriptive data, stability derivatives, and transfer functions for a six degrees of freedom, quasi static model. A compensatory loop structure is applied to coupled longitudinal, lateral and directional equations in such a way that key handling qualities features are examined directly.
Heffley, R. K.↗
Applications notice for participation in the LANDSAT-D image data quality analysis program
The applications notice for the LANDSAT 4 image data quality analysis program is presented. The objectives of the program are to qualify LANDSAT 4 sensor and systems performance from a user applications point of view, and to identify any malfunctions that may impact data applications. Guidelines for preparing proposals and background information are provided.
Source record↗
SECARB-USA: Data Quality Methodology
SECARB-USA Deliverable 4.2.1 utilizes the outputs from the Needs Assessment (Subtask 2.1) to develop a data quality methodology. Many types of data are needed to evaluate a site for technical and financial viability. Several inventories of data types have been produced (for example NETL, 2010). The objective here is to organize the data types so that the needs met are specified. From this cross index (Table 1 in the Appendix), it will be possible in future tasks to (1) determine, on a site-specific basis, how much of each data type is required at each stage of a project to meet the need, and conversely (2) to further specify and define the data collection methods applied such that the data are tailored to fit that need. Derivative tables can then be developed to semi-quantitively evaluate the extent to which need is critical for early go/no-go decision points, or if it is more important than average, requiring faster or larger capitalization and spend to meet the need. In an additional step, the current availability of data for a site can be semi-quantitively assessed. From the table of data criticality and the table of data availability, site-specific cost for meeting the data needs can be determined, and allow a pre-permit spend estimated for a portfolio of projects.
54 ENVIRONMENTAL SCIENCES↗
Federal Automotive Statistical Tool: FY 2022 Federal Fleet Trends and Data Quality [Slides]
This presentation presents a brief overview of the collection of information about the US government's fleet of motor vehicles using the Federal Automotive Statistical Tool (FAST), discusses the makeup and operation of the vehicle fleet during FY 2022 and discusses quality of the submitted data. FAST is a web-based information system sponsored by GSA's Office of Government-wide Policy and DOE's Federal Energy Management Program to collect information about the US federal government's fleet of motor vehicles; FAST is developed, maintained, and supported by DOE's Idaho National Laboratory (INL).
33 ADVANCED PROPULSION SYSTEMS↗
FedFleet 2022: Federal Automotive Statistical Tool - FY 2021 Fleet Trends and Data Quality
This presentation presents a brief overview of the collection of information about the US government's fleet of motor vehicles using the Federal Automotive Statistical Tool (FAST), discusses the makeup and operation of the vehicle fleet during FY 2021, discusses quality of the submitted data, and touches on future aspects of fleet data collection and reporting. FAST is a web-based information system sponsored by GSA's Office of Government-wide Policy and DOE's Federal Energy Management Program to collect information about the US federal government's fleet of motor vehicles; FAST is developed, maintained, and supported by DOE's Idaho National Laboratory (INL).
99 GENERAL AND MISCELLANEOUS↗
Landsat-4 data quality analysis
Landsat-4 satellite Thematic Mapper (TM) and multispectral scanner (MSS) data have been analyzed in order to ascertain data quality and information content. Geometric evaluations have tested band-to-band registration accuracy, and the TM's overall system resolution was evaluated for the case of image objects with high contrast, sharp edge responses. The information content evaluation employed clustering, principal components, and the transformed divergence separability measured on data from Iowa and Chicago, Illinois. The MSS classification analysis compared MSS and TM information contents for a large number of science classes.
Anuta, P.↗