Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality assessment”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 253 records · Page 14

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The ongoing growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Through access to a large number of time series data sets from different power plants, we have found common issues that impede the modeling process. Namely, the time series data are hard to transfer between groups due to differences in variable nomenclature, and the quality of the data sets can vary. We address the issue of variable nomenclature by FAIRifying spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. We have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. With access to large time series data sets across many power plants, we can utilize spatiotemporal coherence between the sites in order to improve the quality of our data. To solve the issue of data missingness, we propose the use of Spatiotemporal-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗

L-PBF High-Throughput Data Pipeline Approach for Multi-modal Integration

Abstract Metal-based additive manufacturing requires active monitoring solutions for assessing part quality. Multiple sensors and data streams, however, generate large heterogeneous data sets that are impractical for manual assessment and characterization. In this work, an automated pipeline is developed that enables feature extraction from high-speed camera video and multi-modal data analysis. The framework removes the need for manual assessment through the utilization of deep learning techniques and training models in a weakly supervised paradigm. We demonstrate this pipeline’s capability over 700,000 high-speed camera frames. The pipeline successfully extracts melt pool and spatter geometries and links them to corresponding pyrometry, radiography, and processparameter information. 715 individual prints are examined to reveal melt pool areas that exceeds 0.07 mm 2 and pyrometry signal over a threshold (375 pyrometry units) were more likely to have defects. These automated processes enable massive throughput of characterization techniques.

36 MATERIALS SCIENCE↗

Mach-Zehnder recording systems for pulsed power diagnostics

Fiber-optic transmission and recording systems, based on Mach-Zehnder modulators, have been developed and installed at the National Ignition Facility (NIF), and are being developed for other pulsed-power facilities such as the Z accelerator at Sandia, with different requirements. We present the design and performance characteristics for the mature analog links, based on the system developed for the Gamma Reaction History diagnostic at the OMEGA laser and at NIF. For a single detector channel, two Mach-Zehnders are used to provide high dynamic range at the full recording bandwidth with no gaps in the coverage. We present laboratory and shot data to estimate upper limits on the radiation effects as they impact recorded data quality. Finally, we will assess the technology readiness level for mature and developing implementations of Mach-Zehnder links for these environments.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Use of ocean color scanner data in water quality mapping

Remotely sensed data, in combination with in situ data, are used in assessing water quality parameters within the San Francisco Bay-Delta. The parameters include suspended solids, chlorophyll, and turbidity. Regression models are developed between each of the water quality parameter measurements and the Ocean Color Scanner (OCS) data. The models are then extended to the entire study area for mapping water quality parameters. The results include a series of color-coded maps, each pertaining to one of the water quality parameters, and the statistical analysis of the OCS data and regression models. It is found that concurrently collected OCS data and surface truth measurements are highly useful in mapping the selected water quality parameters and locating areas having relatively high biological activity. In addition, it is found to be virtually impossible, at least within this test site, to locate such areas on U-2 color and color-infrared photography.

Khorram, S.↗

A Resilient Condition Assessment Monitoring System

An architecture and supporting methods are presented for the implementation of a resilient condition assessment monitoring system that can adaptively accommodate both cyber and physical anomalies to a monitored system under observation. In particular, the architecture includes three layers: information, assessment, and sensor selection. The information layer estimates probability distributions of process variables based on sensor measurements and assessments of the quality of sensor data. Based on these estimates, the assessment layer then employs probabilistic reasoning methods to assess the plant health. The sensor selection layer selects sensors so that assessments of the plant condition can be made within desired time periods. Resilient features of the developed system are then illustrated by simulations of a simplified power plant model, where a large portion of the sensors are under attack.

97 MATHEMATICS AND COMPUTING↗

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Many of these power plants have spatiotemporal coherence that can be utilized for improving model accuracy. We have demonstrated in this paper the FAIRification of spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. Going further, we have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. To solve the issue of data missingness, we propose the use of St-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗

Scan-angle and Detector Effects in Thematic Mapper Radiometry

The performance of the Thematic Mapper (TM) as manifested by the quality of its image data is examined in order to suggest possible improvements for data production and assess the effects of data quality on its utility for land resources applications. The major emphasis is on the radiometric characteristics of TM data, with some attention to spatial and spectral characteristics.

Metzler, M. D.↗

The NPOESS Preparatory Project (NPP) Science Data Segment (SDS) Data Depository and Distribution Element (SD3E) System Architecture

The National Polar-orbiting Operational Environmental Satellite System (NPOESS), the U.S. Government's future low-Earth orbiting satellite system, will monitor global weather and environmental conditions. Serving as a risk reduction for NPOESS, the NPOESS Preparatory Project (NPP) will provide remotely sensed atmospheric, land, ocean, ozone, and sounder data that will serve the meteorological and global climate change scientific communities. The National Aeronautics and Space Administration (NASA) NPP Science Data Segment's (SDS) primary role is to independently assess the quality of the NPP science and environmental data records for their ability to support climate research. The SDS is composed of nine elements; an input element that receives data from the operational agencies and acts as a buffer, a calibration analysis element, five elements devoted to measurement based quality assessment, an element used to test algorithmic improvements, and an element that provides overall science direction. Each element requires a set of sensor specific science data products for their evaluation. There are four NPP sensors that will be flown on the NPP observatory. They are the Visible Infrared Imagining Radiometer Suite (VIIRS), the Advanced Technology Microwave Sounder (ATMS), the Cross-Track Infrared Sounder (CrIS), and the Ozone Mapper/Profiler Suite (OMPS). It is estimated that these four sensors combined will make daily data requests for approximately six terabytes of NPP science products from the operational data providers. As a result, issues associated with duplicate data requests, data transfers of large volumes of diverse products, and data transfer failures raised concerns with respect to the network traffic and bandwidth consumption. Therefore, a central data broker system for receiving and buffering data requests and data products for the SDS was developed. The data element for this system is called the SDS Data Depository and Distribution Element (SD3E). It supports science mission data assessment by assuring the timely and validated acquisition and subsequent transfer of the NPP Science Mission data to the SDS Elements and NPP Science Team. The six science elements that interface with the SD3E span across the NASA Goddard Space Flight Center (GSFC), the NASA Jet Propulsion Laboratory (JPL), and the University of Wisconsin. As the primary communication vehicle for the science elements and science team, the SD3E has an interface to the operational data providers: National Environment Satellite, Data, and Information Service (NESDIS) Interface Data Processing System (IDPS) and the National Oceanic Atmospheric Administration's (NOAA) Comprehensive Large Array-data Stewardship system (CLASS) Archive Data System (ADS), that are responsible for product generation and archive and distribution respectively. The SD3E is designed to be a semi-customizable and semi-automated system. This system is designed to provide flexibility and ease of use for the science users in accessing the latest data products by creating a rolling data cache that temporarily stores the products locally before transferring the data to the SDS Measurement based elements for the land, ocean, atmosphere, sounder, and ozone. This paper describes the design and architecture of one of the nine SDS elements, the SD3E, and how this system has provided a mechanism for efficient data exchange, how it has helped in alleviating some of the network traffic and usage, and how it has contributed to reducing operational costs.

Ho, Evelyn L.↗

FastProNGS: fast preprocessing of next-generation sequencing reads

Background Next-generation sequencing technology is developing rapidly and the vast amount of data that is generated needs to be preprocessed for downstream analyses. Yet, until now, software that can efficiently make all the quality assessments and filtration of raw data is still lacking. Results We developed FastProNGS to integrate the quality control process with automatic adapter removal. Parallel processing was implemented to speed up the process by allocating multiple threads. Compared with similar up-to-date preprocessing tools, FastProNGS is by far the fastest. Read information before and after filtration can be output in plain-text, JSON, or HTML formats with user-friendly visualization. Conclusions FastProNGS is a rapid, standardized, and user-friendly tool for preprocessing next-generation sequencing data within minutes. It is an all-in-one software that is convenient for bulk data analysis. It is additionally very flexible and can implement different functions using different user-set parameter combinations.

97 MATHEMATICS AND COMPUTING↗

FedFleet 2024: Fleet Data Quality - Understanding and Improving Your Agency's FAST Fleet Data Quality Metric? [Slides]

This presentation presents an overview of how the team supporting the Federal Automotive Statistical Tool (FAST) project for DOE and GSA view and assess the quality of fleet data submissions from federal agencies. It also discusses ways in which agencies can improve the quality of their fleet data and of their fleet data submissions. FAST is a web-based information system sponsored by GSA's Office of Government-wide Policy and DOE's Federal Energy Management Program to collect information about the US federal government's fleet of motor vehicles; FAST is developed, maintained, and supported by DOE's Idaho National Laboratory (INL).

99 GENERAL AND MISCELLANEOUS↗

The KTOI Ecosystem Project Relational Database : a Report Prepared by Statistical Consulting Services for KTOI Describing the Key Components and Specifications of the KTOI Relational Database.

Data are the central focus of any research project. Their collection and analysis are crucial to meeting project goals, testing scientific hypotheses, and drawing relevant conclusions. Typical research projects often devote the majority of their resources to the collection, storage and analysis of data. Therefore, issues related to data quality should be of foremost concern. Data quality issues are even more important when conducting multifaceted studies involving several teams of researchers. Without the use of a standardized protocol, for example, independent data collection carried out by separate research efforts can lead to inconsistencies, confusion and errors throughout the larger project. A database management system can be utilized to help avoid all of the aforementioned problems. The centralization of data into a common relational unit, i.e. a relational database, shifts the responsibility for data quality and maintenance from multiple individuals to a single database manager, thus allowing data quality issues to be assessed and corrected in a timely manner. The database system also provides an easy mechanism for standardizing data components, such as variable names and values uniformly across all segments of a project. This is particularly an important issue when data are collected on a number of biological/physical response and explanatory variables from various locations and times. The database system can integrate all segments of a large study into one unit, while providing oversight and accessibility to the data collection process. The quality of all data collected is uniformly maintained and compatibility between research efforts ensured. While the physical database would exist in a central location, access will not be physically limited. Advanced database interfaces are created to operate over the internet utilizing a Web-based relational database, allowing project members to access their data from virtually anywhere. These interfaces provide users with the ability to upload, download, edit, and search data remotely, creating a dynamic system that is continually updated with the most recent information. At the same time, data are protected through user access restrictions, by implementing user profiles and password protected security. This accessibility could be set to any combination of read/write/edit abilities from an administrator capacity with full access to all data, to a highly restricted public access capability limited to general project information. Generation of customized summary reports and basic graphical routines could also be obtained through a Web-based interference. Using these types of features, users could produce summary tables, track trends of specified response variables over time or location, and compare results from various disciplines. Exploration of data in this manner can help users to better define and clarify their research goals and provide a means of integrating various aspects of a larger research project.

13 HYDRO ENERGY↗

Assessing the Feasibility of using PlanetScope Data for Air Quality Applications

Planet is a commercial company known for having one of the largest operational smallsat satellite constellations. This constellation comprises over 120 3U CubeSats called Doves (equipped with a sensor called PlanetScope), which combined, are able to image the entire Earth’s surface daily at a high 3-meter spatial resolution. With Planet data becoming more readily available to scientists due to NASA’s Commercial Smallsat Data Acquisition Program (CSDAP), this study investigates the feasibility of utilizing Planet data to conduct scientific analyses with an emphasis on air quality research. This study offers a comparison of PlanetScope data with MODIS over a variety of land cover types, as well as in response to varying PM2.5 conditions. It also investigates the feasibility of utilizing PlanetScope data for estimating PM2.5 concentrations using deep learning

Jeanne Le Roux↗

Evaluation of NESS soundings used in the FGGE special effort

A special effort to produce high quality edited and enhanced global data sets is being conducted for the two special observing periods of the first GARP global experiment. More than 60 cases were selected for enhancement on the basis of meteorological interest. These cases include situations of blocking, cut off low development, cyclogenesis, and tropical circulations. The sounding data enhancement process consists of supplementing the operational satellite sounding data set with higher resolution sounding in meteorologically active regions and with new soundings where data voids or soundings of questionable quality exist. After the enhancement process is completed, a final editing and quality assessment of the enhanced data are performed. The data is archived for further analyses. Subjective evaluation of the effects of sounding data enhancement indicate that: (1) enhanced and operational IR retrievals are similar in most situations; (2) large improvements in 1000 to 300 mb thickness and mandatory level temperature and intensification of atmospheric thermal gradients occasionally occur; (3) interactive processing can fill in major gaps in the TIROS-N coverage; and (4) assimilation of special effort soundings can result in major modifications to large scale analyses and prognoses.

Atlas, R.↗

Accelerating TEMPO Air Quality Science Through STAQS

Soon after the launch of Tropospheric Emissions: Monitoring of Pollution (TEMPO) mission, NASA is supporting the Synergistic TEMPO Air Quality Science field study (STAQS) in summer 2023. This study’s main objective is to accelerate science with geostationary air quality observations from TEMPO to better understand its use in air pollution research and applications. Science objectives include, but are not limited to, TEMPO L2 product evaluation, the interpretation of the spatiotemporal evolution of TEMPO data during air quality events, assessment of anthropogenic emissions, contributions to chemical transport modeling, and environmental justice applications. These objectives can be accomplished through the integration of satellite data with systematically repeated high-resolution aircraft- and ground-based measurements in multiple urban environments. The primary urban cities considered include Los Angeles, New York City, and Chicago in June-August 2023. Measurements consist of airborne remote sensing observations of air quality constituents of nitrogen dioxide (NO2), formaldehyde (HCHO), ozone, and aerosols using the GeoCape Airborne Simulator and High-Spectral Resolution Lidar 2/Differential Absorption Lidar (HSRL2/DIAL) on the NASA JSC G-V aircraft, greenhouse gas observations from Airborne Visible InfraRed Imaging Spectrometer - Next Generation (AVIRIS-NG) and High Altitude Lidar Observatory (HALO) on the NASA LaRC G-III aircraft, and ground-based remote sensing and in situ observations of ozone, NO2, and HCHO from the Tropospheric Ozone Lidar Network (TOLNet), Pandora spectrometers, and ground-based monitoring networks. Key partnership studies providing in situ airborne observations include the NOAA Atmospheric Emissions and Reactions Observed from Megacities to Marine Areas (AEROMMA) field study on the NASA DC-8 in the same primary target areas as STAQS and Greater New York Oxidant, Tropospheric Halogens, and Aerosol Measurements and Modeling (GOTHAMM) field study on the NSF C-130 near New York City. This presentation will include a current status update of the STAQS mission and an overview of its measurement strategies and science objectives with the goal of promoting continued discussions for building and strengthening collaborations prior to the mission.

Laura Judd↗

Assessment of Altimetry Using Ground-Based GPS Data from the 88S Traverse, Antarctica, in Support of ICESat-2

We conducted a 750 km kinematic GPS survey, referred to as the 88S Traverse, based out of South Pole Station, Antarctica, between December 2017 and January 2018. This ground-based survey was designed to validate space-borne altimetry and airborne altimetry developed at NASA. The 88S Traverse intersects 20% of the ICESat-2 satellite orbits on a route that has been flown by two different Operation IceBridge airborne laser altimeters: the Airborne Topographic Mapper (ATM; 26 October 2014) and the University of Alaska Fairbanks (UAF) Lidar (30 November and 3 December 2017). Here we present an overview of the ground-based GPS data quality and a quantitative assessment of the airborne laser altimetry over a flat section of the ice sheet interior. Results indicate that the GPS data are internally consistent (1:1+- 4:1 cm). Relative to the ground-based 88S Traverse data, the elevation biases for ATM and the UAF lidar range from -9:5 to 3.6 cm, while surface measurement precisions are equal to or better than 14.1 cm. These results suggest that the ground-based GPS data and airborne altimetry data are appropriate for the validation of ICESat-2 surface elevation data.

Brunt, Kelly M.↗

Extraction of ozone and chlorophyll-A distribution from AVIRIS data

The potential of airborne imaging spectrometry for assessing and monitoring natural resources is studied. Therefore, an AVIRIS scene of the NASA's MacEurope 1991 campaign - acquired in Central Switzerland - is used. The test site consists of an urban area, the Lake Zug with its surrounding fields, the Rigi mountain in the center of the test site, and the Lake of Four Cantons. The region is covered by the AVIRIS flight #910705, run 6 and 7 of the NASA ER-2 aircraft resulting in an average nominal pixel size of about 18 m. Simultaneous to the ER-2 overflight spectroradiometric measurements have been taken in various locations. Preselected reference targets were measured in the field with a GER Mark V spectroradiometer, and radiance measurements were taken to the lake using a Li-Cor LI 1800UW specroradiometer below and above the water surface. A comprehensive meteorological data set was obtained by joining the POLLUMET experiment which carried out measurements to investigate the summer smog in Switzerland on the same day. The quality assessment for the actual data set can be found in detail in Meyer et al. A parametric approach calculating the location of the airplane was used to simulate the observation geometry. This parametric preprocessing procedure, which takes care of effects of flight line and attitude variations as well as the pixel-by-pixel topographic corrections is described in Meyer.

Schaepman, M.↗