Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
DOE OSTI2014

Exploring the effects of data quality, data worth, and redundancy of CO2 gas pressure and saturation data on reservoir characterization through PEST Inversion

This study examined the impacts of reservoir properties on CO2 migration after subsurface injection and evaluated the possibility of characterizing reservoir properties using CO2 monitoring data such as saturation distribution. The injection reservoir was assumed to be located 1400-1500 m below the ground surface such that CO2 remained in the supercritical state. The reservoir was assumed to contain layers with alternating conductive and resistive properties, which is analogous to actual geological formations such as the Mount Simon Sandstone unit. The CO2 injection simulation used a cylindrical grid setting in which the injection well was situated at the center of the domain, which extended up to 8000 m from the injection well. The CO2 migration was simulated using the PNNL-developed simulator STOMP-CO2e (the water-salt-CO2 module). We adopted a nonlinear parameter estimation and optimization modeling software package, PEST, for automated reservoir parameter estimation. We explored the effects of data quality, data worth, and data redundancy on the detectability of reservoir parameters using CO2 saturation monitoring data, by comparing PEST inversion results using data with different levels of noises, various numbers of monitoring wells and locations, and different data collection spacing and temporal sampling intervals. This study yielded insight into the use of CO2 saturation monitoring data for reservoir characterization and how to design the monitoring system to optimize data worth and reduce data redundancy.

Carbon sequestration↗
DOE OSTI2022

Using Synchrophasor Status Word as Data Quality Indicator: What to Expect in the Field?

Data quality plays a crucial role in successful applications of synchrophasor data in power system operation and control. This paper presents the results of a data quality analysis of a multi-year field-recorded synchrophasor dataset. The analysis has identified several typical data quality issues encountered in the field data. An examination of the PMU status words included with the dataset has revealed several inconsistent implementations and the lack of correlation between the PMU data quality and the status word, which impacts the usefulness of such information. Our investigation has concluded that the status word alone as found in the recorded field dataset could not be used as a reliable indicator of data quality for field-recorded data. Several recommendations are proposed to improve the usefulness of the PMU status word.

Cheng, Zheyuan↗
DOE OSTI2017

Hunting Data Rogues at Scale: Data Quality Control for Observational Data in Research Infrastructures

Data quality control is one of the most time consuming activities within Research Infrastructures (RIs), especially when involving observational data and multiple data providers. In this work we report on our ongoing development of data rogues, a scalable approach to manage data quality issues for observational data within RIs. The motivation for this work started with the creation of the FLUXNET2015 dataset, which includes carbon, water, and energy fluxes plus micrometeorological and ancillary data measured in over 200 sites around the world. To create an uniform dataset, including derived data products, extensive work on data quality control was needed. The unpredictable nature of observational data quality issues makes the automation of data quality control inherently difficult. Developed based on this experience, the data rogues methodology allows for increased automation of quality control activities by systematically identifying, cataloging, and documenting implementations of solutions to data issues. We believe this methodology can be extended and applied to others domains and types of data, making the automation of data quality control a more tractable problem.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗
DOE OSTI2020

ATLAS data quality operations and performance for 2015–2018 data-taking

The ATLAS detector at the Large Hadron Collider reads out particle collision data from over 100 million electronic channels at a rate of approximately $100$ kHz, with a recording rate for physics events of approximately 1 kHz. Before being certified for physics analysis at computer centres worldwide, the data must be scrutinised to ensure they are clean from any hardware or software related issues that may compromise their integrity. Prompt identification of these issues permits fast action to investigate, correct and potentially prevent future such problems that could render the data unusable. This is achieved through the monitoring of detector-level quantities and reconstructed collision event characteristics at key stages of the data processing chain. This paper presents the monitoring and assessment procedures in place at ATLAS during 2015-2018 data-taking. Through the continuous improvement of operational procedures, ATLAS achieved a high data quality efficiency, with 95.6% of the recorded proton-proton collision data collected at $\sqrt{s}=13$ TeV certified for physics analysis.

43 PARTICLE ACCELERATORS↗
DOE OSTI2024

FedFleet 2024: Fleet Data Quality - Understanding and Improving Your Agency's FAST Fleet Data Quality Metric? [Slides]

This presentation presents an overview of how the team supporting the Federal Automotive Statistical Tool (FAST) project for DOE and GSA view and assess the quality of fleet data submissions from federal agencies. It also discusses ways in which agencies can improve the quality of their fleet data and of their fleet data submissions. FAST is a web-based information system sponsored by GSA's Office of Government-wide Policy and DOE's Federal Energy Management Program to collect information about the US federal government's fleet of motor vehicles; FAST is developed, maintained, and supported by DOE's Idaho National Laboratory (INL).

99 GENERAL AND MISCELLANEOUS↗
DOE OSTI2014

Monitoring and data quality assessment of the ATLAS liquid argon calorimeter

The liquid argon calorimeter is a key component of the ATLAS detector installed at the CERN Large Hadron Collider. The primary purpose of this calorimeter is the measurement of electron and photon kinematic properties. It also provides a crucial input for measuring jets and missing transverse momentum. An advanced data monitoring procedure was designed to quickly identify issues that would affect detector performance and ensure that only the best quality data are used for physics analysis. This article presents the validation procedure developed during the 2011 and 2012 LHC data-taking periods, in which more than 98% of the proton-proton luminosity recorded by ATLAS at a centre-of-mass energy of 7-8 TeV had calorimeter data quality suitable for physics analysis.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2019

Data quality assurance for the Majorana Demonstrator

The Majorana Demonstrator is an experiment constructed to search for neutrinoless double-beta decays in germanium-76 and to demonstrate the feasibility to deploy a large-scale experiment in a phased and modular fashion. It consists of two modular arrays of natural and 7648 Ge-enriched germanium detectors totalling 44.1 kg, located at the 4850' level of the Sanford Underground Research Facility in Lead, South Dakota, USA. Any Neutrinoless double-beta decay search requires a thorough understanding of the background and the signal energy spectra. This talk will discuss the various techniques employed to ensure the integrity of the measured spectra. Data collection is monitored with a thorough set of checks, and subsequent careful analysis is performed to qualify the data for higher level physics analysis. Instrumental background events are tagged for removal, and problematic channels are removed from consideration as necessary.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗
DOE OSTI2021

Perspectives on Citizen Science Data Quality

Information about data quality helps potential data users to determine whether and how data can be used and enables the analysis and interpretation of such data. Providing data quality information improves opportunities for data reuse by increasing the trustworthiness of the data. Recognizing the need for improving the quality of citizen science data, we describe quality assessment and quality control (QA/QC) issues for these data and offer perspectives on aspects of improving or ensuring citizen science data quality and for conducting research on related issues.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2014

Scanning ARM Cloud Radars. Part II: Data Quality Control and Processing

The Scanning ARM Cloud Radars (SACR’s) are the primary instruments for documenting the four-dimensional structure and evolution of clouds within a 20-30 km radius from the ARM fixed and mobile sites. Here, the post-processing of the calibrated SACR measurements is discussed. First, a feature mask algorithm that objectively determines the presence of significant radar returns is described. The feature mask algorithm is based on the statistical properties of radar receiver noise. It accounts for atmospheric emission and is applicable even for SACR profiles with few or no signal-free range gates. Using the nearest-in-time atmospheric sounding, the SACR radar reflectivities are corrected for gaseous attenuation (water vapor and oxygen) using a line-by-line absorption model. Despite having a high pulse repetition frequency, the SACR has a narrow Nyquist velocity limit and thus Doppler velocity folding is commonly observed. An unfolding algorithm that makes use of a first guess for the true Doppler velocity using horizontal wind measurements from the nearest sounding is described. The retrieval of the horizontal wind profile from the HS-RHI SACR scan observations and/or nearest sounding is described. The retrieved horizontal wind profile can be used to adaptively configure SACR scan strategies that depend on wind direction. Several remaining challenges are discussed, including the removal of insect and second-trip echoes. The described algorithms significantly enhance SACR data quality and constitute an important step towards the utilization of SACR measurements for cloud research.

climate↗
DOE OSTI2021

Quality Data Essential for Modeling Water Cycles Effectively

The experimental and observational data is the backbone of effective modeling of all aspects of water cycles. As the computer technology used for these data gathering efforts has improved dramatically in the recent decades, the environmental conditions these instruments are subjected to remain pretty much the same. These measuring instruments are exposed to the sun light, wind, rain, snow and ice; and they are immersed in dirt, water and mud. All of these are damaging to the electronics components used for data gathering, storage, and transmission, which creates many different kinds of data quality issues that affect the quality and reliability of modeling and prediction efforts.

54 ENVIRONMENTAL SCIENCES↗
DOE OSTI2016

SDSS-IV MaNGA IFS GALAXY SURVEY—SURVEY DESIGN, EXECUTION, AND INITIAL DATA QUALITY

The MaNGA Survey (Mapping Nearby Galaxies at Apache Point Observatory) is one of three core programs in the Sloan Digital Sky Survey IV. It is obtaining integral field spectroscopy for 10,000 nearby galaxies at a spectral resolution of R ∼ 2000 from 3622 to 10354 Å. The design of the survey is driven by a set of science requirements on the precision of estimates of the following properties: star formation rate surface density, gas metallicity, stellar population age, metallicity, and abundance ratio, and their gradients; stellar and gas kinematics; and enclosed gravitational mass as a function of radius. We describe how these science requirements set the depth of the observations and dictate sample selection. The majority of targeted galaxies are selected to ensure uniform spatial coverage in units of effective radius (R{sub e}) while maximizing spatial resolution. About two-thirds of the sample is covered out to 1.5 R{sub e} (Primary sample), and one-third of the sample is covered to 2.5 R{sub e} (Secondary sample). We describe the survey execution with details that would be useful in the design of similar future surveys. We also present statistics on the achieved data quality, specifically the point-spread function, sampling uniformity, spectral resolution, sky subtraction, and flux calibration. For our Primary sample, the median r -band signal-to-noise ratio is ∼70 per 1.4 Å pixel for spectra stacked between 1 R{sub e} and 1.5 R{sub e}. Measurements of various galaxy properties from the first-year data show that we are meeting or exceeding the defined requirements for the majority of our science goals.

79 ASTRONOMY AND ASTROPHYSICS↗
DOE OSTI2020

Evolution of the Data Quality Monitoring and Prompt Processing System in the proto DUNE-SP experiment

The DUNE Collaboration currently operates an experimental program based at CERN which includes a beam test and an extended cosmic ray run of two large-scale prototypes of the massive Liquid Argon Time Projection Chamber (LArTPC) for the DUNE Far Detector. The volume of data collected by the single-phase prototype (protoDUNE-SP) amounts to 3PB and the sustained rate of data sent to mass storage is O(100) MB/s. Data Quality Monitoring was implemented by directing a fraction of the data stream to the protoDUNE prompt processing system (p3s) which is optimized for continuous low-latency calculation of the vital detector metrics and various graphics including event displays. It served a crucial role throughout the life cycle of the experiment. We present our experience in leveraging the CERN computing environment and operating the system over an extended period of time, while adapting to evolving requirements and computing platform.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗
DOE OSTI2017

A priori checking of the light-response and data quality before extended data collection in pump–probe photocrystallography experiments

In picosecond and slower pump–probe diffraction experiments, collection of response–ratio correlation sets prior to full data collection provides an invaluable confirmation of the existence of a light-induced signal prior to full data collection. If a response to light exposure is observed, the quality of the data being collected can be assessed. We present a number of such correlation plots both for synchrotron and in-house pump–probe data collection.

Coppens, Philip↗
DOE OSTI2026

Machine Learning-Based Anomaly Detection for PMT Data Quality Monitoring in the SBN and DUNE

Maintaining high-quality detector data is essential for achieving the scientific objectives of the Short-Baseline Neutrino (SBN) Program at Fermilab. Current data quality monitoring (DQM) procedures rely primarily on threshold-based metrics and manual inspection of detector monitoring plots, making the detection of subtle or gradually developing anomalies both time-consuming and dependent on expert interpretation. This project developed and evaluated a machine-learning workflow for automatically identifying anomalous photomultiplier tube (PMT) channels in the Short-Baseline Near Detector (SBND) using optical-hit amplitude data. A Python-based analysis program was developed to process ROOT files, extract statistical features describing individual PMT amplitude distributions, and generate feature vectors for anomaly detection. These features were used to train an Isolation Forest model using data representing normal detector operation. The trained model was subsequently applied to independent detector runs to identify channels exhibiting statistically unusual behavior relative to the learned reference response. To support expert interpretation, the workflow generated complementary diagnostic products, including anomaly score distributions, normalized amplitude comparisons, decision-tree visualizations, and principal component analysis (PCA) projections. This project demonstrated the feasibility of integrating unsupervised machine learning into detector data-quality monitoring and developed a complete workflow for automated PMT performance assessment to aid expert-driven review. Beyond its technical contributions, the VFP appointment fostered a research collaboration between Aurora University and Fermilab and provided direct workforce development benefits by training the visiting faculty member in detector-scale machine-learning methods that are now being incorporated into undergraduate coursework and research. The methodology developed here provides a foundation for future applications to ProtoDUNE and other liquid argon time projection chamber (LArTPC) detectors, contributing to ongoing efforts to improve detector reliability, reduce manual monitoring requirements, and enable scalable data quality monitoring for future large-scale neutrino experiments, including the Deep Underground Neutrino Experiment (DUNE).

Colón Santana, Juan A. [Unlisted, US, IL]↗
DOE OSTI2019

Defining a Use Case for ADMS Testbed: Data Quality Requirements for ADMS Deployment

Advanced distribution management systems (ADMS) provide a suite of tools to meet the needs of a modern grid: increased reliability and power quality, improved resiliency and security, reduced costs, and enhanced customer participation. A critical challenge that utilities face with adoption of ADMS is the quality of models and data that the ADMS uses for making control decisions. Data quality has a two-fold impact on ADMS adoption: 1. Data quality improvement might constitute up to 25% of ADMS deployment costs. 2. The accuracy of data and models used by the ADMS affects the utility's ability to meet its operational objectives. Thus, quantifying the data quality requirements and its impact on performance is critical to reducing the overall cost of deployment, enabling increased adoption and ensuring that the ADMS performs as specified. This paper offers the motivation, methodology, and evaluation strategy to fill this critical gap.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

Provenance–aware workflow for data quality management and improvement for large continuous scientific data streams

Data quality assessment, management and improvement is integral part of any big data intensive scientific research to ensure accurate, reliable and reproducible science discoveries. The task of maintaining the quality of data, however, is non-trivial and pose a challenge for a program like Department of Energy’s Atmospheric Radiation Measurement (ARM) that collects data from hundreds of instruments across the world,and distributes thousands of streaming data products that are continuously growing in near-real-time in an archive 1.7 Petabyte in size and growing. We present a computational data processing workflow to collect the data quality issue via an easy and intuitive web-based portal that allows reporting of any quality issues for any site, facility or instruments at a granularity down to individual variable in the data files. Portal allows instrument specialists and scientists to provide corrective actions in form of symbolic equation. A parallel processing framework applies the data improvement to large volume of data in efficient parallel environment, while optimizing data transfer and file I/O operations. Corrected files are systematically versioned and archived. A provenance tracking module tracks and records any change made to the data during its entire life cycle which are communicated transparently to the scientific users. Developed in Python using open source technologies, software architecture enables efficient and fast management and improvement of data in an operational data center environment.

Kumar, Jitendra↗