Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

A Model-Based Framework for Air Quality Indices and Population Risk Evaluation, with an Application to the Analysis of Scottish Air Quality Data

Summary The paper is devoted to the development of a statistical framework for air quality assessment at the country level and for the evaluation of the ambient population exposure and risk with respect to airborne pollutants. The framework is based on a multivariate space–time model and on aggregated indices defined at different levels of aggregation in space and time. The indices are evaluated, uncertainty included, by considering both the model outputs and the information on the population spatial distribution. The framework is applied to the analysis of air quality data for Scotland for 2009 referring to European and Scottish air quality legislation.

Unbalanced networks↗

Enhancing Data Quality Monitoring at CMS with Interactive Visualization Tools and Automated Reference Run Selection

Current data quality monitoring (DQM) tools at CMS offer granularity limited to per-run analysis. Consequently, issues manifesting at the per-lumisection level can go unnoticed or, even if detectable, often lead to the classification of the whole run as bad, resulting in unnecessary data loss. Additionally, shifters have to evaluate a large set of monitoring elements during their long shifts, increasing the probability of human errors or overlooked problems. In this contribution, we present ongoing work on the development of tools that will provide shifters with an accessible, granularity-enhanced view of DQM data through interactive and dynamic visualizations. Furthermore, we introduce a reference run selection tool currently under development, which will automate the selection based on data-taking conditions and will offer a curated set of training data for machine learning models that will be used for the partial automation of the offline data certification process. These endeavors will be integrated into the DIALS website, enabling enhancements in data certification accuracy and improving the accessibility of DQM at CMS.

72 PHYSICS OF ELEMENTARY PARTICLES AND FIELDS↗

Hydra: Computer Vision for Online Data Quality Monitoring

Hydra is a system utilizing computer vision for near real-time data quality monitoring. Currently operational across all of Jefferson Lab’s experimental halls, it reduces the workload of shift takers by autonomously monitoring diagnostic plots during experiments. Hydra uses "off-the-shelf" supervised learning technologies and is supported by a comprehensive MySQL database. To simplify access, web apps have been developed to facilitate both labeling and monitoring of Hydra’s inferences. Hydra can connect with the alarm system and incorporates complete historical tracking, enabling it to identify issues that shift takers could miss. When issues are detected, a natural first question is: "Why does Hydra think there is a problem?" To answer, Hydra employs Gradient-weighted Class Activation Maps (GradCAM) to identify regions of the image that are important for the specific classification. This interpretive layer enhances transparency and trustworthiness, which is essential for integration with experiment workflows and operation. The Hydra system, results, and sociological considerations for deployment will be discussed.

Jeske, Torri↗

RE Data Explorer is Expanding Public Access to High-Quality Data and Analytics

The Renewable Energy (RE) Data Explorer is an innovative and intuitive web platform from the U.S. Agency for International Development (USAID)-National Renewable Energy Laboratory (NREL) Partnership that enables users to easily access high-quality renewable energy resource data and applied analytics.

ENERGY PLANNING, POLICY, AND ECONOMY↗

Online and Offline Data Quality Monitoring for the Mu2e Calorimeter

This thesis presents the design, implementation, and validation of a calorimeter Data Quality Monitoring (DQM) toolchain for the Mu2e experiment at Fermilab. Mu2e searches for charged lepton flavor violation via coherent muon-to-electron conversion in the field of an aluminum nucleus, $\mu^- Al \rightarrow e^-Al$, a process whose observation would constitute clear evidence of physics beyond the Standard Model. Achieving target sensitivity requires stringent control of detector performance and data integrity during acquisition, as subtle issues in readout configuration, data formatting, or electronics behavior can compromise reconstruction and bias downstream analyzes. To address these challenges, this work develops a multi-layer DQM approach spanning both raw data validation and reconstructed digi-level diagnostics. At the low level, a fragment analysis component performs word- and bit-field decoding of calorimeter readout blocks, enabling sanity checks of the expected structure and producing detailed error and integrity statistics useful for commissioning and troubleshooting. At the digi level, the CaloDigiDQM analyzer is implemented within the art framework and transforms each CaloDigiCollection into a structured hierarchy of ROOT histograms designed for fast drill-down diagnostics. The module generates coherent monitoring views at global, disk, board, and channel granularity, including occupancy, waveform-derived features (baseline, RMS, peak amplitude and position), and left-right sensor consistency metrics. Detector-aware channel-to-electronics mapping is performed through the conditions system (CaloDAQMap), ensuring that diagnostics remain aligned with hardware identifiers used in operations. For end-to-end testing without reliance on live DAQ data, a synthetic CaloDigi producer is developed to generate realistic waveforms with controlled noise and pulse shapes. The resulting system supports both offline ROOT-file production and online operation, including optional histogram streaming through otsdaq via ots::HistoSender. This toolchain provides a practical and scalable foundation for calorimeter commissioning and stable data collection, enabling early detection of anomalies and reducing operational risk for Mu2e.

Vakulenko, Mark [Drew U.] (ORCID:0009000276197818)↗

Hydra: An AI-Based Framework for Interpretable and Portable Data Quality Monitoring

Hydra is an advanced framework designed for training and managing AI models for near real time data quality monitoring at Jefferson Lab. Deployed in all four experimental halls, Hydra has analyzed over 2 million images and has extended its capabilities to offline monitoring and validation. Hydra utilizes computer vision to continually analyze sets of images of monitoring plots generated 24/7 during experiments. Generally, these sets of images are produced at a rate and quantity that is exceedingly difficult for shift crews to effectively monitor. Significant effort has been devoted to enhancing Hydra’s user interface, to ensure that it provides clear, actionable insights for shift workers and other users. Gradient Weighted Class Activation Maps (GradCAM) provide added interpretability, allowing users to visualize important regions of the image for classification. Hydra has been containerized to enable the creation of portable demos and seamless integration with container-based technologies such as Kubernetes and Docker. With the user interface enhancements and containerization, Hydra can be rapidly deployed for new use cases and experiments. This talk will describe the Hydra framework, its user interface and experience, and the challenges inherent in its design and deployment.

Britton, Thomas [Thomas Jefferson National Acceler↗

AI Data Quality Monitoring with Hydra

Hydra is an extensible framework for training and managing AI for near real time monitoring that aims to replace the tedious and repetitive data quality monitoring activities the shift crew and online monitoring coordinator typically perform. It continuously scans incoming data in the form of monitoring plots for signs of problems, flagging them for human review. A web app was developed such that experts can efficiently label images for training. Labels are stored in a database for use in training and model validation. Backed up by a comprehensive database, it utilizes an additional web based front-end for viewing the current monitoring status from anywhere in the world. The system has been in production use for the GlueX experiment at Jefferson Lab for more than 2 years with new features still under active development.

Britton, Thomas↗

Data Quality Objectives Supporting the Environmental Direct Radiation Monitoring Program for the Idaho National Laboratory

This document presents the development of the data quality objectives (DQOs) for the Idaho National Laboratory (INL) Environmental Direct Radiation Monitoring Program and follows the Environmental Protection Agency (EPA) DQO process (EPA 2006). This document also develops and presents the logic to determine the specific number of direct radiation monitoring locations around INL facilities on the desert west of Idaho Falls and in Idaho Falls, at locations bordering the INL Site, and in the surrounding regional area. The selection logic follows the guidance from the Department of Energy (DOE) (2015) for environmental surveillance of DOE facilities.

54 ENVIRONMENTAL SCIENCES↗

Hydra: computer vision for data quality monitoring

Hydra, initially developed for Hall-D in 2019, is a system that utilizes computer vision to perform near real time data quality monitoring. Since then, it has been deployed across all experimental halls at Jefferson Lab, with the CLAS12 collaboration in Hall-B being the first outside of GlueX to fully utilize Hydra. The system comprises back end processes that manage the models, their inferences, and the data flow. Finally, the front-end components, accessible via web pages, allow detector experts and shift crews to view and interact with the system.

47 OTHER INSTRUMENTATION↗

Data Quality Objectives Supporting the Environmental Soil Monitoring Program for the Idaho National Laboratory Site

This document describes the process used to develop data quality objectives for the Idaho National Laboratory (INL) Environmental Soil Monitoring Program in accordance with U.S. Environmental Protection Agency guidance. This document also develops and presents the logic that was used to determine the specific number of soil monitoring locations at the INL Site, at locations bordering the INL Site, and at locations in the surrounding regional area. The monitoring location logic follows the guidance from the U.S. Department of Energy for environmental surveillance of its facilities.

54 ENVIRONMENTAL SCIENCES↗

Historical Data Analysis Supporting the Data Quality Objectives for the INL Site Environmental Soil Monitoring Program

This document represents the initial evaluation and soil monitoring proposed by Battelle Energy Alliance, LLC (BEA) in 2015. The evaluation included analyses of historical soil monitoring data and soil inventories, current emission estimates, and modeled potential deposition/accumulation patterns. The initially proposed monitoring included a 5-year rotation of in-situ gamma measurements augmented by soil sampling with laboratory analyses near each major active and some inactive facilities. It also proposed rotational in-situ gamma measurements and soil sampling at two centrally located onsite air monitoring locations coinciding with sampling at the traditional offsite soil monitoring locations. The chosen alternative includes only physical soil sampling with laboratory analysis and only at the Radioactive Waste Management Complex (RWMC), the two air monitors and the offsite locations as documented in Data Quality Objectives Supporting the Environmental Soil Monitoring Program for the Idaho National Laboratory (INL) Site, INL/EXT-15-34909, Revision 0, February 2016. The data and evaluations in this document are valid for comparisons with future soil data that may be collected in many INL site locations.

54 ENVIRONMENTAL SCIENCES↗

ATLAS muon data quality with 2010 LHC

ATLAS is a large multipurpose detector, presently in the full operational phase at LHC, the CERN Large Hadron Collider accelerator. The ATLAS Muon Spectrometer has been designed to provide standalone measurement of the muon momentum up to a few TeV with a relative precision better that 10% and to trigger on single muons with transverse momentum down to a few GeV, and an efficiency close to 100%, taking into account the high level background environment, the inhomogeneous magnetic field, and the large size of the apparatus (24 m diameter by 44 m length). ATLAS MS detector has being commissioned using all types of physics data available: cosmic rays and events produced during a few days of LHC single beam operations. This first year of data taking has been of great interest, not only for the physics outcome, but also for operating the system under the environment it was designed for. This paper is describing the design, the implementation as well as the data quality performance achieved during this first year of data taking. We also describe how to reduce the cosmic background contamination in early collisions data. (authors)

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

The Prompt Processing System and Data Quality Monitoring in the ProtoDUNE-SP Experiment

The DUNE Collaboration is conducting an experimental program (named protoDUNE) which involves a beam test of two large-scale prototypes of the DUNE Far Detector at CERN operating in 2018-2019. The volume of data to be collected by the protoDUNE-SP (the single-phase detector) will amount to a few petabytes and the sustained rate of data sent to mass storage is in the range of a few hundred MB per second. After collection the data is committed to storage at CERN and transmitted to Fermi National Accelerator Laboratory in the US for processing, analysis and long-term preservation. The protoDUNE experiment requires substantial Data Quality Monitoring capabilities in order to ascertain the condition of the detector and its various subsystems. We present the design of the protoDUNE Prompt Processing System, its deployment at CERN and its performance during the data challenges and actual data taking.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

Characterization of Non-Science Grade DESI CCDs and Creating Additional ICARUS Monitoring Metrics for Data Quality

DESI, otherwise known as the Dark Energy Spectroscopic Instrument, is an astronomical project measuring millions of optical spectra to characterize Dark Energy. The survey has been running for 3 years. My work delves into the data taking/analysis side of the DESI CCDs (Charge-Coupled Devices) which will help in swapping out broken CCDs on the instrument. Additionally, I worked on making new metrics for ICARUS, Imaging Cosmic and Rare Underground Signals for their monitoring of run data. This paper will talk about the types of characterizations done for DESI and additional metrics for data quality and tools used for ICARUS monitoring.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

PMU Data Quality and Sensor Health Monitoring

Phasor Measurement Units (PMUs) play a critical role in the evolution of the electric power industry by providing high-precision, real-time monitoring of essential power system metrics. However, effectively detecting abnormalities and critical events from PMU data is a complex task, complicated by intricate temporal patterns, a scarcity of labeled data for training algo- rithms, and constraints on online computational power. In this study, we apply TranAD, an innovative algorithm that combines transformer architectures with the refinement of adversarial learning, to both synthetic and real-world PMU datasets for developing a data quality and sensor online health monitoring platform for utilities. Our findings reveal that TranAD not only provides efficient detection and localization but also enhances the detail with which abnormalities are detected, marking a a significant step forward in the field of clean data acquisition processes for power system monitoring

deep neural network, machine learning (ML)↗

Waveform Data Quality Assessment

Healthy seismoacoustic sensors and accurate metadata are critical to all the science objectives of both the Source Physics Experiment (SPE) Rock Valley Direct Comparison (RV/DC) and LowYield Physics Experiment-1 (LYNM PE-1). Both projects share similar seismoacoustic stations. We provide a plan on how to move forward with waveform data quality assessments (QA) to the working groups. The identification of stations with instrument response errors will be useful to researchers using amplitudes to study source effects and Earth attenuation. The data QA can also be useful for field technicians to help identify and correct problem sites.

47 OTHER INSTRUMENTATION↗