Search NASA⌕ Search

SEARCH · Search NASA

Results for “data quality issues”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

About a method for compressing X-ray computed microtomography data

The management of scientific data is of high importance especially for experimental techniques that produce big data volumes. Such a technique is x-ray computed tomography (CT) and its community has introduced advanced data formats which allow for better management of experimental data. Rather than the organization of the data and the associated meta-data, the main topic on this work is data compression and its applicability to experimental data collected from a synchrotron-based CT beamline at the Elettra-Sincrotrone Trieste facility (Italy) and studies images acquired from various types of samples. This study covers parallel beam geometry, but it could be easily extended to a cone-beam one. The reconstruction workflow used is the one currently in operation at the beamline. Contrary to standard image compression studies, this manuscript proposes a systematic framework and workflow for the critical examination of different compression techniques and does so by applying it to experimental data. Beyond the methodology framework, this study presents and examines the use of JPEG-XR in combination with HDF5 and TIFF formats providing insights and strategies on data compression and image quality issues that can be used and implemented at other synchrotron facilities and laboratory systems. In conclusion, projection data compression using JPEG-XR appears as a promising, efficient method to reduce data file size and thus to facilitate data handling and image reconstruction

data compression↗

Elucidating and predicting the dynamic evolution of water and land systems due to natural and energy-related forcings

Focal Area(s): 3. Insight gleaned from complex data (both observed and simulated) using AI, big data analytics, and other advanced methods, including explainable AI and physics- or knowledge-guided AI; & 1. Data acquisition and assimilation enabled by machine learning, AI, and advanced methods including experimental/network design/optimization, unsupervised learning (including deep learning), and hardware-related efforts involving AI (e.g., edge computing). Science Challenge: Interactions between water, land, and energy systems are complex and occur on a variety of scales, ranging from local to basinal to regional. Accurately predicting the behavior of ground water and surface water systems for 5-10 years and beyond requires an understanding of the current system and the ability to model both the natural system at scale and human-induced forcings related to energy and other activities. Artificial intelligence and machine learning (AI/ML) combined with modern compilation and integration efforts for U.S. groundwater and surface water systems present potential solutions to bolstering detailed physics-based models of these systems. Big data tied with ML and physics-based modeling can drive breakthroughs in understanding the earth system, but research is often impeded by data access (e.g., privacy issues), quality, formats, gaps, multi-source, multi-scale, integration, and spatiotemporal challenges. Effective integration of real data and simulated (synthetic) data that fill gaps is critical. Overcoming these complex data and model integration challenges will enable a transformational approach to acquiring enhanced understanding of environmental systems.

54 ENVIRONMENTAL SCIENCES↗

Data Quality Reporting

This is a repository for tracking issues with metadata in the National Geothermal Data System. Each page representing a resource in NGDS has a link for the user to report issues with the metadata. NGDS teams also use this issue tracker to indicate problems and manage their resolution.

Source record↗

Data Quality Reporting

This is a repository for tracking issues with metadata in the National Geothermal Data System. Each page representing a resource in NGDS has a link for the user to report issues with the metadata. NGDS teams also use this issue tracker to indicate problems and manage their resolution.

Source record↗

Improved joint X-ray and neutron refinement procedure in Phenix

Neutron diffraction is one of the three crystallographic techniques (X-ray, neutron and electron diffraction) used to determine the atomic structures of molecules. Its particular strengths derive from the fact that H (and D) atoms are strong neutron scatterers, meaning that their positions, and thus protonation states, can be derived from crystallographic maps. However, because of technical limitations and experimental obstacles, the quality of neutron diffraction data is typically much poorer (completeness, resolution and signal to noise) than that of X-ray diffraction data for the same sample. Further, refinement is more complex as it usually requires additional parameters to describe the H (and D) atoms. The increase in the number of parameters may be mitigated by using the `riding hydrogen' refinement strategy, in which the positions of H atoms without a rotational degree of freedom are inferred from their neighboring heavy atoms. However, this does not address the issues related to poor data quality. Therefore, neutron structure determination often relies on the presence of an X-ray data set for joint X-ray and neutron (XN) refinement. In this approach, the X-ray data serve to compensate for the deficiencies of the neutron diffraction data by refining one model simultaneously against the X-ray and neutron data sets. To be applicable, it is assumed that both data sets are highly isomorphous, and preferably collected from the same crystals and at the same temperature. However, the approach has a number of limitations that are discussed in this work by comparing four separately re-refined neutron models. To address the limitations, a new method for joint XN refinement is introduced that optimizes two different models against the different data sets. This approach is tested using neutron models and data deposited in the Protein Data Bank. The efficacy of refining models with H atoms as riding or as individual atoms is also investigated.

36 MATERIALS SCIENCE↗

EPICS-DDS

This paper presents a new extension to EPICS, approaching the Data Distributed Service (DDS) interface based on the Channel Access protocol. DDS is the next generation of middleware industrial standards, bringing a data-centric publish-subscribe paradigm to distributed control systems. In comparison with existing middleware technologies, the data-centric approach is able to provide a consistent consolidated model supporting difference data dissemination scenarios and integrating many important issues, such as quality of service, user-specific data structures, and others. The paper considers different features of the EPICS-DDS layer in the context of the high-level accelerator environment.

43 PARTICLE ACCELERATORS↗

FAIRification, Quality Assessment, and Missingness Pattern Discovery for Spatiotemporal Photovoltaic Data

The ongoing growth of the photovoltaic market has pushed the demand for power forecasting and performance evaluation for a huge population of PV power plants. Through access to a large number of time series data sets from different power plants, we have found common issues that impede the modeling process. Namely, the time series data are hard to transfer between groups due to differences in variable nomenclature, and the quality of the data sets can vary. We address the issue of variable nomenclature by FAIRifying spatiotemporal PV time series data. Through the creation of a solar power plant ontology, we propose standards for the naming and structure of metadata used to describe the data from these power plants. Using the structure from this ontology, we have developed both R and Python packages for the automation of the FAIRification process. We have also developed an R package that automates the analysis of the quality of a data set through the designation of letter grades. With access to large time series data sets across many power plants, we can utilize spatiotemporal coherence between the sites in order to improve the quality of our data. To solve the issue of data missingness, we propose the use of Spatiotemporal-GNN autoencoders to detect and impute missing values from a data set by utilizing data from power plants nearby.

14 SOLAR ENERGY↗

Analysis of rapidly synthesized guest-filled porous complexes with synchrotron radiation: practical guidelines for the crystalline sponge method

This report describes complete practical guidelines and insights for the crystalline sponge method, which have been derived through the first use of synchrotron radiation on these systems, and includes a procedure for faster synthesis of the sponges. These guidelines will be applicable to crystal sponge data collected at synchrotrons or in-house facilities, and will allow researchers to obtain reliable high-quality data and construct chemically and physically sensible models for guest structural determination. A detailed set of synthetic and crystallographic guidelines for the crystalline sponge method based upon the analysis of expediently synthesized crystal sponges using third-generation synchrotron radiation are reported. The procedure for the synthesis of the zinc-based metal–organic framework used in initial crystal sponge reports has been modified to yield competent crystals in 3 days instead of 2 weeks. These crystal sponges were tested on some small molecules, with two being unexpectedly difficult cases for analysis with in-house diffractometers in regard to data quality and proper space-group determination. These issues were easily resolved by the use of synchrotron radiation using data-collection times of less than an hour. One of these guests induced a single-crystal-to-single-crystal transformation to create a larger unit cell with over 500 non-H atoms in the asymmetric unit. This led to a non-trivial refinement scenario that afforded the best Flack x absolute stereochemical determination parameter to date for these systems. The structures did not require the use of PLATON/SQUEEZE or other solvent-masking programs, and are the highest-quality crystalline sponge systems reported to date where the results are strongly supported by the data. A set of guidelines for the entire crystallographic process were developed through these studies. In particular, the refinement guidelines include strategies to refine the host framework, locate guests and determine occupancies, discussion of the proper use of geometric and anisotropic displacement parameter restraints and constraints, and whether to perform solvent squeezing/masking. The single-crystal-to-single-crystal transformation process for the crystal sponges is also discussed. The presented general guidelines will be invaluable for researchers interested in using the crystalline sponge method at in-house diffraction or synchrotron facilities, will facilitate the collection and analysis of reliable high-quality data, and will allow construction of chemically and physically sensible models for guest structural determination.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A Comprehensive Analysis of Real-World Accelerometer Data Quality in a Global Smartphone-based Seismic Network

The proliferation of low-cost sensors in smartphones has facilitated numerous applications; however, large-scale deployments often encounter performance issues. Sensing heterogeneity, which refers to varying data quality due to factors such as device differences and user behaviors, presents a significant challenge. In this research, we perform an extensive analysis of 3-axis accelerometer data from the MyShake system, a global seismic network utilizing smartphones. We systematically evaluate the quality of approximately 22 million 3-axis acceleration waveforms from over 81 thousand smartphone devices worldwide, using metrics that represent sampling rate and noise level. We explore a broad range of factors influencing accelerometer data quality, including smartphone and accelerometer manufacturers, phone specifications (release year, RAM, battery), geolocation, and time. Our findings indicate that multiple factors affect data quality, with accelerometer model and smartphone specifications being the most critical. In addition, we examine the influence of data quality on earthquake parameter estimation and show that removing low-quality accelerometer data enhances the accuracy of earthquake magnitude estimation.

58 GEOSCIENCES↗

Water intensity of photovoltaic module manufacturing at the terawatt scale

As the U.S. ramps photovoltaic (PV) manufacturing to the terawatt scale and emphasizes re-shoring manufacturing, potential regional impacts on the U.S. water supply should be considered, particularly since many PV companies rely almost exclusively on public water supplies for manufacturing. This work surveys the academic literature and PV manufacturer reports to estimate the water intensity of monocrystalline silicon, multicrystalline silicon, and cadmium telluride modules manufactured at the terawatt scale, determining that on average, cadmium telluride manufacturing is less water intensive on a per megawatt scale – this is anticipated to be true for all thin film PV manufacturing. While much lower than the water intensity of thermoelectric (e.g., coal) energy generation, significant issues and gaps with PV manufacturing data quality in academic studies are identified which cause estimates to vary by over 1000x (0.04 – 49 trillion liters/terawatt). Data issues are discussed and the need for accurate accounting of water resources (e.g., via continuous, updated information during PV manufacturing) is highlighted. The opportunity to reconfigure decommissioned thermoelectric sites to PV manufacturing is also explored. Finally, factors that influence PV manufacturing water intensity, from individual manufacturing steps to trends across the PV industry, are examined and water conservation opportunities are presented.

14 SOLAR ENERGY↗

IPN Absorption Coefficients

The dataset contains aerosol light absorption and scattering coefficients measured by three single-wavelength Integrated Photoacoustic Nephelometers (IPN). The absorption coefficients were measured at 405, 721, and 1047 nm, while the scattering coefficients were measured at 405 and 721 nm. The single scattering albedo (SSA) at 405 and 721 nm were also calculated from the IPN measurements. The three IPNs shared a common inlet without impactor/cyclone in July; in August, a PM2.5 cyclone inlet was installed to reduce measurement noise and improve data quality. The 721-nm IPN experienced laser shutdown issues, causing missing of a large fraction of the 721-nm measurements, especially in July. As described before, the 721-nm laser issue was less frequent in August after the cyclone installation, improving the data completeness and quality. The IPN raw data were noisy, making identifying patterns and trends difficult; therefore, the hourly average data are recommended over the raw data. The hourly average data were cross-checked with the filter-based TAP and PSAP measurements, which showed that their patterns, trends, and peaks matched. However, due to the differences in working mechanisms and principles of the filter-based and photoacoustic particle-phase instruments, the IPN measurements are lower than the filter-based measurements despite that they have consistent patterns, trends, and peaks.

54 ENVIRONMENTAL SCIENCES↗

Identifying Suspect Instrument Intervals Using Midnight Noise Time Histories

With an intense work ethic, and high levels of devotion to their craft, seismologists continue to battle the elements and wrestle with complex logistics in their efforts to field instrumentation over the entire globe. We are fortunate to see data volumes accelerating in size, and quality continuing to improve; for example, universal timing issues can now be considered rare. However, instrument response issues are still common, and can hinder research that seeks to understand amplitudes of seismic wavefields. We are interested in using the accumulations of seismic data to develop models that will predict high frequency (0.2-20 Hz, and higher) signal amplitudes for explosion monitoring purposes over broad areas, which requires large data sets and extensive quality control. To identify response and station health issues, we have collected noise time histories for global seismic data, focusing on measurements near (but not restricted to) midnight to eliminate diurnal variations, and have manually determined time intervals that appear inconsistent with background behavior. We assign descriptive labels, but do not attempt to diagnose causes. We use these intervals to discard data. To date, we have examined 39,260 channels from 11,105 stations, heavily weighted toward IRIS holdings, dates through 2017 (depending on station, roughly the date we started our manual review), bands between 1 and 8 Hz, finding 24,733 anomalous time intervals. The great majority (90%) of these intervals appear to be shifts of constant offset, often bounded by times of known instrument changes, likely the result of poor documentation of response parameters at one of many stages between the field and the plotter. We hope these results can be of use to our colleagues, and would encourage community efforts to diagnose anomalous behavior, and fix poor responses. We also hope these results will support automation efforts, including application of supervised learning techniques.

47 OTHER INSTRUMENTATION↗

Detection and attribution of climate extremes in the observed record

We present an overview of practices and challenges related to the detection and attribution of observed changes in climate extremes. Detection is the identification of a statistically significant change in the extreme values of a climate variable over some period of time. Issues in detection discussed include data quality, coverage, and completeness. Attribution takes that detection of a change and uses climate model simulations to evaluate whether a cause can be assigned to that change. Additionally, we discuss a newer field of attribution, event attribution, where individual extreme events are analyzed for the express purpose of assigning some measure of whether that event was directly influenced by anthropogenic forcing of the climate system.

54 ENVIRONMENTAL SCIENCES↗

Automatic 3D image registration for nano-resolution chemical mapping using synchrotron spectro-tomography

Nano-resolution synchrotron X-ray spectro-tomography has been demonstrated as a powerful tool for probing the three-dimensional (3D) structural and chemical heterogeneity of a sample. By reconstructing a number of tomographic data sets recorded at different X-ray energy levels, the energy-dependent intensity variation in every given voxel fingerprints the corresponding local chemistry. The resolution and accuracy of this method, however, could be jeopardized by non-ideal experimental conditions, e.g. instability in the hardware system and/or in the sample itself. Herein is presented one such case, in which unanticipated sample deformation severely degrades the data quality. Here, to address this issue, an automatic 3D image registration method is implemented to evaluate and correct this effect. The method allows the redox heterogeneity in partially delithiated Li x Ta 0.3 Mn 0.4 O 2 battery cathode particles to be revealed with significantly improved fidelity.

3D image registration↗

CHESS 2025: Post-survey report for 2025 NEON AOP Assignable Asset collection of East River and Washington Gulch, Almont and Upper Taylor watersheds at Crested Butte, CO

This report contains details of the National Ecological Observatory Network (NEON) Airborne Observation Platform (AOP) Research Support Services (RSS) Assignable Asset (AA) flights of the East River, Almont and Upper Taylor watersheds near Crested Butte, CO, June–July 2025. The Rocky Mountain Biological Laboratory (RMBL) contracted the NEON AOP AA flights to observe watersheds of interest near Crested Butte with remotely sensed data including high resolution LiDAR, imaging spectroscopy, and high-resolution camera imagery. The report includes a summary of the acquired flight lines over the planned survey areas, results of calibration flights, and results of the acquired data. The report details how the AOP has met the contracted delivery requirements in terms of the data delivered, quality of the data, and describes issues that resulted in data degradation or data loss. CHESS Project Description: The Colorado Headwaters Ecological Spectroscopy Study (CHESS) comprised a multi-week airborne remote sensing and field observation campaign in the Upper Gunnison Basin, Colorado, conducted in June and July of 2025. Airborne remote sensing was conducted by the National Ecological Observatory Network Airborne Observation Platform (NEON AOP), concurrent with a field campaign run by the Rocky Mountain Biological Laboratory (RMBL), the Lawrence Berkeley National Laboratory (LBNL) and SLAC National Accelerator Laboratory Watershed Function Science Focus Area (SFA), and NASA-JPL (Jet Propulsion Laboratory) Earth Surface Mineral Dust Source Investigation (EMIT) program. Between June 10 and July 18, 2025, the NEON AOP flight team collected high-resolution aerial imaging spectroscopy and Light Detection and Ranging (LiDAR) data over three domains: the Upper East River (CRBU), Almont Triangle (ALMO), and the Upper Taylor Basin (UPTA). In coordination with the flights, a field campaign acquired ground-truth observations, including observations of vegetation composition, foliar traits, forest demography, and subsurface properties in 18 core sampling areas within the domains. Additional surface water observations were taken at over 380 point locations. All CHESS campaign datasets can be found within the CHESS ESS-DIVE data portal: https://data.ess-dive.lbl.gov/portals/chess. Funding Acknowledgement: Field and remote-sensing data acquisition was performed under a grant from the National Aeronautics and Space Administration (80NSSC24K1005). This work was also supported by the Watershed Function Science Focus Area at Lawrence Berkeley National Laboratory funded by the US Department of Energy, Office of Science, Biological and Environmental Research under Contract No. DE-AC02-05CH11231.

2018 NEON and 2025 CHESS Campaigns↗

High energy x-ray and neutron studies of disordered energy-related materials at extreme conditions

The fundamental scientific accomplishments are: (1) advances in a general description of the liquid state by employing structural models constrained by measurements to interpret experimental results and extend them to liquids in general, with special emphasis on (2) The structure of the high-temperature crystal and molten UO 2 and 3) water. Specifically, samples of UO 2 and water were probed using high-energy x-rays at the Advanced Photon Source. The high Z of UO 2 , and the 2-3mm diameter droplet shape of the molten sample, means that >100keV X-rays are required to minimize absorption and multiple scattering, which can distort the measured structure factor. A high flux of x-rays is also required to obtain sufficient statistical accuracy in short (a few seconds) measurement times. The scattered x-ray data were analyzed and pair distribution functions, extracted that characterize the local and long-range atomic structure of the material. The measurements of the hot UO 2 solid show a substantial increase in oxygen disorder and, upon melting, the average U-O coordination was found to decrease from 8 to 6.7±0.5. The research incorporated development of diffraction techniques, sample environment optimization and state-of-the-art simulation techniques. The symbiotic nature of the advances in simulation and experiment allowed for a more focused and informed development of future experiments, effective use of expensive beam time and generated new research agendas for the growing number of research groups, within the US and internationally, that focus on the structure of liquids. Molecular dynamics (MD) provided detailed information when combined with high-quality XN data including addressing key issues in liquids; the relationship between cooling path, structure and fictive temperature, and the trade-offs between network over connectedness in liquids containing low-coordination cations.

11 NUCLEAR FUEL CYCLE AND FUEL MATERIALS↗

Recommendations for mass spectrometry data quality metrics for open access data(corollary to the Amsterdam principles)

Policies supporting the rapid and open sharing of proteomic data are being implemented by the leading journals in the field. The proteomics community is taking steps to ensure that data are made publicly accessible and are of high quality, a challenging task that requires the development and deployment of methods for measuring and documenting data quality metrics. On September 18, 2010, the U.S. National Cancer Institute (NCI) convened the 'International Workshop on Proteomic Data Quality Metrics' in Sydney, Australia, to identify and address issues facing the development and use of such methods for open access proteomics data. The stakeholders at the workshop enumerated the key principles underlying a framework for data quality assessment in mass spectrometry data that will meet the needs of the search community, journals, funding agencies, and data repositories. Attendees discussed and agreed upon two primary needs for the wide use of quality metrics: (i)an evolving list of comprehensive quality metrics and (ii)standards accompanied by software analytics. Attendees stressed the importance of increased education and training programs to promote reliable protocols in proteomics. This workshop report explores the historic precedents, key discussions, and necessary next steps to enhance the quality of open access data. By agreement, this article is published simultaneously in Proteomics, Proteomics Clinical Applications, Journal of Proteome Research, and Molecular and Cellular Proteomics, as a public service to the research community.The peer review process was a coordinated effort conducted by a panel of referees selected by the journals.

59 BASIC BIOLOGICAL SCIENCES↗

Autoencoder-Based Anomaly Detection System for Online Data Quality Monitoring of the CMS Electromagnetic Calorimeter

The CMS detector is a general-purpose apparatus that detects high-energy collisions produced at the LHC. Online data quality monitoring of the CMS electromagnetic calorimeter is a vital operational tool that allows detector experts to quickly identify, localize, and diagnose a broad range of detector issues that could affect the quality of physics data. A real-time autoencoder-based anomaly detection system using semi-supervised machine learning is presented enabling the detection of anomalies in the CMS electromagnetic calorimeter data. A novel method is introduced which maximizes the anomaly detection performance by exploiting the time-dependent evolution of anomalies as well as spatial variations in the detector response. The autoencoder-based system is able to efficiently detect anomalies, while maintaining a very low false discovery rate. The performance of the system is validated with anomalies found in 2018 and 2022 LHC collision data. In addition, the first results from deploying the autoencoder-based system in the CMS online data quality monitoring workflow during the beginning of Run 3 of the LHC are presented, showing its ability to detect issues missed by the existing system.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗