Search NASA⌕ Search

SEARCH · Search NASA

Results for “missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Predicting Large‐Scale Systematic Missing Pipe Attributes in Water Distribution Networks

Water distribution network (WDN) models are an essential tool used by water utilities for hydraulic analysis. Unfortunately, missing data and insufficient resources often make creating and maintaining these models unfeasible. Existing methods to address missing pipe properties, like sequential imputation for missing values and reconstruction using graph metrics, are designed to accommodate random patterns of missing information and require a significant percentage of the system's attributes to be known. However, these data completeness assumptions do not always align with real‐world scenarios where large sections of the WDN model have missing data. To address this challenge, this study proposes a data‐driven approach for estimating pipe diameter when considering different spatial patterns and degrees of data completeness (i.e., 0%–90%). Using data from 16 WDNs in Kentucky, this study compares the use of machine learning (ML) using topological and geospatial features against an existing deterministic approach. Results demonstrate that WDN models with pipe diameters predicted by the proposed ML method had comparable hydraulic performance to the ground truth models. Moreover, results showed that ML method performance varies between WDNs of differing topological classification. Insights from this study help advance the ability to leverage partial data to create and maintain WDN models amid uncertainty and inadequate resources.

Poff, Jason W. [Oregon State Univ., Corvallis, OR ↗

Electromechanical Mode On-line Estimation using Regularized Robust RLS Methods

This paper proposes a regularized robust recursive least squares (R3LS) method for on-line estimation of power-system electromechanical modes based on synchronized phasor measurement unit (PMU) data. The proposed method utilizes an autoregressive moving average exogenous (ARMAX) model to account for typical measurement data, which includes low-level pseudo-random probing, ambient, and ringdown data. A robust objective function is utilized to reduce the negative influence from non-typical data, which include outliers and missing data. A dy-namic regularization method is introduced to help include a priori knowledge about the system and reduce the influence of under-determined problems. Based on a 17-machine simulation model, it is shown through the Monte-Carlo method that the proposed R3LS method can estimate and track electromechanical modes by effectively using combined typical and non-typical measurement data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

ARM Climate Research Facility Spectral Surface Albedo Value-Added Product (VAP) Report

This document describes the input requirements, output data products, and methodology for the Spectral Surface Albedo (SURFSPECALB) value-added product (VAP). The SURFSPECALB VAP produces a best-estimate near-continuous high spectral resolution albedo data product using measurements from multifilter radiometers (MFRs). The VAP first identifies best estimates for the MFR downwelling and upwelling shortwave irradiance values, and then calculates narrowband spectral albedo from these best-estimate irradiance values. The methodology for finding the best-estimate values is based on a simple process of screening suspect data and backfilling screened and missing data with estimated values when possible. The resulting best-estimate MFR narrowband spectral albedos are used to determine a daily surface type (snow, 100% vegetation, partial vegetation, or 0% vegetation). For non-snow surfaces, a piecewise continuous function is used to estimate a high spectral resolution albedo at 1 min temporal and 10 cm-1 spectral resolution.

54 ENVIRONMENTAL SCIENCES↗

Methods for Data-based Delineation of Spatial Regions

In data analysis, it is often useful to delineate or segregate areas of interest from the general population of data in order to concentrate further analysis efforts on smaller areas. Three methods are presented here for automatically generating polygons around spatial data of interest. Each method addresses a distinct data type. These methods were developed for and implemented in the sample planning tool called Visual Sample Plan (VSP). Method A is used to delineate areas of elevated values in a rectangular grid of data (raster). The data used for this method are spatially related. Although VSP uses data from a kriging process for this method, it will work for any type of data that is spatially coherent and appears on a regular grid. Method B is used to surround areas of interest characterized by individual data points that are congregated within a certain distance of each other. Areas where data are “clumped” together spatially will be delineated. Method C is used to recreate the original boundary in a raster of data that separated data values from non-values. This is useful when a rectangular raster of data contains non-values (missing data) that indicate they were outside of some original boundary. If the original boundary is not delivered with the raster, this method will approximate the original boundary.

Algorithm↗

Key Nuclear Data Impacting Reactivity in Advanced Reactors

Advanced reactor concepts currently being developed throughout the industry are significantly different from light water reactor (LWR) designs with respect to geometry, materials, and operating conditions, and consequently, with respect to their reactor physics behavior. Given the limited operating experience with non-LWRs, the accurate simulation of reactor physics and the quantification of associated uncertainties are critical for ensuring that advanced reactor concepts operate within the appropriate safety margins. Nuclear data are a major source of input uncertainties in reactor physics analysis. As part of an ongoing project at Oak Ridge National Laboratory (ORNL), the effects of nuclear data uncertainties on key figures of merit associated with advanced reactor safety are being assessed for selected advanced reactor technologies. Key nuclear data relevant for reactor safety analysis for each selected advanced reactor technology were identified, and their impact on important key figures of merit was assessed. Available advanced reactor specifications were reviewed, results from studies performed at ORNL and other research institutions were consulted, and available evaluated nuclear data libraries were analyzed. This report summarizes the key nuclear data for nuclides in the fuel, as well as other significant data, including scattering and neutron capture in various materials for the moderator, coolant, and structure of the considered advanced reactors. For the considered advanced reactors that use low-enriched uranium (LEU) fuel, results from LWR studies provided insight into relevant nuclear data given the lack of available studies specifically addressing these new systems. The major nominal missing data that were identified consist of thermal scattering data and 135m Xe cross section data for molten salt reactor (MSR) analysis. The identified major gaps with respect to nuclear data uncertainties are missing uncertainties of thermal scattering data for high temperature gas-cooled reactors and moderated MSR systems, and incomplete uncertainties on angular distributions in particular for fast spectrum systems, such as sodiumcooled fast reactors, fast molten salt reactors, and heat pipe reactors. Furthermore, it was found that special attention should be paid to cross section and uncertainty differences between different evaluated nuclear data library releases, because significant differences in nuclear data that can lead to major differences in reactivity calculations were found, even for well-known nuclides.

21 SPECIFIC NUCLEAR REACTORS AND ASSOCIATED PLANTS↗

AWARE Shortwave Spectroradiometer Data

Shortwave spectral irradiance was measured at the West Antarctic Ice Sheet (WAIS) Divide Ice Camp as part of the ARM West Antarctic Radiation Experiment (AWARE). A Panalytical (Inc.) FieldSpec Pro Jr spectroradiometer belonging to Scripps Oceanography recorded downwelling spectral irradiance every minute covering the wavelength interval 350-2200 nm . Spectral resolution is 3 nm (10 nm) for wavelengths shorter (longer) than 1000 nm. Temporal coverage is nearly continuous between 8 December 2015 and 18 January 2016, with a few minutes of data missing during each day for data backup and optical collector hygiene. Spectral irradiances, particularly in the near infrared, are sensitive to cloud optical depth, thermodynamic phase, and effective particle size.

54 ENVIRONMENTAL SCIENCES↗

Gaseous Ammonia Measurements at the S2 site for the CoURAGE campaign

This dataset contains measurements of gaseous ammonia (NH3(g)) at the S2 (Mt. Airy) site during the CoURAGE study from April to June 2025. This dataset includes 10-minute and 30-minute average NH3(g) concentrations in units of micrograms per cubic meter. Measurements are conducted with an AiRRmonia NH3 analyzer manufactured by RR Mechatronics. A data flag of -999 indicates missing data due to the instrument being offline (maintenance, calibration, etc.) while a data flag of -888 indicates measurements below the instrument LOD (LOD = 0.1 micrograms per cubic meter). The uncertainty of the measurements is 15%.

AiRRmonia NH3 analyzer↗

Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) Winter Intensive Operational Period: Aerodynamic Particle Sizer Field Campaign Report

An aerodynamic particle sizer spectrometer (APS, model 3321 by TSI Inc., Shoreview, Minnesota) was deployed at the U.S. Department of Energy (DOE) Atmospheric Radiation Measurement (ARM) user facility Eastern North Atlantic (ENA) observatory during the second (winter) intensive operational period (IOP) of the Aerosol and Cloud Experiments in the Eastern North Atlantic (ACE-ENA) field campaign. Data were collected from January 9 through February 26, 2018, covering the second deployment of the ARM Aerial Facility (AAF) aircraft within ACE-ENA. The standard ENA instrument suite does not provide measurements of aerosol particle size distribution for diameters greater than one micron. Properties of coarse-mode (super-micron) aerosols are rarely measured, although super-micron particles can play an important role in the aerosol life cycle, affecting aerosol optical properties and cloud nuclei activity (e.g., Clarke et al. 2003). The APS was deployed to fill this gap in particle size measurements: it is capable of measuring size-resolved concentration of the aerosols in the 0.5-20 μm range (aerodynamic diameter). The APS was installed in the aerosol trailer to sample through the main stack. Special attention was given to selection of the instrument location (just below the main stack) to minimize particle losses in the connecting lines. Most atmospheric aerosols are hygroscopic, so the measurement of the aerosol size distributions should either be performed in a controlled humidity environment or accompanied by concurrent measurement of the temperature and humidity of the sample air. The latter strategy was used during this deployment: a high-precision humidity/temperature transmitter (model EE08 by E+E Elektonik GMBH, Engerwitzdorf, Germany) was mounted in the APS sampling line immediately upstream of the APS inlet. A dedicated laptop computer was used for data collection and communications. The APS was controlled with vendor-supplied software (Aerosol Instrument Manager, ver. 10.1, TSI); aerosol size distribution data were collected over a period of 55 seconds and converted to one-minute averages. Temperature and humidity analog signals were acquired via analog-to-digital converter (ADC) module (model WTAIN-M by Weeder Technologies, Fort Walton Beach, Florida) using vendor-supplied software (ModCom) with 1- sec resolution. In post-processing, the APS data were exported from TSI proprietary format into “comma-separated value” American Standard Code for Information Interchange (ASCII) files and matched with appropriately averaged temperature and humidity data. Two significant periods of missing data were caused by local power outages. Vendor-supplied software was not designed for unattended operation: it needs to be manually restarted including sensor configuration setup.

ACE-ENA, Azores, ultra-high-sensitivity aerosol sp↗

SU-E-T-524: Web-Based Radiation Oncology Incident Reporting and Learning System (ROIRLS)

Purpose: Describe a Web-based Radiation Oncology Incident Reporting and Learning system that has the potential to improve quality of care for radiation therapy patients. This system is an important facet of continuing effort by our community to maintain and improve safety of radiotherapy.Material and Methods: The VA National Radiation Oncology Program office has embarked on a program to electronically collect adverse events and near miss data of radiation treatment of over 25,000 veterans treated with radiotherapy annually. Software used for this program is deployed on the VAs intranet as a Website. All data entry forms (adverse event or near miss reports, work product reports) utilize standard causal, RT process step taxonomies and data dictionaries defined in AAPM and ASTRO reports on error reporting (AAPM Work Group Report on Prevention of Errors and ASTROs safety is no accident report). All reported incidents are investigated by the radiation oncology domain experts. This system encompasses the entire feedback loop of reporting an incident, analyzing it for salient details, and developing interventions to prevent it from happening again. The operational workflow is similar to that of the Aviation Safety Reporting System. This system is also synergistic with ROSIS and SAFRON. Results: The ROIRLS facilitates the collection of data that help in tracking adverse events and near misses and develop new interventions to prevent such incidents. The ROIRLS electronic infrastructure is fully integrated with each registered facility profile data thus minimizing key strokes and multiple entries by the event reporters. Conclusions: OIRLS is expected to improve the quality and safety of a broad spectrum of radiation therapy patients treated in the VA and fulfills our goal of Effecting Quality While Treating Safely The Radiation Oncology Incident Reporting and Learning System software used for this program has been developed, conceptualized and maintained by TSG Innovations Inc. and is deployed on the VA intranet as a Website. The Radiation Oncology Incident Reporting and Learning System software used for this program has been developed, conceptualized and maintained by TSG Innovations Inc. and is deployed on the VA intranet as a Website.

60 APPLIED LIFE SCIENCES↗

SU-E-T-469: Implementation of VAs Web-Based Radiotherapy Incident Reporting and Analysis System (RIRAS)

Purpose: This Web-based Radiotherapy Incident Reporting and Analysis System (RIRAS) is a tool to improve quality of care for radiation therapy patients. This system is an important facet of continuing effort by our community to maintain and improve safety of radiotherapy.Material and Methods: VA’s National Radiation Oncology Program office has embarked on a program to electronically collect adverse events and good-catch data of radiation treatment of over 25,000 veterans treated with radiotherapy annually. This VA-Intranet based software design has made use of dataset taxonomies and data dictionaries defined in AAPM/ASTRO reports on error reporting. We used proven industrial and medical event reporting techniques to avoid several common problems faced in effective data collection such as incomplete data due to data entry fatigue by the reporters, missing data due to data difficult to obtain or not familiar to most reporters, missing reports due to fear of reprisal etc. This system encompasses the entire feedback loop of reporting an incident, analyzing it for salient details, and developing interventions to prevent it from happening again. The analysis reports with corrective, learning actions are shared with the reporter/facility and made public to the community (after deidentification) as part of the learning process. Results: Till date 50 incident/good catches have been reported in RIRAS and we have completed analysis on 100% of these reports. This is done due to the fact that each reported incidents is investigated and a complete analysis/patient-safety-work-product report is generated by radiation oncology domain-experts. Conclusions Because of the completeness of the data, the system has enabled us to analyze process steps and track trends of major errors which in the future will lead to implementing system wide process improvement steps and safe standard operating procedures for each radiotherapy treatment modality/technique and fulfills our goal of “Effecting Quality While Treating Safely”. RIRAS developed and copyrighted by TSG Innovations Inc.

60 APPLIED LIFE SCIENCES↗

Generalized Canonical Polyadic Tensor Decomposition

Tensor decomposition is a fundamental unsupervised machine learning method in data science, with applications including network analysis and sensor data processing. This work develops a generalized canonical polyadic (GCP) low-rank tensor decomposition that allows other loss functions besides squared error. For instance, we can use logistic loss or Kullback--Leibler divergence, enabling tensor decomposition for binary or count data. We present a variety of statistically motivated loss functions for various scenarios. We provide a generalized framework for computing gradients and handling missing data that enables the use of standard optimization methods for fitting the model. Furthermore, we demonstrate the flexibility of the GCP decomposition on several real-world examples including interactions in a social network, neural activity in a mouse, and monthly rainfall measurements in India.

97 MATHEMATICS AND COMPUTING↗

How robust are estimates of key parameters in standard viral dynamic models?

Mathematical models of viral infection have been developed, fitted to data, and provide insight into disease pathogenesis for multiple agents that cause chronic infection, including HIV, hepatitis C, and B virus. However, for agents that cause acute infections or during the acute stage of agents that cause chronic infections, viral load data are often collected after symptoms develop, usually around or after the peak viral load. Consequently, we frequently lack data in the initial phase of viral growth, i.e., when pre-symptomatic transmission events occur. Missing data may make estimating the time of infection, the infectious period, and parameters in viral dynamic models, such as the cell infection rate, difficult. However, having extra information, such as the average time to peak viral load, may improve the robustness of the estimation. Here, we evaluated the robustness of estimates of key model parameters when viral load data prior to the viral load peak is missing, when we know the values of some parameters and/or the time from infection to peak viral load. Although estimates of the time of infection are sensitive to the quality and amount of available data, particularly pre-peak, other parameters important in understanding disease pathogenesis, such as the loss rate of infected cells, are less sensitive. Viral infectivity and the viral production rate are key parameters affecting the robustness of data fits. Fixing their values to literature values can help estimate the remaining model parameters when pre-peak data is missing or limited. We find a lack of data in the pre-peak growth phase underestimates the time to peak viral load by several days, leading to a shorter predicted growth phase. On the other hand, knowing the time of infection (e.g., from epidemiological data) and fixing it results in good estimates of dynamical parameters even in the absence of early data. While we provide ways to approximate model parameters in the absence of early viral load data, our results also suggest that these data, when available, are needed to estimate model parameters more precisely.

59 BASIC BIOLOGICAL SCIENCES↗

OmicsMLMentor: A Web Application for Guided Machine Learning Analysis of Omics Data

Expression-based omics technologies (e.g. proteomics, metabolomics, transcriptomics, etc.) increasingly rely on supervised and unsupervised machine learning (ML) models to find key biomolecules distinguishing conditions, identify natural groupings in biological data, or generate predictions for outcomes of interest. Fitting ML models to omics data presents several challenges, including handling missing data, selecting a normalization method, choosing a valid model, and optimizing hyperparameters, all requiring statistical programming skills to address these challenges. Thus, the open-source web application SLOPE was designed to lower the barrier to ML modeling for omics data. SLOPE supports the fitting of 15 ML models (10 supervised and 5 unsupervised) tailored to omics datasets, such as proteomics, metabolomics, lipidomics, and transcriptomics. SLOPE offers several omics-specific features, including methods for handling missingness (imputation, conversion, removal), normalization tests, ranking of models based on the structure of a user’s data and user input, and optimal hyperparameter selections using cross-validation splits. By streamlining ML workflows for omics analysis, SLOPE address critical gaps in existing online web tools, facilitating a broader adoption of these models for omics research. Here, SLOPE is applied to data from a lignin exposure study to highlight the workflow for fitting both supervised and unsupervised models to data.

lipidomics↗

Cleaning Images with Gaussian Process Regression

Many approaches to astronomical data reduction and analysis cannot tolerate missing data: corrupted pixels must first have their values imputed. This paper presents astrofix, a robust and flexible image imputation algorithm based on Gaussian process regression. Through an optimization process, astrofix chooses and applies a different interpolation kernel to each image, using a training set extracted automatically from that image. It naturally handles clusters of bad pixels and image edges and adapts to various instruments and image types. For bright pixels, the mean absolute error of astrofix is several times smaller than that of median replacement and interpolation by a Gaussian kernel. We demonstrate good performance on both imaging and spectroscopic data, including the SBIG 6303 0.4 m telescope and the FLOYDS spectrograph of Las Cumbres Observatory and the CHARIS integral-field spectrograph on the Subaru Telescope.

42 ENGINEERING↗

A Kriging based spatiotemporal approach for traffic volume data imputation

Along with the rapid development of Intelligent Transportation Systems, traffic data collection technologies have progressed fast. The emergence of innovative data collection technologies such as remote traffic microwave sensor, Bluetooth sensor, GPS-based floating car method, and automated license plate recognition, has significantly increased the variety and volume of traffic data. Despite the development of these technologies, the missing data issue is still a problem that poses great challenge for data based applications such as traffic forecasting, real-time incident detection, dynamic route guidance, and massive evacuation optimization. A thorough literature review suggests most current imputation models either focus on the temporal nature of the traffic data and fail to consider the spatial information of neighboring locations or assume the data follow a certain distribution. These two issues reduce the imputation accuracy and limit the use of the corresponding imputation methods respectively. As a result, this paper presents a Kriging based data imputation approach that is able to fully utilize the spatiotemporal correlation in the traffic data and that does not assume the data follow any distribution. A set of scenarios with different missing rates are used to evaluate the performance of the proposed method. The performance of the proposed method was compared with that of two other widely used methods, historical average and K-nearest neighborhood. Comparison results indicate that the proposed method has the highest imputation accuracy and is more flexible compared to other methods.

Science & Technology - Other Topics↗

SHELXT – Integrated space-group and crystal-structure determination

SHELXT automates routine small-molecule structure determination starting from single-crystal reflection data, the Laue group and a reasonable guess as to which elements might be present. The new computer program SHELXT employs a novel dual-space algorithm to solve the phase problem for single-crystal reflection data expanded to the space group P1. Missing data are taken into account and the resolution extended if necessary. All space groups in the specified Laue group are tested to find which are consistent with the P1 phases. After applying the resulting origin shifts and space-group symmetry, the solutions are subject to further dual-space recycling followed by a peak search and summation of the electron density around each peak. Elements are assigned to give the best fit to the integrated peak densities and if necessary additional elements are considered. An isotropic refinement is followed for non-centrosymmetric space groups by the calculation of a Flack parameter and, if appropriate, inversion of the structure. The structure is assembled to maximize its connectivity and centred optimally in the unit cell. SHELXT has already solved many thousand structures with a high success rate, and is optimized for multiprocessor computers. It is, however, unsuitable for severely disordered and twinned structures because it is based on the assumption that the structure consists of atoms.

75 CONDENSED MATTER PHYSICS, SUPERCONDUCTIVITY AND↗

A Review of the Use of Wearables in Indoor Environmental Quality Studies and an Evaluation of Data Accessibility from a Wearable Device

An understanding of indoor environmental quality (IEQ) and its effects on occupant well-being can inform building system design and operation. The use of wearables in field studies to collect subjective and objective health performance indicators (HPIs) from a large number of occupants could deliver important improvements in IEQ. To facilitate the use of wearables in IEQ studies, there is a need to identify which HPIs should be collected and to evaluate data accessibility from these devices. To address this issue, a literature review of previous IEQ studies was conducted to identify relationships between different IEQ factors and HPIs, with a focus on HPIs that were collected using wearables. A preliminary assessment of data accessibility from a selected wearable device (Fitbit Versa 2) was performed and documented. The review suggested the need to further investigate and collect sleep quality parameters, heart rate, stress response, as well as subjective ratings of comfort using wearables. The data accessibility assessment revealed issues related to missing data points and data resolution from the examined device. A set of recommendations is outlined to inform future studies.

60 APPLIED LIFE SCIENCES↗

AAF_UHSAS-air_CACTI

The UHSAS-air is the bench version of the UHSAS that measures particle size from 0.06 to 1.0 micro-meters. Aerosol passes through a laser beam and the resulting intensity of scattered light is directly correlated to the diameter of the particle passing through the beam. This isntrument has been modified to volumetricaly control the Sheath Flow using an Alicat flow controller. Data is reported at 1.000s and missing data is reported as -9999 Multiple data flags are present in this version. Counts?: The coincident error is deemed minimal until the counts exceed 4000. Counts between 3000 and 4000 are flagged with a caution flag (1). After 4000 counts a bad flag (2) is triggered and the data should not be used. Noise?: Instrument noise detected in 65-75nm size range. 1: Caution slight noise observed. 2: High amount of noise observed data should not be used. Total? Sum of all the flags triggered Max Flag: Max flag triggered by the above QC test CloudFlag: Warns of the presence of a cloud. Clouds can cause shattering on the probe and result in questionable measurements. Data during these periods of thin clouds (1) or heavy clouds (2) should be used with extreme caution.

54 ENVIRONMENTAL SCIENCES↗