SEARCH · Search NASA

Results for “missing data restoration”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

11 records

All sources NASA NTRS DOE OSTI

DOE OSTI2023

Load Profile Inpainting for Missing Load Data Restoration and Baseline Estimation

This paper introduces a Generative Adversarial Nets (GAN) based, Load Profile Inpainting Network (Load-PIN) for restoring missing load data segments and estimating the baseline for a demand response event. The inputs are time series load data before and after the inpainting period together with explanatory variables (e.g., weather data). Here, we propose a Generator structure consisting of a coarse network and a fine-tuning network. The coarse network provides an initial estimation of the data segment in the inpainting period. The fine-tuning network consists of self-attention blocks and gated convolution layers for adjusting the initial estimations. Loss functions are specially designed for the fine-tuning and the discriminator networks to enhance both the point-to-point accuracy and realisticness of the results. We test the Load-PIN on three real-world data sets for two applications: patching missing data and deriving baselines of conservation voltage reduction (CVR) events. We benchmark the performance of Load-PIN with five existing deep-learning methods. Our simulation results show that, compared with the state-of-the-art methods, Load-PIN can handle varying-length missing data events and achieve 15-30% accuracy improvement.

14 SOLAR ENERGY↗

NASA NTRS1992

Restoration of HST images with missing data

Missing data are a fairly common problem when restoring Hubble Space Telescope observations of extended sources. On Wide Field and Planetary Camera images cosmic ray hits and CCD hot spots are the prevalent causes of data losses, whereas on Faint Object Camera images data are lossed due to reseaux marks, blemishes, areas of saturation and the omnipresent frame edges. This contribution discusses a technique for 'filling in' missing data by statistical inference using information from the surrounding pixels. The major gain consists in minimizing adverse spill-over effects to the restoration in areas neighboring those where data are missing. When the mask delineating the support of 'missing data' is made dynamic, cosmic ray hits, etc. can be detected on the fly during restoration.

Adorf, Hans-Martin↗

DOE OSTI2022

Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data

Many distribution network monitoring and control applications - including state estimation, volt/VAR optimization, and network reconfiguration - rely on accurate network models; however, the network models maintained by utilities can become outdated because of restoration activities, network reconfiguration, and missing data. With the widespread deployment of advanced metering infrastructure (AMI), abundant measurement data from low-voltage secondary networks are available. The AMI measurement data can be used for phase identification to improve the network models. Although the existing phase identification techniques work well in passive distribution feeders that do not have photovoltaic (PV) generation, they can fail to accurately identify the phases in the presence of PV. This paper proposes a robust phase identification algorithm based on supervised machine learning that accurately identifies the AMI meter phase connectivity in the presence of significant PV generation. The proposed algorithm does not require network topology information or feeder head measurement data. The algorithm is validated using the AMI measurement data collected in the field and the field-validated phase connectivity database on two real distribution feeders from San Diego Gas & Electric Company that have significant PV generation.

advanced metering infrastructure↗

DOE OSTI2022

Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data

Many distribution network monitoring and control applications - including state estimation, Volt/VAr optimization, and network reconfiguration - rely on accurate network models; however, the network models maintained by utilities can become outdated because of restoration activities, network reconfiguration, and missing data. With the widespread deployment of advanced metering infrastructure (AMI), abundant measurement data from low-voltage secondary networks are available. The AMI measurement data can be used for phase identification to improve the network models. Although the existing phase identification techniques work well in passive distribution feeders that do not have photovoltaic (PV) generation, they can fail to accurately identify the phases in the presence of PV. This paper proposes a robust phase identification algorithm based on supervised machine learning that accurately identifies the AMI meter phase connectivity in the presence of significant PV generation. The proposed algorithm does not require network topology information or feeder-head measurement data. The algorithm is validated using the AMI measurement data collected in the field and the field-validated phase connectivity database on two real distribution feeders from San Diego Gas & Electric Company that have significant PV generation.

advanced metering infrastructure (AMI)↗

DOE OSTI2022

Phase Identification in Real Distribution Networks with High PV Penetration Using Advanced Metering Infrastructure Data: Preprint

advanced metering infrastructure↗

DOE OSTI2024

An iterative bidirectional gradient boosting approach for CVR baseline estimation

Here this paper presents a novel Iterative Bidirectional Gradient Boosting Model (IBi-GBM) for estimating the baseline of Conservation Voltage Reduction (CVR) programs. In contrast to many existing methods, we treat CVR baseline estimation as a missing data retrieval problem. The approach involves dividing the load and its corresponding temperature profiles into three periods: pre-CVR, CVR, and post-CVR. To restore the missing load profile during the CVR period, the method employs a three-step process. First, a forward-pass GBM is executed using data from the pre-CVR period as inputs. Subsequently, a backward-pass GBM is applied using data from the post-CVR period. The two restored load profiles are reconciled, considering pre-calculated weights derived from forecasting accuracy, and only the leftmost and rightmost points are retained. The newly restored points are then included as inputs for the subsequent iteration. This iterative procedure continues until the original load data in the CVR period is fully restored. We develop IBi-GBM using actual smart meter and Supervisory Control and Data Acquisition (SCADA) data. Our results demonstrate that IBi-GBM exhibits robust performance across various data resolutions and in different seasons and outperforms existing methods by achieving a 1-2% reduction in normalized Root Mean Square Error (nRMSE).

42 ENGINEERING↗

NASA NTRS2018

History and Status of ALSEP and the Apollo Lunar Data Project

A suite of automated scientific instruments (the Apollo Lunar Surface Experiment Package, or ALSEP) was installed at each of the landing sites of Apollo 12, 14, 15, 16, and 17 from 1969 to 1972. They operated from deployment until decommissioning on 30 September 1977. These data were continuously transmitted to Earth and saved on the Range Tapes, which were recorded at the Manned Space Flight Network stations. These data were also broken out by experiment and sent to the experiment Principal Investigators on what were called the P.I. Tapes. Starting in April 1973 the Range Tape data were stored in digital format on 7-track magnetic tapes, the ARCSAV Tapes. In February 1976, the handling of the Range Tapes was transferred to UT Galveston. They produced 9-track tapes referred to as the Work Tapes. Following the Apollo program the Range and ARCSAV tapes, which were never archived, were lost. The Work Tapes were archived at the National Space Science Data Center (NSSDC). Some investigators archived their individual experiment data with NSSDC as well, but much of the data had minimal documentation, were not in digital form, or were stored in difficult to translate formats. Data from many experiments were never delivered to the NSSDC. The Lunar Data Project was started to address the problem of both missing and not readily usable data. Our effort has resulted in recovery of some of the ARCSAV tapes, recovery and digitization of a large volume of Apollo scientific and technical documentation, and restoration of many ALSEP and other Apollo data collections. Restoration involves deciphering formats, assembling necessary ancillary data (metadata), and packaging data in digital format to be archived with the Planetary Data System (PDS). Recovery of the data from the ARCSAV tapes involved having the tapes read on special equipment and extracting the individual experiment data out of the integrated data stream. We will report on the history and status of the various recovery efforts.

Work Tapes↗

NASA NTRS2017

Synergistic Use of Nighttime Satellite Data, Electric Utility Infrastructure, and Ambient Population to Improve Power Outage Detections in Urban Areas

Natural and anthropogenic hazards are frequently responsible for disaster events, leading to damaged physical infrastructure, which can result in loss of electrical power for affected locations. Remotely-sensed, nighttime satellite imagery from the Suomi National Polar-orbiting Partnership (Suomi-NPP) Visible Infrared Imaging Radiometer Suite (VIIRS) Day/Night Band (DNB) can monitor power outages in disaster-affected areas through the identification of missing city lights. When combined with locally-relevant geospatial information, these observations can be used to estimate power outages, defined as geographic locations requiring manual intervention to restore power. In this study, we produced a power outage product based on Suomi-NPP VIIRS DNB observations to estimate power outages following Hurricane Sandy in 2012. This product, combined with known power outage data and ambient population estimates, was then used to predict power outages in a layered, feedforward neural network model. We believe this is the first attempt to synergistically combine such data sources to quantitatively estimate power outages. The VIIRS DNB power outage product was able to identify initial loss of light following Hurricane Sandy, as well as the gradual restoration of electrical power. The neural network model predicted power outages with reasonable spatial accuracy, achieving Pearson coefficients (r) between 0.48 and 0.58 across all folds. Our results show promise for producing a continental United States (CONUS)- or global-scale power outage monitoring network using satellite imagery and locally-relevant geospatial data.

da/night band↗

DOE OSTI2023

Sequential Image Recovery Using Joint Hierarchical Bayesian Learning

Abstract Recovering temporal image sequences (videos) based on indirect, noisy, or incomplete data is an essential yet challenging task. We specifically consider the case where each data set is missing vital information, which prevents the accurate recovery of the individual images. Although some recent (variational) methods have demonstrated high-resolution image recovery based on jointly recovering sequential images, there remain robustness issues due to parameter tuning and restrictions on the type of sequential images. Here, we present a method based on hierarchical Bayesian learning for the joint recovery of sequential images that incorporates prior intra- and inter-image information. Our method restores the missing information in each image by “borrowing” it from the other images. More precisely, we couple sequential images by penalizing their pixel-wise difference. The corresponding penalty terms (one for each pixel and pair of subsequent images) are treated as weakly-informative random variables that favor small pixel-wise differences but allow occasional outliers. As a result, all of the individual reconstructions yield improved accuracy. Our method can be used for various data acquisitions and allows for uncertainty quantification. Some preliminary results indicate its potential use for sequential deblurring and magnetic resonance imaging.

Xiao, Yao↗

DOE OSTI2023

Human DNA polymerase η promotes RNA-templated error-free repair of DNA double-strand breaks

A growing body of evidence indicates that RNA plays a critical role in orchestrating DNA double-strand break repair (DSBR). Recently, we showed that homologous nascent RNA can be used as a template for error-free repair of double-strand breaks (DSBs) in the transcribed genome and to restore the missing sequence at the break site via the transcription-coupled classical nonhomologous end-joining (TC-NHEJ) pathway. TC-NHEJ is a complex multistep process in which a reverse transcriptase (RT) is essential for synthesizing the DNA strand from template RNA. However, the identity of the RT involved in the TC-NHEJ pathway remained unknown. Here, we report that DNA polymerase eta (Pol η), known to possess RT activity, plays a critical role in TC-NHEJ. We found that Pol η forms a multiprotein complex with RNAP II and other TC-NHEJ factors, while also associating with nascent RNA. Moreover, purified Pol η, along with DSBR proteins PNKP, XRCC4, and Ligase IV can fully repair RNA templated 3'-phosphate-containing gapped DNA substrate. In addition, we demonstrate here that Pol η deficiency leads to accumulation of R-loops and persistent strand breaks in the transcribed genes. Finally, we determined that, in Pol η depleted but not in control cells, TC-NHEJ-mediated repair was severely abrogated when a reporter plasmid containing a DSB with several nucleotide deletion within the E. coli lacZ gene was introduced for repair in lacZ-expressing mammalian cells. Thus, our data strongly suggest that RT activity of Pol η is required in error-free DSBR.

59 BASIC BIOLOGICAL SCIENCES↗

NASA NTRS1998

A novel calmodulin-regulated Ca2+-ATPase (ACA2) from Arabidopsis with an N-terminal autoinhibitory domain

To study transporters involved in regulating intracellular Ca2+, we isolated a full-length cDNA encoding a Ca2+-ATPase from a model plant, Arabidopsis, and named it ACA2 (Arabidopsis Ca2+-ATPase, isoform 2). ACA2p is most similar to a "plasma membrane-type" Ca2+-ATPase, but is smaller (110 kDa), contains a unique N-terminal domain, and is missing a long C-terminal calmodulin-binding regulatory domain. In addition, ACA2p is localized to an endomembrane system and not the plasma membrane, as shown by aqueous-two phase fractionation of microsomal membranes. ACA2p was expressed in yeast as both a full-length protein (ACA2-1p) and an N-terminal truncation mutant (ACA2-2p; Delta residues 2-80). Only the truncation mutant restored the growth on Ca2+-depleted medium of a yeast mutant defective in both endogenous Ca2+ pumps, PMR1 and PMC1. Although basal Ca2+-ATPase activity of the full-length protein was low, it was stimulated 5-fold by calmodulin (50% activation around 30 nM). In contrast, the truncated pump was fully active and insensitive to calmodulin. A calmodulin-binding sequence was identified within the first 36 residues of the N-terminal domain, as shown by calmodulin gel overlays on fusion proteins. Thus, ACA2 encodes a novel calmodulin-regulated Ca2+-ATPase distinguished by a unique N-terminal regulatory domain and a non-plasma membrane localization.

Non-NASA Center↗