Search NASA⌕ Search

SEARCH · Search NASA

Results for “Missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 361 records · Page 20

Statistical Performance of Forced Oscillation Detectors in the Presence of Missing Measurements

In bulk power systems, measurement-based monitoring for large oscillations can help maintain system reliability. One of the challenges encountered in a recent field demonstration was the unavailability of measurements due to underlying measurement quality or communication problems. During the demonstration, the oscillation detector ignored a measurement location if even 10 seconds of data was missing. To extend the detector's ability to operate in these conditions, this paper evaluates the impact of three methods for addressing missing data. The strengths and weaknesses of each approach are evaluated using theoretical expressions for the probability of detection along with results from simulated data and publicly available field measurements. Based on these results, a suitable approach is identified that can extend the oscillation detector's performance when large segments of data are missing.

Follum, James D.↗

Satellite Embedding-Based Population Imputation for Areas with Missing Building Footprint Data: A Computer Vision-Based Approach

High-resolution population modeling is important for supporting effective decision-making across diverse sectors. LandScan Mosaic generates population estimates at the level of individual buildings and aggregates them to 3 arc-second grids, and this approach performs well in regions where building footprint data are comprehensive and reliable. However, large portions of the globe still suffer from incomplete, sparse, or entirely missing building stock datasets, creating a structural limitation for strictly building-based population models. To address this research gap, this study proposes a computer vision-based framework that employs Google Earth Engine satellite embeddings and UNet, which allows us to directly impute grid-level population estimates in building-data-deficient areas. Applied to Taiwan as a case study, the framework achieved strong predictive performance with R$^{2}$ of 0.89, RMSE of 18.70, and MAE of 8.41, outperforming traditional machine learning approaches. Notably, the proposed framework effectively addressed building false-positive errors inherent in Global Human Settlement Layer (GHSL) data, correctly identifying uninhabited areas that were erroneously classified as populated. The framework also offers significant advantages for global population mapping, particularly in terms of scalability and temporal consistency, thereby extending the coverage and accuracy of high-resolution population products in data-scarce regions worldwide. Urban planners, decision makers, and related stakeholders can obtain granular population distributions to support more accurate and targeted infrastructure investment, service delivery, resource allocation, and risk assessment decisions.

97 MATHEMATICS AND COMPUTING↗

Scalable tensor factorizations with incomplete data.

The problem of incomplete data - i.e., data with missing or unknown values - in multi-way arrays is ubiquitous in biomedical signal processing, network traffic analysis, bibliometrics, social network analysis, chemometrics, computer vision, communication networks, etc. We consider the problem of how to factorize data sets with missing values with the goal of capturing the underlying latent structure of the data and possibly reconstructing missing values (i.e., tensor completion). We focus on one of the most well-known tensor factorizations that captures multi-linear structure, CANDECOMP/PARAFAC (CP). In the presence of missing data, CP can be formulated as a weighted least squares problem that models only the known entries. We develop an algorithm called CP-WOPT (CP Weighted OPTimization) that uses a first-order optimization approach to solve the weighted least squares problem. Based on extensive numerical experiments, our algorithm is shown to successfully factorize tensors with noise and up to 99% missing data. A unique aspect of our approach is that it scales to sparse large-scale data, e.g., 1000 x 1000 x 1000 with five million known entries (0.5% dense). We further demonstrate the usefulness of CP-WOPT on two real-world applications: a novel EEG (electroencephalogram) application where missing data is frequently encountered due to disconnections of electrodes and the problem of modeling computer network traffic where data may be absent due to the expense of the data collection process.

99 GENERAL AND MISCELLANEOUS↗

Atomic Oxygen Erosion Data from the MISSE 2-8 Missions

Polymers and other oxidizable materials on the exterior of spacecraft in the low Earth orbit (LEO) space environment can be eroded from reaction with atomic oxygen (AO). Therefore, in order to design durable spacecraft it is important to know the extent of erosion that will occur during a mission. This can be determined by knowing the LEO AO erosion yield, E(sub y) (volume loss per incident oxygen atom), of materials susceptible to AO reaction. In addition, recent flight experiments have shown that the AO E(sub y) can vary with the AO fluence and/or solar exposure. Therefore obtaining AO E(sub y) data for materials flown on various spaceflight missions is important. NASA Glenn Research Center has flown numerous experiments as part of the Materials International Space Station Experiment (MISSE) missions on the exterior of the International Space Station to characterize the LEO E(sub y) of polymers, composites, protective coatings, and other spacecraft materials. This report provides a summary of the erosion data for ram samples from six Glenn polymer experiments flown as part of MISSE 2, 4, 6, 7, and 8. A total of 71 types of materials with 111 E(sub y) values are provided. The E(sub y) values for uncoated polymers range from 3.81×10(exp –27) cu cm/atom for DC 93-500 silicone exposed to an AO fluence of 4.62×10(exp 21) atoms/sq cm on MISSE 8 to 9.14×10(exp –24) cu cm/atom for polyoxymethylene (POM) exposed to an AO fluence of 8.43×10(exp 21) atoms/sq cm on MISSE 2. One polymer, Triton oxygen resistant, low modulus (TOR(TM) LM), experienced mass gain when exposed to an AO fluence of 2.15×10(exp 21) atoms/sq cm on MISSE 4. In many cases the same material was flown on numerous missions so that trends for E(sub y) versus AO fluence and/or solar exposure can be determined, along with temperature effects.

Erosion↗

Evaluating the Effects of Missing Values and Mixed Data Types on Social Sequence Clustering Using t-SNE Visualization

© 2019 Association for Computing Machinery. The goal of this work is to investigate the impact of missing values in clustering joint categorical social sequences. Identifying patterns in sociodemographic longitudinal data is important in a number of social science settings. However, performing analytical operations, such as clustering on life course trajectories, is challenging due to the categorical and multidimensional nature of the data, their mixed data types, and corruption by missing and inconsistent values. Data quality issues were investigated previously on single variable sequences. To understand their effects on multivariate sequence analysis, we employ a dataset of mixed data types and missing values, a dissimilarity measure designed for joint categorical sequence data, together with dimensionality reduction methodologies in a systematic design of sequence clustering experiments. Given the categorical nature of our data, we employ an “edit” distance using optimal matching. Because each data record has multiple variables of different types, we investigate the impact of mixing these variables in a single dissimilarity measure. Between variables with binary values and those with multiple nominal values, we find that the ability to overcome missing data problems is more difficult in the nominal domain than in the binary domain. Additionally, alignment of leading missing values can result in systematic biases in dissimilarity matrices and subsequently introduce both artificial clusters and unrealistic interpretations of associated data domains. We demonstrate the usage of t-distributed stochastic neighborhood embedding to visually guide mitigation of such biases by tuning the missing value substitution cost parameter or determining an optimal sequence span.

Lazar, Alina↗

Erosion Data from the MISSE 8 Polymers Experiment After 2 Years of Space Exposure on the International Space Station

The Polymers Experiment was exposed to the low Earth orbit (LEO) space environment for 2.14 and 2.0 years as part of the Materials International Space Station Experiment 8 (MISSE 8) and the Optical Reflector Materials Experiment-III (ORMatE-III), respectively. The experiment contained 42 samples, which were flown in either ram, wake, or zenith orientations. The primary objective was to determine the effect of solar exposure on the atomic oxygen erosion yield (Ey) of fluoropolymers. This paper provides an overview of the experiment with details on the polymers flown, the characterization techniques used, the atomic oxygen fluence for each exposure orientation, and the LEO Ey results. The Ey values for the fluoropolymers range from 1.45 x 10(exp -25) cm(exp 3)/atom for white Tedlar Registered Trademark (polyvinyl fluoride with white titanium dioxide pigment) flown in the ram orientation to 6.32 x 10(exp -24) cm(exp 3)/atom for aluminized-Teflon Registered Trademark fluorinated ethylene propylene (Al-FEP) flown in the zenith orientation. Erosion yield data for FEP flown in ram, wake and zenith orientations are compared, and the Ey was found to be highly dependent on orientation, hence environmental exposure. Teflon FEP had an order of magnitude higher Ey when flown in the zenith direction (6.32 x10(exp -24) cm(exp3)/atom) as compared to the ram direction (2.37 x 10(exp -25) cm(exp 3)/atom). The Ey of FEP was found to increase with a direct correlation to the solar exposure/AO fluence ratio showing the effect of solar radiation and/or heating due to solar exposure on FEP erosion. In addition, back-surface carbon painted FEP (C-FEP) flown in the zenith orientation had a significantly higher Ey than clear FEP or Al-FEP further indicating that heating has a significant impact on the erosion of FEP, particularly in the zenith orientation.

Solar radiation↗

XTE Observations of the Intraday Variable BL LAC S5 0716+714 in a coordinated Multiwavelength Study

S5 0716+714 was observed with the Rossi X-ray Timing Explorer (RXTE) between 6 and 22 April 1996 as part of a multifrequency monitoring campaign for this source observed to show extreme intra-day variability (IDV) at a number of frequencies. There were 24 distinct epochs of observation ('views') with the RXTE Proportional Counter Array (PCA), each lasting on average 3-4 ks. There were a number of technical problems with the data. During view 10 no Proportional Counter Unit (PCU) was on. A gain change (from gain epoch 2 to gain epoch 3) took place during view 11, on 15 April 1996. No data were received for view 18. Views 10, 11 and 18 have been omitted from our analysis. There were two distinct pointings of the telescope in views 4 and 17 (all the other observations have only one pointing), and in many of the pointings the data are broken by periods where the high voltage was turned off during South Atlantic Anomaly (SAA) passages and Earth occultations. Originally no filter file was available for views 21 and 22, and for the second pointing of view 17, because housekeeping data were missing, but these data were supplied when the observations were reprocessed in early 1998. For most of the remaining observations only three PCUs were on; the exceptions are views 23, 24 (four PCUs) and 17, 19, 20, 21 and 22 (five PCUs). All working PCUs have been analyzed. The third part of the first pointing of view 17 was found to be very short and is discarded from our analysis. The second part of observation 2 is pointed far off-axis and is discarded.

Worrall, D. M.↗

Data and scripts from: “Denoising autoencoder for reconstructing sensor observation data and predicting evapotranspiration: noisy and missing values repair and uncertainty quantification”

This data package includes data and scripts from the manuscript “Denoising autoencoder for reconstructing sensor observation data and predicting evapotranspiration: noisy and missing values repair and uncertainty quantification”.The study addressed common challenges faced in environmental sensing and modeling, including uncertain input data, missing sensor observations, and high-dimensional datasets with interrelated but redundant variables. Point-scaled meteorological and soil sensor observations were perturbed with noises and missing values, and denoising autoencoder (DAE) neural networks were developed to reconstruct the perturbed data and further predict evapotranspiration. This study concluded that (1) the reconstruction quality of each variable depends on its cross-correlation and alignment to the underlying data structure, (2) uncertainties from the models were overall stronger than those from the data corruption, and (3) there was a tradeoff between reducing bias and reducing variance when evaluating the uncertainty of the machine learning models.This package includes:(1) Four ipython scripts (.ipynb): “DAE_train.ipynb” trains and evaluates DAE neural networks, “DAE_predict.ipynb” makes predictions from the trained DAE models, “ET_train.ipynb” trains and evaluates ET prediction neural networks, and “ET_predict.ipynb” makes predictions from trained ET models.(2) One python file (.py): “methods.py” includes all user-defined functions and python codes used in the ipython scripts.(3) A “sub_models” folder that includes five trained DAE neural networks (in pytorch format, .pt), which could be used to ingest input data before being fed to the downstream ET models in ‘ET_train.ipynb” or ‘ET_predict.ipynb’.(4) Two data files (.csv). Daily meteorological, vegetation, and soil data is in “df_data.csv”, where “df_meta.csv” contains the location and time information of “df_data.csv”. Each row (index) in “df_meta.csv” corresponds to each row in “df_data.csv”. These data files are formatted to follow the data structure requirements and be directly used in the ipython scripts, and they have been shuffled chronologically to train machine learning models. The meteorological and soil data was collected using point sensors between 2019-2023 at(4.a) Three shrub-dominated field sites in East River, Colorado (named “ph1”, “ph2” and “sg5” in “df_meta.csv”, where “ph1” and “ph2” were located at PumpHouse Hillslopes, and “sg5” was at Snodgrass Mountain meadow) and(4.b) One outdoor, mesoscale, and herbaceous-dominated experiment in Berkeley, California (named “tb” in “df_meta.csv”, short for Smartsoils Testbed at Lawrence Berkeley National Lab).- See "df_data_dd.csv" and "df_meta_dd.csv" for variable descriptions and the Methods section for additional data processing steps. See "flmd.csv" and "README.txt" for brief file descriptions.- All ipython scripts and python files are written in and require PYTHON language software.

54 ENVIRONMENTAL SCIENCES↗

Measurement of the 12C(e,e'p)11B Two-Body Breakup Reaction at High Missing Momentum Values

The five-fold differential cross section for the 12C(e,e'p)11B reaction was determined over a missing momentum range of 200-400 MeV/c, in a kinematics regime with Bjorken x > 1 and Q2 = 2.0 (GeV/c)2. A comparison of the results and theoretical models and previous lower missing momentum data is shown. The theoretical calculations agree well with the data up to a missing momentum value of 325 MeV/c and then diverge for larger missing momenta. The extracted distorted momentum distribution is shown to be consistent with previous data and extends the range of available data up to 400 MeV/c.

Monaghan, P↗

Free-Flying Magnetometer Data System

The Free-Flying Magnetometer (FFM) is an autonomous "sensorcraft" developed at the Jet Propulsion Laboratory (JPL) for the Enstrophy sounding rocket mission. This mission was a collaborative project between the University of New Hampshire, Cornell University and JPL. The science goal of the mission was the study of current filamentation phenomena in the northern auroral region through multipoint measurements of magnetic field. The technical objective of the mission was the proof of concept of the JPL FFM design and the demonstration of an in-situ multipoint measurement technique employing many free-flying spacecraft. Four FFMs were successfully deployed from a sounding rocket launched from Poker Flats, Alaska on February 11, 1999. These hockey-puck-sized (80 mm diameter, 38 mm. height, 250 gram mass) free flyers each carry a miniature 3-axis flux-gate magnetometer that output +/- 2 V signals corresponding to a +/- 60,000 nT measurement range for each axis. The FFM uses a synchronized four-channel Sigma(Delta) Analog-to-Digital Converter (ADC) having a dynamic range of +/- 2.5V and converting at a rate of 279 samples/second/channel. Three channels are used to digitize the magnetometer signals to 17-bit (1.144 nT/bit) resolution. The fourth ADC channel is multiplexed for system monitoring of four temperature sensors and two battery voltages. The FFM also contains two sun sensors, a laser diode which emits a fan-shaped beam, a miniature S-band transmitter for direct communication to the ground station antennas, an ultra-stable Temperature Compensated Crystal Oscillator (TCXO) clock, an integrated data subsystem implemented in a Field-Programmable Gate Array (FPGA), a 4 Mbit Static Random Access Memory (SRAM) for data storage and Lithium Thionyl Chloride batteries for power. Communicating commands to the FFM prior to deployment is achieved with an infrared (IR) link. The FFM IR receiver responds to 9-bit pulse coded signals that are generated by an IR Light Emitting Diode (LED) in the payload for turning FFM power on or off and placing the FFM in a test mode or flight mode. The IR links are also used to synchronize (zero) the clocks onboard all the FFMs through a reset pulse originating from the payload GPS receiver that is issued when the FFMs are in flight mode. The FPGA based data subsystem manages continuous data collection from the four ADC channels and sun sensors, formatting and storing the data to SRAM, and controlling downlink transmission. The transmitter is powered only after a 2547 frame SRAM buffer has been filled (approx. 5 minutes of data). The data is Viterbi encoded and sent to the S-band transmitter via a First-In-First-Out (FIFO) buffer who's output is clocked at 100 bits/second. After the 26-second transmission, the transmitter is turned off to reduce noise coupling to the sensitive magnetometer. The data subsystem control consists of a master state machine that performs data flow management and is interfaced through a prioritized interrupt scheme to state machines that service the ADC, sun sensors and transmitter FIFO. Continuous data collection prevents the missing of data during transmission and provides implicit time tagging of the data acquired by the ADC because of synchronization with the TCXO clock.

Blaes, B.↗

PPC750 Performance Monitor

The PPC750 Performance Monitor (Perfmon) is a computer program that helps the user to assess the performance characteristics of application programs running under the Wind River VxWorks real-time operating system on a PPC750 computer. Perfmon generates a user-friendly interface and collects performance data by use of performance registers provided by the PPC750 architecture. It processes and presents run-time statistics on a per-task basis over a repeating time interval (typically, several seconds or minutes) specified by the user. When the Perfmon software module is loaded with the user s software modules, it is available for use through Perfmon commands, without any modification of the user s code and at negligible performance penalty. Per-task run-time performance data made available by Perfmon include percentage time, number of instructions executed per unit time, dispatch ratio, stack high water mark, and level-1 instruction and data cache miss rates. The performance data are written to a file specified by the user or to the serial port of the computer

Meyer, Donald↗

GRIP: Constraint-based Explanation of Missing Answers for Graph Queries

Abstract: A useful feature in graph query engines is to clarify “Why certain entities (nodes, attribute values or edges) are missing” in query answers. This task is even more challenging when the relevant data is already missing in the underlying data source. Missing data, on the other hand, can be inferred by enforcing data constraints for graphs. We demonstrate GRIP, a system that exploits data constraints to clarify missing answers for graph queries. (1) Constraint-based ex- planation. Given a desired yet missing entity in the query answer, GRIP ensures to generate finite and minimal sequences of data con- strains (an “explanation”) that should be consecutively enforced to ?? to ensure its occurrence for the same query. (2) Answering “why” and“how” questions. Users can query GRIP with both“Why”(“Why” the element is missing) and “How” questions (“How” to refine the graph to include the missing answer). GRIP engine supports run- time generation of explanations by incrementally maintaining a set of bi-directional search trees. (3) Interactive exploration. GRIP provides a user-friendly GUI to support interactive ad visual exploration of explanations, including both automated generation and step-by-step inspection of graph manipulations.

graphs↗

Assessment of BSRN radiation records for the computation of monthly means

The accuracy of the Baseline Surface Radiation Network (BSRN) radiation monthly averages are assessed by investigating data gap frequency due to missing or discarded high time resolution data. The monthly statistics, especially means are considered to be important and useful values for climate research, model performance evaluations and for assessing the quality of satellite computations. The study investigates the spread in different algorithms that have been applied for the computation of monthly means from minute values. We found that a substantial fraction of the radiation fluxes observed at BSRN sites is either missing or flagged as questionable. The percentage of missing data is 4.4%, 13.0%, and 6.5% for global radiation, direct shortwave radiation, and downwelling longwave radiation, respectively. Most flagged data in the shortwave are due to nighttime instrumental noise and can reasonably be set to zero after correcting for thermal offsets in the daytime data. The paper reveals that the computation of monthly means from 1-minute observations distinctly depends on the selected method. The intra-method difference generally increases with an increasing fraction of missing data. The study demonstrates that the handling of flagged data clearly impacts on monthly mean estimates obtained with different methods. We showed that the spread of monthly shortwave fluxes are generally clearly higher than for downwelling longwave radiation. Overall, BSRN observation provide sufficient accuracy and completeness for reliable estimates of monthly mean values. However, we should aim at further increasing the data quality by reducing the data gap frequency and the number of outliers. The authors suggest using a standardized method for the computation of monthly means in order to avoid a mismatch of published monthly mean radiation fluxes from BSRN.

54 ENVIRONMENTAL SCIENCES↗

Augmenting machine learning of Grad–Shafranov equilibrium reconstruction with Green's functions

This work presents a method for predicting plasma equilibria in tokamak fusion experiments and reactors. The approach involves representing the plasma current as a linear combination of basis functions using principal component analysis of plasma toroidal current densities (J t ) from the EFIT-AI equilibrium database. Then utilizing EFIT's Green's function tables, basis functions are created for the poloidal flux (ψ) and diagnostics generated from the toroidal current (J t ). Similar to the idea of a physics-informed neural network (NN), this physically enforces consistency between ψ, J t , and the synthetic diagnostics. First, the predictive capability of a least squares technique to minimize the error on the synthetic diagnostics is employed. The results show that the method achieves high accuracy in predicting ψ and moderate accuracy in predicting J t with median R 2 = 0.9993 and R 2 = 0.978, respectively. A comprehensive NN using a network architecture search is also employed to predict the coefficients of the basis functions. The NN demonstrates significantly better performance compared to the least squares method with median R 2 = 0.9997 and 0.9916 for J t and ψ, respectively. The robustness of the method is evaluated by handling missing or incorrect data through the least squares filling of missing data, which shows that the NN prediction remains strong even with a reduced number of diagnostics. Additionally, the method is tested on plasmas outside of the training range showing reasonable results.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗

Integrative analysis of transcriptomic and proteomic data of Shewanella oneidensis: missing value imputation using temporal datasets

Despite significant improvements in recent years, proteomic datasets currently available still suffer large number of missing values. Integrative analyses based upon incomplete proteomic and transcriptomic da-tasets could seriously bias the biological interpretation. In this study, we applied a non-linear data-driven stochastic gradient boosted trees (GBT) model to impute missing proteomic values for proteins experi-mentally undetected, using a temporal transcriptomic and proteomic dataset of Shewanella oneidensis. In this dataset, genes expression was measured after the cells were exposed to 1 mM potassium chromate for 5-, 30-, 60-, and 90-min, while protein abundance was measured only for 45- and 90-min samples. With the goal of elucidating the relationship between temporal gene expression and protein abundance data, and then using it to impute missing proteomic values for samples of 45-min (which does not have cognate transcriptomic data) and 90-min, we initially used nonlinear Smoothing Splines Curve Fitting (SSCF) to identify temporal relationships among transcriptomic data at different time points and then imputed missing gene expression measurements for the sample at 45-min. After the imputation was validated by biological constrains (i.e. operons), we used a data-driven Gradient Boosted Trees (GBT) model to uncover possible non-linear relationships between temporal transcriptomic and proteomic data, and to impute protein abundance for the proteins experimentally undetected in the 45- and 90-min sam-ples, based on relevant predictors such as temporal mRNA gene expression data, cellular roles, molecular weight, sequence length, protein length, guanine-cytosine (GC) content and triple codon counts. The imputed protein values were validated using biological constraints such as operon, regulon and pathway information. Finally, we demonstrated that such missing value imputation improved characterization of the temporal response of S. oneidensis to chromate.

SBR mercury↗