Search NASA⌕ Search

SEARCH · Search NASA

Results for “missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 271 records · Page 15

A DATA-DRIVEN MODEL FOR SPECTRA: FINDING DOUBLE REDSHIFTS IN THE SLOAN DIGITAL SKY SURVEY

We present a data-driven method-heteroscedastic matrix factorization, a kind of probabilistic factor analysis-for modeling or performing dimensionality reduction on observed spectra or other high-dimensional data with known but non-uniform observational uncertainties. The method uses an iterative inverse-variance-weighted least-squares minimization procedure to generate a best set of basis functions. The method is similar to principal components analysis (PCA), but with the substantial advantage that it uses measurement uncertainties in a responsible way and accounts naturally for poorly measured and missing data; it models the variance in the noise-deconvolved data space. A regularization can be applied, in the form of a smoothness prior (inspired by Gaussian processes) or a non-negative constraint, without making the method prohibitively slow. Because the method optimizes a justified scalar (related to the likelihood), the basis provides a better fit to the data in a probabilistic sense than any PCA basis. We test the method on Sloan Digital Sky Survey (SDSS) spectra, concentrating on spectra known to contain two redshift components: these are spectra of gravitational lens candidates and massive black hole binaries. We apply a hypothesis test to compare one-redshift and two-redshift models for these spectra, utilizing the data-driven model trained on a random subset of all SDSS spectra. This test confirms 129 of the 131 lens candidates in our sample and all of the known binary candidates, and turns up very few false positives.

79 ASTRONOMY AND ASTROPHYSICS↗

Tomographic methods in flow diagnostics

This report presents a viewpoint of tomography that should be well adapted to currently available optical measurement technology as well as the needs of computational and experimental fluid dynamists. The goals in mind are to record data with the fastest optical array sensors; process the data with the fastest parallel processing technology available for small computers; and generate results for both experimental and theoretical data. An in-depth example treats interferometric data as it might be recorded in an aeronautics test facility, but the results are applicable whenever fluid properties are to be measured or applied from projections of those properties. The paper discusses both computed and neural net calibration tomography. The report also contains an overview of key definitions and computational methods, key references, computational problems such as ill-posedness, artifacts, missing data, and some possible and current research topics.

Decker, Arthur J.↗

A Hybrid Framework Combining Model-Based and Data-Driven Methods for Hierarchical Decentralized Robust Dynamic State Estimation

This paper combines model-based and data-driven methods to develop a hierarchical, decentralized, robust dynamic state estimator (DSE). A two-level hierarchy is proposed, where the lower level consists of robust, model-based, decentralized DSEs. The state estimates sent from the lower level are received at the upper level, where they are filtered by a robust data-driven DSE after a principled sparse selection. This selection allows us to shrink the dimension of the problem at the upper level and hence significantly speed up the computational time. The proposed hybrid framework does not depend on the centralized infrastructure of the control centers; thus it can be completely embedded into the wide-area measurement systems. This feature will ultimately facilitate the placement of hierarchical decentralized control schemes at the phasor data concentrator locations. Also, the network model is not necessary; thus, a topology processor is not required. Finally, there is no assumption on the dynamics of the electric loads. The proposed framework is tested on the 2,000-bus synthetic Texas system, and shown to be capable of reconstructing the dynamic states of the generators with high accuracy, and of forecasting in the advent of missing data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Ground surface temperature derived Snow Cover Properties, Seward Peninsula, Alaska, 2019-2023

Snow-ground interface temperatures have been collected at the Teller mile marker 27 and Kougarok mile marker 64 field sites on the Seward Peninsula, Alaska from 2019 through 2023 (with data missing from Fall 2020 through Summer 2021 due to COVID). Temperatures were measured using iButton Link DS1921G-F5# Thermochron miniature temperature sensors and Tinytag TGP-4017 internal sensors deployed across the Kougarok 64 and Teller 27 field sites. These sensors are a cost-efficient way to collect snow-ground interface temperatures at a high spatial resolution, and when paired with air temperature data these measurements can provide insight into fine-scale variability in snowpack characteristics across the study sites. From this data, snow process metrics were calculated at each sensor location based on the methods outlined in Staub and Delaloye, 2017. Metrics are calculated daily for each sensor as well as over the entire season. These metrics include ground surface temperature (°C), the number of days under snow cover (number of days), the insulation effect of snow (unitless), the length of the transitional snow periods (number of days), as well as intermediaries such as temperature variability. Calculating these snow processes relies on the assumption that when snow covers a temperature sensor, it is buffered from diurnal fluctuations in air temperature by the insulating snow layer. More information on the calculated metrics can be found in the User Guide of this dataset, as well as in Staub and Delaloye’s 2017 publication Using Near-Surface Ground Temperature Data to Derive Snow Insulation and Melt Indices for Mountain Permafrost Applications. This dataset includes one daily and one seasonal *.csv file of metrics for every year of data, a daily and a seasonal *.csv data dictionary, and one User Guide document (*.pdf) describing data collection and processing.The Next-Generation Ecosystem Experiments: Arctic (NGEE Arctic), was a research effort to reduce uncertainty in Earth System Models by developing a predictive understanding of carbon-rich Arctic ecosystems and feedbacks to climate. NGEE Arctic was supported by the Department of Energy's Office of Biological and Environmental Research.The NGEE Arctic project had two field research sites: 1) located within the Arctic polygonal tundra coastal region on the Barrow Environmental Observatory (BEO) and the North Slope near Utqiagvik (Barrow), Alaska and 2) multiple areas on the discontinuous permafrost region of the Seward Peninsula north of Nome, Alaska.Through observations, experiments, and synthesis with existing datasets, NGEE Arctic provided an enhanced knowledge base for multi-scale modeling and contributed to improved process representation at global pan-Arctic scales within the Department of Energy's Earth system Model (the Energy Exascale Earth System Model, or E3SM), and specifically within the E3SM Land Model component (ELM).

54 ENVIRONMENTAL SCIENCES↗

Application of the Koopman Operator-Theoretic Framework to Power System Dynamic State Estimation

Model-based and data-driven methods are combined to develop a hierarchical decentralized, robust dynamic state estimator (DSE). A two-level hierarchy is proposed, where the lower level consists of robust, model-based, decentralized DSEs. The state estimates sent from the lower level are received at the upper level, where they are filtered by a robust data-driven DSE. The proposed hybrid framework does not depend on the centralized infrastructure of the control centers; thus it can be completely embedded into the wide-area measurement systems. This feature will ultimately facilitate the placement of hierarchical decentralized control schemes at the phasor data concentrator locations. Also, the network model is not necessary; thus, a topology processor is not required. Finally, there is no assumption on the dynamics of the electric loads. The proposed framework is tested on the 2,000-bus synthetic Texas system and shown to be capable of reconstructing the dynamic states of the generators with high accuracy, and of forecasting in the advent of missing data.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Interrelationships Between Receiver/Relative Operating Characteristics Display, Binomial, Logit, and Bayes' Rule Probability of Detection Methodologies

Unknown risks are introduced into failure critical systems when probability of detection (POD) capabilities are accepted without a complete understanding of the statistical method applied and the interpretation of the statistical results. The presence of this risk in the nondestructive evaluation (NDE) community is revealed in common statements about POD. These statements are often interpreted in a variety of ways and therefore, the very existence of the statements identifies the need for a more comprehensive understanding of POD methodologies. Statistical methodologies have data requirements to be met, procedures to be followed, and requirements for validation or demonstration of adequacy of the POD estimates. Risks are further enhanced due to the wide range of statistical methodologies used for determining the POD capability. Receiver/Relative Operating Characteristics (ROC) Display, simple binomial, logistic regression, and Bayes' rule POD methodologies are widely used in determining POD capability. This work focuses on Hit-Miss data to reveal the framework of the interrelationships between Receiver/Relative Operating Characteristics Display, simple binomial, logistic regression, and Bayes' Rule methodologies as they are applied to POD. Knowledge of these interrelationships leads to an intuitive and global understanding of the statistical data, procedural and validation requirements for establishing credible POD estimates.

Generazio, Edward R.↗

Coherent structure in solar wind C{sup 6+}/C{sup 4+} ionic composition data during the quiet-sun conditions of 2008

This analysis offers evidence of characteristic scale sizes in solar wind charge state data measured in situ for 13 quiet-Sun Carrington rotations in 2008. Using a previously established novel methodology, we analyze the wavelet power spectrum of the charge state ratio C{sup 6+}/C{sup 4+} measured in situ by ACE/SWICS for 2 hr and 12 minute cadence. We construct a statistical significance level in the wavelet power spectrum to quantify the interference effects arising from filling missing data in the time series, allowing extraction of significant power from the measured data to a resolution of 24 minutes. We analyze each wavelet power spectrum for transient coherency and global periodicities resulting from the superposition of repeating coherent structures. From the significant wavelet power spectra, we find evidence for a general upper limit on individual transient coherency of ∼10 days. We find evidence for a set of global periodicities between 4-5 hr and 35-45 days. We find evidence for the distribution of individual transient coherency scales consisting of two distinct populations. Below the ∼2 day timescale, the distribution is reasonably approximated by an inverse power law, whereas for scales ≳2 days, the distribution levels off, showing discrete peaks at common coherency scales. In addition, by organizing the transient coherency scale distributions by wind type, we find that these larger, common coherency scales are more prevalent and well defined in coronal hole wind. Finally, we discuss the implications of our results for current theories of solar wind generation and describe future work for determining the relationship between the coherent structures in our ionic composition data and the structure of the coronal magnetic field.

79 ASTRONOMY AND ASTROPHYSICS↗

Statistics Clinic

Do you have elevated p‐values? Is the data analysis process getting you down? Do you experience anxiety when you need to respond to criticism of statistical methods in your manuscript? You may be suffering from Insufficient Statistical Support Syndrome (ISSS). For symptomatic relief of ISSS, come for a free consultation with JSC biostatisticians at our help desk during the poster sessions at the HRP Investigators Workshop. Get answers to common questions about sample size, missing data, multiple testing, when to trust the results of your analyses and more. Side effects may include sudden loss of statistics anxiety, improved interpretation of your data, and increased confidence in your results.

Feiveson, Alan H.↗

A New Method of Deriving Time-Averaged Tropospheric Column Ozone over the Tropics Using Total Ozone Mapping Spectrometer (TOMS) Radiances: Intercomparison and Analysis Using TRACE A Data

Error analysis of archived total 03 from total ozone mapping spectrometer (TOMS) (version 6) presented. Daily total 03 maps for the tropics, from the period October 6-21, 1992, are derived from TOMS radiances following correction for these errors. These daily maps, averaged together, show a wavelike feature, which is observed in all latitude bands, underlying sharp peaks which occur at different longitudes depending on the latitude. The wave pattern is used to derive both time-averaged stratospheric and tropospheric 03 fields. The nature of the wave pattern (stratospheric or tropospheric) cannot be determined with certainty due to missing data (no Pacific sondes, no lower stratospheric Stratospheric Aerosol and Gas Experiment (SAGE) ozone for 18 months after the Mt. Pinatubo eruption) and significant uncertainties in the corroborative satellite record in the lower stratosphere (solar backscattered ultraviolet (SBUV), microwave limb sounder (MLS)). However, the time- averaged tropospheric ozone field, based on the assumption that the wave feature is stratospheric, agrees within 10% with ultraviolet differential absorption laser Transport and Atmospheric Chemistry near the Equator-Atlantic) (TRACE A) 03 measurements from the DC-8 and with ozonesonde measurements over Brazzaville, Congo, Ascension Island, and Natal, Brazil, for the period October 6-21, 1992. The derived background (nonpolluted) Indian Ocean tropospheric ozone amount, 26 Dobson units (DU), agrees with the cleanest African ozonesonde profiles for September-October 1992. The assumption of a totally tropospheric wave (flat stratosphere) gives 38 DU above the western Indian Ocean and 15-40% disagreements with the sondes. Tropospheric column 03 is high from South America to Africa, owing to interaction of dynamics with biomass burning emissions. Comparison with fire distributions from advanced very high resolution radiometer (AVHHR) during October 1992 suggests that tropospheric 03 produced from biomass burning in South America and Africa dominates the 03 budget in the tropical southern hemisphere during the study period.

Kim, J. H.↗

Using Concurrent Cardiovascular Information to Augment Survival Time Data from Orthostatic Tilt Tests

Orthostatic Intolerance (OI) is the propensity to develop symptoms of fainting during upright standing. OI is associated with changes in heart rate, blood pressure and other measures of cardiac function. Problem: NASA astronauts have shown increased susceptibility to OI on return from space missions. Current methods for counteracting OI in astronauts include fluid loading and the use of compression garments. Multivariate trajectory spread is greater as OI increases. Pairwise comparisons at the same time within subjects allows incorporation of pass/fail outcomes. Path length, convex hull area, and covariance matrix determinant do well as statistics to summarize this spread Missing data problems Time series analysis need many more time points per OTT session treatment of trend? how incorporate survival information?

Feiveson, Alan H.↗

Dark energy survey year 3 results: miscentring calibration and X-ray-richness scaling relations in redMaPPer clusters

ABSTRACT We use Dark Energy Survey Year 3 (DES Y3) clusters with archival XMM–Newton and Chandra X-ray data to assess the centring performance of the redMaPPer cluster finder and to measure key richness observable scaling relations. We find that 10–20 per cent of redMaPPer clusters are miscentred, both when comparing to the X-ray peak position and to the visually identified central cluster galaxy. We find no significant difference in miscentring in bins of low versus high richness or redshift. The dominant reasons for miscentring include masked or missing data and the presence of other bright galaxies in the cluster. For half of the miscentred clusters, the correct central was one of the possible centrals identified by redMaPPer, while for ∼40 per cent of miscentred clusters, the correct central is not a redMaPPer member mostly due to masking. Additionally, we fit scaling relations of X-ray temperature and luminosity with richness. We find a TX–λ scatter of $0.21\pm 0.01$. While the scatter in TX–λ is consistent in redshift bins, we find modestly different slopes, with high-redshift clusters displaying a somewhat shallower relation. Splitting based on richness, we find a marginally larger scatter for our lowest richness bin, 20 < λ < 40. We note that the robustness of the scaling relations at lower richnesses is limited by the unknown selection function, but at λ > 75, we detect nearly all of the clusters falling within existing X-ray pointings. The X-ray properties of detected, serendipitous clusters are generally consistent with those of targeted clusters.

79 ASTRONOMY AND ASTROPHYSICS↗

On the stability of robotic systems with random communication rates

Control problems of sampled data systems which are subject to random sample rate variations and delays are studied. Due to the rapid growth of the use of computers more and more systems are controlled digitally. Complex systems such as space telerobotic systems require the integration of a number of subsystems at different hierarchical levels. While many subsystems may run on a single processor, some subsystems require their own processor or processors. The subsystems are integrated into functioning systems through communications. Communications between processes sharing a single processor are also subject to random delays due to memory management and interrupt latency. Communications between processors involve random delays due to network access and to data collisions. Furthermore, all control processes involve delays due to casual factors in measuring devices and to signal processing. Traditionally, sampling rates are chosen to meet the worst case communication delay. Such a strategy is wasteful as the processors are then idle a great proportion of the time; sample rates are not as high as possible resulting in poor performance or in the over specification of control processors; there is the possibility of missing data no matter how low the sample rate is picked. Asymptotical stability with probability one for randomly sampled multi-dimensional linear systems is studied. A sufficient condition for the stability is obtained. This condition is so simple that it can be applied to practical systems. A design procedure is also shown.

Kobayashi, H.↗

Offshore wind energy forecasting sensitivity to sea surface temperature input in the Mid-Atlantic

Abstract. As offshore wind farm development expands, accurate wind resource forecasting over the ocean is needed. One important yet relatively unexplored aspect of offshore wind resource assessment is the role of sea surface temperature (SST). Models are generally forced with reanalysis data sets, which employ daily SST products. Compared with observations, significant variations in SSTs that occur on finer timescales are often not captured. Consequently, shorter-lived events such as sea breezes and low-level jets (among others), which are influenced by SSTs, may not be correctly represented in model results. The use of hourly SST products may improve the forecasting of these events. In this study, we examine the sensitivity of model output from the Weather Research and Forecasting model (WRF) 4.2.1 to different SST products. We first evaluate three different data sets: the Multiscale Ultrahigh Resolution (MUR25) SST analysis, a daily, 0.25∘ × 0.25∘ resolution product; the Operational Sea Surface Temperature and Ice Analysis (OSTIA), a daily, 0.054∘ × 0.054∘ resolution product; and SSTs from the Geostationary Operational Environmental Satellite 16 (GOES-16), an hourly, 0.02∘ × 0.02∘ resolution product. GOES-16 is not processed at the same level as OSTIA and MUR25; therefore, the product requires gap-filling using an interpolation method to create a complete map with no missing data points. OSTIA and GOES-16 SSTs validate markedly better against buoy observations than MUR25, so these two products are selected for use with model simulations, while MUR25 is at this point removed from consideration. We run the model for June and July of 2020 and find that for this time period, in the Mid-Atlantic, although OSTIA SSTs overall validate better against in situ observations taken via a buoy array in the area, the two products result in comparable hub-height (140 m) wind characterization performance on monthly timescales. Additionally, during hours-long flagged events (< 30 h each) that show statistically significant wind speed deviations between the two simulations, both simulations once again demonstrate similar validation performance (differences in bias, earth mover's distance, correlation, and root mean square error on the order of 10−1 or less), with GOES-16 winds validating nominally better than OSTIA winds. With a more refined GOES-16 product, which has been not only gap-filled but also assimilated with in situ SST measurements in the region, it is likely that hub-height winds characterized by GOES-16-informed simulations would definitively validate better than those informed by OSTIA SSTs.

17 WIND ENERGY↗

Comparing ICME simulations with scaled laboratory experiment

In stellar physics and astrophysics, numerical simulations and laboratory experiments are often compared to observational data to support their representation of the real world. However, there is also merit in comparing numerical simulations to properly scaled experiments, especially when the experiment and the simulation are both emulating the solar phenomena. Confirming the credibility of scaled experiments and their scaling with well-validated models is important to expand our knowledge of the associated physical phenomena. This is significant because experiments and simulations can be performed frequently, whereas observations may be limited by location, field of view, and missing data. In this work, we use the Alfvén Wave Solar atmosphere Model, a well-validated magnetohydrodynamic model, to simulate an interplanetary coronal mass ejection (ICME) and compare it to an experiment which provides a scaled analog to a physical ICME. The experiment was performed on the Big Red Ball facility and scaled using dimensionless parameters such as plasma β and magnetosonic Mach number to reproduce the main structure of an ICME. We compare the model-simulated temperature, density, and magnetic field to those from the experiment, as well as the scaling parameters used in the experiment, to those calculated from the simulation. This comparison is performed to further justify the scaling arguments made by the experiment. Additionally, the comparison would lead to the development of stronger scaling arguments for future experiments.

Bryant, K. [University of Michigan, Ann Arbor, MI ↗

Intersection-Controller Software Module

An important part of the emergency-vehicle traffic-light-preemption system summarized in the preceding article is a software module executed by a microcontroller in each intersection controller. This module monitors the broadcasts from all nearby participating emergency vehicles and intersections. It gathers the broadcast data pertaining to the positions and velocities of the vehicles and the timing of traffic and pedestrian lights and processes the data into predictions of the future positions of the vehicles. Analyzing the predictions by a combination of proximity tests, map-matching techniques, and statistical calculations designed to minimize the adverse effects of uncertainties in vehicle positions and headings, the module decides whether to preempt and issues the appropriate commands to the traffic lights, pedestrian lights, and electronic warning signs at the intersection. The module also broadcasts its state to all nearby vehicles and intersections. The module is designed to mitigate the effects of missing data and of unpredictable delays in the system. It has been intensively tested and refined so that it fails to warn in very few cases and issues very few false warnings.

Bachelder, Aaron↗

Recent Improvements in the National Solar Radiation Database (NSRDB)

The National Solar Radiation Database (NSRDB) has significantly evolved since the first release of the point source database in 1993. The NSRDB has been widely used by the solar energy industry to provide long-term time-series solar resource data for various project phases. The NSRDB represents the state of the art in the satellite-based estimation of solar resource information and uses a unique physics-based modeling approach that allows improvements in accuracy with the deployment of the next-generation geostationary satellites. This poster provides an overview of (1) the improved spatiotemporal resolution; (2) on-demand services and their applications; (3) future Improvements, such as a new direct normal irradiance model and new methods to gap-fill missing data using physics-guided machine learning; (4) data quality; and (5) data dissemination.

MATHEMATICS AND COMPUTING,SOLAR ENERGY↗

Benefits and Challenges of CCSDS File Delivery Protocol as Applied to Europa Clipper

—This paper describes the use of the Consultative Committee for Space Data Systems (CCSDS) File Delivery Protocol (CFDP) on the Europa Clipper mission for both uplink and downlink of files. It includes an overview of CFDP, the history of why CFDP was chosen, how it benefits mission operations, some of the mission scenarios that stress CFDP, operability aspects, the best practices that Clipper adopted from other missions and some of the technical challenges with implementation, and verification and validation. The benefits to mission operations accrue because CFDP reduces the need for manual management of file transfer, including retransmission of missing data, and deletion of files only after confirmation of receipt by the ground. The challenges occur because CFDP is a round-trip protocol – it requires messages in both directions to complete a file transfer, and because it uses timers to ensure that control messages are resent if needed to prevent transactions from going stale. Any situations where communication is restricted to a single direction, interrupted, reordered, or backlogged can pose a challenge. There are also implementation challenges. Europa Clipper is the first mission at the Jet Propulsion Laboratory (JPL) to adopt class 2, fully acknowledged, CFDP for both uplink and downlink. The implementation needed new software, requirements and operational procedures. The experience of the Applied Physics Laboratory (APL) with CFDP from their previous missions was crucial to success for Europa Clipper. Because CFDP relies on timers and messages travel in both directions, verification and validation (V&V) requires new approaches. For certain scenarios, a live ground system talking to a live flight system with realistic simulated one-way light times, data rates and data outages must be used.

Albers, Joshua↗

Filling the Gaps: A Bayesian Mixture Model for Imputing Missing Soil Water Content Data

ABSTRACT Soil water content (SWC) data are central to evaluating how soil moisture varies over time and space and influences critical plant and ecosystem functions, especially in water‐limited drylands. However, sensors that record SWC at high frequencies often malfunction, leading to incomplete timeseries and limiting our understanding of dryland ecosystem dynamics. We developed an analytical approach to impute missing SWC data, which we tested at six eddy flux tower sites along an elevation gradient in the southwestern United States. We impute missing data as a mixture of linearly interpolated SWC between the observed endpoints of a missing data gap and SWC simulated by an ecosystem water balance model (SOILWAT2). Within a Bayesian framework, we allowed the relative utility (mixture weight) of each component (linearly interpolated vs. SOILWAT2) to vary by depth, site and gap characteristics. We explored “fixed” weights versus “dynamic” weights that vary as a function of cumulative precipitation, average temperature, and time since the start of the gap. Both models estimated missing SWC data well ( R 2 = 0.70–0.88 vs. 0.75–0.91 for fixed vs. dynamic weights, respectively), but the utility of linearly interpolated versus SOILWAT2 values depended on site and depth. SOILWAT2 was more useful for more arid sites, shallower depths, longer and warmer gaps and gaps that received greater precipitation. Overall, the mixture model reliably gap‐fills SWC, while lending insight into processes governing SWC dynamics. This approach to impute missing data could be adapted to accommodate more than two mixture components and other types of environmental timeseries.

Ogle, Kiona [School of Informatics, Computing, and↗