Search NASA⌕ Search

SEARCH · Search NASA

Results for “Imputation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 73 records · Page 4

A hybrid computational strategy to address WGS variant analysis in >5000 samples

Abstract Background The decreasing costs of sequencing are driving the need for cost effective and real time variant calling of whole genome sequencing data. The scale of these projects are far beyond the capacity of typical computing resources available with most research labs. Other infrastructures like the cloud AWS environment and supercomputers also have limitations due to which large scale joint variant calling becomes infeasible, and infrastructure specific variant calling strategies either fail to scale up to large datasets or abandon joint calling strategies. Results We present a high throughput framework including multiple variant callers for single nucleotide variant (SNV) calling, which leverages hybrid computing infrastructure consisting of cloud AWS, supercomputers and local high performance computing infrastructures. We present a novel binning approach for large scale joint variant calling and imputation which can scale up to over 10,000 samples while producing SNV callsets with high sensitivity and specificity. As a proof of principle, we present results of analysis on Cohorts for Heart And Aging Research in Genomic Epidemiology (CHARGE) WGS freeze 3 dataset in which joint calling, imputation and phasing of over 5300 whole genome samples was produced in under 6 weeks using four state-of-the-art callers. The callers used were SNPTools, GATK-HaplotypeCaller, GATK-UnifiedGenotyper and GotCloud. We used Amazon AWS, a 4000-core in-house cluster at Baylor College of Medicine, IBM power PC Blue BioU at Rice and Rhea at Oak Ridge National Laboratory (ORNL) for the computation. AWS was used for joint calling of 180 TB of BAM files, and ORNL and Rice supercomputers were used for the imputation and phasing step. All other steps were carried out on the local compute cluster. The entire operation used 5.2 million core hours and only transferred a total of 6 TB of data across the platforms. Conclusions Even with increasing sizes of whole genome datasets, ensemble joint calling of SNVs for low coverage data can be accomplished in a scalable, cost effective and fast manner by using heterogeneous computing platforms without compromising on the quality of variants.

59 BASIC BIOLOGICAL SCIENCES↗

Soybean ( Glycine max ) Haplotype Map (GmHapMap): a universal resource for soybean translational and functional genomics

Here, we describe a worldwide haplotype map for soybean (GmHapMap) constructed using whole-genome sequence data for 1007 Glycine max accessions and yielding 14.9 million variants as well as 4.3 M tag single-nucleotide polymorphisms (SNPs). When sampling random subsets of these accessions, the number of variants and tag SNPs plateaued beyond approximately 800 and 600 accessions, respectively. This suggests extensive coverage of diversity within the cultivated soybean. GmHapMap variants were imputed onto 21 618 previously genotyped accessions with up to 96% success for common alleles. A local association analysis was performed with the imputed data using markers located in a 1-Mb region known to contribute to seed oil content and enabled us to identify a candidate causal SNP residing in the NPC1 gene. We determined gene-centric haplotypes (407 867 GCHs) for the 55 589 genes and showed that such haplotypes can help to identify alleles that differ in the resulting phenotype. Finally, we predicted 18 031 putative loss-of-function (LOF) mutations in 10 662 genes and illustrated how such a resource can be used to explore gene function. The GmHapMap provides a unique worldwide resource for applied soybean genomics and breeding.

54 ENVIRONMENTAL SCIENCES↗

Scripts and raw/processed precipitation NRCS network data, Upper Colorado, 2008-2017

This data package consists of scripts and data that were used to generate all the results and figures for Mital et al., 2020 (https://doi.org/10.3389/frwa.2020.00020). The purpose of the study was to develop a new algorithm to impute (or gap-fill) missing daily precipitation data. Study area was the Upper Colorado Water Resources Region (UCWRR), over a time period of 2008-2017. In terms of data, the package consists of raw precipitation measurements from Natural Resources Conservation Service (NRCS) network. These raw measurements are compiled into a single csv file (NRCS_dates_2008_2017.csv). The corresponding metadata are compiled in Metadata_processed.csv. The package also consists of outputs generated during Baseline and Sequential imputation runs as documented in Mital et al., 2020. Scripts used to generate all the results and figures for Mital et al., 2020 are also uploaded. A readme file documenting the layout of the data archive is also uploaded, and a KML file shows the geographic range for the data.

54 ENVIRONMENTAL SCIENCES↗

Enhancing approximate modular Bayesian inference by emulating the conditional posterior

In modular Bayesian analyses, complex models are composed of distinct modules, each representing different aspects of the data or prior information. In this context, fully Bayesian approaches can sometimes lead to undesirable feedback between modules, compromising the integrity of the inference. The “cut-distribution” prevents unwanted influence between modules by “cutting” feedback. The direct sampling (DS) algorithm is standard practice for approximating the cut-distribution, but it can be computationally intensive, especially when the number of imputations required is large. An enhanced method is proposed, the Emulating the Conditional Posterior (ECP) algorithm, which leverages emulation to increase the number of imputations. Through numerical experiment it is demonstrated that the ECP algorithm outperforms the traditional DS approach in terms of accuracy and computational efficiency, particularly when resources are constrained. Here, it is also shown how the DS algorithm can be improved using ideas from design of experiments. Some practical recommendations are given for algorithm choice in modular Bayesian analyses.

97 MATHEMATICS AND COMPUTING↗

Developing custom fire behavior fuel models from ecologically complex fuel structures for upper Atlantic Coastal Plain forests.

Currently geospatial fire behavior analyses are performed with an array of fire behavior modeling systems such as FARSITE, FlamMap, and the Large Fire Simulation System. These systems currently require standard or customized surface fire behavior fuel models as inputs that are often assigned through remote sensing information. The ability to handle hundreds or thousands of measured surface fuelbeds representing the fine scale variation in fire behavior on the landscape is constrained in terms of creating compatible custom fire behavior fuel models. In this study, we demonstrate an objective method for taking ecologically complex fuelbeds from inventory observations and converting those into a set of custom fuel models that can be mapped to the original landscape. We use an original set of 629 fuel inventory plots measured on an 80,000 ha contiguous landscape in the upper Atlantic Coastal Plain of the southeastern United States. From models linking stand conditions to component fuel loads, we impute fuelbeds for over 6000 stands. These imputed fuelbeds were then converted to fire behavior parameters under extreme fuel moisture and wind conditions (97th percentile) using the fuel characteristic classification system (FCCS) to estimate surface fire rate of spread, surface fire flame length, shrub layer reaction intensity (heat load), non-woody layer reaction intensity, woody layer reaction intensity, and litter-lichen-moss layer reaction intensity. We performed hierarchical cluster analysis of the stands based on the values of the fire behavior parameters. The resulting 7 clusters were the basis for the development of 7 custom fire behavior fuel models from the cluster centroids that were calibrated against the FCCS point data for wind and fuel moisture. The latter process resulted in calibration against flame length as it was difficult to obtain a simultaneous calibration against both rate of spread and flame length. The clusters based on FCCS fire behavior parameters represent reasonably identifiable stand conditions, being: (1) pine dominated stands with more litter and down woody debriscomponents than other stands, (2) hardwood and pine stands with no shrubs, (3) hardwood dominated stands with low shrub and high non-woody biomass and high down woody debris, (4) stands with high grass and forb (i.e., non-woody) biomass as well as substantial shrub biomass, (5) stands with both high shrub and litter biomass, (6) pine-mixed hardwood stands with moderate litter biomass and low shrub biomass, and (7) baldcypress-tupelo stands. Models representing these stand clusters generated flame lengths from 0.6 to 2.3 musing a 30 km h{sub 1} wind speed and fireline intensities of 100-1500 kW m{sub 1} that are typical within the range of experience on this landscape. The fuel models ranked 1 < 2 < 7 < 5 < 4 < 3 < 6 in terms of both flame length and fireline intensity. The method allows for ecologically complex data to be utilized in order to create a landscape representative of measured fuel conditions and to create models that interface with geospatial fire models.

09 BIOMASS FUELS↗

Geostatistical interpolation of streambed hydrologic attributes with addition of left censored data and anisotropy

Spatial geostatistical interpolation of point measurements of streambed attributes in the hyporheic zone may be constrained by the streambed anisotropy, and data density and spatial distribution may significantly impact the results. Spatial clustering and low spatial data density can be caused by bedrock outcropping at the streambed limiting installation of in-stream piezometers. This study examines parameter error variability of the geostatistical interpolation using anisotropic interpolation methods and increasing the data density by adding left censored values (i.e., data below measurement limit) to locations where measurements were limited by exposed bedrock lining the streambed. The reduction in relative standard error of the interpolation was determined for the spatial distributions of streambed attributes including hydraulic conductivity, seepage flux, and mercury solute flux measured in two different years along a study reach in East Fork Poplar Creek, Tennessee, USA. Here, two methods to impute the left censored values were compared including the conventional half the detection limit substitution method, and the Stochastic Approximation of Expectation-Maximization (SAEM) algorithm, which both had comparable results. Imputing left censored data increased the data density to recommended ranges, reduced data clustering, increased the spatial dependence for some attributes, and reduced the standard error for each of the three attributes. For the reach considered herein, addition of the left censored values resulted in a larger error reduction than the consideration of anisotropy within the interpolation, which confirms the benefit of data addition to increase data density within data-limited river corridors.

58 GEOSCIENCES↗

Estimation of missing values in solar radiation data using piecewise interpolation methods: Case study at Penang city

The solar radiation values have been composed by automatic weather station using the device that namely pyranometer. The device is functions to records all the radiation values that have been dispersed, and these data are very useful for it experimental works and solar device’s development. In addition, for modeling and designing on solar radiation system application is needed for complete data observation. Unfortunately, lack for obtained the complete solar radiation data frequently occur due to several technical problems, which mainly contributed by monitoring device. Into encountering this matter, estimation missing values in an effort to substitute absent values with imputed data. This paper aimed to evaluate several piecewise interpolation techniques likes linear, splines, cubic, and nearest neighbor into dealing missing values in hourly solar radiation data. Then, proposed an extendable work into investigating the potential used of cubic Bezier technique and cubic Said-ball method as estimator tools. As result, methods for cubic Bezier and Said-ball perform the best compare to another piecewise imputation technique.

14 SOLAR ENERGY↗

Market-based Policies for Non-Proliferation

Access to weapons-useable fissile material—highly enriched uranium and separated plutonium—is the primary technical barrier to nuclear weapons development. Nuclear non-proliferation policy is largely concerned with the secure distribution and use of these materials and the technologies that produce them. To the extent that both materials and technologies are traded in markets, the problem may be viewed as one of regulating markets to give an acceptable set of allocative outcomes. While a laissez-faire approach to these markets was rejected long ago, most current policy is based on prescribed or proscribed behavior rather than market incentives. “Market-based” non-proliferation policies would act through incentives in the nuclear materials and technology markets rather than by prescribing outcomes or behavior. Possible polices of this sort include subsidized enrichment services, credits to impute value to spent nuclear fuel, and new markets for special scarce nuclear materials. While market-based policies will not replace command-and-control policies entirely, they could lead to efficient regulation which suffices for most market participants, reserving special, more discriminatory instruments for the few hard cases.

nulcear fuel markets, nonproliferation, SWU↗

A Statistical Framework for Protein Quantitation in Bottom-Up MS-Based Proteomics

Motivation: Quantitative mass spectrometry-based proteomics requires protein-level estimates and associated confidence measures. Challenges include the presence of low quality or incorrectly identified peptides and informative missingness. Furthermore, models are required for rolling peptide-level information up to the protein level. Results: We present a statistical model that carefully accounts for informative missingness in peak intensities and allows unbiased, model-based, protein-level estimation and inference. The model is applicable to both label-based and label-free quantitation experiments. We also provide automated, model-based, algorithms for filtering of proteins and peptides as well as imputation of missing values. Two LC/MS datasets are used to illustrate the methods. In simulation studies, our methods are shown to achieve substantially more discoveries than standard alternatives. Availability: The software has been made available in the opensource proteomics platform DAnTE (http://omics.pnl.gov/software/). Contact: adabney@stat.tamu.edu Supplementary information: Supplementary data are available at Bioinformatics online.

59 BASIC BIOLOGICAL SCIENCES↗

Radiation Doses to Members of the U.S. Population from Ubiquitous Radionuclides in the Body: Part 3, Results, Variability, and Uncertainty

This paper is part three of a three-part series investigating annual effective doses to residents of the United States from intakes of ubiquitous radionuclides, including radionuclides occurring naturally, radionuclides whose concentrations are technologically enhanced, and anthropogenic radionuclides. The radionuclides of interest are the 238U series (14 nuclides), the actinium series (headed by 235U; 11 nuclides), and the 232Th series (11 nuclides); primordial radionuclides 87Rb and 40K; cosmogenic and fallout radionuclides 14C and 3H; and purely anthropogenic radionuclides 137Cs-137mBa, 129I and 90Sr-90Y. This series of papers explicitly excludes intakes from inhaling 222Rn, 220Rn, and their short-lived decay products; it also excludes intakes of radionuclides in occupational and medical settings. Part one reviewed, summarized, characterized, and grouped all published and some unpublished data for U.S. residents on ubiquitous radionuclide concentrations in tissues and organs. Part two described the methods used to organize the data collected in part one and segregate it into the ages and genders defined by the study, imputed missing values from the existing data, apportioned activity in bone, and imputed activity in hollow organ contents and the remainder of the body. This paper estimates equivalent doses to target tissues from source regions and maps target tissues to lists of tissues with International Commission on Radiation Protection (ICRP) tissue-weighting factors or to surrogate tissue regions when there is no direct match. Effective doses, using ICRP tissue-weighting factors recommended in 1977, 1990, and 2007, are then calculated, and an upper bound of variability of the effective dose is estimated by calculating the average coefficients of variation (CV), assuming all variance is due to variability. Most of the data were for adult males, whose average annual effective dose is estimated to be 337 μSv (CV = 0.65, geometric mean = 283 μSv, geometric standard deviation sG = 1.81) using 2007 ICRP tissue-weighting factors. This result is between the National Council on Radiation Protection & Measurements’ 1987 estimate of 390 μSv (using 1977 wTs) and its 2009 estimate of 285 μSv (using 2007 wTs) and is higher than the United Nations Scientific Committee on the Effects of Atomic Radiation’s 2000 estimate of 310 μSv (using 1990 wTs). The methods and software developed for this project are sufficiently detailed and sufficiently general to be usable with autopsy data from any or all countries.

59 BASIC BIOLOGICAL SCIENCES↗

Low-rank Tensor Completion for PMU Data Recovery

This paper proposes a tensor completion method for the recovery of missing phasor measurement unit (PMU) measurements. Tensor completion as the general case of matrix completion has attracted increasing attention in recent years. The imputation accuracy for the existing matrix completion methods may be significantly reduced when there are consecutive data losses across multiple data channels. To tackle this issue, we explore the multi-way characteristics of PMU measurements by using a tensor model. We leverage the low-rank property of the PMU measurements and formulate the missing PMU data recovery problem as a low-rank tensor completion problem. An efficient algorithm based on alternating direction method of multipliers (ADMM) is developed to solve the tensor completion problem. The experiments using the real PMU dataset show that the proposed method exhibits better imputation accuracy compared with the conventional data recovery methods.

Ghasemkhani, Amir↗

Latent Neural ODE for Integrating Multi-Timescale Measurements in Smart Distribution Grids

Under a smart grid paradigm, there has been an increase in sensor installations to enhance situational awareness. The measurements from these sensors can be leveraged for real-time monitoring, control, and protection. However, these measurements are typically irregularly sampled. These measure-ments may also be intermittent due to communication bandwidth limitations. To tackle this problem, this paper proposes a novel latent neural ordinary differential equations (LODE) approach to aggregate the unevenly sampled multivariate time-series measurements. The proposed approach is flexible in performing both imputations and predictions while being computationally efficient. Simulation results on IEEE 37 bus test systems illustrate the efficiency of the proposed approach.

multi time-scale measurements↗

On the Impact of Income, Age, and Travel Distance on the Value of Time

The value of time (VOT) is a fundamental component used in transportation modeling, policy analysis, and economic appraisal. Decades of research and practice have empirically estimated the VOT across many factors (e.g., mode, purpose, time, comfort, etc.), yet little is known about its underlying form. Although it is well established that VOT can vary, it is still unclear whether patterns exist in this variation. The objective of this paper is not to merely estimate the VOT, but to model the VOT across multiple continuous and interacting variables. The purpose is to reveal its functional form with respect to mode, age, gender, purpose, income, and time of day to provide a generalizable understanding for future research and practice. Such an understanding can help develop simpler models and reduce the need for bespoke estimations for every conceivable variable perturbation. This research utilized a household travel survey containing 14,159 reported trips with imputed travel time and costs for the alternative mode choices. The average overall estimated VOT is 40.32 $/h, with results showing VOT varying log-linearly with income and trip distance, but following a Gaussian function (normal curve) with age. Overall, the results show that travel distance dominates VOT variation, which increases exponentially at a rate that is 3.61 times higher per mile of distance than per $10,000 of income, and that VOT by age peaks at age 54. This basic understanding of how the VOT varies sets the foundation for answering the subsequent question for why it might vary.

Engineering↗

LandScan Mosaic

The LandScan program at Oak Ridge National Laboratory (ORNL), in collaboration with the National Geospatial-Intelligence Agency (NGA), continues to deliver the most accurate and up to date global, high resolution gridded population data. Additionally, the latest advancements in the LandScan HD methodology led to reduced latency in development of rapid updates for geopolitical events. With momentum towards reporting more up to date population estimates, feedback from the user community expressed interest in reporting population estimates in ranges - whether to express a level of uncertainty or confirm to leadership and stakeholders the modeled data are estimates. Building upon the need to understand uncertainty or confidence in the modeled data and report ranges at the global scale, LandScan Mosaic was developed. LandScan Mosaic represents the next generation of high-resolution population modeling, building upon the established success of previous LandScan HD iterations. While LandScan HD employed a deterministic big data fusion approach, LandScan Mosaic enhances this methodology by integrating advanced machine learning techniques to impute missing, yet crucial, population model parameters. This advancement allows for probabilistic modeling of building occupancy and population distribution, incorporating uncertainty quantification through Monte Carlo sampling methods. By combining big data fusion with machine learning-driven imputation and stochastic modeling, LandScan Mosaic provides a more comprehensive and robust representation of population dynamics. LandScan Mosaic will be following the in the footsteps of its longstanding counterpart LandScan Global and releasing a global gridded population raster, at the 3-arcsecond resolution. This technical report documents the current stage of development of LandScan Mosaic, detailing the methodologies and data sources behind the modeling. Stakeholders are encouraged to use this document as an authoritative reference for insight into Mosaic’s data development processes. However, readers should note that LandScan Mosaic remains in a late-stage research and development phase, and methodologies and data presented here are subject to refinements ahead of the anticipated global release in Summer 2025. Feedback and inquiries from users and stakeholders are welcomed as we continue to refine and enhance this important population resource.

97 MATHEMATICS AND COMPUTING↗

Cleaning Images with Gaussian Process Regression

Many approaches to astronomical data reduction and analysis cannot tolerate missing data: corrupted pixels must first have their values imputed. This paper presents astrofix, a robust and flexible image imputation algorithm based on Gaussian process regression. Through an optimization process, astrofix chooses and applies a different interpolation kernel to each image, using a training set extracted automatically from that image. It naturally handles clusters of bad pixels and image edges and adapts to various instruments and image types. For bright pixels, the mean absolute error of astrofix is several times smaller than that of median replacement and interpolation by a Gaussian kernel. We demonstrate good performance on both imaging and spectroscopic data, including the SBIG 6303 0.4 m telescope and the FLOYDS spectrograph of Las Cumbres Observatory and the CHARIS integral-field spectrograph on the Subaru Telescope.

42 ENGINEERING↗

A consistent dataset for the net income distribution for 190 countries and aggregated to 32 geographical regions from 1958 to 2015

Abstract. Data on income distributions within and across countries are becoming increasingly important for informing analysis of income inequality and understanding the distributional consequences of climate change. While datasets on income distribution collected from household surveys are available for multiple countries, these datasets often do not represent the same concept of inequality (or income concept) and therefore make comparisons across countries, over time and across datasets difficult. Here, we present a consistent dataset of income distributions across 190 countries from 1958 to 2015 measured in terms of net income. We complement the observed values in this dataset with values imputed from a summary measure of the income distribution, specifically the Gini coefficient. For the imputation, we use a recently developed nonparametric principal-component-based approach that shows an excellent fit to data on income distributions compared to other approaches. We also present another version of this dataset aggregated from the country level to 32 geographical regions. Our dataset is developed for the purpose of calibrating models such as integrated human–Earth system models with detailed data on income distributions. This dataset will enable more robust analysis of income distribution at multiple scales. The latest version of our data are available on Zenodo: https://doi.org/10.5281/zenodo.7093997 (Narayan et al., 2022b).

97 MATHEMATICS AND COMPUTING↗

Full-Coverage High-Resolution Daily PM(sub 2.5) Estimation using MAIAC AOD in the Yangtze River Delta of China

Satellite aerosol optical depth (AOD) has been used to assess population exposure to fine particulate matter (PM (sub 2.5)). The emerging high-resolution satellite aerosol product, Multi-Angle Implementation of Atmospheric Correction(MAIAC), provides a valuable opportunity to characterize local-scale PM(sub 2.5) at 1-km resolution. However, non-random missing AOD due to cloud snow cover or high surface reflectance makes this task challenging. Previous studies filled the data gap by spatially interpolating neighboring PM(sub 2.5) measurements or predictions. This strategy ignored the effect of cloud cover on aerosol loadings and has been shown to exhibit poor performance when monitoring stations are sparse or when there is seasonal large-scale missngness. Using the Yangtze River Delta of China as an example, we present a Multiple Imputation (MI) method that combines the MAIAC high-resolution satellite retrievals with chemical transport model (CTM) simulations to fill missing AOD. A two-stage statistical model driven by gap-filled AOD, meteorology and land use information was then fitted to estimate daily ground PM(sub 2.5) concentrations in 2013 and 2014 at 1 km resolution with complete coverage in space and time. The daily MI models have an average R(exp 2) of 0.77, with an inter-quartile range of 0.71 to 0.82 across days. The overall Ml model 10-fold cross-validation R(exp 2) (root mean square error) were 0.81 (25 gm(exp 3)) and 0.73 (18 gm(exp 3)) for year 2013 and 2014, respectively. Predictions with only observational AOD or only imputed AOD showed similar accuracy.Comparing with previous gap-filling methods, our MI method presented in this study performed bette rwith higher coverage, higher accuracy, and the ability to fill missing PM(sub 2.5) predictions without ground PM(sub 2.5) measurements. This method can provide reliable PM(sub 2.5)predictions with complete coverage that can reduce biasin exposure assessment in air pollution and health studies.

PM2.5↗