Search NASA⌕ Search

SEARCH · Search NASA

Results for “Imputation”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Classifying with confidence from incomplete information.

For this paper, we consider the problem of classifying a test sample given incomplete information. This problem arises naturally when data about a test sample is collected over time, or when costs must be incurred to compute the classification features. For example, in a distributed sensor network only a fraction of the sensors may have reported measurements at a certain time, and additional time, power, and bandwidth is needed to collect the complete data to classify. A practical goal is to assign a class label as soon as enough data is available to make a good decision. We formalize this goal through the notion of reliability—the probability that a label assigned given incomplete data would be the same as the label assigned given the complete data, and we propose a method to classify incomplete data only if some reliability threshold is met. Our approach models the complete data as a random variable whose distribution is dependent on the current incomplete data and the (complete) training data. The method differs from standard imputation strategies in that our focus is on determining the reliability of the classification decision, rather than just the class label. We show that the method provides useful reliability estimates of the correctness of the imputed class labels on a set of experiments on time-series data sets, where the goal is to classify the time-series as early as possible while still guaranteeing that the reliability threshold is met.

96 KNOWLEDGE MANAGEMENT AND PRESERVATION↗

Augmenting system reliability analyses with observation priors

Occasionally, a system may fail a test without an obvious component being at fault. Instead, experts may know that at least one of a set of components has failed, but there is uncertainty about which members in the set were the actual failures. When no further information is available, this missing data may be imputed using standard data augmentation (DA). This process is already used in the current implementation of the JMP complex-system reliability modeling codes. In some cases when this situation arises, there may be some supplemental information about the nature of the failure that suggests which subset of components are more likely to have failed. the behavior of the system during the failure may make certain components more likely candidates, and lead the engineering experts to have certain prior beliefs about what occurred. In this case, it is still known that at least one of a set of components failed, but the experts have some idea that certain failure scenarios are more likely than others. This white paper addresses this situation by modifying the imputation process of data augmentation through the use of an observation prior. This prior is specific to particular observations, and a given outcome which is repeated several times could potentially have different observation priors associated with each occurrence.

97 MATHEMATICS AND COMPUTING↗

Linkage map construction using limited parental genotypic information

Abstract Genetic linkage maps based on single nucleotide polymorphisms (SNPs) represent an essential tool for a variety of genomic analyses. Today, next-generation sequencing (NGS) enables rapid genotyping of different mapping populations based on thousands of SNPs and the construction of highly saturated linkage maps. Nevertheless, missing data in the genotyping of the parental lines creates a bottleneck that determines the number of SNPs that can be used for the linkage map. As a proof of concept, a highly saturated genetic linkage map was constructed using the imputed genotypic data of a recombinant inbred line (RIL) population and the limited genotypic information of its parental lines. Two ABH genotype files were created from a pseudo-parental genotypic data set that includes all the SNPs present in the RIL population. In the first ABH file pseudo-parental 1 was considered parental A, while in the second pseudo-parental 1 was considered parental B. These two duplicate ABH genotype files were merged by chromosome and subjected to linkage map analysis. Since the ABH data were duplicated, two mirrored linkage groups were generated per chromosome. The correct linkage map was identified and selected based on the partial genotypic data of the parental lines. This strategy was effective for constructing a highly saturated linkage map of 33,421 SNPs based on the genotyping of 205 RILs and a limited number of 100 SNPs present in the parental lines. This strategy enables the use of all the NGS SNP data obtained from a low-coverage sequencing experiment in the mapping population.

59 BASIC BIOLOGICAL SCIENCES↗

Derivation of Lanthanide Series Crystal Field Parameters From First Principles.

Two series of lanthanide complexes were chosen to analyze the trends of the magnetic properties along the series: the highly symmetric LnZn 16 (picHA) 16 series with Ln = Tb, Dy, Ho, Er, Yb and picHA=picoline hydroxamic, and the [Ln(DPA) 3 ](C 3 H 5 N 2 ) 3 · 3H 2 O series with DPA = 2,6-dipicolinic acid and Ln = Ce–Yb and the approximate threefold symmetry. The first series present a compressed coordination sphere of 8 oxygen atoms environment while in the 2nd series, the coordination sphere is formed by an elongated coordination sphere formed by 6 oxygen atoms. Crystal field parameters (CFPs) are deduced from ab initio calculations using two methods: the AILFT (Ab Initio Ligand Field Theory) which determines the parameters at the orbital level, and the ITO (Irreducible Technique Operator) decomposition which treats the problem at the many-electron level. It is shown that the CFPs are transferable from one derivative to the other, within a given series. The sign of the 2nd order parameter $B_0^2$ differs in the two series reflecting the difference of environment. It is shown that the use of strength parameter $S$ permits a easy comparison between complexes. Furthermore, in both cases, the parameters are found to decrease in magnitude along the series and this decrease is imputed to covalent effects.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

Predictive analytics of selections of russet potatoes

We explore the application of machine learning algorithms specifically to enhance the selection process of Russet potato (Solanum tuberosum L.) clones in breeding trials by predicting their suitability for advancement. This study addresses the challenge of efficiently identifying high-yield, disease-resistant, and climate-resilient potato varieties that meet processing industry standards. Leveraging manually collected data from trials in the state of Oregon, we investigate the potential of a wide variety of state-of-the-art binary classification models. The dataset includes 1086 clones, with data on 38 attributes recorded for each clone, focusing on yield, size, appearance, and frying characteristics, with several control varieties planted consistently across four Oregon regions from 2013 to 2021. We conduct a comprehensive analysis of the dataset that includes preprocessing, feature engineering, and imputation to address missing values. We focus on several key metrics such as accuracy, F1-score, and Matthews correlation coefficient (MCC) for model evaluation. The top-performing models, namely a feedforward neural network classifier (Neural Net), a histogram-based gradient boosting classifier (HGBC), and a support vector machine classifier (SVM), demonstrate consistent and significant results. To further validate our findings, we conducted a simulation study using the aims, data-generating mechanisms, estimands, methods, and performance measures (ADEMP) framework, simulating different data-generating scenarios to assess model robustness and performance through true positive, true negative, false positive, and false negative distributions, area under the receiver operating characteristic curve (AUC-ROC) and MCC. The simulation results highlight that non-linear models like SVM and HGBC consistently show higher AUC-ROC and MCC than logistic regression, thus outperforming the traditional linear model across various distributions, and emphasizing the importance of model selection and tuning in agricultural trials. Variable selection further enhances model performance and identifies influential features in predicting trial outcomes. The findings emphasize the potential of machine learning in streamlining the selection process for potato varieties, offering benefits such as increased efficiency, substantial cost savings, and judicious resource utilization. Our study contributes insights into precision agriculture and showcases the relevance of advanced technologies for informed decision-making in breeding programs.

60 APPLIED LIFE SCIENCES↗

Does the leaf economic spectrum hold within plant functional types? A Bayesian multivariate trait meta-analysis

The leaf economic spectrum is a widely-studied axis of plant trait variability that defines a trade-off between leaf longevity and productivity. While this has been investigated at the global scale, where it is robust, and at local scales, where deviations from it are common, it has received less attention at the intermediate scale of PFTs. We investigated whether global leaf economic relationships are also present within the scale of plant functional types (PFTs) commonly used by Earth System models, and the extent to which this global-PFT hierarchy can be used to constrain trait estimates. We developed a hierarchical multivariate Bayesian model that assumes separate means and covariance structures within and across PFTs and fit this model to seven leaf traits from the TRY database related to leaf longevity, morphology, biochemistry, and photosynthetic metabolism. Although patterns of trait covariation were generally consistent with the leaf economic spectrum, we found three approximate tiers to this consistency. Relationships among morphological and biochemical traits (SLA, N, P) were the most robust within and across PFTs, suggesting that covariation in these traits is driven by universal leaf construction trade-offs and stoichiometry. Relationships among metabolic traits (R_d, V_{c,max}, J_max) were slightly less consistent, reflecting in part their much sparser sampling (especially for high-latitude PFTs), but also pointing to more flexible plasticity in plant metabolistm. Finally, relationships involving leaf lifespan were the least consistent, indicating that leaf economic relationships related to leaf lifespan are dominated by across-PFT differences and that within-PFT variation in leaf lifespan is more complex and idiosyncratic. Across all traits, these covariance were an important source of information, as evidenced by the improved imputation accuracy and reduced predictive uncertainty in multivariate models compared to univariate models. Ultimately, our study reaffirms the value of studying not just individual traits but the multivariate trait space and the utility of hierarchical modeling for studying the scale dependence of trait relationships.

biodiversity, ecological modeling, ecology, leaf b↗

Challenging problems of quality assurance and quality control (QA/QC) of meteorological time series data

Abstract Representativeness and quality of collected meteorological data impact accuracy and precision of climate, hydrological, and biogeochemical analyses and predictions. We developed a comprehensive Quality Assurance (QA) and Quality Control (QC) statistical framework, consisting of three major phases: Phase I—Preliminary data exploration, i.e., processing of raw datasets, with the challenging problems of time formatting and combining datasets of different lengths and different time intervals; Phase II—QA of the datasets, including detecting and flagging of duplicates, outliers, and extreme data; and Phase III—the development of time series of a desired frequency, imputation of missing values, visualization and a final statistical summary. The paper includes two use cases based on the time series data collected at the Billy Barr meteorological station (East River Watershed, Colorado), and the Barro Colorado Island (BCI, Panama) meteorological station. The developed statistical framework is suitable for both real-time and post-data-collection QA/QC analysis of meteorological datasets.

54 ENVIRONMENTAL SCIENCES↗

Nuclear Forensics Analysis with Missing and Uncertain Data

We have applied a new imputation-based method for analyzing incomplete data, called Monte Carlo Bayesian Database Generation (MCBDG), to the Spent Fuel Isotopic Composition (SFCOMPO) database. About 60% of the entries are absent for SFCOMPO. The method estimates missing values of a property from a probability distribution created from the existing data for the property, and then generates multiple instances of the completed database for training a machine learning algorithm. Uncertainty in the data is represented by an empirical or an assumed error distribution. The method makes few assumptions about the underlying data, and compares favorably against results obtained by replacing missing information with constant values.

97 MATHEMATICS AND COMPUTING↗

Efficiently embedding QUBO problems on adiabatic quantum computers

Adiabatic quantum computers like the D-Wave 2000Q can approximately solve the QUBO problem, which is an NP-hard problem, and have been shown to outperform classical computers on several instances. Solving the QUBO problem literally means solving virtually any NP-hard problem like the traveling salesman problem, airline scheduling problem, protein folding problem, genotype imputation problem, thereby enabling significant scientific progress, and potentially saving millions/billions of dollars in logistics, airlines, healthcare and many other industries. Yet, before QUBO problems are solved on quantum computers, they must be embedded (or compiled) onto the hardware of quantum computers, which in itself is a very hard problem. Here, we propose an efficient embedding algorithm, that lets us embed QUBO problems fast, uses less qubits and gets the objective function value close to the global minimum value. We then compare the performance of our embedding algorithm to that of D-Wave’s embedding algorithm, which is the current state of the art, and show that our embedding algorithm convincingly outperforms D-Wave’s embedding algorithm. Our embedding approach works with perfect Chimera graphs, i.e., Chimera graphs with no missing qubits.

97 MATHEMATICS AND COMPUTING↗

Visibility-enhanced model-free deep reinforcement learning algorithm for voltage control in realistic distribution systems using smart inverters

Increasing integration of distributed solar photovoltaic (PV) into distribution networks could result in adverse effects on grid operation. Traditional model-based control algorithms require accurate model information that is difficult to acquire and thus are challenging to implement in practice. Here, this paper proposes a surrogate model-enabled grid visibility scheme to empower deep reinforcement learning (DRL) approach for distribution network voltage regulation using PV inverters with minimal system knowledge. In contrast to existing DRL methods, this paper presents and corroborates the adverse impact of missing load information on DRL performance and, based on this finding, proposes a surrogate model methodology to impute load information utilizing observable data. Additionally, a multi-fidelity neural network is utilized to construct the DRL training environment, chosen for its efficient data utilization and enhanced robustness to data uncertainty. The feasibility and effectiveness of the proposed algorithm are assessed by considering DRL testing across varying degrees of observable load information and diverse training environments on a realistic power system.

14 SOLAR ENERGY↗

Machine learning models of intermittent operation of RO wellhead water treatment for salinity reduction and nitrate removal

Machine learning models were developed for intermittent multi-mode operation of a wellhead reverse osmosis water purification and desalination system to predict salt passage, nitrate passage, and permeate flux. The models, based on long short-term memory (LSTM) recurrent neural network (RNN) architecture, included an attention mechanism to increase model performance in proximity of the regulatory limit for nitrate. Training and testing of the models for the Startup, Production, Shutdown and Flushing operational modes were based on operational data (consisting of 22 process variables per data sample) acquired every 2–5 s over a six-month period. The significant sets of model input attributes for the different operational modes were assessed via Spearman ranking correlation, Self-Organizing Map (SOM) analysis and feed forward feature selection (FFFS). Although the variability of nitrate passage, salt passage and permeate flux was significant over the four operational modes, prediction performance for the three outcomes were with R2 and Average Absolute Relative Error (AARE) of 0.78–0.95 and 2.96–6.16 %, respectively. Model updates post membrane elements replacement demonstrated similar levels of prediction accuracy. The study results suggest that there is merit in exploring the utility of multi-mode models for sensor fault detection, data imputation, and for potential use in model-predictive control.

Intermittent RO operation↗

Conditional distribution estimation of building characteristics with diffusion models for urban energy modeling

Understanding current energy consumption behavior in communities is critical for informing future energy use decisions and enabling efficient energy management. Urban energy models, which are used to simulate these energy use patterns, require large datasets with detailed building characteristics for accurate outcomes. However, such detailed characteristics at the individual building level are often unknown and costly to acquire, or unavailable. Through this work, we propose using a generative modeling approach to generate realistic building attributes to fill in the data gaps and finally provide complete characteristics as inputs to energy models. Our model learns complex, building-level patterns from training on a large-scale residential building stock model containing 2.2 million buildings. We employ a tabular diffusion-based framework that is designed to handle heterogeneous (discrete and continuous) features in tabular building data, such as occupancy, floor area, heating, cooling, and other equipment details. We develop a capability for conditional diffusion, enabling the imputation of missing building characteristics conditioned on known attributes. We conduct a comprehensive validation of our conditional diffusion model, firstly by comparing the generated conditional distributions against the underlying data distribution, and secondly, by performing a case study for a Baltimore residential region, showing the practical utility of our approach. Our work is one of the first to demonstrate the potential of generative modeling to accelerate building energy modeling workflows.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Estimation of TETRA radio use in the Airwave Health Monitoring Study of the British police forces

Highlights: • Operator-derived records alongside self-reported information to estimate radio use as a proxy for RF-EMF exposure. • Imputed average TETRA radio usage correlates better with objective data than self-reported use. • Higher personal radio usage seen in younger and male participants as well as in police officers compared with staff. • The first large observational study with an estimate of monthly average personal radio usage for all participants. The Airwave Health Monitoring Study aims to investigate the possible long-term health effects of Terrestrial Trunked Radio (TETRA) use among the police forces in Great Britain. Here, we investigate whether objective data from the network operator could be used to correct for misreporting in self-reported data and expand the radio usage availability in our cohort.

61 RADIATION PROTECTION AND DOSIMETRY↗

Development of RTOG Consensus Guidelines for the Definition of the Clinical Target Volume for Postoperative Conformal Radiation Therapy for Prostate Cancer

Purpose: To define a prostate fossa clinical target volume (PF-CTV) for Radiation Therapy Oncology Group (RTOG) trials using postoperative radiotherapy for prostate cancer. Methods and Materials: An RTOG-sponsored meeting was held to define an appropriate PF-CTV after radical prostatectomy. Data were presented describing radiographic failure patterns after surgery. Target volumes used in previous trials were reviewed. Using contours independently submitted by 13 radiation oncologists, a statistical imputation method derived a preliminary 'consensus' PF-CTV. Results: Starting from the model-derived CTV, consensus was reached for a CT image-based PF-CTV. The PF-CTV should extend superiorly from the level of the caudal vas deferens remnant to >8-12 mm inferior to vesicourethral anastomosis (VUA). Below the superior border of the pubic symphysis, the anterior border extends to the posterior aspect of the pubis and posteriorly to the rectum, where it may be concave at the level of the VUA. At this level, the lateral border extends to the levator ani. Above the pubic symphysis, the anterior border should encompass the posterior 1-2 cm of the bladder wall; posteriorly, it is bounded by the mesorectal fascia. At this level, the lateral border is the sacrorectogenitopubic fascia. Seminal vesicle remnants, if present, should be included in the CTV if there is pathologic evidence of their involvement. Conclusions: Consensus on postoperative PF-CTV for RT after prostatectomy was reached and is available as a CT image atlas on the RTOG website. This will allow uniformity in defining PF-CTV for clinical trials that include postprostatectomy RT.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Outcomes in a Multi-institutional Cohort of Patients Treated With Intraoperative Radiation Therapy for Advanced or Recurrent Renal Cell Carcinoma

Purpose/Objective(s): This study aimed to analyze outcomes in a multi-institutional cohort of patients with advanced or recurrent renal cell carcinoma (RCC) who were treated with intraoperative radiation therapy (IORT). Methods and Materials: Between 1985 and 2010, 98 patients received IORT for advanced or locally recurrent RCC at 9 institutions. The median follow-up time for surviving patients was 3.5 years. Overall survival (OS), disease-specific survival (DSS), and disease-free survival (DFS) were estimated with the Kaplan-Meier method. Chained imputation accounted for missing data, and multivariate Cox hazards regression tested significance. Results: IORT was delivered during nephrectomy for advanced disease (28%) or during resection of locally recurrent RCC in the renal fossa (72%). Sixty-nine percent of the patients were male, and the median age was 58 years. At the time of primary resection, the T stages were as follows: 17% T1, 12% T2, 55% T3, and 16% T4. Eighty-seven percent of the patients had a visibly complete resection of tumor. Preoperative or postoperative external beam radiation therapy was administered to 27% and 35% of patients, respectively. The 5-year OS was 37% for advanced disease and 55% for locally recurrent disease. The respective 5-year DSS was 41% and 60%. The respective 5-year DFS was 39% and 52%. Initial nodal involvement (hazard ratio [HR] 2.9-3.6, P<.01), presence of sarcomatoid features (HR 3.7-6.9, P<.05), and higher IORT dose (HR 1.3, P<.001) were statistically significantly associated with decreased survival. Adjuvant systemic therapy was associated with decreased DSS (HR 2.4, P=.03). For locally recurrent tumors, positive margin status (HR 2.6, P=.01) was associated with decreased OS. Conclusions: We report the largest known cohort of patients with RCC managed by IORT and have identified several factors associated with survival. The outcomes for patients receiving IORT in the setting of local recurrence compare favorably to similar cohorts treated by local resection alone suggesting the potential for improved DFS with IORT.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Nodal Clearance Rate and Long-Term Efficacy of Individualized Sentinel Node–Based Pelvic Intensity Modulated Radiation Therapy for High-Risk Prostate Cancer

Purpose: To assess the efficacy of individual sentinel node (SN)-guided pelvic intensity modulated radiation therapy (IMRT) by determining nodal clearance rate [(n expected nodal involvement − n observed regional recurrences)/n expected nodal involvement] in comparison with surgically staged patients. Methods and Materials: Data on 475 high-risk prostate cancer patients were examined. Sixty-one consecutive patients received pelvic SN-based IMRT (5 × 1.8 Gy/wk to 50.4 Gy [pelvic nodes + individual SN] and an integrated boost with 5 × 2.0 Gy/wk to 70.0 Gy to prostate + [base of] seminal vesicles) and neo-/adjuvant long-term androgen deprivation therapy; 414 patients after SN–pelvic lymph node dissection were used to calculate the expected nodal involvement rate for the radiation therapy sample. Biochemical control and overall survival were estimated for the SN-IMRT patients using the Kaplan-Meier method. The expected frequency of nodal involvement in the radiation therapy group was estimated by imputing frequencies of node-positive patients in the surgical sample to the pattern of Gleason, prostate-specific antigen, and T category in the radiation therapy sample. Results: After a median follow-up of 61 months, 5-year OS after SN-guided IMRT reached 84.4%. Biochemical control according to the Phoenix definition was 73.8%. The nodal clearance rate of SN-IMRT reached 94%. Retrospective follow-up evaluation is the main limitation. Conclusions: Radiation treatment of pelvic nodes individualized by inclusion of SNs is an effective regional treatment modality in high-risk prostate cancer patients. The pattern of relapse indicates that the SN-based target volume concept correctly covers individual pelvic nodes. Thus, this SN-based approach justifies further evaluation, including current dose-escalation strategies to the prostate in a larger prospective series.

62 RADIOLOGY AND NUCLEAR MEDICINE↗

Hydrologic connectivity and dynamics of solute transport in a mountain stream: Insights from a long-term tracer test and multiscale transport modeling informed by machine learning

The movement of solutes in a watershed is a complex process with multiple interactions and feedbacks across spatial and temporal scales. Modeling the dynamics of solute transport along diverse hydrologic pathways within watersheds – from hillslopes to stream channels and in and out of the hyporheic zones – is challenging but critically important, as these processes integrate and contribute to the biogeochemical functioning of the river corridor up to the river network scale. Here we use results from a long-term network-scale tracer test at the H.J. Andrews experimental forest in western Cascade Mountains, Oregon, USA to inform a multiscale framework for transport in stream corridors. The framework uses a Lagrangian-based subgrid model to represent the effects of hyporheic exchange flow and advective transport at stream network scales. The spatially and temporally resolved stream discharge needed for the transport model is imputed across the river system by an entity-aware long short-term memory network. Modeled concentrations show good agreements with the observations and exhibit power scaling laws indicative of a very wide range of timescales over which hyporheic exchange flow occurs. Our results demonstrate a data-informed modeling framework that links dynamical processes occurring at small scales to a network context to help understand how changes at reach scale cascade into network-scale effects, providing a useful tool for sustainable river basin management.

54 ENVIRONMENTAL SCIENCES↗

Impact of duration and missing data on the long-term photovoltaic degradation rate estimation

Accurate quantification of photovoltaic (PV) system degradation rate (R D ) is essential for lifetime yield predictions. Although R D is a critical parameter, its estimation lacks a standardized methodology that can be applied on outdoor field data. The purpose of this paper is to investigate the impact of time period duration and missing data on R D by analyzing the performance of different techniques applied to synthetic PV system data at different linear R D patterns and known noise conditions. The analysis includes the application of different techniques to a 10-year synthetic dataset of a crystalline Silicon PV system, with emulated degradation levels and imputed missing data. Here, the analysis demonstrated that the accuracy of ordinary least squares (OLS), year-on-year (YOY), autoregressive integrated moving average (ARIMA) and robust principal component analysis (RPCA) techniques is affected by the evaluation duration with all techniques converging to lower R D deviations over the 10-year evaluation, apart from RPCA at high degradation levels. Moreover, the estimated R D is strongly affected by the amount of missing data. Filtering out the corrupted data yielded more accurate R D results for all techniques. It is proven that the application of a change-point detection stage is necessary and guidelines for accurate R D estimation are provided.

14 SOLAR ENERGY↗