Search NASA⌕ Search

SEARCH · Search NASA

Results for “missing data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 307 records · Page 17

Handling Large and Complex Data in a Photovoltaic Research Institution Using a Custom Laboratory Information Management System

Twenty-five years ago the desktop computer started becoming ubiquitous in the scientific lab. Researchers were delighted with its ability to both control instrumentation and acquire data on a single system, but they were not completely satisfied. There were often gaps in knowledge that they thought might be gained if they just had more data and they could get the data faster. Computer technology has evolved in keeping with Moore’s Law meeting those desires; however those improvements have of late become both a boon and bane for researchers. Computers are now capable of producing high speed data streams containing terabytes of information; capabilities that evolved faster than envisioned last century. Software to handle large scientific data sets has not kept up. How much information might be lost through accidental mismanagement or how many discoveries are missed through data overload are now vital questions. An important new task in most scientific disciplines involves developing methods to address those issues and to create the software that can handle large data sets with an eye towards scalability. This software must create archived, indexed, and searchable data from heterogeneous instrumentation for the implementation of a strong data-driven materials development strategy. At the National Center for Photovoltaics in the National Renewable Energy Laboratory, we began development a few years ago on a Laboratory Information Management System (LIMS) designed to handle lab-wide scientific data acquisition, management, processing and mining needs for physics and materials science data, and with a specific focus towards future scalability for new equipment or research focuses. We will present the decisions, processes, and problems we went through while building our LIMS system for materials research, its current operational state and our steps for future development.

14 SOLAR ENERGY↗

Graph identification of proteins in tomograms ( GRIP‐Tomo )

Abstract In this study, we present a method of pattern mining based on network theory that enables the identification of protein structures or complexes from synthetic volume densities, without the knowledge of predefined templates or human biases for refinement. We hypothesized that the topological connectivity of protein structures is invariant, and they are distinctive for the purpose of protein identification from distorted data presented in volume densities. Three‐dimensional densities of a protein or a complex from simulated tomographic volumes were transformed into mathematical graphs as observables. We systematically introduced data distortion or defects such as missing fullness of data, the tumbling effect, and the missing wedge effect into the simulated volumes, and varied the distance cutoffs in pixels to capture the varying connectivity between the density cluster centroids in the presence of defects. A similarity score between the graphs from the simulated volumes and the graphs transformed from the physical protein structures in point data was calculated by comparing their network theory order parameters including node degrees, betweenness centrality, and graph densities. By capturing the essential topological features defining the heterogeneous morphologies of a network, we were able to accurately identify proteins and homo‐multimeric complexes from 10 topologically distinctive samples without realistic noise added. Our approach empowers future developments of tomogram processing by providing pattern mining with interpretability, to enable the classification of single‐domain protein native topologies as well as distinct single‐domain proteins from multimeric complexes within noisy volumes.

59 BASIC BIOLOGICAL SCIENCES↗

National Energy Water Treatment & Speciation (NEWTS): A Water & Critical Mineral Database and Dashboard

The scarcity of water resources, the need for beneficial water reuse, and the challenges of wastewater treatment are becoming increasingly pressing in economic, social, and environmental domains. Addressing these concerns requires effective treatment strategies to manage wastewater streams and tackle environmental and economic issues. Furthermore, the recovery of critical minerals from the waste streams associated with energy production holds the promise of offsetting treatment costs and securing local sources of valuable minerals. However, relevant data on these waste streams are dispersed and challenging to locate. The process of ingesting such data into modeling software often involves multiple steps, requiring data restructuring to meet software-input requirements. The non-standardized reporting of water data makes data aggregation and reformatting a time-consuming process. Additionally, essential attributes necessary for modeling water treatment and mineral scale formation are frequently missing. Moreover, data gaps vary depending on the region of interest. Consequently, there is a pressing need for high-quality energy-water composition data that can be easily imported into water chemistry modeling software. To address this need, the National Energy Technology Laboratory has created the National Energy Water Treatment and Speciation (NEWTS) Database and Dashboard—a free online tool catering to community leaders and water researchers. NEWTS facilitates a comprehensive understanding of the composition of energy-related wastewater streams in the United States. The datasets provide detailed concentrations and speciation of major and minor aqueous compounds in energy-related wastewater streams, including power plant leachate, acid mine drainage, brackish water, and oil and gas produced water across the United States. Many of the aqueous species are critical minerals (Li, REEs) in high demand to modernize the world’s energy infrastructure. Many of the datasets also contain volumetric flow-rates needed to model the treatment and reuse scenarios in advanced aqueous chemistry software programs. The NEWTS Database and Dashboard offer public access to hitherto challenging-to-access datasets, presented in a standardized format that is tailored for easy input into aqueous chemistry modeling software. By performing the work needed to transform dispersed, disparate data sources into unified, model-ready datasets, NEWTS serves as an essential resource in advancing water treatment research and sustainable water resource management.

produced water management↗

Long-term missing value imputation for time series data using deep neural networks

We present an approach that uses a deep learning model, in particular, a MultiLayer Perceptron, for estimating the missing values of a variable in multivariate time series data. We focus on filling a long continuous gap (e.g., multiple months of missing daily observations) rather than on individual randomly missing observations. Our proposed gap filling algorithm uses an automated method for determining the optimal MLP model architecture, thus allowing for optimal prediction performance for the given time series. We tested our approach by filling gaps of various lengths (three months to three years) in three environmental datasets with different time series characteristics, namely daily groundwater levels, daily soil moisture, and hourly Net Ecosystem Exchange. We compared the accuracy of the gap-filled values obtained with our approach to the widely used R-based time series gap filling methods ImputeTS and mtsdi. The results indicate that using an MLP for filling a large gap leads to better results, especially when the data behave nonlinearly. Thus, our approach enables the use of datasets that have a large gap in one variable, which is common in many long-term environmental monitoring observations.

97 MATHEMATICS AND COMPUTING↗

Constructing a Simulation Surrogate with Partially Observed Output

Gaussian process surrogates are a popular alternative to directly using computationally expensive simulation models. When the simulation output consists of many responses, dimension-reduction techniques are often employed to construct these surrogates. However, surrogate methods with dimension reduction generally rely on complete output training data. This article proposes a new Gaussian process surrogate method that permits the use of partially observed output while remaining computationally efficient. The new method involves the imputation of missing values and the adjustment of the covariance matrix used for Gaussian process inference. The resulting surrogate represents the available responses, disregards the missing responses, and provides meaningful uncertainty quantification. In conclusion, the proposed approach is shown to offer sharper inference than alternatives in a simulation study and a case study where an energy density functional model that frequently returns incomplete output is calibrated.

42 ENGINEERING↗

Bayesian High-Rank Hankel Matrix Completion for Nonlinear Synchrophasor Data Recovery

Phasor measurement units (PMUs) provide high temporal-resolution synchrophasor measurements for power system monitoring and control. The frequent data quality issues, such as missing and bad data, prevent the incorporation of synchrophasor data in real-time operations. Most existing data-driven data recovery methods assume the power system dynamics can be approximated by a linear dynamical system, and the recovery performance degrades significantly when the power system is experiencing nonlinear dynamics during significant events. Here, this paper proposes a data-driven Bayesian nonlinear synchrophasor data recovery method (Ba-NSDR) that can recover a consecutive time period of simultaneous data losses or errors across all channels, even when the underlying system is highly nonlinear. The idea is to lift the Hankel matrix of the spatial-temporal synchrophasor data to a higher dimension such that the lifted Hankel matrix is low-rank in that space and can be processed with the kernel trick. Our proposed Bayesian method then infers the probabilistic distributions of synchrophasor from the partial observations. Some distinctive features of Ba-NSDR include an uncertainty index to measure the accuracy of the recovery result and the robustness to parameter selections. Our method is verified on both synthetic and recorded event datasets.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

ORNL Neutron Cross-Section Measurements Activities

To support the Nuclear Criticality Safety Program, neutron cross-section measurements have been initiated using GELINA at the European Commission Joint Research Center IRMM. Concerns about data deficiencies in some existing cross-section evaluations from libraries such as ENDF/B, JEFF, or JENDL for nuclear criticality calculations have been the prime motivator for new cross section measurements. Over the past years, many problems with existing nuclear data have been emerging. An example is the use of the evaluated neutron cross-section data for tungsten in nuclear criticality safety calculations, which exhibit discrepancies in benchmark calculations and show the need for reliable covariance data. We recently measured the neutron total and capture cross sections of 182,183,184,186 W, 63,65 Cu, and natural calcium in the neutron energy range from 100 eV to several hundred keV. This will help to improve the representation of the cross sections since most of the available evaluated data rely only on old measurements. Besides, most of the evaluations in the nuclear data libraries are missing high-fidelity covariance data. New covariance data for the libraries can only be created by a reevaluation of the cross-section data; however, sometimes this impossible because the data used in the analysis and evaluations are no longer available. In such cases, this shortfall can be overcome only by new experimental data included in a new evaluation.

97 MATHEMATICS AND COMPUTING↗

SU-G-JeP1-15: Sliding Window Prior Data Assisted Compressed Sensing for MRI Lung Tumor Tracking

Purpose: Prior Data Assisted Compressed Sensing (PDACS) is a partial k-space acquisition and reconstruction method for mobile tumour (i.e. lung) tracking using on-line MRI in radiotherapy. PDACS partially relies on prior data acquired at the beginning of dynamic scans, and is therefore susceptible to artifacts in longer duration scan due to slow drifts in MR signal. A novel sliding window strategy is presented to mitigate this effect. Methods: MRI acceleration is simulated by retrospective removal of data from the fully sampled sets. Six lung cancer patients were scanned (clinical 3T MRI) using a balanced steady state free precession (bSSFP) sequence for 3 minutes at approximately 4 frames per second, for a total of 650 dynamics. PDACS acceleration is achieved by undersampling of k-space in a single pseudo-random pattern. Reconstruction iteratively minimizes the total variations while constraining the images to satisfy both the currently acquired data and the prior data in missing k-space. Our novel sliding window technique (SW-PDACS), uses a series of distinct pseudo-random under-sampling patterns of partial k-space – with the prior data drawn from a sliding window of the most recent data available. Under-sampled data, simulating 2 – 5x acceleration are reconstructed using PDACS and SW-PDACS. Three quantitative metrics: artifact power, centroid error and Dice’s coefficient are computed for comparison. Results: Quantitively metric values from all 6 patients are averaged in 3 bins, each containing approximately one minute of dynamic data. For the first minute bin, PDACS and SW-PDACS give comparable results. Progressive decline in image quality metrics in bins 2 and 3 are observed for PDACS. No decline in image quality is observed for SW-PDACS. Conclusion: The novel approach presented (SW-PDACS) is a more robust for accelerating longer duration (>1 minute) dynamic MRI scans for tracking lung tumour motion using on-line MRI in radiotherapy. B.G. Fallone is a co-founder and CEO of MagnetTx Oncology Solutions (under discussions to license Alberta bi-planar linac MR for commercialization).

60 APPLIED LIFE SCIENCES↗

Virtual Metering for Monitoring Building Energy Consumption

The United States Department of Energy (DOE) has standard metering requirements of commercial buildings for optimizing energy performance. The guiding principles are to continuously track and optimize energy performance and install building-level meters for electricity, natural gas, and steam. Some buildings at Los Alamos National Laboratory (LANL) have physical submeters monitoring their energy consumption, but these meters have proven to be unreliable. And, in most cases, replacing them has proven to be a slow process. Installing new submeters also requires a temporary lockout of the circuit on which they are being installed. Many buildings at LANL contain laboratories with ongoing experiments or data centers, which makes an equipment power outage nearly impossible to plan. This inability to plan power outages results in long-term submeter failures. Although most submeters are eventually replaced, failures lead to missing consumption data for some unpredictable, extended time. A building automation system (BAS) is a system that provides control and monitoring on a building to maintain the operational performance of the building and occupancy comfort. Many buildings at LANL currently have a BAS, and all new renovations and installs will include installing a BAS if one does not already exist. The intended purpose for a BAS is primarily to monitor the health and efficiency of a building; however, it is also possible to calculate equipment power and energy consumption using BAS information. This project aims to use virtual meters to monitor building energy consumption as a cost-effective and minimally labor-intensive alternative to installing physical submeters. The fault detection and diagnostics tool, SkySpark, provides a centralized database for all the data from the various BAS that are active at LANL. This data includes the information that is needed to create virtual meters for heating, ventilating, and air conditioning (HVAC) systems in most buildings, including heating and cooling loads. 9 This report begins with a detailed summary of the project, including the reasoning, procedure, and results. The specific processes of creating the various virtual meters are then identified. Then the limitations are discussed. And, lastly, the results and future potential are presented.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

An Assessment of High-Resolution Gridded Temperature Datasets over California

Abstract High-resolution gridded datasets are in high demand because they are spatially complete and include important finescale details. Previous assessments have been limited to two to three gridded datasets or analyzed the datasets only at the station locations. Here, eight high-resolution gridded temperature datasets are assessed two ways: at the stations, by comparing with Global Historical Climatology Network–Daily data; and away from the stations, using physical principles. This assessment includes six station-based datasets, one interpolated reanalysis, and one dynamically downscaled reanalysis. California is used as a test domain because of its complex terrain and coastlines, features known to differentiate gridded datasets. As expected, climatologies of station-based datasets agree closely with station data. However, away from stations, spread in climatologies can exceed 6°C. Some station-based datasets are very likely biased near the coast and in complex terrain, due to inaccurate lapse rates. Many station-based datasets have large unphysical trends (>1°C decade−1) due to unhomogenized or missing station data—an issue that has been fixed in some datasets by using homogenization algorithms. Meanwhile, reanalysis-based gridded datasets have systematic biases relative to station data. Dynamically downscaled reanalysis has smaller biases than interpolated reanalysis, and has more realistic variability and trends. Dynamical downscaling also captures snow–albedo feedback, which station-based datasets miss. Overall, these results indicate that 1) gridded dataset choice can be a substantial source of uncertainty, and 2) some datasets are better suited for certain applications.

54 ENVIRONMENTAL SCIENCES↗

Dynamic influence of pinning points on marine ice-sheet stability: a numerical study in Dronning Maud Land, East Antarctica

The East Antarctic ice sheet is likely more stable than its West Antarctic counterpart because its bed is largely lying above sea level. However, the ice sheet in Dronning Maud Land, East Antarctica, contains marine sectors that are in contact with the ocean through overdeepened marine basins interspersed by grounded ice promontories and ice rises, pinning and stabilising the ice shelves. In this paper, we use the ice-sheet model BISICLES to investigate the effect of sub-ice-shelf melting, using a series of scenarios compliant with current values, on the ice-dynamic stability of the outlet glaciers between the Lazarev and Roi Baudouin ice shelves over the next millennium. Overall, the sub-ice-shelf melting substantially impacts the sea-level contribution. Locally, we predict a short-term rapid grounding-line retreat of the overdeepened outlet glacier Hansenbreen, which further induces the transition of the bordering ice promontories into ice rises. Furthermore, our analysis demonstrated that the onset of the marine ice-sheet retreat and subsequent promontory transition into ice rise is controlled by small pinning points, mostly uncharted in pan-Antarctic datasets. Pinning points have a twofold impact on marine ice sheets. They decrease the ice discharge by buttressing effect, and they play a crucial role in initialising marine ice sheets through data assimilation, leading to errors in ice-shelf rheology when omitted. Our results show that unpinning increases the sea-level rise by 10%, while omitting the same pinning point in data assimilation decreases it by 10%, but the more striking effect is in the promontory transition time, advanced by two centuries for unpinning and delayed by almost half a millennium when the pinning point is missing in data assimilation. As a result, pinning points exert a subtle influence on ice dynamics at the kilometre scale, which calls for a better knowledge of the Antarctic margins.

58 GEOSCIENCES↗

Brownian bridge-based speed imputation technique for truck energy consumption and emissions estimation

The available truck Global Positioning System (GPS) data, typically collected with large time gaps, rely on imputation techniques to obtain second-by-second data that are required in models for estimating truck energy consumption and emissions. However, existing speed imputation techniques either require a large amount of high-resolution data for model training or rely on special movement assumptions. Here, to fill the gap and effectively apply the low-resolution truck GPS datasets, this paper proposes a simple imputation technique that adopts the Brownian bridge structure to impute missing speed data. The proposed technique introduces a feasible imputation region and a combined drift into the imputation procedure to capture vehicle acceleration constraint, travel distance constraint, and speed volatility. The calibrated model is applied to a set of low-resolution truck GPS data. The results demonstrate the robustness of the proposed technique in enhancing estimation accuracy when using low-resolution GPS data to estimate fuel consumption and emissions.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

Introducing a multiscale feature integration network for inpainting with applications to enhanced CMB map reconstruction

We introduce a novel neural network, SkyReconNet, which combines the expanded receptive fields of dilated convolutional layers along with standard convolutions, to capture both the global and local features for reconstructing the missing information in an image. We implement our network to inpaint the masked regions in a full-sky cosmic microwave background (CMB) map. Inpainting CMB maps is a particularly formidable challenge when dealing with extensive and irregular masks, such as galactic masks which can obscure substantial fractions of the sky. The hybrid design of SkyReconNet leverages the strengths of standard and dilated convolutions to accurately predict CMB fluctuations in the masked regions by effectively utilizing the information from surrounding unmasked areas. During training, the network optimizes its weights by minimizing a composite loss function that combines the structural similarity index measure (SSIM) and mean squared error (MSE). SSIM preserves the essential structural features of the CMB, ensuring an accurate and coherent reconstruction of the missing CMB fluctuations, while MSE minimizes the pixelwise deviations, thus enhancing the overall accuracy of the predictions. The predicted CMB maps and their corresponding angular power spectra align closely with the targets, achieving the performance limited only by the fundamental uncertainty of cosmic variance. The network’s generic architecture enables application to other physics-based challenges involving data with missing or defective pixels, systematic artifacts, etc. In conclusion, our results demonstrate its effectiveness in addressing the challenges posed by large irregular masks, offering a significant inpainting tool not only for CMB analyses but also for image-based experiments across disciplines where such data imperfections are prevalent.

Cosmic microwave background↗

Uncertainty-Guided Prediction Horizon of Phase-Resolved Ocean Wave Forecasting Under Data Sparsity: Experimental and Numerical Evaluation

Accurate short-term wave forecasting is critical for the safe and efficient operation of marine structures that rely on real-time, phase-resolved ocean wave information for control and monitoring purposes (e.g., digital twins). These systems often depend on environmental sensors (e.g., waverider buoys, wave-sensing LIDAR). Challenges arise when upstream sensor data are missing, sparse, or phase-shifted due to drift. This study investigates the performance of two machine learning models, time-series dense encoder (TiDE) and long short-term memory (LSTM), for forecasting phase-resolved ocean surface elevations under varying degrees of data degradation. We introduce the τ-trimming algorithm, which adapts the prediction horizon based on uncertainty thresholds derived from historical forecasts. Numerical wave tank (NWT) and wave basin experiments are used to benchmark model performance under short- and long-term data masking, spatially coarse sensor grids, and upstream phase shifts. Results show under a 50% probability of upstream data loss, the τ-trimmed TiDE model achieves a 46% reduction in error at the most upstream target, compared to 22% for LSTM. Furthermore, phase misalignment in upstream data introduces a near-linear increase in forecast error. Under moderate model settings, a ±3 s misalignment increases the mean absolute error by approximately 0.5 m, while the same error is accumulated at ±4 s using the more conservative approach. These findings inform the design of resilient, uncertainty-aware wave forecasting systems suited for realistic offshore sensing environments.

42 ENGINEERING↗

A Robust Event Diagnostics Platform: Integrating Tensor Analytics and Machine Learning into Real-time Grid Monitoring

The objective of this project is to develop a robust event diagnostics (RED) platform by integrating state-of-the-art tensor analytics and machine learning into real-time grid monitoring. The proposed platform can effectively analyze and discover the information hiding within the provided PMU data for effective real-time grid monitoring. The proposed RED platform provides a set of robust diagnostics tools for grid operation and management, including 1) data quality assessment, 2) data completion, 3) event detection, and 4) robust event classification. All the functionalities of the RED platform can help the operator to make informed decisions and respond in a timely manner. The developed RED platform will serve as an innovative advisory tool to reliably identify key events and discover new insights about the events and grid characteristics in the PMU data, and contribute to the efficient, safe, reliable operation and design of the nation’s electric system.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Hydrogen shift isomerizations in the kinetics of the second oxidation mechanism of alkane combustion. Reactions of the hydroperoxypentylperoxy OOQOOH radical

Hydroperoxyalkylperoxy species are important intermediates that are generated during the autoignition of transport fuels. In combustion, the fate of hydroperoxyalkylperoxy is important for the performance of advanced combustion engines, especially for autoignition. A key fate of the hydroperoxyalkylperoxy is a 1,5 H-shift, for which kinetics data are experimentally unavailable. In the present work, we study 1-hydroperoxypentan-3-yl)dioxidanyl (CH 3 CH 2 CH(OO)CH 2 CH 2 OOH) as a model compound to clarify the kinetics of 1,5 H-shift of hydroperoxyalkylperoxy species, in particular α-H isomerization and alternative competitive pathways. With a combination of electronic structure calculations, we determine previously missing thermochemical data, and with multipath variational transition state theory (MP-VTST), a multidimensional tunneling (MT) approximation, multiple-structure anharmonicity, and torsional potential anharmonicity, we obtained much more accurate rate constants than the ones that can computed by conventional single-structure harmonic transition state theory (TST) and than the empirically estimated rate constants that are currently used in combustion modeling. The roles of various factors in determining the rates are elucidated. The pressure-dependent rate constants for these competitive reactions are computed using system-specific quantum RRK theory. The calculated temperature range is 298–1500 K, and the pressure range is 0.01–100 atm. The accurate thermodynamic and kinetics data determined in this work are indispensable in the detailed understanding and prediction of ignition properties of hydrocarbons and alternative fuels.

37 INORGANIC, ORGANIC, PHYSICAL, AND ANALYTICAL CH↗

The role of moisture in MgCl 2 salt: A multiscale approach to TES performance

This study presents a comprehensive multiscale analysis to evaluate the influence of moisture on the thermal performance of thermal energy storage systems using magnesium chloride (MgCl 2 ) as the phase-changing material. The system uses graphite foam with 90% relative density to enhance thermal conductivity. The analysis includes thermal conductivity calculations and specific heat capacity for different hydrate phases of MgCl 2 : Anhydrous, Mono, Di, Tetra, and Hexa. These properties were evaluated for the first time using the phonon density of states from Density Functional Theory simulations. Results showed that thermal conductivity decreased, while specific heat capacity increased by a factor of two as the phase changed from anhydrous to hexahydrate. Meso-scale models were created to account for the anisotropy of graphite foam and property variations of the MgCl 2 hydrate phases. Asymptotic Expansion homogenization simulations determined the anisotropic thermal conductivity for all phases. This unique methodology improved simulation accuracy, which matched experimental data for anhydrous MgCl 2 . A parametric study examined various operating conditions and their effect on TES performance. It revealed that higher charging temperatures did not enhance exergy efficiency, but increased discharge mass flow rates improved it due to better heat transfer. Thermal performance evaluated by the exergy efficiency remained consistent across hydrate systems under the tested conditions. Furthermore, the study suggests that this uniformity is linked to missing key data, particularly latent heat and melting point for different hydrates. Overall, it highlights the importance of multiscale effects and accurate material properties in designing and optimizing TES systems.

Molten salt degradation↗