SEARCH · Search NASA
Results for “data recovery”
Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.
Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.
Data recovery via covert cognizance for unattended operational resilience
One of the important premises of unattended operation, a highly promoted characteristic of fission batteries and advanced microreactors, is the ability to automate the analysis of sensors data used in support of operational monitoring and control. Here, to meet this vision, this work proposes a new monitoring and data recovery paradigm to ensure resilience against data corruption which may be the result of malicious intrusion into the reactor operational network. This is paramount to ensure 100% availability under contingency scenarios such as cyberattacks. In support of this vision, earlier work has presented the concept of covert cognizance and demonstrated its mathematical ability to identify and embed cognizance parameters under the noise-dominated null space of the sensors data. This work extends this concept and applies it in real-time to demonstrate three key characteristics: zero-impact, zero-observability, and data recovery, where the first characteristic is to ensure no impact on operation, the second is immunity to discovery by pattern recognition techniques, and the third is to allow recovery of corrupt or falsified data. Recognizing that fission batteries are designed to operate under steady state most of the time, we elect to employ a small modular reactor model under transient operational conditions to demonstrate the operational resilience enabled by the covert cognizance paradigm. Specifically, the PI controller is augmented with the covert cognizance modules to develop self-awareness and enable automatic data recovery. The developed modules are expected to be equally applicable to a wide range of advanced reactor technologies relying on full or partial unattended control.
Probabilistic neural networks for fluid flow surrogate modeling and data recovery
We consider the use of probabilistic neural networks for fluid flow surrogate modeling and data recovery. This framework is constructed by assuming that the target variables are sampled from a Gaussian distribution conditioned on the inputs. Consequently, the overall formulation sets up a procedure to predict the hyperparameters of this distribution which are then used to compute an objective function given training data. We demonstrate that this framework has the ability to provide for prediction confidence intervals based on the assumption of a probabilistic posterior, given an appropriate model architecture and adequate training data. The applicability of the present framework to cases with noisy measurements and limited observations is also assessed. To demonstrate the capabilities of this framework, we consider canonical regression problems of fluid dynamics from the viewpoint of reduced-order modeling and spatial data recovery for four canonical data sets. The examples considered in this study arise from (i) the shallow-water equations, (ii) a two-dimensional cylinder flow, (iii) the wake of a NACA0012 airfoil with a Gurney flap, and (iv) the NOAA sea surface temperature data set. Furthermore, the present results indicate that the probabilistic neural network not only produces a machine-learning-based fluid flow surrogate model but also systematically quantifies the uncertainty therein to assist with model interpretability.
Bayesian High-Rank Hankel Matrix Completion for Nonlinear Synchrophasor Data Recovery
Phasor measurement units (PMUs) provide high temporal-resolution synchrophasor measurements for power system monitoring and control. The frequent data quality issues, such as missing and bad data, prevent the incorporation of synchrophasor data in real-time operations. Most existing data-driven data recovery methods assume the power system dynamics can be approximated by a linear dynamical system, and the recovery performance degrades significantly when the power system is experiencing nonlinear dynamics during significant events. Here, this paper proposes a data-driven Bayesian nonlinear synchrophasor data recovery method (Ba-NSDR) that can recover a consecutive time period of simultaneous data losses or errors across all channels, even when the underlying system is highly nonlinear. The idea is to lift the Hankel matrix of the spatial-temporal synchrophasor data to a higher dimension such that the lifted Hankel matrix is low-rank in that space and can be processed with the kernel trick. Our proposed Bayesian method then infers the probabilistic distributions of synchrophasor from the partial observations. Some distinctive features of Ba-NSDR include an uncertainty index to measure the accuracy of the recovery result and the robustness to parameter selections. Our method is verified on both synthetic and recorded event datasets.
A Deep Learning Approach for In-Network Synchrophasor Missing Data Recovery Using Programmable Network Switches
Phasor measurement unit (PMU) networks deliver accurate and timely measurements, which is essential for managing today’s electric power systems. To ensure data quality and enhance the cyber-resilience of PMU networks against malicious attacks and data errors, this study presents an online PMU missing data recovery scheme by leveraging P4 programmable switches. The data plane incorporates a customized PMU protocol parser that abstracts the necessary payload data for recovery. Recovery processes are executed in the control plane using a pre-trained machine learning model. Both traditional and advanced ML models, such as transformer and TimeGPT, are explicitly employed for data prediction. This approach ensures rapid and precise data recovery. Performance evaluations focus on recovery speed and accuracy, using a real dataset from a campus microgrid. With 20% missing PMU data, the mean absolute percentage error for voltage magnitude is 0.0384%, and the phase angle error discrepancy is approximately 0.4064%.
Low-rank Tensor Completion for PMU Data Recovery
This paper proposes a tensor completion method for the recovery of missing phasor measurement unit (PMU) measurements. Tensor completion as the general case of matrix completion has attracted increasing attention in recent years. The imputation accuracy for the existing matrix completion methods may be significantly reduced when there are consecutive data losses across multiple data channels. To tackle this issue, we explore the multi-way characteristics of PMU measurements by using a tensor model. We leverage the low-rank property of the PMU measurements and formulate the missing PMU data recovery problem as a low-rank tensor completion problem. An efficient algorithm based on alternating direction method of multipliers (ADMM) is developed to solve the tensor completion problem. The experiments using the real PMU dataset show that the proposed method exhibits better imputation accuracy compared with the conventional data recovery methods.
A machine-learning-aided data recovery approach for predicting multi-material thermal behaviors in advanced test reactor capsules
Instrumented experiments conducted at test reactors are essential to the deployment of new advanced reactor systems. Designing new experiments and generating data on specific reactor conditions require significant investments in terms of both time and cost. Finite element analysis software can be used to create high-fidelity models of experiment environments in order to support the actual experiments, but computation time remains a concern in terms of applying outcomes to real-time usage of data (e.g., a digital twin [DT]). Here, the present research proposes a machine-learning (ML) aided approach to making temperature and displacement predictions based on the thickness of the outer gas gap on the experimental capsule used for in-pile demonstration of a novel new thermal conductivity probe in the Advanced Test Reactor (ATR). This capsule consisted of U10Zr fuel, a rodlet, sodium, and inner and outer capsules. Gas gaps existed between the fuel and the rodlet, and between the inner and the outer capsule. The learning data pertained to an experimental capsule's radial distributions of temperature and displacement, as obtained based on Abaqus and the physical features. For the first step of ML sequence, the temperature was predicted using three positional parameters. Next, the displacement was predicted using seven additional parameters. Each physical feature was normalized in order to be both nondimensional and standardized. The temperature and displacement predictions showed good agreement with the simulation results in all cases involving interpolation and extrapolation. Furthermore, data similarity enhancement increased the similarity between the training and the target data, thereby increasing the predictive accuracy of the ML models. In certain extrapolation cases involving limited original ML model accuracy, data similarity enhancement and data recovery was able to somewhat improve this accuracy.
Recent Metallic Fuel Data Recovery in FIPD
The Metallic Fuels Irradiation and Physics Database (FIPD) [1] is an organized collection of metallic fuel test pin data (U-xPu-yZr, = 0 ~ 28; y = 2 ~ 10) and documentation available to industry. FIPD mainly contains three types of data: (1) Fuel pin fabrication data, including fuel slug diameter, fuel slug length, cladding diameter, smear density, etc. (2) Fuel pin operation conditions, including axial distributions for power, temperatures, fluences, burnup, and isotopic densities, etc. and (3) Fuel pin post-irradiation examination (PIE) data, including fission gas release and gas chemistry, profilometry, and neutron radiography, etc. The operating conditions for pins with PIE data available in FIPD span significant ranges across key parameters. The fuel peak burnup extends from less than 5% up to 20 at%. The cladding peak temperature varies from about 490°C to 660°C. Finally, the cladding peak DPA shows a wide range from less than 5 to 120. These broad ranges reflect the diverse testing conditions and operational parameters captured in the available PIE data. More detail about FIPD can be found in ref. [2]. The database development is an ongoing effort covering metallic fuel experiments from the Experimental Breeder Reactor II (EBR-II) and the Fast Flux Test Facility (FFTF). As reported in the ref. [3, 4], most of the PIE data generated during the IFR program [5] has been collected, reviewed, processed, and integrated into FIPD. The most recently added PIE data can be found in ref. [4], which shows the collection of over 95% of the PIE data by the time of this paper. The recent improvements to the database are summarized in this paper.
Report on local data recovery approaches suitable for weather and climate prediction (Deliverable 1.3) (V.1.0)
Numerical weather and climate prediction rates as one of the scientific applications whose accuracy improvements greatly depend on the growth of the available computing power. As the number of cores in top computing facilities pushes into the millions, increasing average frequency of hardware and software failures forces users to review their algorithms and systems in order to protect simulations from breakdown. This report surveys approaches for fault-tolerance in numerical algorithms and system resilience in parallel simulations from the perspective of numerical weather and climate prediction systems. A selection of existing strategies is analyzed, featuring interpolation-restart and compressed checkpointing for the numerics, in-memory checkpointing, user-level failure mitigation-based and backup-based methods for the systems. Numerical examples showcase the performance of the techniques in addressing faults, with particular emphasis on iterative solvers for linear systems, a staple of atmospheric fluid flow solvers. The potential impact of these strategies is discussed in relation to current development of numerical weather prediction algorithms and systems towards the exascale. Trade-offs between performance, efficiency and effectiveness of resiliency strategies are analyzed and some recommendations outlined for future developments.
Techniques for storing data to enhance recovery and detection of data corruption errors
Often there are errors when reading data from computer memory. To detect and correct these errors, there are multiple types of error correction codes. Disclosed is an error correction architecture that creates a codeword having a data portion and an error correction code portion. Swizzling rearranges the order of bits and distributes the bits among different codewords. Because the data is redistributed, a potential memory error of up to N contiguous bits, where N for example equals 2 times the number of codewords swizzled together, only affects up to, at most, two bits per swizzled codeword. This keeps the error within the error detecting capabilities of the error correction architecture. Furthermore, this can allow improved error correction and detection without requiring a change to error correcting code generators and checkers.
Techniques for storing data to enhance recovery and detection of data corruption errors
Often there are errors when reading data from computer memory. To detect and correct these errors, there are multiple types of error correction codes. Disclosed is an error correction architecture that creates a codeword having a data portion and an error correction code portion. Swizzling rearranges the order of bits and distributes the bits among different codewords. Because the data is redistributed, a potential memory error of up to N contiguous bits, where N for example equals 2 times the number of codewords swizzled together, only affects up to, at most, two bits per swizzled codeword. This keeps the error within the error detecting capabilities of the error correction architecture. Furthermore, this can allow improved error correction and detection without requiring a change to error correcting code generators and checkers.
Application of machine learning techniques for fast MeV x-ray spectra unfolding from filter stack spectrometer data
Recovery of MeV x-ray spectra from detector signals is difficult because the response matrix inversion is ill-conditioned and current methods are too slow for high-repetition-rate experiments. In this work, we make use of neural networks to unfold MeV x-ray spectra from measurements obtained with a filter stack spectrometer at rates of near 40 Hz. The neural network was trained on synthetic data and tested on both synthetic and experimental data, the latter obtained in two separate experiments performed at the Omega EP laser facility. We show here that this unfolding method has good performance on synthetic data and that it is a promising option for experimental data of up to 40 MeV. The accuracy on experimental data is verified by using a simple forward model to compare against measured values.
General Analysis of Data Collected from DOE Lidar Buoy Deployments Off Virginia and New Jersey
Pacific Northwest National Laboratory (PNNL) operates two AXYS WindSentinel lidar buoys for the U.S. Department of Energy’s Wind Energy Technologies. The purpose of these buoys is to collect hub-height winds and supporting meteorological and oceanographic information to facilitate the development of offshore wind energy in the U.S. In general, each buoy is deployed for a year or more at a given location in order to capture at least a full annual cycle of weather conditions. The initial deployment for one buoy was off the coast of Virginia beginning in 2015, and the other buoy was first deployed off the coast of New Jersey beginning in 2016. Over the last two years, PNNL has had an opportunity to analyze the data collected during these first two deployments. This report describes a substantial analysis of data collected by the two lidar buoys operated off the coasts of Virginia and New Jersey. The centerpiece instrument for each buoy as deployed off Virginia and New Jersey is a lidar system, which is designed to measure the horizontal wind vector from approximately 40 m to 200 m above the sea surface with vertical resolution of 40 m. Since the initial deployments, the original lidars have been replaced with more powerful Leosphere 866 v2 systems. The analyses in this report will apply to the original Vindicator systems. In addition to the wind profiles from the lidars, the buoys collect near-surface measurements of wind speed and direction, air temperature, relative humidity, barometric pressure, and solar irradiance. Oceanographic variables measured include the two-dimensional wave spectrum, water temperature and conductivity, and ocean current vectors to a depth of 90 m. An assessment of overall data recovery and a basic analysis of the data collected was provided in a previous report This report substantially extends that analysis. The various sections describe the development and application of an inertial measurement unit (IMU) data recovery scheme for the New Jersey deployment; a climatological analysis of winds at hub height and at the surface together with thermodynamic variables measured at the surface for the full deployment periods; an analysis of oceanographic observations describe sea state; the development of a refinement for NOAA’s WaveWatch III model to allow its application to near-shore areas; a basic climatology of ocean currents observed from the buoys; an analysis of observed winds in the framework of Monin-Obukhov Similarity Theory; and the development and evaluation of techniques to extract turbulence intensity and turbulence kinetic energy from the lidars. The analyses contained in this report provide a great deal of new information about offshore conditions on the U.S. East Coast. In addition, the experience gained will inform both configurations and analysis of data from future deployments of these lidar buoy systems.
Data-driven recovery of hidden physics in reduced order modeling of fluid flows
In this article, we introduce a modular hybrid analysis and modeling (HAM) approach to account for hidden physics in reduced order modeling (ROM) of parameterized systems relevant to fluid dynamics. The hybrid ROM framework is based on using first principles to model the known physics in conjunction with utilizing the data-driven machine learning tools to model the remaining residual that is hidden in data. This framework employs proper orthogonal decomposition as a compression tool to construct orthonormal bases and Galerkin projection (GP) as a model to build the dynamical core of the system. Our proposed methodology hence compensates structural or epistemic uncertainties in models and utilizes the observed data snapshots to compute true modal coefficients spanned by these bases. The GP model is then corrected at every time step with a data-driven rectification using a long short-term memory (LSTM) neural network architecture to incorporate hidden physics. A Grassmannian manifold approach is also adopted for interpolating basis functions to unseen parametric conditions. The control parameter governing the system's behavior is thus implicitly considered through true modal coefficients as input features to the LSTM network. The effectiveness of the HAM approach is then discussed through illustrative examples that are generated synthetically to take hidden physics into account. Furthermore, our approach thus provides insights addressing a fundamental limitation of the physics-based models when the governing equations are incomplete to represent underlying physical processes.
Development of an Encoding Method on an Co-simulation Platform for Mitigating the Impact of Unreliable Communication
This report presents a hardware-in-the-loop (HIL) based modeling approach for simulating impacts of unreliable communication on the performance of centralized volt-var control and for developing an encoding method to mitigate the impacts. First, an asynchronous real-time HIL simulation platform is introduced to enable multi-rate co-simulation of a distribution system with many inverter-based distributed energy resources (DERs). The distribution system is modeled by milliseconds phasor-based models and the DERs are modeled by micro-seconds power electronic models. Communication connections between a centralized volt-var controller (modeled externally to the HIL testbed) and smart inverters are built by implementing Modbus links and the Long Term Evolution network. On this co-simulation platform, an enhanced, augmented Lagrangian multiplier based encoded data recovery (EALM-EDR) algorithm for mitigating the impact of unreliable communication is developed and validated. Simulation results demonstrate the efficacy of using the HIL-based co-simulation platform as a power grid digital twin for developing algorithms that coordinate a large number of heterogeneous control systems through wired and wireless communication links.
Mitigating Smart Meter Asynchrony Error Via Multi-Objective Low Rank Matrix Recovery
Not provided.
SpectraCodec: A Hilbert curve-based method for encoding metadata in mass spectra for machine learning applications (SpectraCodec) v1
Machine learning approaches to mass spectrometry (MS) data analysis require structured metadata for optimal performance. However, current MS file formats necessitate external metadata sources, creating integration challenges that impede analytical workflows. Here, we present a novel approach for encoding metadata directly within mzML files using one-hot encoding of ASCII characters mapped via Hilbert space-filling curves. This strategy embeds metadata in the first spectrum's m/z-intensity space, ensuring persistence with the primary data, eliminating the need for external metadata files, and maintaining compatibility with existing MS software. We demonstrate that the Hilbert curve mapping efficiently utilizes the two-dimensional spectral space while maintaining robust data recovery. This method offers a practical solution for machine learning applications in mass spectrometry by ensuring metadata and spectral data remain unified through all stages of analysis.
CoREC: Scalable and Resilient In-memory Data Staging for In-situ Workflows
The dramatic increase in the scale of current and planned high-end HPC systems is leading new challenges, such as the growing costs of data movement and IO, and the reduced mean time between failures (MTBF) of system components. In-situ workflows, i.e., executing the entire application workflows on the HPC system, have emerged as an attractive approach to address data-related challenges by moving computations closer to the data, and staging-based frameworks have been effectively used to support in-situ workflows at scale. However, the resilience of these staging-based solutions has not been addressed, and they remain susceptible to expensive data failures. Furthermore, naive use of data resilience techniques such as n-way replication and erasure codes can impact latency and/or result in significant storage overheads. In this article, we present CoREC, a scalable and resilient in-memory data staging runtime for large-scale in-situ workflows. CoREC uses a novel hybrid approach that combines dynamic replication with erasure coding based on data access patterns. It also leverages multiple levels of replications and erasure coding to support diverse data resiliency requirements. Furthermore, the article presents optimizations for load balancing and conflict-avoiding encoding, and a low overhead, lazy data recovery scheme. We have implemented the CoREC runtime and have deployed with the DataSpaces staging service on leadership class computing machines and present an experimental evaluation in the article. Here, the experiments demonstrate that CoREC can tolerate in-memory data failures while maintaining low latency and sustaining high overall storage efficiency at large scales.