Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bad data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 55 records · Page 3
All sourcesNASA NTRSDOE OSTI
NASA NTRS1989

Extended observability of linear time-invariant systems under recurrent loss of output data

Recurrent loss of sensor data in integrated control systems of an advanced aircraft may occur under different operating conditions that include detected frame errors and queue saturation in computer networks, and bad data suppression in signal processing. This paper presents an extension of the concept of observability based on a set of randomly selected nonconsecutive outputs in finite-dimensional, linear, time-invariant systems. Conditions for testing extended observability have been established.

Luck, Rogelio↗
NASA NTRS1976

PROCAMS - A second generation multispectral-multitemporal data processing system for agricultural mensuration

PROCAMS (Prototype Classification and Mensuration System) has been designed for the classification and mensuration of agricultural crops (specifically small grains including wheat, rye, oats, and barley) through the use of data provided by Landsat. The system includes signature extension as a major feature and incorporates multitemporal as well as early season unitemporal approaches for using multiple training sites. Also addressed are partial cloud cover and cloud shadows, bad data points and lines, as well as changing sun angle and atmospheric state variations.

Erickson, J. D.↗
NASA NTRS2000

Satellite Data Sets in the Polar Regions

We have generated about two decades of consistently derived geophysical parameters in the polar regions. The key parameters are sea ice concentration, surface temperature, albedo, and cloud cover statistics. Sea ice concentrations were derived from the Scanning Multichannel Microwave Radiometer (SMMR) data and the Special Scanning Cl Microwave Imager (SSM/I) data from several platforms using the enhanced Bootstrap Algorithm for the period 1978 through 1999. The new algorithm reduces the errors associated with spatial and temporal variations in the emissivity and surface temperatures of sea ice. Also, bad data at ocean/land interfaces are identified and deleted in an unsupervised manner. Surface ice temperature, albedo and cloud cover statistics are derived simultaneously from the Advanced Very High Resolution Radiometer (AVHRR) data from 1981 through 1999 and mapped at a higher resolution but the same format as the ice concentration data. The technique makes use these co-registered ice concentration maps to enable cloud masking to be done separately for open ocean, sea ice and land areas. The effect of inversion is minimized by taking into consideration the expected changes in the effect of inversion with altitude, especially in the Antarctic. A technique for ice type regional classification has also been developed using multichannel cluster analysis and a neural network. This provide a means to identify large areas of thin ice, first year ice, and older ice types. The data sets have been shown to be coherent with each other and provide a powerful tool for in depth studies of the currently changing Arctic and Antarctic environment.

Comiso, Josefino C.↗
NASA NTRS1989

A fast 3-D object recognition algorithm for the vision system of a special-purpose dexterous manipulator

A fast 3-D object recognition algorithm that can be used as a quick-look subsystem to the vision system for the Special-Purpose Dexterous Manipulator (SPDM) is described. Global features that can be easily computed from range data are used to characterize the images of a viewer-centered model of an object. This algorithm will speed up the processing by eliminating the low level processing whenever possible. It may identify the object, reject a set of bad data in the early stage, or create a better environment for a more powerful algorithm to carry the work further.

Hung, Stephen H. Y.↗
NASA NTRS2004

The Micro-Pulse Lidar Network (MPLNET): A Federated Network of Micro-pulse Lidars and AERONET Sunphotometers

We present the formation of a new global-ground based eye-safe lidar network, the NASA Micro-Pulse Lidar Network (MPLNET). The aim of MPLNET is to acquire long- term observations of aerosol and cloud vertical profiles at unique geographic sites within the NASA Aerosol Robotic Network (AERONET). MPLNET utilizes standard instrumentation and data processing algorithms for efficient network operations and direct comparison of data between each site. The micro-pulse lidar is eye-safe, compact, and commercially available, and most easily allows growth of the network without sacrificing standardized instrumentation goals. Network growth follows a federated approach, pioneered by AERONET, wherein independent research groups may join MPLNET with their own instrument and site. MPLNET sites produce not only vertical profile data, but also column-averaged products already available from AERONET (aerosol optical depth, sky radiance, size distributions). Algorithms are presented for each MPLNET data product. Real-time Level 1 data products (next-day) include daily lidar signal images from the surface to -2Okm, and Level 1.5 aerosol extinction profiles at times co-incident with AERONET observations. Quality assured Level 2 aerosol extinction profiles are generated after screening the Level 1.5 results and removing bad data. Level 3 products include continuous day/night aerosol extinction profiles, and are produced using Level 2 calibration data. Rigorous uncertainty calculations are presented for all data products. Analysis of MPLNET data show the MPL and our analysis routines are capable of successfully retrieving aerosol profiles, with the strenuous accounting of uncertainty necessary for accurate interpretation of the results.

Welton, Ellsworth J.↗
DOE OSTI2021

Analyzing the Effects of Cyberattacks on Distribution System State Estimation

Key components of power systems—such as energy management systems, automatic generation control, and state estimation—are under serious vulnerability from cyberattacks. Cyber threats in electric grids have increased significantly because of the increased interconnectivity of supervisory control and data acquisition systems and public network infrastructure. As the penetration level of distributed energy resources increases, it is imperative to employ system-monitoring techniques such as state estimation for the reliable operation of distribution systems. Recently, multiple methods have been developed that exploit the low rank property of distribution system state matrix and are robust to bad data, such as matrix completion. This paper analyzes the impact of various realistic cyberattack scenarios on matrix completion. Realistic cyberattack scenarios are converted into data corruption models that are used in an extensive simulation of a custom IEEE 123-bus system.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2020

Analyzing the Effects of Cyberattacks on Distribution System State Estimation: Preprint

Key components of power systems—such as energy management systems, automatic generation control, and state estimation—are under serious vulnerability from cyber attacks. Cyber threats in electric grids have increased significantly because of the increased interconnectivity of supervisory control and data acquisition systems and public network infrastructure. As the penetration level of distributed energy resources increases, it is imperative to employ system-monitoring techniques such as state estimation for the reliable operation of distribution systems. Recently, multiple methods have been developed that exploit the low rank property of distribution system state matrix and are robust to bad data, such as matrix completion. This paper analyzes the impact of various realistic cyber attack scenarios on matrix completion. Realistic cyber attack scenarios are converted into data corruption models that are used in an extensive simulation of a custom IEEE 123-bus system.

41 EE - Solar Energy Technologies Office (EE-4S)↗
DOE OSTI2023

Robust Distribution State Estimation for Reliable Locational Marginal Pricing under Cyber-Attacks

Here this paper examines the impact of false data injection (FDI) cyber-attacks on distribution system state estimation (DSSE) and the resulting distribution locational marginal price (DLMP) in power markets. Two robust high-breakdown regression estimators, namely S- and MM- estimators, are implemented to provide resistance against FDI attacks targeting measurements and grid topology, creating leverage points. The introduced estimators are compared to the weighted least squares (WLS) with a bad data detection and rejection module (BDD) and the robust Huber M-estimator. The proposed estimators are shown to be effective and compare favorably to both existing Huber M- and the WLS with BDD in the presence of topology FDI attacks. Both the S- and MM-estimators provide good performance in the case of clean and corrupted measurements. Their performance is comparable in this case to the Huber M- and the WLS, followed by a BDD module. The simulation considered a modified distribution IEEE 13 and 34-bus systems where the impact of FDI attack scenarios is shown on the state and the DLMP pricing in the presence of distributed Generation.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2020

Robust Medium-Voltage Distribution System State Estimation using Multi-Source Data

Due to the lack of sufficient online measurements for distribution system observability, pseudo-measurements from short-term load or distributed renewable energy resources (DERs) forecasting are used. However, the accuracy of them is low and thus significantly limits the performance of distribution system state estimation (DSSE). In this paper, a robust DSSE that integrates multi-source measurement data is proposed. Specifically, the historical low-voltage (LV) side smart meters are used to forecast load and DERs injections via the support vector machine (SVM) with optimally tuned parameters. By contrast, the online smart meters at LV side are utilized to derive equivalent power injections at the MV/LV transformers, yielding more accurate pseudo-measurements compared to the forecasted injections. Furthermore, to deal with bad data caused by communication loss, instrumental errors and cyber attacks, robust DSSE that relies on generalized maximum-likelihood (GM)-estimation criterion is developed. The projection statistics are developed to adjust the weights of each measurement, leading to better balance between pseudo- and real-time measurements. Numerical results conducted on modified IEEE 33-bus system with DG integration demonstrate the effectiveness and robustness of the proposed method.

distribution system state estimation↗
DOE OSTI2019

Sample Complexity of Power System State Estimation Using Matrix Completion

In this paper, we propose an analytical framework to quantify the amount of data samples needed to obtain accurate state estimation in a power system - a problem known as sample complexity analysis in computer science. Motivated by the increasing adoption of distributed energy resources into the distribution-level grids, it becomes imperative to estimate the state of distribution grids in order to ensure stable operation. Traditional power system state estimation techniques mainly focus on the transmission network which involve solving an overdetermined system and eliminating bad data. However, distribution networks are typically underdetermined due to the large number of connection points and high cost of pervasive installation of measurement devices. In this paper, we consider the recently proposed state-estimation method for underdetermined systems that is based on matrix completion. In particular, a constrained matrix completion algorithm was proposed, wherein the standard matrix completion problem is augmented with additional equality constraints representing the physics (namely power-flow constraints). We analyze the sample complexity of this general method by proving an upper bound on the sample complexity that depends directly on the properties of these constraints that can lower number of needed samples as compared to the unconstrained problem. To demonstrate the improvement that the constraints add to state estimation, we test the method on a 141-bus distribution network case study and compare it to the traditional least squares minimization state estimation method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2019

Sample Complexity of Power System State Estimation using Matrix Completion: Preprint

In this paper, we propose an analytical framework to quantify the amount of data samples needed to obtain accurate state estimation in a power system-a problem known as \emph{sample complexity} analysis in computer science. Motivated by the increasing adoption of distributed energy resources into the distribution-level grids, it becomes imperative to estimate the state of distribution grids in order to ensure stable operation. Traditional power system state estimation techniques mainly focus on the transmission network which involve solving an overdetermined system and eliminating bad data. However, distribution networks are typically \emph{underdetermined} due to the large number of connection points and high cost of pervasive installation of measurement devices. In this paper, we consider the recently proposed state-estimation method for underdetermined systems that is based on \emph{matrix completion}. In particular, a \emph{constrained} matrix completion algorithm was proposed, wherein the standard matrix completion problem is augmented with additional equality constraints representing the physics (namely power-flow constraints). We analyze the sample complexity of this general method by proving an upper bound on the sample complexity that depends directly on the properties of these constraints that can lower number of needed samples as compared to the unconstrained problem. To demonstrate the improvement that the constraints add to state estimation, we test the method on a 141-bus distribution network case study and compare it to the traditional least squares minimization state estimation method.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS2008

Data Assimilation Cycling for Weather Analysis

This software package runs the atmospheric model MM5 in data assimilation cycling mode to produce an optimized weather analysis, including the ability to insert or adjust a hurricane vortex. The program runs MM5 through a cycle of short forecasts every three hours where the vortex is adjusted to match the observed hurricane location and storm intensity. This technique adjusts the surrounding environment so that the proper steering current and environmental shear are achieved. MM5cycle uses a Cressman analysis to blend observation into model fields to get a more accurate weather analysis. Quality control of observations is also done in every cycle to remove bad data that may contaminate the analysis. This technique can assimilate and propagate data in time from intermittent and infrequent observations while maintaining the atmospheric field in a dynamically balanced state. The software consists of a C-shell script (MM5cycle.driver) and three FORTRAN programs (splitMM5files.F, comRegrid.F, and insert_vortex.F), and are contained in the pre-processor component of MM5 called "Regridder." The model is first initialized with data from a global model such as the Global Forecast System (GFS), which also provides lateral boundary conditions. These data are separated into single-time files using splitMM5.F. The hurricane vortex is then bogussed in the correct location and with the correct wind field using insert_vortex.F. The modified initial and boundary conditions are then recombined into the model fields using comRegrid.F. The model then makes a three-hour forecast. The three-hour forecast data from MM5 now become the analysis for the next short forecast run, where the vortex will again be adjusted. The process repeats itself until the desired time of analysis is achieved. This code can also assimilate observations if desired.

Tran, Nam↗
DOE OSTI2024

Protorheology in practice: Avoiding misinterpretation

Protorheology is the paradigm that any observed flow or deformation is a chance to infer quantitative rheological properties. While this creates many opportunities for insight, there is significant risk of misunderstanding the physics involved, e.g. misinterpreting a liquid as a solid or mistaking viscous flow time as viscoelastic relaxation time. We describe these and other potential mistakes, use case studies to show how serious the problems can be, and contrast misinterpretations with correct approaches and interpretations. Some issues are especially important with materials involving colloidal particles and flows involving surface tension. Whether the reader is making inference from a tilted vial, time-lapse gravity-driven flow, a bounce test, die swell, or any other protorheology observation, the examples here serve as a guide for avoiding bad data in protorheology.

42 ENGINEERING↗
DOE OSTI2024

Specifying angular reference for robust three-phase state estimation

While the phase angle of any of the bus voltages can be chosen as the angular reference in the state estimation formulation of positive sequence networks, the same approach does not readily extend to three-phase network state estimation problem. It is commonly assumed that there is at least one bus where the bus voltages are perfectly balanced with phase angles displaced ±120° and these balanced three phase voltages are used as the three-phase reference in solving the three-phase state estimation problem. This assumption may be quite realistic in transmission networks, and for distribution networks with a strong transmission system connection. However, it might not be realistic to assume existence of a perfectly balanced reference bus in today’s distribution systems with ever increasing penetration of renewable sources or for isolated operation of microgrids. In this paper, a novel state estimation formulation will be presented which facilitates correct solution irrespective of the existence of buses with perfectly balanced voltages. The new formulation is general, and lends itself to bad data processing. It yields accurate results in any three-phase power system irrespective of its operating conditions (balanced or highly unbalanced), configuration (isolated microgrid, connected to transmission system, etc.) and whether or not it contains any synchronous generators. The performance of the method is validated using the IEEE 123 bus three-phase system.

24 POWER TRANSMISSION AND DISTRIBUTION↗
DOE OSTI2021

Intelligent Prediction of States in Multi-port Autonomous Reconfigurable Solar power plant (MARS)

In power electronics, prediction of states may be used for identification of faults, determination of aging of components, identification of bad data measurements, among others. Prediction of states in power electronics have broadly been based on: (a) physics-based models, (b) data-driven models, and (c) hybrid models. In this paper, data-driven approaches are presented for intelligent prediction of states in multi-port autonomous reconfigurable solar power plant (MARS) and compared. The data-set needed to train the data-driven models based on artificial intelligence (AI) algorithms has been identified and the trained models are evaluated under different extrapolated normal and abnormal operating conditions. The AI algorithms include nonlinear auto-regressive exogenous model (NARX), spiking neural networks (SNN), and decision tree. The models are compared and contrasted. The best model (NARX) is evaluated under different normal and abnormal operating conditions that have indicated accurate prediction.

Debnath, Suman↗
DOE OSTI2020

Net Load Redistribution Attacks on Nodal Voltage Magnitude Estimation in AC Distribution Networks

A high penetration level of smart devices and communication networks increases the threat of cyber-attacks in the distribution system. In this paper, we model a hidden, coordinated, net load redistribution attack (NLRA) in an AC distribution system. Based on local information of an attack region, the attacker’s goal is to create violations in nodal voltage magnitude estimation. Acting as a system operator equipped with global AC state estimation and bad data detection, we validate the stealthiness of the hidden NLRA in multiple attack cases. Simulation results on a modified PG&E 69-node distribution system show the validity of the proposed NLRA. The influence of NLRA on the distribution system is assessed and the impact of attack regions, attack timing, and system observability is also revealed.

Zhang, Hang↗
DOE OSTI2020

2020 IEEE PES Innovative Smart Grid Technologies Europe (ISGT-Europe)

Recent proliferation of distributed energy sources in distribution or sub-transmission systems necessitates close monitoring of these three-phase power grids which typically operate under unbalanced loading conditions. Unlike the transmission systems where the network equations are commonly based on the positive sequence component models, a detailed three phase model will have to be used in implementing network applications for these systems. In the specific case of the state estimator, where measurement and parameter errors may bias the solution, bad data and parameter error detection algorithms should also be incorporated. Implementing the state estimator and error detection algorithms for three-phase systems impose additional computational burden and modifications to the state estimation code. This paper proposes a practical solution to avoid these issues by using synchronized phasor measurements and modal decoupling. The previously developed parameter error detection algorithm based on the normalized Lagrange multipliers (NLM) test is applied to the measurements independently in each mode in parallel, not only saving CPU time but also avoiding new code development for a three-phase estimator. Different parameter error scenarios are created and tested to verify the effectiveness of the proposed error detection approach.

Khalili, Ramtin↗
DOE OSTI2022

Prediction of Power Measurements Using Adaptive Filters

With the advent of smart grid concept, Internet of Things (IoT) and the deployment of smart meters, the cyberattack threats on power networks have increased due to the use of communication systems that can be accessed by adversaries. Attackers will have the ability to manipulate the outcomes of smart meters which in turn influence the core application of Energy Management System 9EMS): State Estimation (SE). Bad data analytic tools may fail to detect some attacks into measurements. Meanwhile, Machine Learning (ML) solutions have been proposed for detecting False Data Injection (FDI) attacks. However, there is a lack of ML time-series solutions presented in the state-of-the-art that is yet to be complex. In signal processing, time-series solutions do not only consider the signal, but also the statistics of the signal over time. Therefore, in this paper, a machine learning for time-series solutions is presented as an application to model the measurements of the power grid that are used in SE. The presented model takes into account adaptive linear and non-linear filters: Finite Impulse Response (FIR), and Infinite Impulse Response (IIR). The presented models are implemented and performed on the IEEE-118 bus system. The results indicate the advantage of applying those filters over the state-of-the-art machine learning solutions.

Hamad, Khaled↗