Search NASA⌕ Search

SEARCH · Search NASA

Results for “Bad data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 91 records · Page 5

Remote-Sensing Time Series Analysis, a Vegetation Monitoring Tool

The Time Series Product Tool (TSPT) is software, developed in MATLAB , which creates and displays high signal-to- noise Vegetation Indices imagery and other higher-level products derived from remotely sensed data. This tool enables automated, rapid, large-scale regional surveillance of crops, forests, and other vegetation. TSPT temporally processes high-revisit-rate satellite imagery produced by the Moderate Resolution Imaging Spectroradiometer (MODIS) and by other remote-sensing systems. Although MODIS imagery is acquired daily, cloudiness and other sources of noise can greatly reduce the effective temporal resolution. To improve cloud statistics, the TSPT combines MODIS data from multiple satellites (Aqua and Terra). The TSPT produces MODIS products as single time-frame and multitemporal change images, as time-series plots at a selected location, or as temporally processed image videos. Using the TSPT program, MODIS metadata is used to remove and/or correct bad and suspect data. Bad pixel removal, multiple satellite data fusion, and temporal processing techniques create high-quality plots and animated image video sequences that depict changes in vegetation greenness. This tool provides several temporal processing options not found in other comparable imaging software tools. Because the framework to generate and use other algorithms is established, small modifications to this tool will enable the use of a large range of remotely sensed data types. An effective remote-sensing crop monitoring system must be able to detect subtle changes in plant health in the earliest stages, before the effects of a disease outbreak or other adverse environmental conditions can become widespread and devastating. The integration of the time series analysis tool with ground-based information, soil types, crop types, meteorological data, and crop growth models in a Geographic Information System, could provide the foundation for a large-area crop-surveillance system that could identify a variety of plant phenomena and improve monitoring capabilities.

McKellip, Rodney↗

De-Trending Techniques: Methods for Cleaning Questionable Shock Data

Not all zero shifted acceleration data can De-trended using this technique. DC shifts, improper AC coupling, Circuit noise/EMI/EMR, Equivalent RC circuit gain response/Circuit saturation(Slew Rate Limited), fixture grounding and wiring losses can all contribute to bad shock data being recorded. Some data that is zero-shifted or exhibit large instantaneous velocity shifts is inherently bad and a retest is warranted. Clean Acceleration-Time history data can be bad upon examining the Velocity & Displacement profiles. Laser Vibrometers provide a high level of accuracy for pyrotechnic shock testing. Engineering judgment and experience will determine the validity of Shock data.

Grillo, Vincent J.↗

In situ data

The evidence indicates that November 1979 was a very climatological month, certainly in the Atlantic. This must make us particularly wary of deviations from climatology anywhere in the globe during this period. The McLain and Pazan SST analyses are identical to within a degree. The Pazan ship data need to be edited against climatology in order to remove bad ship data.

Pazan, S.↗

Nuclear Data 101: The Good, The Bad, and The Ugly

This lecture is an attempt to cover in one hour the fundamentals of what nuclear data are, where they come from and how they affect nuclear science applications.

73 NUCLEAR PHYSICS AND RADIATION PHYSICS↗

3D Visualisation of Un-georeferenced, Coordinate-poor Spatial Data in Complex D and D and WM Environments - 18294

This paper describes the development of an algorithm-based utility tool using Visual Basic that examines un-georeferenced or GPS-deprived spatial datasets (3D datasets lacking one or more of the key x, y and z components). Taking into account any known geometrical parameters which are input via a user-friendly front-end, the new code automatically evaluates attributes, including any spatial coordinate components, detects invalid survey data points and fills in bad or missing data points utilising data-specific interpolation (e.g. cubic or linear spline) of valid data points, such that local extrema are not created. Previously troublesome 2D data can be easily converted to realistic 3D contour display, integrated with other time-series data to create 4D movies, and incorporated into other existing 3D datasets. More importantly those processed and 'more complete' data can be easily stored in 3D or 4D spatial database management systems for future interrogation by using clear, realistic and intuitive 3D visualisation. Test cases are presented to demonstrate the value and versatility of the newly developed utility tool which include the 3D visualisation of: surveyed wall thicknesses of a helical coil, radiation mapping of a plant and aerial survey mapping of radiation in a region near Fukushima. (authors)

12 MANAGEMENT OF RADIOACTIVE AND NON-RADIOACTIVE W↗

The good, the bad, and the ugly: Data-driven load profile discord identification in a large building portfolio

Reducing the overall energy consumption and associated greenhouse gas emissions in the building sector is essential for meeting our future sustainability goals. Recently, smart energy metering facilities have been deployed to enable monitoring of energy consumption data with hourly or subhourly temporal resolution. This unprecedented data collection has created various opportunities for advanced data analytics involving load profiles (e.g., building energy benchmarking programs, building-to-grid integration, and calibration of urban-scale energy models). These applications often need preprocessing steps to detect daily load profile discords, such as: 1) outliers due to system malfunctions (the bad) and 2) irregular energy consumption patterns, such as those resulting from holidays (the ugly) compared to normal consumption patterns (the good). However, current preprocessing methods predominantly focus on filtering using statistical threshold values, which fail to capture the contextual discords of daily profiles. In addition, discord detection algorithms in building research are often aimed at finding individual building-level discords, which are not suitable at a large scale. Thus, here, we develop a method for automated load profile discord identification (ALDI) in a large portfolio of buildings (more than 100 buildings). Specifically, ALDI 1) uses the matrix profile (MP) method to quantify the similarities of daily subsequences in time series meter data, 2) compares daily MP values with typical-day MP distributions using the Kolmogorov-Smirnov test, and 3) identifies daily load profile discords in a large building portfolio. We evaluate ALDI using the metering data of both an academic campus and a residential neighborhood. Our results demonstrate that ALDI efficiently discovers measurement errors by system malfunctions and low energy consumption days in the academic campus portfolio, and it detects unique load shape patterns likely driven by occupant behavior and extreme weather conditions in the residential neighborhood.

32 ENERGY CONSERVATION, CONSUMPTION, AND UTILIZATI↗

A comparison of in situ measurements of vector-E and - vector-V x vector-B from Dynamics Explorer 2

Dynamics Explorer-2 provided the first opportunity to make a direct comparison of in situ measurements of the high-latitude convection electric field by two distinctly different techniques. The vector electric field instrument (VEFI) used antennae to measure the intrinsic electric fields and the ion drift meter (IDM) and retarding potential analyzer (RPA) measured the ion drift velocity vector, from which the convection electric field can be deduced. The data from three orbits having large electric fields at high latitude are presented, one at high, one at medium, and one at low altitudes. The general agreement between the two measurements of electric field is very good, with typical differences at high latitudes of the order of a few millivolts per meter, but there are some regions where the particle fluxes are extremely large (e.g., the cusp) and the disagreement is worse, probably because of IDM difficulties. The auroral zone potential patterns derived from the two devices are in excellent agreement for two of the cases, but not in the third, where bad attitude data may be the problem. At low latitudes there are persistent differences in the measurements of a few millivolts per meter, though these differences are quite constant from orbit to orbit. This problem seems to arise from some shortcoming in the VEFI measurments. Overall, however, these measurements confirm the concept of `frozen-in' plasma that drifts with velocity vector-E x vector-B/B(exp 2) within the measurement errors of the two techniques.

Hanson, W. B.↗

A Game-Theoretic Data-Driven Approach for Pseudo-Measurement Generation in Distribution System State Estimation

In this paper, we present an efficient computational framework with the purpose of generating weighted pseudo-measurements to improve the quality of distribution system state estimation (DSSE) and provide observability with advanced metering infrastructure (AMI) against unobservable customers and missing data. The proposed technique is based on a game-theoretic expansion of relevance vector machines (RVMs). This platform is able to estimate the nodal power consumption and quantify its uncertainty while reducing the prohibitive computational burden of model training for large AMI datasets. To achieve this objective, the large training set is decomposed and distributed among multiple parallel learning entities. The resulting estimations from the parallel RVMs are then combined using a game-theoretic model based on the idea of repeated games with vector payoff. It is observed that through this approach and by exploiting the seasonal changes in customers’ behavior the accuracy of pseudo-measurements can be considerably improved, while introducing robustness against bad training data samples. The proposed pseudo-measurement generation model is integrated into a DSSE using a closed-loop information system, which takes advantage of a branch current state estimator (BCSE) to further improve the performance of the designed machine learning framework. This method has been tested on a practical distribution feeder model with smart meter data for verification.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Multi-area parameter error identification for large power systems

Power grid model parameters may contain errors due to various reasons. Detecting and correcting parameter errors typically requires significant computational effort due to the size and complexity of the parameter database. While the normalized Lagrange multiplier (NLM) method can effectively detect, identify and correct parameter errors, its computational burden could rapidly grow with increasing system size. This paper addresses this issue by proposing a multi-area parameter error identification method. Each area has its own outlier detection tool for detecting the incorrect parameters and measurements within the area. On the other hand, due to the reduced redundancy at area boundaries, parameter errors on branches incident to boundary buses may not be detected. Such errors are subsequently detected by a coordination level estimator completing the system-wide parameter detection procedure. In conclusion, performance of the developed method is demonstrated using the IEEE 118-bus and 2000-bus Texas synthetic systems.

24 POWER TRANSMISSION AND DISTRIBUTION↗

Power System Dynamic State Estimation: Motivations, Definitions, Methodologies and Future Work

This paper summarizes the technical activities of the Task Force on Power System Dynamic State and Parameter Estimation. This Task Force was established by the IEEE Working Group on State Estimation Algorithms to investigate the added benefits of dynamic state and parameter estimation for the enhancement of the reliability, security, and resilience of electric power systems. The motivations and engineering values of dynamic state estimation (DSE) are discussed in detail. Then, a set of potential applications that will rely on DSE is presented and discussed. Furthermore, a unified framework is proposed to clarify the important concepts related to DSE, forecasting-aided state estimation, tracking state estimation and static state estimation. An overview of the current progress in DSE and dynamic parameter estimation is provided. The paper also provides future research needs and directions for the power engineering community.

task force, dynamic state estimation, Parameter Es↗

A robust dynamic state estimation approach against model errors caused by load changes

Dynamic state estimation (DSE) plays an important role in power system security monitoring and online control. In practice, there are two approaches to implementing DSE. The first approach is distributed DSE, which is based on the assumption that the terminal bus of each generator can be measured by PMUs (phasor measurement units). The assumption cannot be satisfied currently, however, because PMUs usually are installed at important high-voltage buses such as 500-kV buses installed in portions of the grid overseen by the Western Electricity Coordinating Council. Another issue of this approach is that performance of DSE is vulnerable to bad measurement data. The reason for this vulnerability is that DSE is performed separately through measurements at each terminal bus, and measurements at terminal buses are the only measurement upon which DSE can rely. Therefore, important redundant measurements are not included in this approach. The second approach is centralized DSE. This approach does not have the requirement for PMU location, and redundant measurements can be considered fully. However, load changes and grid topology changes impact centralized DSE. In this paper, we propose a new approach for handling the impact of load changes on DSE. We have developed a new algorithm that includes two sequential steps. In the first step, errors caused by load changes are detected by analyzing the difference between prediction results and measured results. In the second step, once model error is detected, a model optimization procedure is run to correct the error so the state estimation error can be mitigated. Simulation results from the IEEE 68 bus system show that the proposed approach can effectively handle model errors caused by load changes.

robust dynamic state estimation, load change, powe↗

Robust Distributed State Estimator for Interconnected Transmission and Distribution Networks (Final Report RPPR-1)

This project’s objective is to develop a combined transmission and distribution state estimator which accounts for very large system size and model complexity (by way of distributing the computations) and large number of solar PV units connected to the distribution system on multiple feeders. The project not only provides a robust formulation and solution to this problem but also tests the solution by implementing it on a well-established large utility system. It introduces several improvements with respect to the state of the art in existing state estimation software: (a) The developed state estimator (SE) allows robust and accurate monitoring of bidirectional flows in distribution systems which result due to the distributed energy sources which are not observable and thus not incorporated in generation dispatch; (b) Large utility systems with tens of thousands of transmission buses and hundreds of thousands of distribution nodes are difficult to model as a single integrated system. This shortcoming is addressed by developing a “scalable distributed computational framework” which allows splitting the ultra large system models into several small subsystems and coordinating their solution by a robust and practical state estimation formulation; (c) Measurement errors irrespective of their locations are detected and removed by the developed state estimator. Historically, transmission and distribution systems were analyzed and operated as two independent systems. Given the non-transposed short feeder sections, unevenly loaded phases, strictly radial configuration and unidirectional power flows in the absence of remote generation, distribution system analysis was customized to account for these characteristics. However, some of these assumptions are no longer valid (non-radial configuration, bidirectional power flows) and thus distribution system analysis should be revisited. Furthermore, in the past, the interaction between the transmission and distribution systems was quite passive, where distribution substations were modeled as lumped loads in the transmission system model. With substantial generation injected by renewable generation located in the distribution systems, such modeling will no longer be accurate. The developed state estimator facilitates proper monitoring of the interactions between the transmission and distribution systems and enables smart dispatch of these units which are made observable by the state estimator.

24 POWER TRANSMISSION AND DISTRIBUTION↗

An analysis of redundancy management algorithms for asynchronous fault tolerant control systems

Redundancy management algorithms, commonly referred to as voters, are algorithms used in fault-tolerant control systems to vote on incoming redundant data, isolate bad signals, and output a single good value. In a synchronous environment, this algorithm is a straightforward signal-to-signal comparison with relatively low complexity. The technology of asynchronous control systems, recently realized in the Ultrareliable Fault Tolerant Control System research program at NASA Ames Research Center, requires more complex algorithms for fault detection and signal selection. A variety of algorithms used for this process, a means of testing them, and their basic performance under a simulated environment of the ultrareliable fault-tolerant control system are presented.

Davis, Gloria J.↗

Characterizing Uranus with an Ice giant Planetary Origins Probe (Ice-POP)

We now know from studies of planetary transits and microlensing that Neptune-mass planets are ubitquitous and may be the most common class of planets in the Galaxy. As such it is crucial that we understand the formation and evolution of the ice giant planets in our own solar system so that we can better understand planet formation throughout the galaxy. An entry probe mission to Uranus would help accomplish this goal. In fact the Planetary Decadal Survey recommended a Uranus orbiter with entry probe but did not explore in detail the specifications for the entry probe. NASA Ames is currently studying thermal protection system requirements for such a mission and this has led to questions regarding the minimum interesting science payload of such an entry probe. The single most important in-situ measurement for an ice giant entry probe is a measurement of atmospheric composition. For Uranus this would specifically include the methane and noble gas abundances. An in situ measurement of the methane abundance, from below the methane cloud, would constrain the atmospheric carbon abundance, which is believed to be roughly 30 to 50 times solar. There are hints from the transiting planets that extrasolar ice giants show comparable or even greater enhancements of heavy elements compared to their primary stars. However the origin of this carbon enhancement is controversial. Is Uranus a "failed core" of a larger gas giant or was the atmosphere enhanced by accretion of icy planetesimals' Constraining atmospheric abundances of C and perhaps S or even N from below 5 bars would provide badly needed data to address such issues. A measurement of the N abundance would provide clues on the origin of the planetesimals that formed Uranus. Low N-abundance indicates planetesimals from 'warmer' regions where N was mainly in form of NH3, whereas a strong enrichment could indicate planetesimals / cometary material from the colder outer regions of the nebula. Furthermore CO and HCN have been detected in Neptune but not in Uranus. A measurement of the abundance of either would constrain the source mechanisms for these molecules (exogenic or internal). A major surprise from the Galileo Entry Probe was that the heavier noble gases Ar, Kr, and Xe are enhanced in Jupiter's atmosphere at a level comparable to what was seen for the chemically active volatiles N, C, and S. It had been generally expected that Ar, Kr, and Xe would be present in solar abundances, as all were expected to accrete with hydrogen during the gravitational capture of nebular gases. Enhanced abundances of Ar, Kr, and Xe is equivalent to saying that these noble gases have been separated from hydrogen. There are several mechanisms that could accomplish this but these hypotheses require further testing. Measurement of noble gas abundances in an ice giant would constrain the planetary formation and nebular mechanisms responsible for this enhancement. Standard three-layer models of Uranus find that the outer, predominantly H/He layer of Uranus does not reach pressures high enough (approximately 1 Mbar) for H2 to transition to liquid metallic hydrogen. However, valid models can also be constructed with a smaller intermediate water-rich layer, with hydrogen then reaching the metallic hydrogen phase. If this occurs, He should phase separate from the hydrogen and ``rain out," taking along a substantial abundance of Ne, as suggested for Jupiter (and likely also for Saturn). Hence He and Ne depletions could be probes of the planet's structure in the much deeper interior. A determination of Uranus' atmospheric abundances, particularly of the noble gasses, is thus critical to understanding the formation of Uranus, and giant planets in general. These measurements can only be performed with an entry probe. The second key measurement would be a temperature-pressure sounding to provide ground truth for remote measurements of atmospheric temperature and composition and to constrain the internal heat flow. This would also establish that the methane abundance measurements have indeed been made below any possible methane cloud. Finally an ultra stable oscillator would measure wind speeds and constrain atmospheric dynamics. In our presentation we will discuss the importance of all of these measurements and argue that an entry probe is a crucial component of any ice giant mission.

Marley, Mark S.↗

Accurate determination of ferric iron in garnets

Numerous techniques are available to determine the amount of Fe 2+ and Fe 3+ in minerals. Calculating Fe 2+ and Fe 3+ by charge-balance using electron probe microanalysis (EPMA) data is the most common method, but several studies question the usefulness and accuracy of this approach (Canil and O’Neill 1996; Dyar et al. 1993, 2012; Lalonde et al. 1998; Li et al. 2005; McGuire et al. 1989; Schingaro et al. 2016; Schmid et al. 2003; Sobolev et al. 2011). We compile and compare data for natural garnets that have been analyzed by both EPMA and Mössbauer spectroscopy. Comparison of Fe3+/ΣFe determined by charge-balance vs. Mössbauer spectroscopy shows an approximate 1:1 correlation. The EPMA data set of Dyar et al. (2012) is reexamined and it is shown that disagreement between EPMA and Mössbauer for their data is not nearly as bad as reported. Finally, data for charge-balance vs. Mössbauer spectroscopy are compared and show that the EPMA/charge-balance approach provides a suitable alternative when other methods are not practical.

58 GEOSCIENCES↗

Raw soil carbon dioxide, moisture, temperature and micrometeorological data in the East River Watershed, Colorado June 2021-June 2024. (DE-SC0021139)

This dataset contains raw data from four tripod stations along an elevation gradient on Snodgrass Mountain in the East River Watershed, CO, USA. Each station contains a datalogger connected to 3 soil Carbon Dioxide CO2 gas probes, 3 soil temperature/moisture sensors and a micrometeorological station. Sensors are scanned every minute, and the 30 minute average is reported. The file snodgrass_soil_ESS.csv contains raw data, a row of column descriptors, and units of measurements. some data processing and QA/QC was done to filter out data from sensors that went bad and extreme outliers. CO2 sensors that went bad were replaced with new sensors as soon as possible. This research was performed to investigate the ecohydrological linkages of belowground carbon processes in the East River watershed forested communities to better understand how these ecosystems will respond to a changing cold-season moisture input. This is the second version of this data set and was modified on 10/01/2024. The primary change in the data was the addition of data from the fall of 2022 to June of 2024. In addition, minor QA/QC was done to filter out data from sensors that went bad and extreme outliers. THe filtered data are now NA's in this data frame and primarily the CO2 sensors. Limited to no QA/QC has been done on the other environmental data. This is now the third version of the data set, and was modified 03/25/2026. The primary change in the data was the addition of data from the June of 2024 to December 2025. Further r QA/QC was done with the new data to filter out bad data from faulty sensors and extreme outliers. The filtered data are now NA's in this data frame and primarily the CO2 sensors. Limited to no QA/QC has been done on the other environmental data. ##This additional data was funded under DE-SC0024218( Responses of Plant and Microbial Respiration Sources to Changing Cold Season Climate Drivers in the East River Watershed)

54 ENVIRONMENTAL SCIENCES↗

A Graph Convolutional Network for Active Distribution System Anomaly Detection Considering Measurement Spatial-Temporal Correlations

The accuracy of distribution system state estimation may be significantly impacted by the existence of bad measure-ments and unexpected topology errors. This paper proposes a data-driven Graph Convolutional Network (GCN) for anomaly detection, including bad measurements and topology change events. Compared to many existing machine learning approaches, the proposed approach embeds both spatial-temporal measure-ment correlations, which allows us to detect and distinguish different anomalies. Numerical results carried out on the IEEE 37-node system demonstrate that the proposed-based method can obtain high accuracy in detecting bad data and topology changes as compared to other approaches, even in the presence of high PV penetrations.

active distribution system↗