Search NASA⌕ Search

SEARCH · Search NASA

Results for “false data”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 163 records · Page 9

Evaluating resilience of DNP3-controlled SCADA systems against event buffer flooding

The DNP3 protocol is widely used in SCADA systems (particularly electrical power) as a means of communicating observed sensor state information back to a control center. Typical architectures using DNP3 have a two level hierarchy, where a specialized data aggregator device receives observed state from devices within a local region, and the control center collects the aggregated state from the data aggregator. The DNP3 communication between control center and data aggregator is asynchronous with the DNP3 communication between data aggregator and relays; this leads to the possibility of completely filling a data aggregator's buffer of pending events, when a relay is compromised or spoofed and sends overly many (false) events to the data aggregator. This paper investigates how a real-world SCADA device responds to event buffer flooding. A Discrete-Time Markov Chain (DTMC) model is developed for understanding this. The DTMC model is validated by a Moebius simulation model and data collected on real SCADA testbed.

42 ENGINEERING↗

Verification, Validation, and Calibration Through a Causal Lens

While typical validation and verification approaches focus on identifying the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on identifying causal relationships between data elements. Statistical and machine-learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between data sets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify, and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles, it is known as a directed acyclic graph. A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts can identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗

VALIDATION, VERIFICATION, AND CALIBRATION THROUGH A CAUSAL LENS

This paper presents an alternative method based on causal inference to perform validation, verification, and calibration of simulation models. While classical validation and verification approaches focus on the identification of the associations between data elements using statistical and machine learning methods, the novel methods in this paper focus instead on the identification of causal relationships between data elements. Statistical and machine learning-based approaches are strictly data-driven, meaning that they provide quantitative comparison measures between datasets without explicitly considering the hypotheses behind them. This can lead to the erroneous conclusion that, if two data sets are close enough, then the models that generated them are similar. In addition, when experimental and simulated data differ to an extent that fails to meet the acceptance criteria, calibration techniques are used to tweak simulation model parameters to reduce the gap between the two types of data. This produces the false expectation that a simulation model will match reality. The methods presented in this paper move away from these strictly data-driven methods for validation and calibration toward more robust, model-driven methods based on causal inference. Causal inference aims to identify the possible mechanisms that might have generated data. Thus, this analysis targets the prediction of the effects when one (or more) of the identified mechanisms are altered. There are many approaches to identify, quantify and illustrate causal relationships. For the scope of this paper, directed graphs are employed as causal models. If the directed graph lacks cycles it is known as a directed acyclic graph (DAG). A node in such a graph represents an observed data element while a directed edge connecting two nodes represents a causal relationship between two variables. The developed causal methods are designed to extract causal models from simulation models and from experimental data. Causal models capture the causal relationships between data elements (e.g., simulated and experimental data). In this context, validation and verification are performed by comparing causal models. The proposed approach does not only inform system analysts on how a simulation model matches real-world data, but also identifies elements of the simulation model that should be revised when discrepancies between simulation and experimental data are observed. Through these causal methods, analysts have a means to identify the portion of the model equation(s) that are behind an edge connecting two variables. Hence, once the structural differences between causal models have been determined, model calibration can occur by changing only those model parameters that impact the identified causal relationships.

97 MATHEMATICS AND COMPUTING↗

Timestamp Shift Detection for Synchrophasor Data Based on Similarity Analysis between Relative Phase Angle and Frequency

Timestamp Shift (TS) in synchrophasor data can lead to false operation of synchrophasor-based wide-area monitoring, protection and control applications. The letter presents a method to detect TS of synchrophasor data based on similarity analysis between its own relative phase angle and frequency. Here, a linear correlation component, caused by TS, between relative phase angle and measured frequency is discovered via theoretical derivation. Then a detection method is developed utilizing this linear component for TS diagnosis. Firstly, system phase angle is leveraged as angle reference and then relative phase angle is calculated. Secondly, the correlation coefficient between relative phase angle and frequency is calculated for similarity analysis. Thirdly, the TS is detected via threshold criterion and corresponding time error is estimated. The effectiveness of this method is validated via experiments with actual synchrophasor data in FNET/Grideye.

97 MATHEMATICS AND COMPUTING↗

THE THIRD SWIFT BURST ALERT TELESCOPE GAMMA-RAY BURST CATALOG

To date, the Burst Alert Telescope (BAT) onboard Swift has detected ∼1000 gamma-ray bursts (GRBs), of which ∼360 GRBs have redshift measurements, ranging from z = 0.03 to z = 9.38. We present the analyses of the BAT-detected GRBs for the past ∼11 years up through GRB 151027B. We report summaries of both the temporal and spectral analyses of the GRB characteristics using event data (i.e., data for each photon within approximately 250 s before and 950 s after the BAT trigger time), and discuss the instrumental sensitivity and selection effects of GRB detections. We also explore the GRB properties with redshift when possible. The result summaries and data products are available at http://swift.gsfc.nasa.gov/results/batgrbcat/index.html. In addition, we perform searches for GRB emissions before or after the event data using the BAT survey data. We estimate the false detection rate to be only one false detection in this sample. There are 15 ultra-long GRBs (∼2% of the BAT GRBs) in this search with confirmed emission beyond ∼1000 s of event data, and only two GRBs (GRB 100316D and GRB 101024A) with detections in the survey data prior to the starting of event data.

79 ASTRONOMY AND ASTROPHYSICS↗

Contamination in the Kepler field. Identification of 685 KOIs as false positives via ephemeris matching based ON Q1-Q12 data

The Kepler mission has to date found almost 6000 planetary transit-like signals, utilizing three years of data for over 170,000 stars at extremely high photometric precision. Due to its design, contamination from eclipsing binaries, variable stars, and other transiting planets results in a significant number of these signals being false positives (FPs). This directly affects the determination of the occurrence rate of Earth-like planets in our Galaxy, as well as other planet population statistics. In order to detect as many of these FPs as possible, we perform ephemeris matching among all transiting planet, eclipsing binary, and variable star sources. We find that 685 Kepler Objects of Interest (KOIs)—12% of all those analyzed—are FPs as a result of contamination, due to 409 unique parent sources. Of these, 118 have not previously been identified by other methods. We estimate that ∼35% of KOIs are FPs due to contamination, when performing a first-order correction for observational bias. Comparing single-planet candidate KOIs to multi-planet candidate KOIs, we find an observed FP fraction due to contamination of 16% and 2.4% respectively, bolstering the existing evidence that multi-planet KOIs are significantly less likely to be FPs. We also analyze the parameter distributions of the ephemeris matches and derive a simple model for the most common type of contamination in the Kepler field. We find that the ephemeris matching technique is able to identify low signal-to-noise FPs that are difficult to identify with other vetting techniques. We expect FP KOIs to become more frequent when analyzing more quarters of Kepler data, and note that many of them will not be able to be identified based on Kepler data alone.

79 ASTRONOMY AND ASTROPHYSICS↗

Assimilation of Time-Averaged Pseudoproxies for Climate Reconstruction

The efficacy of a novel ensemble data assimilation (DA) technique is examined in the climate field reconstruction (CFR) of surface temperature. A minimalistic, computationally inexpensive DA technique is employed that requires only a static ensemble of climatologically plausible states. Pseudoproxy experiments are performed with both general circulation model (GCM) and Twentieth Century Reanalysis (20CR) data by reconstructing surface temperature fields from a sparse network of noisy pseudoproxies. The DA approach is compared to a conventional CFR approach based on principal component analysis (PCA) for experiments on global domains. DA outperforms PCA in reconstructing global-mean temperature in all experiments and is more consistent across experiments, with a range of time series correlations of 0.69–0.94 compared to 0.19–0.87 for the PCA method. DA improvements are even more evident in spatial reconstruction skill, especially in sparsely sampled pseudoproxy regions and for 20CR experiments. It is hypothesized that DA improves spatial reconstructions because it relies on coherent, spatially local temperature patterns, which remain robust even when glacial states are used to reconstruct nonglacial states and vice versa. These local relationships, as utilized by DA, appear to be more robust than the orthogonal patterns of variability utilized by PCA. Comparing results for GCM and 20CR data indicates that pseudoproxy experiments that rely solely on GCM data may give a false impression of reconstruction skill.

54 ENVIRONMENTAL SCIENCES↗

A Comparison of the Detection Sensitivity of the Poisson Clutter Split and Region of Interest Algorithms on the RadMAP Mobile System

This analysis uses source injection into background data collected by the Radiological Multi-sensor Analysis Platform (RadMAP) to characterize the performance of the Poisson Clutter Split algorithm and compare it with a region-of-interest algorithm. This comparison is performed for varying detector array sizes and false alarm rates using data from Sodium Iodide and High Purity Germanium detector arrays. The application of the Poisson Clutter Split algorithm is found to yield significant performance gains for both medium- and high-resolution detector arrays. Furthermore, trade-offs between energy resolution, array size, cost, and detection performance are explored. In doing so, it is shown that the choice of detection algorithm is a key factor in determining the overall system performance and should be an important consideration in system design.

Joshi, T. H.↗

Subspace-Driven Learning for Anomaly Detection in Process Transients

Nuclear power plant (NPP) monitoring and diagnostic centers are actively investigating and implementing automated anomaly detection algorithms to help plants catch anomalies sooner, thereby preventing or reducing the duration of unexpected shutdowns. Current machine learning-based anomaly detection methods are expected to be highly effective during stable, full-power operations because NPPs typically operate as baseload power generators, meaning there are extensive operating data available from plant equipment. However, it is expected that anomaly detection methods will face significant challenges during transient conditions (i.e., when power output falls below full power) because plants only occasionally operate at these lower power levels, generating sparse transient operational data, and resulting in false alarms or missed detections. Here, to address this issue, transfer learning is used, which for this problem leverages knowledge (in the form of learned features) from stable, full-power operations to improve detection accuracy during transient conditions, even with limited data. In this effort, a novel subspace approach is developed to transfer a subset of the data features from full power operation to transients. This approach is validated through experiments using synthetic data and was found to outperform two baseline transfer learning approaches in anomaly detection performance across a range of amounts of transient data used in the training process.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

Rolling Root Mean Square Based Multimodal Anomaly Detection for Real Time Monitoring of Smart Grid

Reliable real-time monitoring is valuable for maintaining the operational integrity of modern electrical smart grids. Deployment of heterogeneous sensing technologies in substations has enabled high-resolution, multichannel waveform monitoring, but also introduces challenges for anomaly detection due to noise, baseline drift, and modality-dependent signal characteristics. In this work, we present a computationally efficient unsupervised method for multimodal event detection based on Rolling Root Mean Square based Event Detection (RRMSED). The method is developed using in-house, field deployed sensors collecting data at a utility substation. The sensing system comprises voltage and current sensors, triaxial accelerometers, and magnetometers, collectively capturing electrical, vibrational, and magnetic waveform measurements at high temporal resolution. RRMSED operates by extracting rolling RMS energy features and their first-order temporal differences from consecutive waveform segments for each channel and then applying channel-specific statistical thresholds learned from historical data. A persistence-based exceedance logic is employed to robustly identify transient events while suppressing impulsive noise, and to provide precise temporal localization with high resolution. The framework is designed for continuous server-side operation and can be deployed in real time without requiring complex models. Experiments on simulated waveform data with known ground truth demonstrate low false positive (FP) and false negative (FN) rates. Application to real substation data shows RRMSED to identify events that are not captured by conventional monitoring indicators including fast transient detection algorithm currently deployed in the system. These results indicate that rolling RMS based features provide an effective and practical basis for real-time multimodal event detection in smart-grid substations.

Mukherjee, Subrata [ORNL] (ORCID:0000000309930338)↗

The Use of a Multichannel Analyzer to Investigate Effects of Experimental Factors on Gross-Counting Gamma and Neutron Detectors

Radiation detection technology is invaluable to many fields of study in identifying nuclear materials. However, many detectors use gross-counting methods that give only a relative count rate. Without a spectrum (information on counts over time vs energy), it can be more difficult to discern if an alarm is false, innocent, or real. In particular, we would like to understand better the effect of experimental factors (i.e., external conditions and equipment parameters) on detector data, with possible implications for false alarms. To more thoroughly characterize detector technology, a multichannel analyzer (MCA) was used to record spectra from neutron (helium-3 tubes) and gamma (photomultiplier tubes) gross-counting detectors. Several factors could affect the signal-to-noise ratio of sources. For example, we examined the effects of neutron detector high voltage setting on the appearance of a californium-252 spectrum, the effect of discriminator values on integrated counts in neutron detection, and the effect of gain changes on the gamma spectra from several sources. Possible implications of ambient temperature of the experiment on the data collected will be discussed. The input impedance of the MCA must be considered to ensure that data are not affected by the measurement itself. Moreover, a calibration on the MCA was performed to verify the conversion of a MCA channel number to a voltage. In summary, the series of source spectra collected on an MCA with a variety of experimental conditions allow us to understand factors that affect data better, and assure us that gross-counting neutron and gamma detectors will have minimal false alarms.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

RADIOISOTOPE IDENTIFICATION OF SHIELDED AND MASKED SNM RDD MATERIALS

Sonar and speech techniques have been investigated to improve functionality and enable handheld and other man-portable, mobile, and portal systems to positively detect and identify illicit nuclear materials, with minimal data and with minimal false positives and false negatives. RadSonar isotope detection and identification is an algorithm development project funded by NA-22 and employing the resources of Savannah River National Laboratory and three University Laboratories (JHU-APL, UT-ARL, and UW-APL). Algorithms have been developed that improve the probability of detection and decrease the number of false positives and negatives. Two algorithms have been developed and tested. The first algorithm uses support vector machine (SVM) classifiers to determine the most prevalent nuclide(s) in a spectrum. It then uses a constrained weighted least squares fit to estimate and remove the contribution of these nuclide(s) to the spectrum, iterating classification and fitting until there is nothing of significance left. If any Special Nuclear Materials (SNMs) were detected in this process, a second tier of more stringent classifiers are used to make the final SNM alert decision. The second algorithm is looking at identifying existing feature sets that would be relevant in the radioisotope identification context. The underlying philosophy here is to identify parallels between the physics and/or the structures present in the data for the two applications (speech analysis and gamma spectroscopy). The expectation is that similar approaches may work in both cases. The mel-frequency cepstral representation of spectra is widely used in speech, particularly for two reasons: approximation of the response of the human ear, and simplicity of channel effect separation (in this context, a 'channel' is a method of signal transport that affects the signal, examples being vocal tract shape, room echoes, and microphone response). Measured and simulated gamma-ray spectra from a hand-held Radioisotope Identification Device were used to evaluate the algorithms. This paper will present and discuss results of the Test and Evaluation performed on two algorithms produced from the project.

46 INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AND ↗

A Statistical Framework for Operational lnfrasound Monitoring [Slides]

Historically, much infrasound research has been event-based, with known events providing the ground-truth required to inform the search for associated signals. For monitoring nuclear explosions in the atmosphere or underground, reliable techniques are required for operational infrasound data processing that effectively minimize both false alarms and missed detections. A difficulty in working with infrasound data is to adequately account for dynamic variations in the atmosphere, which occur over a variety of temporal and spatial scales. However, contrary to one's intuition, it is not necessary to have a high fidelity atmospheric model in order to perform robust detection, association and location. We present detection, association and location algorithms that assume no a priori knowledge of the atmosphere; however, where knowledge is available we show how it can be used to reduce the associated uncertainties. Rather than exporting seismic techniques to the problem of infrasound monitoring, our techniques were developed from the ground-up to properly account for the unique considerations of atmospheric acoustics. In contrast with other routinely used infrasound detectors, we utilize a contextual detection hypothesis that adaptively accounts for temporally variable correlated ambient noise. Our location scheme allows for the location of infrasound events without any meteorological data through a Bayesian framework. We demonstrate the application of the techniques presented here for a series of ground-truth events, and for routine data processing of large amounts of regional infrasound data.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗

ASSESSING THE SIGNIFICANCE OF APPARENT CORRELATIONS BETWEEN RADIO AND GAMMA-RAY BLAZAR FLUXES

Whether or not a correlation exists between the radio and gamma-ray flux densities of blazars is a long-standing question, and one that is difficult to answer confidently because of various observational biases, which may either dilute or apparently enhance any intrinsic correlation between radio and gamma-ray luminosities. We introduce a novel method of data randomization to evaluate quantitatively the effect of these biases and to assess the intrinsic significance of an apparent correlation between radio and gamma-ray flux densities of blazars. The novelty of the method lies in a combination of data randomization in luminosity space (to ensure that the randomized data are intrinsically, and not just apparently, uncorrelated) and significance assessment in flux space (to explicitly avoid Malmquist bias and automatically account for the limited dynamical range in both frequencies). The method is applicable even to small samples that are not selected with strict statistical criteria. For larger samples we describe a variation of the method in which the sample is split in redshift bins, and the randomization is applied in each bin individually; this variation is designed to yield the equivalent to luminosity-function sampling of the underlying population in the limit of very large, statistically complete samples. We show that for a smaller number of redshift bins, the method yields a worse significance, and in this way it is conservative: although it may fail to confirm an existing intrinsic correlation in a small sample that cannot be split into many redshift bins, it will not assign a stronger, artificially enhanced significance. We demonstrate how our test performs as a function of number of sources, strength of correlation, and number of redshift bins used, and we show that while our test is robust against common-distance biases and associated false positives for uncorrelated data, it retains the power of other methods in rejecting the null hypothesis of no correlation for correlated data.

79 ASTRONOMY AND ASTROPHYSICS↗

Analysis of Fire Calls to an Industrial Complex Over a 12-year Period

This paper gives an analysis of fire calls from over a decade of operations at a process facility. These data include valid alarms, unwanted or false alarms, ambulance calls, and hazardous material cleanup calls. Of special interest are false alarms, which are not only a nuisance to facility productivity but are detrimental to public and facility safety. Of the fire calls listed here, over half were from false alarms. The results given are compared to National Fire Protection Association data. The data presented can serve as exemplar data for future facilities and can be compared to other operating facilities’ experiences.

99 GENERAL AND MISCELLANEOUS↗

Improving Peptide-Level Mass Spectrometry Analysis via Double Competition

A critical statistical task in the analysis of shotgun proteomics data involves controlling the false discovery rate (FDR) among the reported set of discoveries. This task is most commonly solved at the peptide-spectrum match (PSM) level by using target-decoy competition (TDC), in which a set of observed spectra are searched against a database containing a mixture of real (target) and decoy peptides. The PSM-level procedure can be adapted to the peptide level by selecting the top-scoring PSM per peptide prior to FDR estimation. Here, we investigate both PSM-level and peptide-level FDR control methods and come to two conclusions. First, although the TDC procedure is provably correct under certain assumptions, we observe that one of these assumptions - that incorrect PSMs are independent of one another - is frequently violated. Hence, we empirically demonstrate that TDC-based PSM-level FDR estimates can be liberally biased. Further, we propose that researchers avoid PSM-level results and instead focus on peptide-level analysis. Second, we investigate three ways to carry out peptide level TDC and show that the most common method ("PSM-only") offers the lowest statistical power in practice. The most powerful method, peptide-level FDR with PSM competition ("PSM-and-peptide"), carries out competition first at the PSM level and then again at the peptide level. In our experiments, this approach yields an average increase of 17% more discovered peptides at a1% FDR threshold relative to the PSM-only method.

59 BASIC BIOLOGICAL SCIENCES↗

Interpreting internal consistency of DES measurements

ABSTRACT Bayesian evidence ratios are widely used to quantify the statistical consistency between different experiments. However, since the evidence ratio is prior dependent, the precise translation between its value and the degree of concordance/discordance requires additional information. The most commonly adopted metric, the Jeffreys scale, can falsely suggest agreement between data sets when priors are chosen to be sufficiently wide. This work examines evidence ratios in a DES-Y1 simulated analysis, focusing on the internal consistency between weak lensing and galaxy clustering. We study two scenarios using simulated data in controlled experiments. First, we calibrate the expected evidence ratio distribution given noise realizations around the best-fitting DES-Y1 ΛCDM cosmology. Secondly, we show the behaviour of evidence ratios for noiseless fiducial data vectors simulated using a modified gravity model, which generates internal tension in the LCDM analysis. We find that the evidence ratio of noise realizations generated at all confidence levels was biased towards agreement and show, with a modified gravity model, that the choice of prior could conceal the discrepancies between weak lensing and galaxy clustering induced by prior effects in unlike cosmological models, concluding that the evidence ratio in a DES-Y1 study is, indeed, biased towards agreement. Boundary effects can also influence conclusions about the inconsistency induced by modified gravity, even in a noiseless data vector simulation.

79 ASTRONOMY AND ASTROPHYSICS↗

Neural Network Approaches for Mobile Spectroscopic Gamma-Ray Source Detection

Artificial neural networks (ANNs) for performing spectroscopic gamma-ray source identification have been previously introduced, primarily for applications in controlled laboratory settings. To understand the utility of these methods in scenarios and environments more relevant to nuclear safety and security, this work examines the use of ANNs for mobile detection, which involves highly variable gamma-ray background, low signal-to-noise ratio measurements, and low false alarm rates. Simulated data from a 2” × 4” × 16” NaI(Tl) detector are used in this work for demonstrating these concepts, and the minimum detectable activity (MDA) is used as a performance metric in assessing model performance.In addition to examining simultaneous detection and identification, binary spectral anomaly detection using autoencoders is introduced in this work, and benchmarked using detection methods based on Non-negative Matrix Factorization (NMF) and Principal Component Analysis (PCA). On average, the autoencoder provides a 12% and 23% improvement over NMF- and PCA-based detection methods, respectively. Additionally, source identification using ANNs is extended to leverage temporal dynamics by means of recurrent neural networks, and these time-dependent models outperform their time-independent counterparts by 17% for the analysis examined here. The paper concludes with a discussion on tradeoffs between the ANN-based approaches and the benchmark methods examined here.

Bilton, Kyle J. (ORCID:0000000184553689)↗