Search NASA⌕ Search

SEARCH · Search NASA

Results for “false alarm rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 19 records
All sourcesNASA NTRSDOE OSTI
NASA NTRS1983

SETI Pulse Detection Algorithm: Analysis of False-alarm Rates

Some earlier work by the Search for Extraterrestrial Intelligence (SETI) Science Working Group (SWG) on the derivation of spectrum analyzer thresholds for a pulse detection algorithm based on an analysis of false alarm rates is extended. The algorithm previously analyzed was intended to detect a finite sequence of i periodically spaced pulses that did not necessarily occupy the entire observation interval. This algorithm would recognize the presence of such a signal only if all i-received pulse powers exceeded a threshold T(i): these thresholds were selected to achieve a desired false alarm rate, independent of i. To simplify the analysis, it was assumed that the pulses were synchronous with the spectrum sample times. This analysis extends the earlier effort to include infinite and/or asynchronous pulse trains. Furthermore, to decrease the possibility of missing an extraterrestrial intelligence signal, the algorithm was modified to detect a pulse train even if some of the received pulse powers fall below the threshold. The analysis employs geometrical arguments that make it conceptually easy to incorporate boundary conditions imposed on the derivation of the false alarm rates. While the exact results can be somewhat complex, simple closed form approximations are derived that produce a negligible loss of accuracy.

Levitt, B. K.↗
NASA NTRSDate not supplied

Monte Carlo Simulations for Evaluating the Accuracy of GLM Detection Efficiency and False Alarm Rate Retrievals

The performance of the Geostationary Lightning Mapper has been evaluated through comparison with other satellite lightning sensors and ground-based networks. However, because the performance of these reference sensors is both imperfect and imperfectly known, the true performance of GLM can only be estimated through such a process. In particular, GLM performance metrics such as detection efficiency (DE) and false alarm rate (FAR) retrieved through comparison with reference networks are affected by those networks’ own DE, FAR, and spatiotemporal accuracy, as well as the selected flash matching criteria. This study presents follow-on results from an ongoing analysis that uses a Monte Carlo simulation-based inversion technique to quantify the effect on retrieved GLM performance of a reasonable range of reference network DE, FAR, spatiotemporal accuracy, and the associated geographic patterns of each. These results help quantify and bound the actual performance of GLM and the attendant uncertainties when comparing GLM to imperfect reference networks.

Katrina Virts↗
DOE OSTI2021

Characterizing Human Performance: Detecting Targets at High False Alarm Rates [Slides]

Analysts develop a “no threat” bias with high false alarms. If only shown alarms for actual attacks, may never actually see an alarm. We see this in the laboratory, but not often studied in applied environments. (TSA is an exception.) In this work, near-operational paradigms are useful, but difficult to construct well. Pilot testing is critical before engaging time-limited professionals. Experimental control is difficult to balance with operational realism. Grounding near-operational experiments in basic research paradigms has both advantages and disadvantages. Despite shortcomings in our second experiment, we now have a platform for experimental investigations into the human element of physical security systems.

45 MILITARY TECHNOLOGY, WEAPONRY, AND NATIONAL DEF↗
NASA NTRS2006

Biochemical Detection and Identification False Alarm Rate Dependence on Wavelength Using Laser Induced Fluorescence

Most organic and many inorganic materials absorb strongly in specific wavelength ranges in the deep UV between about 220nm and 300nm. Excitation within these absorption bands results in native fluorescence emission. Each compound or composite material, such as a bacterial spore, has a unique excitation-emission fingerprint that can be used to provide information about the material. The sensitivity and specificity with which these materials can be detected and identified depends on the excitation wavelength and the number and location of observation wavelengths.We will present data on our deep ultraviolet Targeted Ultraviolet Chemical Sensors that demonstrate the sensitivity and specificity of the sensors. In particular, we will demonstrate the ability to quantitatively differentiate a wide range of biochemical agent targets against a wide range of background materials. We will describe the relationship between spectral resolution and specificity in target identification, as well as simple, fast, algorithms to identify materials.Hand-held, battery operated instruments using a deep UV laser and multi-band detection have been developed and deployed on missions to the Antarctic, the Arctic, and the deep ocean with the capability of detecting a single bacterial spore and to differentiate a wide range of organic and biological compounds.

native flourescence↗
NASA NTRS2001

Radiometry Measurements of Mars at 1064 nm Using the Mars Orbiter Laser Altimeter

Measurements by the Mars Orbiter Laser Altimeter (MOLA) on board the Mars Global Surveyor (MGS) may be used to provides a radiometric measurement of Mars in addition to the topographic measurement. We will describe the principle of operation, a mathematical model, and the receiver calibration in this presentation. MOLA was designed primarily to measure Mars topography, surface roughness end the bidirectional reflectance to the laser beam. To achieve the highest sensitivity the receiver detection threshold is dynamically adjusted to be as low as possible while keeping a predetermined false alarm rate. The average false alarm rate 29 monitored in real time on board MOLA via a noise counter, whose output is fed to the threshold control loop. The false alarm rate at a given threshold is a function of the detector output noise which is the sum of the photo detector, shot noise due to the background light seen by the detector and the dark noise. A mathematical model has been developed that can be used to numerically solve for the optical background power given the MOLA threshold setting and the average noise count. The radiance of Mars can then be determined by dividing the optical power by the solid angle subtended by the MOLA receiver, the receiver optical band-width, end the Mars surface area within the receiver field of view. The phase angle which is the sun-Mars-MOLA angle is available from the MGS database. MOLA also measures simultaneously the bidirectional reflectance of Mars vie its 106-lum loser beam at nadir with nearly zero phase angle. The optical bandwidth of the MOLA receiver is 2um full width at half maximum (FWHM) and centered at 106-lum. The receiver field of view is 0.95mrad FWHM. The nominated spacecraft altitude is 100km and the ground track speed is about 3km/s. Under normal operation, the noise counter are read and the threshold levels are updated at 1Hz. The receiver sensitivity is limited by the detector dark noise to about 0.1nW, which corresponds to less than 2% the maximum radiance during daytime from the brightest area on Mars. The results from the mathematical model agree well with the prelaunch measurements at several calibrated optical power levels. The radiance of sunlit Mars estimated with this technique correlates well with the measurement from the MGS. Thermal Emission Spectrometer (TES) and the Hubble Space Telescope at similar wavelength.

Sun, Xiao-Li↗
NASA NTRS1994

Synthetic aperture radar target detection, feature extraction, and image formation techniques

This report presents new algorithms for target detection, feature extraction, and image formation with the synthetic aperture radar (SAR) technology. For target detection, we consider target detection with SAR and coherent subtraction. We also study how the image false alarm rates are related to the target template false alarm rates when target templates are used for target detection. For feature extraction from SAR images, we present a computationally efficient eigenstructure-based 2D-MODE algorithm for two-dimensional frequency estimation. For SAR image formation, we present a robust parametric data model for estimating high resolution range signatures of radar targets and for forming high resolution SAR images.

Li, Jian↗
NASA NTRS2005

Global Positioning System (GPS) Precipitable Water in Forecasting Lightning at Spaceport Canaveral

Using meteorology data, focusing on precipitable water (PW), obtained during the 2000-2003 thunderstorm seasons in Central Florida, this paper will, one, assess the skill and accuracy measurements of the current Mazany forecasting tool and, two, provide additional forecasting tools that can be used in predicting lightning. Kennedy Space Center (KSC) and Cape Canaveral Air Force Station (CCAFS) are located in east Central Florida. KSC and CCAFS process and launch manned (NASA Space Shuttle) and unmanned (NASA and Air Force Expendable Launch Vehicles) space vehicles. One of the biggest cost impacts is unplanned launch scrubs due to inclement weather conditions such as thunderstorms. Each launch delay/scrub costs over a quarter million dollars, and the need to land the Shuttle at another landing site and return to KSC costs approximately $ 1M. Given the amount of time lost and costs incurred, the ability to accurately forecast (predict) when lightning will occur can result in significant cost and time savings. All lightning prediction models were developed using binary logistic regression. Lightning is the dependent variable and is binary. The independent variables are the Precipitable Water (PW) value for a given time of the day, the change in PW up to 12 hours, the electric field mill value, and the K-index value. In comparing the Mazany model results for the 1999 period B against actual observations for the 2000-2003 thunderstorm seasons, differences were found in the False Alarm Rate (FAR), Probability of Detection (POD) and Hit Rate (H). On average, the False Alarm Rate (FAR) increased by 58%, the Probability of Detection (POD) decreased by 31% and the Hit Rate decreased by 20%. In comparing the performance of the 6 hour forecast period to the performance of the 1.5 hour forecast period for the Mazany model, the FAR was lower by 15% and the Hit Rate was higher by 7%. However, the POD for the 6 hour forecast period was lower by 16% as compared to the POD of the 1.5 hour forecast period. Neither forecast period performed at the accuracy measures expected. A 2-Hr Forecasting Tool was developed to support a Phase I Lightning Advisory, which requires a 30-minute lead time for predicting lightning.

Kehrer, Kristen↗
NASA NTRS2001

Design for Vibration Monitoring: A Methodology for Reliable and Cost-Effective Vibration Monitoring

The purpose of health monitoring systems is to detect failures or defects for increased safety and performance and to provide on-condition maintenance with reduced costs. The problems associated with health monitoring systems include high rates of false alarms and missed failures, which make monitoring an unreliable and costly task. The reason for this is that unaccounted variations invalidate signal modeling assumptions. Our approach was to focus on vibration monitoring of rotating components. We analyzed baseline signals to determine statistical variations, identify and model factors that influence vibrations (pre-production vs. post-production variations), determine hit and false alarm rates with baseline flight data, model and predict effects of defects and variations on vibrations, and develop algorithms and metrics for failure and anomaly detection in the presence of variations.

Tumer, Irem Y.↗
DOE OSTI2022

A data-centric weak supervised learning for highway traffic incident detection

Using the data from loop detector sensors for near-real-time detection of traffic incidents on highways is crucial to averting major traffic congestion. While recent supervised machine learning methods offer solutions to incident detection by leveraging human-labeled incident data, the false alarm rate is often too high to be used in practice. Specifically, the inconsistency in the human labeling of the incidents significantly affects the performance of supervised learning models. To that end, we focus on a data-centric approach to improve the accuracy and reduce the false alarm rate of traffic incident detection on highways. We develop a weak supervised learning workflow to generate high-quality training labels for the incident data without the ground truth labels, and we use those generated labels in the supervised learning setup for final detection. This approach comprises three stages. First, we introduce a data preprocessing and curation pipeline that processes traffic sensor data to generate high-quality training data through leveraging labeling functions, which can be domain knowledge-related or simple heuristic rules. Second, we evaluate the training data generated by weak supervision using three supervised learning models-random forest, k-nearest neighbors, and a support vector machine ensemble-and long short-term memory classifiers. The results show that the accuracy of all of the models improves significantly after using the training data generated by weak supervision. Third, we develop an online real-time incident detection approach that leverages the model ensemble and the uncertainty quantification while detecting incidents. Finally, we show that our proposed weak supervised learning workflow achieves a high incident detection rate (0.90) and low false alarm rate (0.08).

97 MATHEMATICS AND COMPUTING↗
DOE OSTI2025

Real-Time, Adaptive Radiological Anomaly Detection and Isotope Identification Using Non-Negative Matrix Factorization

Spectroscopic anomaly detection and isotope identification algorithms are integral components in nuclear nonproliferation applications such as search operations. The task is especially challenging in the case of mobile detector systems because the observed gamma-ray background changes more than for a static detector system, and a pretrained background model can easily find itself out of domain. The result is that algorithms may exceed their intended false alarm rate or sacrifice detection sensitivity to maintain the desired false alarm rate. Non-negative matrix factorization (NMF) is a powerful tool for spectral anomaly detection and identification, but, like many similar algorithms that rely on data-driven background models, in its conventional implementation, it is unable to update in real time to account for environmental changes that affect the background spectroscopic signature. Here, we have developed a novel NMF-based algorithm that periodically updates its background model to accommodate changing environmental conditions. The adaptive NMF algorithm involves fewer assumptions about its environment, making it more generalizable than existing NMF-based methods while maintaining or exceeding detection performance on simulated and real-world datasets.

Anomaly detection↗
DOE OSTI2020

Leak detection in a subcritical boiler

Thermal power plants experience cycling duty leading to the fatigue of the boiler and heat exchanger tubes. As a result, tube failures occur frequently in coal fired fleets leading to forced outages. Furthermore, because the tube leaks have been the major source of unwanted shutdowns and the number of outages is increasing, present work focuses on the detection and isolation of the leak in a subcritical boiler based upon the process data from a commercial coalfired power plant. The mass balance equation around the steam drum was analyzed using timeseries data collected from a 300 MW power plant. The ratio of the feed water mass flow rate to the steam mass flow rate was defined as a key parameter for detecting leaks. The difference in slope between the feedwater and steam mass flow rate during the normal and faulty operations was established as the upper control limit for real time monitoring. To reduce false alarm rates that arise when raw signal is directly compared against the threshold due to common process fluctuations, an optimal filter was derived for smoothing. It was found that the optimal filter reacted much more quickly to process changes than an exponential moving average filter, around 8 h earlier on average. Occurrence of relatively high false alarm rates even in the filtered responses was related to the cycling of the boiler from the base load condition. Variable threshold was established to keep false alarm rates to the minimum while maintaining the leak detection rate. Finally, the leak was located at the economizer and this could readily be isolated by investigating the magnitude of the mass flow rates ratio and the temperature at the economizer outlet.

42 ENGINEERING↗
DOE OSTI2022

WISP: Watching grid Infrastructure Stealthily through Proxies (Final Technical Report)

The complex interdependencies of cyber systems (sensors and communications), physical grids and associated electricity market operations make protecting electric power grids a significant challenge. The energy sector is constantly under new, targeted, advanced and dangerous cyber-attacks that have the potential to result in the loss of human life. These threats are further exacerbated by our need to modernize the grid. One focus of cyber security research in smart grids is the securing of the SCADA system through advanced intrusion detection systems (IDS) and bad data detection algorithms in state estimation. These methods either require full knowledge of the system topology and parameters or fail to understand the physical behaviors under attack. WISP (Watching grid Infrastructure Stealthily through Proxies) is designed to provide additional protection to the power grid using only publicly available data. In particular, WISP exploits the spatio-temporal nature of the real time locational marginal prices (LMPs), in conjunction with other information such as bids, weather, outages and load data to analyze anomalous power pricing behaviors and then correlate those observations to localize regions of interest and identify potential cyber events. WISP is non-intrusive as the tool is deployed as a service in the Cloud or on premise and provides reliable information to system operators for enhanced situational awareness, without impeding energy delivery functions. The WISP technology comprises three modules: the data-driven anomaly detection core, the vulnerability and risk analysis and the root cause analysis. The data-driven anomaly detection core performs the tasks of feature selection, anomaly detection and attack region localization. The vulnerability and risk analysis module provides system level information of the vulnerable variables and times, assisting the operators in selecting monitoring and protection nodes. The root cause analysis module takes the detection results and identifies potential operational conditions that contribute to the detected anomalies. In Phase I, we have demonstrated the feasibility and effectiveness of WISP. We developed a realistic electricity market simulator capable of generating normal and attack market data under various operational conditions. We developed a series of cyber-attack detection and analysis algorithms and evaluated them under multiple data sources. Finally, we integrated all modules into an end-to-end software, providing functions for data management, data analytics and visualization. Specifically, we have achieved: (i) real-time data acceptance from external utility interfaces with >99% acceptance rate; (ii) high performance anomaly detection algorithms with >98% detection accuracy and <0.1% false alarm rate; and (iii) ultra-low computing delay <50 milliseconds. Additionally, our team developed algorithms to identify the vulnerable variables in electricity market operations and root cause analysis functions to identify major contributors to the price spikes. These ancillary modules are necessary when deploying WISP in real world industry environment. In Phase II, we have demonstrated the effectiveness of WISP software on realistic largescale power systems. We performed red team testing for the Phase I WISP software and identified software vulnerabilities and implemented corresponding mitigation solutions. We adapted the electricity market simulator for the Texas synthetic 2000-bus system and generated datasets for the false data injection attacks. We created database and visualization interfaces for the Texas system and the ISO New England system. We performed software optimization in terms of operation efficiency, computing speed and detection accuracy. Finally, we tested the software on the Texas system and the ISO New England system and evaluated the detection performance. Overall, we achieved above 89% detection rate, below 3% false alarm rate and below 37 seconds of end-to-end detection delay.

24 POWER TRANSMISSION AND DISTRIBUTION↗
NASA NTRS1998

Measures and Interpretations of Vigilance Performance: Evidence Against the Detection Criterion

Operators' performance in a vigilance task is often assumed to depend on their choice of a detection criterion. When the signal rate is low this criterion is set high, causing the hit and false alarm rates to be low. With increasing time on task the criterion presumably tends to increase even further, thereby further decreasing the hit and false alarm rates. Virtually all of the empirical evidence for this simple interpretation is based on estimates of the bias measure Beta from signal detection theory. In this article, I describe a new approach to studying decision making that does not require the technical assumptions of signal detection theory. The results of this new analysis suggest that the detection criterion is never biased toward either response, even when the signal rate is low and the time on task is long. Two modifications of the signal detection theory framework are considered to account for this seemingly paradoxical result. The first assumes that the signal rate affects the relative sizes of the variances of the information distributions; the second assumes that the signal rate affects the logic of the operator's stopping rule. Actual or potential applications of this research include the improved training and performance assessment of operators in areas such as product quality control, air traffic control, and medical and clinical diagnosis.

Balakrishnan, J. D.↗