SEARCH · Search NASA

Results for “false alarm rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 145 records · Page 8

All sources NASA NTRS DOE OSTI

DOE OSTI2024

Efficient Anomaly Detection Driven By Different Machine Learning Architectures And Models

The rapid growth and ubiquitous adoption of the internet and cyber-physical systems (CPS) have fundamentally transformed modern communication, work, and human-system interactions. While networks now form the backbone of critical digital ecosystems, enabling seamless data transmission across diverse, interconnected systems, this increased connectivity also expands the attack surface, making real-time detection of network intrusions and anomalies a pressing challenge. Detecting unusual activities within network infrastructure requires advanced data traffic analysis to differentiate between legitimate and malicious interactions. Traditional approaches to network anomaly detectionâ??such as rule-based and signature-based systemsâ??often depend on predefined patterns to identify known anomalies, limiting their effectiveness against emerging, stealthy, or previously unseen threats. These conventional methods suffer from high false alarm rates and fail to adapt to the ever-evolving nature of network traffic, particularly in large-scale, decentralized environments where data volume, velocity, and variety are constantly increasing. This dissertation presents artificial intelligence (AI)-driven approaches to anomaly detection that leverage graphics processing unit (GPU)-enabled high-performance computing (HPC) platforms for processing massive network traffic data and monitoring the components of cyber-physical systems (CPS) for potentially hazardous conditions. The research advances several key contributions: (1) Designing efficient machine learning techniques for CPS condition monitoring and anomaly detection; (2) enabling federated learning (FL) frameworks that enable distributed detection while preserving data privacy and system resilience; (3) exploring graph-based methodologies combining graph neural networks (GNN) and graph machine learning (ML) approaches for the Internet of Things (IoT) and automotive network security, and (4) performing distributed edge computing optimizations that integrate FL with scalable technologies for reduced communication overhead. Through extensive experiments, these methodologies demonstrate that complex anomaly detection and condition monitoring tasks can be achieved while balancing computational efficiency and detection accuracy through fine-grained network information processing. The frameworks developed in this research establish a robust foundation for network anomaly detection, providing scalable, adaptive, and privacy-preserving solutions for safeguarding CPS and IoT networks in an increasingly interconnected digital landscape. The practical implications of these research findings are significant, as they can inform the development of next-generation network security systems and contribute to the protection of critical infrastructure against sophisticated cyber attacks.

Marfo, William↗

DOE OSTI2020

Automated Vehicle Detection in a Nuclear Facility Using Low-Frequency Acoustic Sensors

This article presents an analysis of the method of construction and results for a classifier intended to identify vehicles using low-frequency acoustic data collected by a dis-tributed sensor network. This data is collected as part of a venture intended to explore data analytics and multisensor fusion techniques for the monitoring of activities at a test bed nuclear facility located at Oak Ridge National Laboratory in Oak Ridge, Tennessee. We describe the associated target signature and design a classifier based on a multilayer perceptron, followed by an analysis of its results. We discuss how overall accuracy is not the only consideration in constructing this classifier, and how for this application, it is actually desirable to operate at a lower level of accuracy in exchange for a reduction in the false alarm rate, as well as how this relates to the actual deployment of the classifier in practical use.

46 - INSTRUMENTATION RELATED TO NUCLEAR SCIENCE AN↗

NASA NTRS2001

Toward Failure Modeling In Complex Dynamic Systems: Impact of Design and Manufacturing Variations

When designing vehicle vibration monitoring systems for aerospace devices, it is common to use well-established models of vibration features to determine whether failures or defects exist. Most of the algorithms used for failure detection rely on these models to detect significant changes during a flight environment. In actual practice, however, most vehicle vibration monitoring systems are corrupted by high rates of false alarms and missed detections. Research conducted at the NASA Ames Research Center has determined that a major reason for the high rates of false alarms and missed detections is the numerous sources of statistical variations that are not taken into account in the. modeling assumptions. In this paper, we address one such source of variations, namely, those caused during the design and manufacturing of rotating machinery components that make up aerospace systems. We present a novel way of modeling the vibration response by including design variations via probabilistic methods. The results demonstrate initial feasibility of the method, showing great promise in developing a general methodology for designing more accurate aerospace vehicle vibration monitoring systems.

Tumer, Irem Y.↗

NASA NTRS2024

New Release Products: Combining the Relativistic Electron Alert System for Exploration With Other Early Indicators of Solar Energetic Particle Events

The Relativistic Electron Alert System for Exploration (“REleASE”) exploits the prompt arrival of near-relativistic electrons in the Earth/moon system to rapidly warn human explorers of the imminent arrival of hazardous energetic ions from solar energetic particle (SEP) events. Since 2008, the REleASE system has been implemented in near-real-time, initially with electron observations from SOHO, and since 2017 with additional observations from ACE as “HESPERIA REleASE”: upcoming opportunities will utilize IMAP and SWFO observations. REleASE is undergoing performance validation as part of the NASA/CCMC SEP scoreboarding process (https://ccmc.gsfc.nasa.gov/scoreboards/sep/). The ongoing return of STEREO-A to the vicinity of Earth provides observations of radio waves in near-real-time and allows REleASE to be combined with the identification of type-III radio bursts, which provide additional evidence of particle escape from the Sun (the use of type-III bursts for SEP forecasting was pioneered by “ESPERTA” developed by Laurenza et al., 2009). The aim of this planned “REleASE+” system is to further reduce the rate of false alarms. Another opportunity, facilitated by the CLEAR space weather center (PI L. Zhao, Univ. Michigan) will combine REleASE with “SEPSTER” (Richardson et al., 2018) to reduce the high rate of false alarms produced by SEPSTER, which is triggered by reports of CMEs, the vast majority of which do not result in detected SEP ion events. This presentation will give a status update on these objectives.

solar particle events↗

NASA NTRS2002

Towards Comprehensive Variation Models for Designing Vehicle Monitoring Systems

When designing vehicle vibration monitoring systems for aerospace devices, it is common to use well-established models of vibration features to determine whether failures or defects exist. Most of the algorithms used for failure detection rely on these models to detect significant changes in a flight environment. In actual practice, however, most vehicle vibration monitoring systems are corrupted by high rates of false alarms and missed detections. This crucial roadblock makes their implementation in real vehicles (e.g., helicopter transmissions and aircraft engines) difficult, making their operation costly and unreliable. Research conducted at the NASA Ames Research Center has determined that a major reason for the high rates of false alarms and missed detections is the numerous sources of statistical variations that are not taken into account in the modeling assumptions. In this paper, we address one such source of variations, namely, those caused during the design and manufacturing of rotating machinery components that make up aerospace systems. We present a novel way of modeling the vibration response by including design variations via probabilistic methods. Using such models, we develop a methodology to account for design and manufacturing variations, and explore the changes in the vibration response to determine its stochastic nature. We explore the potential of the methodology using a nonlinear cam-follower model, where the spring stiffness values are assumed to follow a normal distribution. The results demonstrate initial feasibility of the method, showing great promise in developing a general methodology for designing more accurate aerospace vehicle monitoring systems.

McAdams, Daniel A.↗

NASA NTRS2018

Verification of Real-Time WSA-ENLIL+Cone Simulations of CME Arrival-Time at the CCMC from 2010 to 2016

The Wang-Sheeley-Arge (WSA)-ENLIL+Cone model is used extensively in space weather operations world-wide to model coronal mass ejection (CME) propagation. As such, it is important to assess its performance. We present validation results of the WSA-ENLIL+Cone model installed at the Community Coordinated Modeling Center (CCMC) and executed in real-time by the CCMC space weather team. CCMC uses the WSA-ENLIL+Cone model to predict CME arrivals at NASA missions throughout the inner heliosphere. In this work we compare model predicted CME arrival-times to in situ interplanetary coronal mass ejection leading edge measurements at Solar TErrestrial RElations Observatory-Ahead (STEREO-A), Solar TErrestrial RElations Observatory-Behind (STEREO-B), and Earth (Wind and ACE) for simulations completed between March 2010 and December 2016 (over 1,800 CMEs). We report hit, miss, false alarm, and correct rejection statistics for all three locations. For all predicted CME arrivals, the hit rate is 0.5, and the false alarm rate is 0.1. For the 273 events where the CME was predicted to arrive at Earth, STEREO-A, or STEREO-B, and was actually observed (hit event), the mean absolute arrival-time prediction error was 10.4 +/- 0.9 h, with a tendency to early prediction error of -4.0 h. We show the dependence of the arrival-time error on CME input parameters. We also explore the impact of the multi-spacecraft observations used to initialize the model CME inputs by comparing model verification results before and after the STEREO-B communication loss (since September 2014) and STEREO-A sidelobe operations (August 2014- December 2015). There is an increase of 1.7 h in the CME arrival time error during single, or limited two-viewpoint periods, compared to the three-spacecraft viewpoint period. This trend would apply to a future space weather mission at L5 or L4 as another coronagraph viewpoint to reduce CME arrival time errors compared to a single L1 viewpoint.

MHD / modeling / validation / forecasting / corona↗

DOE OSTI2025

Radiation portal monitor data file format for comprehensive background radiation monitoring

Radiation portal monitors (RPMs) are widely used at border security checkpoints to detect the presence of radioactive materials in people, vehicles, and cargo. Typically, RPM detection systems consist of two pillars equipped with gamma and neutron detectors. To improve detection efficiency, RPMs employ techniques such as a limited energy window, dynamic alarm thresholds, and lead shielding. However, without continuous monitoring of background radiation, signal interpretation can be compromised, because environmental factors and mechanical failures can cause fluctuations. Here, we introduce a daily file format that logs gamma background and neutron background radiation levels continuously over a 24 h period; this format is different from traditional formats that record data only when the RPM is active or occupied. The approach enables RPM operators and analysts to (1) identify and diagnose malfunctioning components, (2) adjust system settings to account for dynamic environmental factors, and (3) use the recorded data to characterize outer space phenomena. Continuous background reporting is essential for identifying issues such as faulty connections, voltage divider failures, and errors in background updates. Continuous background reporting also enables the detection of external influences, including nearby X-ray scanners, temperature fluctuations, rainfall, cosmic radiation, and lunar phase changes. These data files are designed to be easily evaluated and parsed using common tools, and a quick review by an expert is often sufficient for problem diagnosis. We anticipate that continuous background radiation monitoring and these new strategies will significantly improve the accuracy and reliability of RPM systems, reducing the rate of false alarms and enhancing overall system performance.

Background radiation monitoring↗

NASA NTRS1983

Reliability analysis and fault-tolerant system development for a redundant strapdown inertial measurement unit

A methodology is developed and applied for quantitatively analyzing the reliability of a dual, fail-operational redundant strapdown inertial measurement unit (RSDIMU). A Markov evaluation model is defined in terms of the operational states of the RSDIMU to predict system reliability. A 27 state model is defined based upon a candidate redundancy management system which can detect and isolate a spectrum of failure magnitudes. The results of parametric studies are presented which show the effect on reliability of the gyro failure rate, both the gyro and accelerometer failure rates together, false alarms, probability of failure detection, probability of failure isolation, and probability of damage effects and mission time. A technique is developed and evaluated for generating dynamic thresholds for detecting and isolating failures of the dual, separated IMU. Special emphasis is given to the detection of multiple, nonconcurrent failures. Digital simulation time histories are presented which show the thresholds obtained and their effectiveness in detecting and isolating sensor failures.

Motyka, P.↗

NASA NTRS2001

Principal Components Analysis of Triaxial Vibration Data From Helicopter Transmissions

Research on the nature of the vibration data collected from helicopter transmissions during flight experiments has led to several crucial observations believed to be responsible for the high rates of false alarms and missed detections in aircraft vibration monitoring systems. This work focuses on one such finding, namely, the need to consider additional sources of information about system vibrations. In this light, helicopter transmission vibration data, collected using triaxial accelerometers, were explored in three different directions, analyzed for content, and then combined using Principal Components Analysis (PCA) to analyze changes in directionality. In this paper, the PCA transformation is applied to 176 test conditions/data sets collected from an OH58C helicopter to derive the overall experiment-wide covariance matrix and its principal eigenvectors. The experiment-wide eigenvectors. are then projected onto the individual test conditions to evaluate changes and similarities in their directionality based on the various experimental factors. The paper will present the foundations of the proposed approach, addressing the question of whether experiment-wide eigenvectors accurately model the vibration modes in individual test conditions. The results will further determine the value of using directionality and triaxial accelerometers for vibration monitoring and anomaly detection.

Tumer, Irem Y.↗

NASA NTRS2011

Optimization of Adaboost Algorithm for Sonar Target Detection in a Multi-Stage ATR System

JPL has developed a multi-stage Automated Target Recognition (ATR) system to locate objects in images. First, input images are preprocessed and sent to a Grayscale Optical Correlator (GOC) filter to identify possible regions-of-interest (ROIs). Second, feature extraction operations are performed using Texton filters and Principal Component Analysis (PCA). Finally, the features are fed to a classifier, to identify ROIs that contain the targets. Previous work used the Feed-forward Back-propagation Neural Network for classification. In this project we investigate a version of Adaboost as a classifier for comparison. The version we used is known as GentleBoost. We used the boosted decision tree as the weak classifier. We have tested our ATR system against real-world sonar images using the Adaboost approach. Results indicate an improvement in performance over a single Neural Network design.

Adaboost↗

NASA NTRS2013

Aquarius RFI Detection and Mitigation Algorithm: Assessment and Examples

Aquarius is an L-band radiometer system designed to map sea surface salinity from space. This is a sensitive measurement, and protection from radio frequency interference (RFI) is important for success. An initial look at the performance of the Aquarius RFI detection and mitigation algorithm is reported together with examples of the global distribution of RFI at the L-band. To protect against RFI, Aquarius employs rapid sampling (10 ms) and a "glitch" detection algorithm that looks for outliers among the samples. Samples identified as RFI are removed, and the remainder is averaged to produce an RFI-free signal for the salinity retrieval algorithm. The RFI detection algorithm appears to work well over the ocean with modest rates for false alarms (5%) and missed detection. The global distribution of RFI coincides well with population centers and is consistent with observations reported by the Soil Moisture and Ocean Salinity mission.

remote sensing↗

NASA NTRS2012

Current-State Constrained Filter Bank for Wald Testing of Spacecraft Conjunctions

We propose a filter bank consisting of an ordinary current-state extended Kalman filter, and two similar but constrained filters: one is constrained by a null hypothesis that the miss distance between two conjuncting spacecraft is inside their combined hard body radius at the predicted time of closest approach, and one is constrained by an alternative complementary hypothesis. The unconstrained filter is the basis of an initial screening for close approaches of interest. Once the initial screening detects a possibly risky conjunction, the unconstrained filter also governs measurement editing for all three filters, and predicts the time of closest approach. The constrained filters operate only when conjunctions of interest occur. The computed likelihoods of the innovations of the two constrained filters form a ratio for a Wald sequential probability ratio test. The Wald test guides risk mitigation maneuver decisions based on explicit false alarm and missed detection criteria. Since only current-state Kalman filtering is required to compute the innovations for the likelihood ratio, the present approach does not require the mapping of probability density forward to the time of closest approach. Instead, the hard-body constraint manifold is mapped to the filter update time by applying a sigma-point transformation to a projection function. Although many projectors are available, we choose one based on Lambert-style differential correction of the current-state velocity. We have tested our method using a scenario based on the Magnetospheric Multi-Scale mission, scheduled for launch in late 2014. This mission involves formation flight in highly elliptical orbits of four spinning spacecraft equipped with antennas extending 120 meters tip-to-tip. Eccentricities range from 0.82 to 0.91, and close approaches generally occur in the vicinity of perigee, where rapid changes in geometry may occur. Testing the method using two 12,000-case Monte Carlo simulations, we found the method achieved a missed detection rate of 0.1%, and a false alarm rate of 2%.

Carpenter, J. Russell↗

NASA NTRS2005

Aircraft Fault Detection and Classification Using Multi-Level Immune Learning Detection

This work is an extension of a recently developed software tool called MILD (Multi-level Immune Learning Detection), which implements a negative selection algorithm for anomaly and fault detection that is inspired by the human immune system. The immunity-based approach can detect a broad spectrum of known and unforeseen faults. We extend MILD by applying a neural network classifier to identify the pattern of fault detectors that are activated during fault detection. Consequently, MILD now performs fault detection and identification of the system under investigation. This paper describes the application of MILD to detect and classify faults of a generic transport aircraft augmented with an intelligent flight controller. The intelligent control architecture is designed to accommodate faults without the need to explicitly identify them. Adding knowledge about the existence and type of a fault will improve the handling qualities of a degraded aircraft and impact tactical and strategic maneuvering decisions. In addition, providing fault information to the pilot is important for maintaining situational awareness so that he can avoid performing an action that might lead to unexpected behavior - e.g., an action that exceeds the remaining control authority of the damaged aircraft. We discuss the detection and classification results of simulated failures of the aircraft's control system and show that MILD is effective at determining the problem with low false alarm and misclassification rates.

Wong, Derek↗

NASA NTRS2005

Design Methods and Practices for Fault Prevention and Management in Spacecraft

Integrated Systems Health Management (ISHM) is intended to become a critical capability for all space, lunar and planetary exploration vehicles and systems at NASA. Monitoring and managing the health state of diverse components, subsystems, and systems is a difficult task that will become more challenging when implemented for long-term, evolving deployments. A key technical challenge will be to ensure that the ISHM technologies are reliable, effective, and low cost, resulting in turn in safe, reliable, and affordable missions. To ensure safety and reliability, ISHM functionality, decisions and knowledge have to be incorporated into the product lifecycle as early as possible, and ISHM must be considered as an essential element of models developed and used in various stages during system design. During early stage design, many decisions and tasks are still open, including sensor and measurement point selection, modeling and model-checking, diagnosis, signature and data fusion schemes, presenting the best opportunity to catch and prevent potential failures and anomalies in a cost-effective way. Using appropriate formal methods during early design, the design teams can systematically explore risks without committing to design decisions too early. However, the nature of ISHM knowledge and data is detailed, relying on high-fidelity, detailed models, whereas the earlier stages of the product lifecycle utilize low-fidelity, high-level models of systems and their functionality. We currently lack the tools and processes necessary for integrating ISHM into the vehicle system/subsystem design. As a result, most existing ISHM-like technologies are retrofits that were done after the system design was completed. It is very expensive, and sometimes futile, to retrofit a system health management capability into existing systems. Last-minute retrofits result in unreliable systems, ineffective solutions, and excessive costs (e.g., Space Shuttle TPS monitoring which was considered only after 110 flights and the Columbia disaster). High false alarm or false negative rates due to substandard implementations hurt the credibility of the ISHM discipline. This paper presents an overview of the current state of ISHM design,and a review of formal design methods to make recommendations about possible approaches to enable the ISHM capabilities to be designed-in at the system-level, from the very beginning of the vehicle design process.

Tumer, Irem Y.↗

NASA NTRS2017

Structured Light-Based Hazard Detection For Planetary Surface Navigation

This paper describes a structured light-based sensor for hazard avoidance in planetary environments. The system presented here can also be used in terrestrial applications constrained by reduced onboard power and computational complexity and low illumination conditions. The sensor is on a calibrated camera and laser dot projector system. The onboard hazard avoidance system determines the position of the projected dots in the image and through a triangulation process detects potential hazards. The paper presents the design parameters for this sensor and describes the image based solution for hazard avoidance. The system presented here was tested extensively in day and night conditions in Lunar analogue environments. The current system achieves over 97 detection rate with 1.7 false alarms over 2000 images.

Nefian, Ara↗

NASA NTRS1990

Current Emergency Locator Transmitter (ELT) deficiencies and potential improvements utilizing TSO-C91a ELTs

An analysis was conducted of current ELT problems and potential improvements that could be made by employing the TSO-C91a ELTs to replace the current TSO-C91 ELTs. The scope of the study included the following: (1) validate the problems; (2) determine specific failure causes; (3) determine false alarm causes; (4) estimate improvements from TSO-C91a; (5) estimate benefits from replacement of the current ELTs; and (6) determine need and benefits for improved ELT inspection and maintenance. A detailed comparison between the two requirements documents (TSO-C91 and -91a) was made to assess improved performance of the ELT in each category of failure cause and each cause of false alarms. The comparison and analysis resulted in projecting a success of operation rate approximately 3 times the current rate and a reduction in false alarms to 0.25 of those generated by TSO-C91 ELTs. These improvements led to a projection of benefits of approximately 25 additional lives to be saved each year with TSO-C91a ELTs and an improved inspection and maintenance program.

Trudell, Bernard J.↗

NASA NTRS1991

User acceptance of intelligent avionics: A study of automatic-aided target recognition

User acceptance of new support systems typically was evaluated after the systems were specified, designed, and built. The current study attempts to assess user acceptance of an Automatic-Aided Target Recognition (ATR) system using an emulation of such a proposed system. The detection accuracy and false alarm level of the ATR system were varied systematically, and subjects rated the tactical value of systems exhibiting different performance levels. Both detection accuracy and false alarm level affected the subjects' ratings. The data from two experiments suggest a cut-off point in ATR performance below which the subjects saw little tactical value in the system. An ATR system seems to have obvious tactical value only if it functions at a correct detection rate of 0.7 or better with a false alarm level of 0.167 false alarms per square degree or fewer.

Becker, Curtis A.↗

DOE OSTI2021

Alfvén eigenmode classification based on ECE diagnostics at DIII-D using deep recurrent neural networks

Modern tokamaks have achieved significant fusion production, but further progress towards steady-state operation has been stymied by a host of kinetic and MHD instabilities. Control and identification of these instabilities is often complicated, warranting the application of data-driven methods to complement and improve physical understanding. In particular, Alfvén eigenmodes are a class of ubiquitous mixed kinetic and MHD instabilities that are important to identify and control because they can lead to loss of confinement and potential damage to the walls of a plasma device. In the present work, we use reservoir computing networks to classify Alfvén eigenmodes in a large labeled database of DIII-D discharges, covering a broad range of operational parameter space. However, despite the large parameter space, we show excellent classification and prediction performance, with an average hit rate of 91% and false alarm ratio of 7%, indicating promise for future implementation with additional diagnostic data and consolidation into a real-time control strategy.

70 PLASMA PHYSICS AND FUSION TECHNOLOGY↗