SEARCH · Search NASA

Results for “False Positive Rate”

Search indexed NASA NTRS and DOE OSTI research on propulsion, heat transfer, battery materials and energy systems. Follow report and document links to the original sources.

SourceResearch connectionFrom yearThrough year

Quote a phrase for an exact phrase match. Source license links do not imply unrestricted reuse.

At least 37 records · Page 2

All sources NASA NTRS DOE OSTI

NASA NTRS2010

Multi-Stage System for Automatic Target Recognition

A multi-stage automated target recognition (ATR) system has been designed to perform computer vision tasks with adequate proficiency in mimicking human vision. The system is able to detect, identify, and track targets of interest. Potential regions of interest (ROIs) are first identified by the detection stage using an Optimum Trade-off Maximum Average Correlation Height (OT-MACH) filter combined with a wavelet transform. False positives are then eliminated by the verification stage using feature extraction methods in conjunction with neural networks. Feature extraction transforms the ROIs using filtering and binning algorithms to create feature vectors. A feedforward back-propagation neural network (NN) is then trained to classify each feature vector and to remove false positives. The system parameter optimizations process has been developed to adapt to various targets and datasets. The objective was to design an efficient computer vision system that can learn to detect multiple targets in large images with unknown backgrounds. Because the target size is small relative to the image size in this problem, there are many regions of the image that could potentially contain the target. A cursory analysis of every region can be computationally efficient, but may yield too many false positives. On the other hand, a detailed analysis of every region can yield better results, but may be computationally inefficient. The multi-stage ATR system was designed to achieve an optimal balance between accuracy and computational efficiency by incorporating both models. The detection stage first identifies potential ROIs where the target may be present by performing a fast Fourier domain OT-MACH filter-based correlation. Because threshold for this stage is chosen with the goal of detecting all true positives, a number of false positives are also detected as ROIs. The verification stage then transforms the regions of interest into feature space, and eliminates false positives using an artificial neural network classifier. The multi-stage system allows tuning the detection sensitivity and the identification specificity individually in each stage. It is easier to achieve optimized ATR operation based on its specific goal. The test results show that the system was successful in substantially reducing the false positive rate when tested on a sonar and video image datasets.

Chao, Tien-Hsin↗

NASA NTRS2012

Modeling and Detection of Ice Particle Accretion in Aircraft Engine Compression Systems

The accretion of ice particles in the core of commercial aircraft engines has been an ongoing aviation safety challenge. While no accidents have resulted from this phenomenon to date, numerous engine power loss events ranging from uneventful recoveries to forced landings have been recorded. As a first step to enabling mitigation strategies during ice accretion, a detection scheme must be developed that is capable of being implemented on board modern engines. In this paper, a simple detection scheme is developed and tested using a realistic engine simulation with approximate ice accretion models based on data from a compressor design tool. These accretion models are implemented as modified Low Pressure Compressor maps and have the capability to shift engine performance based on a specified level of ice blockage. Based on results from this model, it is possible to detect the accretion of ice in the engine core by observing shifts in the typical sensed engine outputs. Results are presented in which, for a 0.1 percent false positive rate, a true positive detection rate of 98 percent is achieved.

May, Ryan D.↗

NASA NTRS2009

Neural Network Target Identification System for False Alarm Reduction

ATR↗

NASA NTRS2021

Cloud Detection over Sunglint Regions with Observations from the Earth Polychromatic Imaging Camera (EPIC)

With the ability to observe the entire sunlit side of the Earth, EPIC data have become an important resource for studying cloud daily variability. Inaccurate cloud masking is a great source of uncertainty. One main region that is prone to error in cloud masking is the sunglint area over ocean surfaces. Cloud detection over these regions is challenging for the EPIC instrument because of its limited spectral channels. Clear sky ocean surface reflectance from visible channels over sunglint is much larger than that over the non-glint areas and can exceed reflectance from thin clouds. This paper presents an improved EPIC ocean cloud masking algorithm (Version 3). Over sunglint regions (glint angle ≤ 25°), the algorithm utilizes EPIC’s oxygen (O2) A-band ratio (764nm to 780 nm) in addition to the 780 nm reflectance observations in masking tests. Outside the sunglint regions, a dynamic reflectance threshold for the Rayleigh corrected 780 nm reflectance is applied. The thresholds are derived as a function of glint angle. When compared with co-located data from the geosynchronous Earth orbit (GEO) and the low Earth orbit (LEO) observations, the consistency of the new ocean cloud mask algorithm has increased by 4~10% and 4~6% in the glint center and granule edges respectively. The false positive rate is reduced by 10~17%. Overall global ocean cloud detection consistency increases by 2%. This algorithm, along with other improvements to the EPIC cloud masks, has been implemented in the EPIC cloud products Version 3. This algorithm will improve the cloud daily variability analysis by removing the artificial peak at local noon time in the glint center latitudes and reducing biases in the early morning and late afternoon cloud fraction over ocean surfaces.

EPIC↗

NASA NTRS2011

Daytime Water Detection Based on Sky Reflections

Robust water detection is a critical perception requirement for unmanned ground vehicle (UGV) autonomous navigation. This is particularly true in wide-open areas where water can collect in naturally occurring terrain depressions during periods of heavy precipitation and form large water bodies. One of the properties of water useful for detecting it is that its surface acts as a horizontal mirror at large incidence angles. Water bodies can be indirectly detected by detecting reflections of the sky below the horizon in color imagery. The Jet Propulsion Laboratory (JPL) has implemented a water detector based on sky reflections that geometrically locates the pixel in the sky that is reflecting on a candidate water pixel on the ground and predicts if the ground pixel is water based on color similarity and local terrain features. This software detects water bodies in wide-open areas on cross-country terrain at mid- to far-range using imagery acquired from a forward-looking stereo pair of color cameras mounted on a terrestrial UGV. In three test sequences approaching a pond under a clear, overcast, and cloudy sky, the true positive detection rate was 100% when the UGV was beyond 7 meters of the water's leading edge and the largest false positive detection rate was 0.58%. The sky reflection based water detector has been integrated on an experimental unmanned vehicle and field tested at Ft. Indiantown Gap, PA, USA.

sky reflections↗

NASA NTRS1981

The effect of abnormal cell proportion on specimen classifier performance

An analysis is presented of the results obtained from a cell classifier which is confronted with an abnormal/normal cell ratio which is different from the ratio assumed in the calibration of the classifier. False negative and false positive error rates are determined in advance for classifier operation, along with the necessary sample size in order to validate the predicted distributions. Changes are demonstrated to happen only regarding the false negative rate, where reductions in the abnormal cell rate below the expected rates would cause totally unreliable data. Substantial overproduction of abnormal cells would be quickly noticeable, while production rates beyond, but close to, the expected rates would only require more extensive sampling. Classifier systems for 10% proportions of abnormal cells are concluded to be possible, but difficulties are present with much lower rates

Castleman, K. R.↗

NASA NTRS2007

Characterization of Polar Stratospheric Clouds With Spaceborne Lidar: CALIPSO and the 2006 Antarctic Season

The role of polar stratospheric clouds in polar ozone loss has been well documented. The CALIPSO satellite mission offers a new opportunity to characterize PSCs on spatial and temporal scales previously unavailable. A PSC detection algorithm based on a single wavelength threshold approach has been developed for CALIPSO. The method appears to accurately detect PSCs of all opacities, including tenuous clouds, with a very low rate of false positives and few missed clouds. We applied the algorithm to CALIPSO data acquired during the 2006 Antarctic winter season from 13 June through 31 October. The spatial and temporal distribution of CALIPSO PSC observations is illustrated with weekly maps of PSC occurrence. The evolution of the 2006 PSC season is depicted by time series of daily PSC frequency as a function of altitude. Comparisons with virtual solar occultation data indicate that CALIPSO provides a different view of the PSC season than attained with previous solar occultation satellites. Measurement-based time series of PSC areal coverage and vertically-integrated PSC volume are computed from the CALIPSO data. The observed area covered with PSCs is significantly smaller than would be inferred from a temperature-based proxy such as TNAT but is similar in magnitude to that inferred from TSTS. The potential of CALIPSO measurements for investigating PSC microphysics is illustrated using combinations of lidar backscatter coefficient and volume depolarization to infer composition for two CALIPSO PSC scenes.

Pitts, Michael C.↗

NASA NTRS2008

Science requirements for LISA

Historically, gravitational wave antennas have been characterized by their detection capability. This is measured in terms of signal-to-noise ratio, and implies a rate of false positives and false negatives. But to do useful astrophysics, one would like to measure - or more properly, estimate - astrophysical parameters of the gravitational wave sources. In the interest of strengthening the connection between science objectives and a specific instrument performance, the LISA community has reformulated the Laser Interferometer Space Antenna (LlSA) science requirements around the anticipated uncertainty in astrophysical parameter estimation. The rationale for this characterization of LlSA and a summary of the astrophysics and fundamental physics that LISA can do will be given. LISA will be able to make precision measurements of sources out to z approximately equal to 10.

Stebbins, Robin T.↗

NASA NTRS2023

SafeAeroBERT: Towards a Safety-Informed Aerospace-Specific Language Model

As aviation systems continue to operate with high traffic, large amounts of documents containing safety-relevant data continue to be generated via reporting systems such as the ASRS. Advanced natural language processing techniques, specifically pre-trained language models, have shown great success in domain-specific applications; however, the text in aviation safety reports is inundated with jargon and thus not fully utilized by general pre-trained models. In this research, we work towards developing a safety-informed aerospace-specific language model by pre-training a Bidirectional Encoder Representations from Transformer (BERT) model on reports from the Aviation Safety Reporting System and the National Transportation Safety Board. The resulting model, called SafeAeroBERT, is fine-tuned for the specific task of document classification, and can be further tuned for named-entity recognition, relation detection, information retrieval, and summarization. Results from the classification task are compared between SafeAeroBERT, the base BERT, and SciBERT models and show SafeAeroBERT outperforms the general BERT and SciBERT on classifying reports about human factors, aircraft, and procedure. SafeAeroBERT can be used on custom tasks, not limited to document classification, and is intended to aid an intelligent knowledge manager for safety report repositories.

Aviation↗

NASA NTRS2010

Support for Systematic Code Reviews with the SCRUB Tool

SCRUB is a code review tool that supports both large, team-based software development efforts (e.g., for mission software) as well as individual tasks. The tool was developed at JPL to support a new, streamlined code review process that combines human-generated review reports with program-generated review reports from a customizable range of state-of-the-art source code analyzers. The leading commercial tools include Codesonar, Coverity, and Klocwork, each of which can achieve a reasonably low rate of false-positives in the warnings that they generate. The time required to analyze code with these tools can vary greatly. In each case, however, the tools produce results that would be difficult to realize with human code inspections alone. There is little overlap in the results produced by the different analyzers, and each analyzer used generally increases the effectiveness of the overall effort. The SCRUB tool allows all reports to be accessed through a single, uniform interface (see figure) that facilitates brows ing code and reports. Improvements over existing software include significant simplification, and leveraging of a range of commercial, static source code analyzers in a single, uniform framework. The tool runs as a small stand-alone application, avoiding the security problems related to tools based on Web browsers. A developer or reviewer, for instance, must have already obtained access rights to a code base before that code can be browsed and reviewed with the SCRUB tool. The tool cannot open any files or folders to which the user does not already have access. This means that the tool does not need to enforce or administer any additional security policies. The analysis results presented through the SCRUB tool s user interface are always computed off-line, given that, especially for larger projects, this computation can take longer than appropriate for interactive tool use. The recommended code review process that is supported by the SCRUB tool consists of three phases: Code Review, Developer Response, and Closeout Resolution. In the Code Review phase, all tool-based analysis reports are generated, and specific comments from expert code reviewers are entered into the SCRUB tool. In the second phase, Developer Response, the developer is asked to respond to each comment and tool-report that was produced, either agreeing or disagreeing to provide a fix that addresses the issue that was raised. In the third phase, Closeout Resolution, all disagreements are discussed in a meeting of all parties involved, and a resolution is made for all disagreements. The first two phases generally take one week each, and the third phase is concluded in a single closeout meeting.

Holzmann, Gerald J.↗

NASA NTRS2009

Optimization of OT-MACH Filter Generation for Target Recognition

An automatic Optimum Trade-off Maximum Average Correlation Height (OT-MACH) filter generator for use in a gray-scale optical correlator (GOC) has been developed for improved target detection at JPL. While the OT-MACH filter has been shown to be an optimal filter for target detection, actually solving for the optimum is too computationally intensive for multiple targets. Instead, an adaptive step gradient descent method was tested to iteratively optimize the three OT-MACH parameters, alpha, beta, and gamma. The feedback for the gradient descent method was a composite of the performance measures, correlation peak height and peak to side lobe ratio. The automated method generated and tested multiple filters in order to approach the optimal filter quicker and more reliably than the current manual method. Initial usage and testing has shown preliminary success at finding an approximation of the optimal filter, in terms of alpha, beta, gamma values. This corresponded to a substantial improvement in detection performance where the true positive rate increased for the same average false positives per image.

false alarm reduction↗

NASA NTRSDate not supplied

Acoustic Emission Methods for the Health Evaluation of Integrated Pressure Components

At the request of the Pressure Systems Senior Discipline Expert, to overcome the access and detectable flaw size limitations of other nondestructive evaluation (NDE) methods for integrated components, a technique akin to seismology was advanced to perceive the earliest signs of cracking in titanium components before they escalate into serious issues. In this project we achieved a landmark in Acoustic Emission (AE) testing with the first NASA Probability of Detection (POD) study. This is an important first step to the certification of the AE method for flight components and was based on the testing of notched panels. By working closely with industry allies and employing approved methods from the boiler and pressure vessel sector, we not only met but surpassed all POD benchmarks with our test and acceptance criteria. In experiments involving solution heat-treated and aged 6Al-4V-Ti alloy tested with broadband sensors, we recorded a 96% success rate, with zero false positives reported. Our research shows that in some cases, AE testing can locate smaller cracks than other NDE methods. An industry tool for scaling results to full pressure vessels is highlighted. However, before applying our findings to NASA components, tests to validate the transfer functions are advised.

acoustic↗

NASA NTRS2010

Daytime Water Detection Based on Color Variation

Robust water detection is a critical perception requirement for unmanned ground vehicle (UGV) autonomous navigation. This is particularly true in wide open areas where water can collect in naturally occurring terrain depressions during periods of heavy precipitation and form large water bodies (such as ponds). At far range, reflections of the sky provide a strong cue for water. But at close range, the color coming out of a water body dominates sky reflections and the water cue from sky reflections is of marginal use. We model this behavior by using water body intensity data from multiple frames of RGB imagery to estimate the total reflection coefficient contribution from surface reflections and the combination of all other factors. Then we describe an algorithm that uses one of the color cameras in a forward- looking, UGV-mounted stereo-vision perception system to detect water bodies in wide open areas. This detector exploits the knowledge that the change in saturation-to-brightness ratio across a water body from the leading to trailing edge is uniform and distinct from other terrain types. In test sequences approaching a pond under clear, overcast, and cloudy sky conditions, the true positive and false negative water detection rates were (95.76%, 96.71%, 98.77%) and (0.45%, 0.60%, 0.62%), respectively. This software has been integrated on an experimental unmanned vehicle and field tested at Ft. Indiantown Gap, PA.

reflection coefficient↗

NASA NTRS2020

A Demonstration that Correcting for Completeness and Reliability Is Critical for Robust Occurrence Rates

A measurement of planetary occurrence rates based on a planet catalog should be robust against details of how initial detections were classified as planets or false positives. This is accomplished by supplying the catalog’s rate of missed planets (completeness) and rate of non-planets incorrectly called planets (reliability). The final Kepler data release (DR25) includes products that can be used with the DR25 planet candidate catalog to correct for completeness and reliability in occurrence rate estimates. This is made possible by the Kepler Robovetter, which algorithmically and uniformly selects planets based on a variety of metrics and thresholds. Completeness, reliability, and occurrence rates potentially depend on these Robovetter thresholds. We study the impact of varying these vetting thresholds using the techniques of Bryson et al. 2019 (arXiv:1906.03575). We explore sets of thresholds that result in more or fewer planets (trading off completeness for reliability), as well as thresholds tuned to pass DR25 false positives identified as possible planets by the Kepler False Positive Working Group. We find that when correcting only for completeness, and not reliability, the resulting occurrence rates have a strong dependence on these threshold sets. For example, the value of SAG13 eta-Earth varies by over a factor of 4 when not corrected for reliability. However, when correcting for both completeness and reliability, occurrence rates using our threshold sets are statistically indistinguishable, with differences being well inside 1-sigma error bars. We present occurrence rates integrated over several period-radius ranges. For example, SAG13 eta-Earth is consistent with 0.127 (+0.094)(-0.054) (from Bryson et al. 2019) for all the Robovetter threshold sets. This result emphasizes the importance of correcting occurrence rates for both completeness and reliability. This suggests that inconsistent completeness and reliability correction may be a significant contributor to the large variation of occurrence rates in recent literature. We plan to make the Robovetter results for our threshold sets available, and encourage the community to use them to examine whether other occurrence rate methods yield similarly robust results.

Bryson, S.↗

NASA NTRS2005

Intelligent, Self-Diagnostic Thermal Protection System for Future Spacecraft

The goal of this project is to provide self-diagnostic capabilities to the thermal protection systems (TPS) of future spacecraft. Self-diagnosis is especially important in thermal protection systems (TPS), where large numbers of parts must survive extreme conditions after weeks or years in space. In-service inspections of these systems are difficult or impossible, yet their reliability must be ensured before atmospheric entry. In fact, TPS represents the greatest risk factor after propulsion for any transatmospheric mission. The concepts and much of the technology would be applicable not only to the Crew Exploration Vehicle (CEV), but also to ablative thermal protection for aerocapture and planetary exploration. Monitoring a thermal protection system on a Shuttle-sized vehicle is a daunting task: there are more than 26,000 components whose integrity must be verified with very low rates of both missed faults and false positives. The large number of monitored components precludes conventional approaches based on centralized data collection over separate wires; a distributed approach is necessary to limit the power, mass, and volume of the health monitoring system. Distributed intelligence with self-diagnosis further improves capability, scalability, robustness, and reliability of the monitoring subsystem. A distributed system of intelligent sensors can provide an assurance of the integrity of the system, diagnosis of faults, and condition-based maintenance, all with provable bounds on errors.

Hyers, Robert W.↗

NASA NTRS2010

Testing of Haar-Like Feature in Region of Interest Detection for Automated Target Recognition (ATR) System

The objectives of this project were to develop a ROI (Region of Interest) detector using Haar-like feature similar to the face detection in Intel's OpenCV library, implement it in Matlab code, and test the performance of the new ROI detector against the existing ROI detector that uses Optimal Trade-off Maximum Average Correlation Height filter (OTMACH). The ROI detector included 3 parts: 1, Automated Haar-like feature selection in finding a small set of the most relevant Haar-like features for detecting ROIs that contained a target. 2, Having the small set of Haar-like features from the last step, a neural network needed to be trained to recognize ROIs with targets by taking the Haar-like features as inputs. 3, using the trained neural network from the last step, a filtering method needed to be developed to process the neural network responses into a small set of regions of interests. This needed to be coded in Matlab. All the 3 parts needed to be coded in Matlab. The parameters in the detector needed to be trained by machine learning and tested with specific datasets. Since OpenCV library and Haar-like feature were not available in Matlab, the Haar-like feature calculation needed to be implemented in Matlab. The codes for Adaptive Boosting and max/min filters in Matlab could to be found from the Internet but needed to be integrated to serve the purpose of this project. The performance of the new detector was tested by comparing the accuracy and the speed of the new detector against the existing OTMACH detector. The speed was referred as the average speed to find the regions of interests in an image. The accuracy was measured by the number of false positives (false alarms) at the same detection rate between the two detectors.

neural network↗

NASA NTRS2011

Modeling Kepler Transit Light Curves as False Positives: Rejection of Blend Scenarios for Kepler-9, and Validation of Kepler-9 d, a Super-Earth-Size Planet in a Multiple System

Light curves from the Kepler Mission contain valuable information on the nature of the phenomena producing the transit-like signals. To assist in exploring the possibility that they are due to an astrophysical false positive we describe a procedure (BLENDER) to model the photometry in terms of a blend rather than a planet orbiting a star. A blend may consist of a background or foreground eclipsing binary (or star-planet pair) whose eclipses are attenuated by the light of the candidate and possibly other stars within the photometric aperture. We apply BLENDER to the case of Kepler-9 (KIC 3323887), a target harboring two previously confirmed Saturn-size planets (Kepler-9 b and Kepler-9 c) showing transit timing variations, and an additional shallower signal with a 1.59 day period suggesting the presence of a super-Earth-size planet. Using BLENDER together with constraints from other follow-up observations we are able to rule out all blends for the two deeper signals and provide independent validation of their planetary nature. For the shallower signal, we rule out a large fraction of the false positives that might mimic the transits. The false alarm rate for remaining blends depends in part (and inversely) on the unknown frequency of small-size planets. Based on several realistic estimates of this frequency, we conclude with very high confidence that this small signal is due to a super-Earth-size planet (Kepler-9 d) in a multiple system, rather than a false positive. The radius is determined to be 1.64(exp)(sub-14),R, and current spectroscopic observations are as yet insufficient to establish its mass.

PLANETARY SYSTEMS↗

NASA NTRS2003

Optical landmark detection for spacecraft navigation

In this paper, a new crater detection algorithm is suggested. Experimental studies show that this new algorithm can achieve sub-pixel accuracy in position, its detection rate is better than 90% and its false alarm rate is less than 5%. These good characteristics indicate that it is an ideal crater detection algorithm for spacecraft optical navigation.

optical↗