Results for “probability of detection” — Page 18

NASA NTRS2011

An Overview of the Total Lightning Jump Algorithm: Past, Present and Future Work

Rapid increases in total lightning prior to the onset of severe and hazardous weather have been observed for several decades. These rapid increases are known as lightning jumps and can precede the occurrence of severe weather by tens of minutes. Over the past decade, a significant effort has been made to quantify lightning jump behavior in relation to its utility as a predictor of severe and hazardous weather. Based on a study of 34 thunderstorms that occurred in the Tennessee Valley, early work conducted in our group at Huntsville determined that it was indeed possible to create a reasonable operational lightning jump algorithm (LJA) based on a statistical framework relying on the variance behavior of the lightning trending signal. We the expanded this framework and tested several variance-related LJA configurations on a much larger sample of 87 severe and non severe thunderstorms. This study determined that a configuration named the "2(sigma)" algorithm had the most promise in development of the operational LJA with a probability of detection (POD) of 87%, a false alarm rate (FAR) of 33%, a Heidke Skill Score (HSS) of 0.75. The 2(sigma) algorithm was then tested on an even larger sample of 711 thunderstorms of all types from four regions of the country where total lightning measurement capability existed. The result was very encouraging.Despite the larger number of storms and the inclusion of different regions of the country, the POD remained high (79%), the FAR was low (36%) and HSS was solid (0.71). Average lead time from jump to severe weather occurrence was 20.65 minutes, with a standard deviation of +/- 15 minutes. Also, trends in total lightning were compared to cloud to ground (CG) lightning trends, and it was determined that total lightning trends had a higher POD (79% vs 66%), lower FAR (36% vs 54 %) and a better HSS (0.71 vs 0.55). From the 711-storm case study it was determined that a majority of missed events were due to severe weather producing thunderstorms in low flashing environments. The latest efforts have been geared toward examining these low flashing storms in order to adjust the algorithm for such storms, thus enhancing the capability of the LJA. Future work will test the algorithm in real time using current satellite and radar based cell tracking methods, as well as, comparing total lightning jump occurrence to both satellite based and ground base observations of thunderstorms to create correlations between lightning jumps and the observed structures within thunderstorms. Finally this algorithm will need to be tested using Geostationary Lightning Mapper proxy data to transition the algorithm from VHF ground based lightning measurements to lower frequency space-based lightning measurements.

Schultz, Christopher J.↗

NASA NTRS2009

Ultrasonic Phased Array Simulations of Welded Components at NASA

Comprehensive and accurate inspections of welded components have become of increasing importance as NASA develops new hardware such as Ares rocket segments for future exploration missions. Simulation and modeling will play an increasing role in the future for nondestructive evaluation in order to better understand the physics of the inspection process, to prove or disprove the feasibility for an inspection method or inspection scenario, for inspection optimization, for better understanding of experimental results, and for assessment of probability of detection. This study presents simulation and experimental results for an ultrasonic phased array inspection of a critical welded structure important for NASA future exploration vehicles. Keywords: nondestructive evaluation, computational simulation, ultrasonics, weld, modeling, phased array

Roth, D. J.↗

NASA NTRS2014

Lightning Jump Algorithm Development for the GOES·R Geostationary Lightning Mapper

Current work on the lightning jump algorithm to be used in GOES‐R Geostationary Lightning Mapper (GLM)'s data stream is multifaceted due to the intricate interplay between the storm tracking, GLM proxy data, and the performance of the lightning jump itself. This work outlines the progress of the last year, where analysis and performance of the lightning jump algorithm with automated storm tracking and GLM proxy data were assessed using over 700 storms from North Alabama. The cases analyzed coincide with previous semi‐objective work performed using total lightning mapping array (LMA) measurements in Schultz et al. (2011). Analysis shows that key components of the algorithm (flash rate and sigma thresholds) have the greatest influence on the performance of the algorithm when validating using severe storm reports. Automated objective analysis using the GLM proxy data has shown probability of detection (POD) values around 60% with false alarm rates (FAR) around 73% using similar methodology to Schultz et al. (2011). However, when applying verification methods similar to those employed by the National Weather Service, POD values increase slightly (69%) and FAR values decrease (63%). The relationship between storm tracking and lightning jump has also been tested in a real‐time framework at NSSL. This system includes fully automated tracking by radar alone, real‐time LMA and radar observations and the lightning jump. Results indicate that the POD is strong at 65%. However, the FAR is significantly higher than in Schultz et al. (2011) (50‐80% depending on various tracking/lightning jump parameters) when using storm reports for verification. Given known issues with Storm Data, the performance of the real‐time jump algorithm is also being tested with high density radar and surface observations from the NSSL Severe Hazards Analysis & Verification Experiment (SHAVE).

Schultz. E.↗

NASA NTRS2014

MAG4 Versus Alternative Techniques for Forecasting Active-Region Flare Productivity

MAG4 is a technique of forecasting an active region's rate of production of major flares in the coming few days from a free-magnetic-energy proxy. We present a statistical method of measuring the difference in performance between MAG4 and comparable alternative techniques that forecast an active region's major-flare productivity from alternative observed aspects of the active region. We demonstrate the method by measuring the difference in performance between the "Present MAG4" technique and each of three alternative techniques, called "McIntosh Active-Region Class," "Total Magnetic Flux," and "Next MAG4." We do this by using (1) the MAG4 database of magnetograms and major-flare histories of sunspot active regions, (2) the NOAA table of the major-flare productivity of each of 60 McIntosh active-region classes of sunspot active regions, and (3) five technique-performance metrics (Heidke Skill Score, True Skill Score, Percent Correct, Probability of Detection, and False Alarm Rate) evaluated from 2000 random two-by-two contingency tables obtained from the databases. We find that (1) Present MAG4 far outperforms both McIntosh Active-Region Class and Total Magnetic Flux, (2) Next MAG4 significantly outperforms Present MAG4, (3) the performance of Next MAG4 is insensitive to the forward and backward temporal windows used, in the range of one to a few days, and (4) forecasting from the free-energy proxy in combination with either any broad category of McIntosh active-region classes or any Mount Wilson active-region class gives no significant performance improvement over forecasting from the free-energy proxy alone (Present MAG4).

Falconer, David A.↗

NASA NTRS2015

NASA DOEPOD NDE Capabilities Data Book

This data book contains the Directed Design of Experiments for Validating Probability of Detection (POD) Capability of NDE Systems (DOEPOD) analyses of the nondestructive inspection data presented in the NTIAC, Nondestructive Evaluation (NDE) Capabilities Data Book. DOEPOD is designed as a decision support system to validate inspection system, personnel, and protocol demonstrating 0.90 POD with 95% confidence at critical flaw sizes, a90/95. Although 0.90 POD with 95% confidence at critical flaw sizes is often stated as an inspection requirement in inspection documents, including NASA Standards, NASA critical aerospace applications have historically only accepted 0.978 POD or better with a 95% one-sided lower confidence bound exceeding 0.90 at critical flaw sizes, a90/95.

Generazio, Edward R.↗

NASA NTRS2014

MAG4 Versus Alternative Techniques for Forecasting Active-Region Flare Productivity

MAG4 is a technique of forecasting an active region's rate of production of major flares in the coming few days from a free-magnetic-energy proxy. We present a statistical method of measuring the difference in performance between MAG4 and comparable alternative techniques that forecast an active region's major-flare productivity from alternative observed aspects of the active region. We demonstrate the method by measuring the difference in performance between the "Present MAG4" technique and each of three alternative techniques, called "McIntosh Active-Region Class," "Total Magnetic Flux," and "Next MAG4." We do this by using (1) the MAG4 database of magnetograms and major-flare histories of sunspot active regions, (2) the NOAA table of the major-flare productivity of each of 60 McIntosh active-region classes of sunspot active regions, and (3) five technique-performance metrics (Heidke Skill Score, True Skill Score, Percent Correct, Probability of Detection, and False Alarm Rate) evaluated from 2000 random two-by-two contingency tables obtained from the databases. We find that (1) Present MAG4 far outperforms both McIntosh Active-Region Class and Total Magnetic Flux, (2) Next MAG4 significantly outperforms Present MAG4, (3) the performance of Next MAG4 is insensitive to the forward and backward temporal windows used, in the range of one to a few days, and (4) forecasting from the free-energy proxy in combination with either any broad category of McIntosh active-region classes or any Mount Wilson active-region class gives no significant performance improvement over forecasting from the free-energy proxy alone (Present MAG4).

Falconer, David A.↗

NASA NTRS2016

Normalized Temperature Contrast Processing in Flash Infrared Thermography

The paper presents further development in normalized contrast processing of flash infrared thermography method by the author given in US 8,577,120 B1. The method of computing normalized image or pixel intensity contrast, and normalized temperature contrast are provided, including converting one from the other. Methods of assessing emissivity of the object, afterglow heat flux, reflection temperature change and temperature video imaging during flash thermography are provided. Temperature imaging and normalized temperature contrast imaging provide certain advantages over pixel intensity normalized contrast processing by reducing effect of reflected energy in images and measurements, providing better quantitative data. The subject matter for this paper mostly comes from US 9,066,028 B1 by the author. Examples of normalized image processing video images and normalized temperature processing video images are provided. Examples of surface temperature video images, surface temperature rise video images and simple contrast video images area also provided. Temperature video imaging in flash infrared thermography allows better comparison with flash thermography simulation using commercial software which provides temperature video as the output. Temperature imaging also allows easy comparison of surface temperature change to camera temperature sensitivity or noise equivalent temperature difference (NETD) to assess probability of detecting (POD) anomalies.

Koshti, Ajay M.↗

NASA NTRS2015

NASA NDE Program

The current activities in the National Aeronautics and Space Administration Nondestructive Evaluation (NDE) Program are presented. The topics covered include organizational communications, orbital weld inspection, electric field imaging, fracture critical probability of detection validation, monitoring of thermal protection systems, physical and document standards, image quality indicators, integrity of composite pressure vessels, and NDE for additively manufactured components.

Generazio, Ed↗

NASA NTRS2016

Automated Storm Tracking and the Lightning Jump Algorithm Using GOES-R Geostationary Lightning Mapper (GLM) Proxy Data

This study develops a fully automated lightning jump system encompassing objective storm tracking, Geostationary Lightning Mapper proxy data, and the lightning jump algorithm (LJA), which are important elements in the transition of the LJA concept from a research to an operational based algorithm. Storm cluster tracking is based on a product created from the combination of a radar parameter (vertically integrated liquid, VIL), and lightning information (flash rate density). Evaluations showed that the spatial scale of tracked features or storm clusters had a large impact on the lightning jump system performance, where increasing spatial scale size resulted in decreased dynamic range of the system's performance. This framework will also serve as a means to refine the LJA itself to enhance its operational applicability. Parameters within the system are isolated and the system's performance is evaluated with adjustments to parameter sensitivity. The system's performance is evaluated using the probability of detection (POD) and false alarm ratio (FAR) statistics. Of the algorithm parameters tested, sigma-level (metric of lightning jump strength) and flash rate threshold influenced the system's performance the most. Finally, verification methodologies are investigated. It is discovered that minor changes in verification methodology can dramatically impact the evaluation of the lightning jump system.

lightning jump↗

NASA NTRS2018

Optimization of Second Fault Detection Thresholds to Maximize Mission Probability of Success

No abstract available

Anzalone, Evan↗

NASA NTRS2019

Predicting Well-Connected SEP Events from Observations of Solar EUVs and Energetic Protons

This study shows a quantitative assessment of the use of Extreme Ultraviolet (EUV) observations in the prediction of Solar Energetic Proton (SEP) events. The UMASEP scheme (Space Weather, 9, S07003, 2011; 13, 2015, 807-819) forecasts the occurrence and the intensity of the first hours of SEP events. in order to predict well-connected events, this scheme correlates Solar Soft X-rays (SXR) with differential proton fluxes of the GOES satellites. In this study, we explore the use of the EUV time history from GOES-EUVS and SDO-AIA instruments in the UMASEP scheme. This study presents the results of the prediction of the occurrence of well-connected >10 MeV SEP events, for the period from May 2010 to December 2017, in terms of Probability of Detection (POD), False Alarm Ratio (FAR), Crticial Success Index (CSI), and the average and median of the warning times. The UMASEP/EUV-based models were calibrated using GOES and SDO data from May 2010 to October 2014, and validated using out-of-sample SDO data from November 2014 to December 2017. The best results were obtained by those models that used EUV data in the range 50-340 angstroms. We conclude that the UMASEP/EUV-based models yield similar or better POD results, and similar or worse FAR results, than those of the current real-time UMSEP/SXR-based model. The reason for the higher POD of the UMASEP/EUV-based models in the range of 50-340 angstroms was due to the high percentage of successful predictions of well-connected SEP events associated with 10 MeV SEP events, improves the overall performance, obtaining a POD of 92.9% (39/42) compared with 81% (34/42) of the current tool, and a slightly worse FAR of 31.6% (18/57) compared with 29.2% (14/58) of the current tool.

Nunez, Marlon↗

NASA NTRS2021

Modeling of Guided Waves for Aerospace Applications

Advancements in computer hardware has led to new possibilities for rapid modeling and simulation capabilities across many scientific fields. Nondestructive evaluation (NDE) can benefit from increased use of simulation tools to guide optimization of inspection and health monitoring methods, enhance understanding of data, aid in development of defect characterization methods, and generate data sets for use with machine learning and model-assisted probability of detection. Recent work at NASA has entailed development and benchmarking of both custom simulation codes and commercial simulation tools for ultrasonic wave propagation. This paper describes recent work at NASA in modeling of guided waves in composites and other aerospace materials. Results and computational speeds for a composite benchmark case are reported for a custom finite difference Rotated Staggered Grid code and for the commercial finite element software package, Pogo. Recent progress in linking NDE models to parametric analysis tools is also discussed.

Nondestructive evaluation↗

NASA NTRS2021

Investigating the potential of a global precipitation forecast to inform landslide prediction

Extreme rainfall events within landslide-prone areas can be catastrophic, resulting in loss of property, infrastructure, and life. A global Landslide Hazard Assessment for Situational Awareness (LHASA) model provides routine near-real time estimates of landslide hazard using Integrated Multi-Satellite Precipitation Retrievals for the Global Precipitation Mission (IMERG). However, it does not provide information on potential landslide hazard in the future. Forecasting potential landslide events at a global scale presents an area of open research. This study compares a global precipitation forecast provided by NASA's Goddard Earth Observing System (GEOS) with near-real time satellite precipitation estimates. The Multi-Radar Multi-Sensor gauge corrected (MRMS-GC) reference is used to assess the performance of both satellite and model-based precipitation products over the contiguous United States (CONUS). The forecast lead time of 24hrs is considered, with a focus on extreme precipitation events. The performance of IMERG and GEOS-Forecast products is assessed in terms of the probability of detection, success ratio, critical success index and hit bias as well as continuous statistics. The results show that seasonality influences the performance of both satellite and model-based precipitation products. Comparison of IMERG and GEOS-Forecast globally as well as in several event case studies (Colombia, southeast Asia, and Tajikistan) reveals that GEOS-Forecast detects extreme rainfall more frequently relative to IMERG for these specific analyses. For recent landslide points across the globe, the 24hr accumulated precipitation forecast >100 mm corresponds well with near-real time daily accumulated IMERG precipitation estimates. GEOS-Forecast and IMERG precipitation match more closely for tropical cyclones than for other types of storms. The main intention of this study is to assess the viability of using a global forecast for landslide predictions and understand the extent of the variability between these products to inform where we would expect the landslide modeling results to most prominently diverge. Results of this study will be used to inform how forecasted precipitation estimates can be incorporated into the LHASA model to provide the first global predictive view of landslide hazards.

S. Khan↗

NASA NTRSDate not supplied

A Markov Random Field Model for Texture-based Segmentation of Small Cracks in Thin InconelTubes

Markov random fields have been used for image segmentation since their introduction in the1980s. This work applies a method from Principal Component Thermography to enhance thecontrast in damage regions in 3D images derived from X-ray computed tomography (CT)inspections. The developed method is applied to sizing of small cracks in thin Inconel tubes designed as probability of detection (POD) samples for radiographic inspection.Misclassification errors arising from artifacts due to beam-hardening are reduced by fitting the boundary of the segmented damage region with the arc of an ellipse. Results are comparedagainst those obtained through manual inspection.

Image segmentation↗

NASA NTRS2021

Investigating the Potential of A Global Precipitation Forecast to Inform Landslide Prediction

Extreme rainfall events within landslide-prone areas can be catastrophic, resulting in loss of property, infra-structure, and life. A global Landslide Hazard Assessment for Situational Awareness (LHASA) model provides routine near-real time estimates of landslide hazard using Integrated Multi-Satellite Precipitation Retrievals for the Global Precipitation Mission (IMERG). However, it does not provide information on potential landslide hazard in the future. Forecasting potential landslide events at a global scale presents an area of open research. This study compares a global precipitation forecast provided by NASA’s Goddard Earth Observing System (GEOS) with near-real time satellite precipitation estimates. The Multi-Radar Multi-Sensor gauge corrected (MRMS-GC) reference is used to assess the performance of both satellite and model-based precipitation products over the contiguous United States (CONUS). The forecast lead time of 24hrs is considered, with a focus on extreme precipitation events. The performance of IMERG and GEOS-Forecast products is assessed in terms of the probability of detection, success ratio, critical success index and hit bias as well as continuous statistics. The results show that seasonality influences the performance of both satellite and model-based precipitation products. Comparison of IMERG and GEOS-Forecast globally as well as in several event case studies (Colombia, southeast Asia, and Tajikistan) reveals that GEOS-Forecast detects extreme rainfall more frequently relative to IMERG for these specific analyses. For recent landslide points across the globe, the 24hr accumulated precipitation forecast >100 mm corresponds well with near-real time daily accumulated IMERG precipitation estimates. GEOS-Forecast and IMERG precipitation match more closely for tropical cyclones than for other types of storms. The main intention of this study is to assess the viability of using a global forecast for landslide predictions and understand the extent of the variability between these products to inform where we would expect the landslide modeling results to most prominently diverge. Results of this study will be used to inform how forecasted precipitation estimates can be incorporated into the LHASA model to provide the first global predictive view of landslide hazards.

S. Khan↗

NASA NTRSDate not supplied

Role of NDE and In-Situ Process Monitoring in Managing Risk of AM Space Hardware

The recently published NASA-STD-6030 defines the Additive Manufacturing (AM) Requirements for Spaceflight Systems. Key aspects of the certification approach include the development of a qualified material process (QMP) and material characterization determined by part classification. Nondestructive evaluation (NDE) of the full surface and volume is required for all part classifications except those with negligible risk. NASA is exploring the use of in-process monitoring data to improve risk posture and supplement post-build inspection for complex parts. Currently, the most challenging obstacle to overcome is linking the indications in the monitoring data to the physics of the process and the final material state of the finished part. NASA is undertaking studies to understand and quantify this relationship for various monitoring methods. The desired goal is to develop a protocol to establish this correlation for any monitoring method. Once this correlation is known, the critical defect size can be linked to a representative indication in the monitoring data, and the capability of the monitoring system can be tested using the 90/95 probability of detection requirement for NDE methods. This would enable the use of in-process monitoring as a defect screening activity for AM part certification. Many high-criticality components built with AM have high complexity and therefore limited inspectability, so using in-process monitoring can help address this certification gap. The use of adaptive, closed-loop monitoring systems that alter the locked process will require a new approach to the QMP.

additive manufacturing↗

NASA NTRSDate not supplied

Role of NDE and In-Situ Process Monitoring in Managing Risk of AM Space Hardware

The recently published NASA-STD-6030 defines the Additive Manufacturing (AM) Requirements for Spaceflight Systems. Key aspects of the certification approach include the development of a qualified material process (QMP) and material characterization determined by part classification. Nondestructive evaluation (NDE) of the full surface and volume is required for all part classifications except those with negligible risk. NASA is exploring the use of in-process monitoring data to improve risk posture and supplement post-build inspection for complex parts. Currently, the most challenging obstacle to overcome is linking the indications in the monitoring data to the physics of the process and the final material state of the finished part. NASA is undertaking studies to understand and quantify this relationship for various monitoring methods. The desired goal is to develop a protocol to establish this correlation for any monitoring method. Once this correlation is known, the critical defect size can be linked to a representative indication in the monitoring data, and the capability of the monitoring system can be tested using the 90/95 probability of detection requirement for NDE methods. This would enable the use of in-process monitoring as a defect screening activity for AM part certification. Many high-criticality components built with AM have high complexity and therefore limited inspectability, so using in-process monitoring can help address this certification gap. The use of adaptive, closed-loop monitoring systems that alter the locked process will require a new approach to the QMP.

advanced manufacturing↗

NASA NTRS2021

Evaluation of VIIRS and MODIS snow cover fraction in High-Mountain Asia using Landsat 8 OLI

We present thefirst application of the Snow Covered Area and Grain size model (SCAG) tothe Visible Infrared imaging Radiometer Suite (VIIRS) and assess these retrievals withfiner-resolution fractional snow cover maps from Landsat 8 Operational Land Imager (OLI).Because Landsat 8 OLI avoids saturation issues common to Landsat 1–7 in the visiblewavelengths, we re-assess the accuracy of the SCAG fractional snow cover maps fromModerate Resolution Imaging Spectroradiometer (MODIS) that were previously evaluatedusing data from earlier Landsat sensors. Use of the fractional snow cover maps fromLandsat 8 OLI shows a negative bias of−0.5% for MODSCAG and−1.3% for VIIRSCAG,whereas previous MODSCAG evaluations found a bias of−7.6% in the Himalaya. Wefindsimilar root mean squared error (RMSE) values of 0.133 and 0.125 for MODIS and VIIRS,respectively. The Recall statistic (probability of detection) for cells with more than 15%snow cover in this challenging steep topography was found to be 0.90 for both MODSCAGand VIIRSCAG, significantly higher than previous evaluations based on Landsat 5Thematic Mapper (TM) and 7 Enhanced Thematic Mapper Plus (ETM+). In addition,daily retrievals from MODIS and VIIRS are consistent across gradients of elevation, slope,and aspect. Different native resolutions of the gridded products at 1 km and 500 m forVIIRS and MODIS, respectively, result in snow cover maps showing a slightly differentdistribution of values with VIIRS having more mixed pixels and MODIS having 7% morepure snow pixels. Despite the resolution differences, the snow maps from both sensorsproduce similar total snow-covered areas and snow-line elevations in this region, withR2values of 0.98 and 0.88, respectively. Wefind that the SCAG algorithm performsconsistently across various spatial resolutions and that fractional snow cover mapsfrom the VIIRS instruments aboard Suomi NPP, JPPS–1, and JPPS–2 can be asuitable replacement as MODIS sensors reach their ends of life.

Karl Rittger↗